Fast ReActions: Planning and Reasoning Quickly with LLMs
One of the most important factors of Copilot products that utilize the chat interface is response latency. The expectation of low latency in chat makes it difficult to deliver high-quality results of complex actions like API calls and text summaries. This talk will quickly describe how Continual tackled some of these challenges when building an AI copilot platform.
