GitHub user coderplay added a comment to the discussion: Short-Term Memory Design
My 2 cents: 1. It looks like the data types for short-term memory are primitives and nested objects, correct? If we want to review the history of the last 5 runs, how would we do that? 2. Flink state is persisted permanently and uploaded to blob storage after a checkpoint. What is the retention period for our short-term memory? How does the user define this retention? 3. Conversation history is the most common form of short-term memory, but long conversations (in our case, which means large number of runs) are challenging for current LLMs. A full history may not fit within an LLM’s context window, leading to unrecoverable errors. Even if the LLM supports the full context length, most models still perform poorly over long contexts, becoming “distracted” by stale or off-topic content while also suffering from slower response times and higher costs. What mechanisms does our design provide to address this issue? GitHub link: https://github.com/apache/flink-agents/discussions/40#discussioncomment-13715117 ---- This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected]
