GitHub user coderplay edited a comment on the discussion: Short-Term Memory Design
A couple of questions: 1. What is typically used as the partition key here? Could you give an example? For instance, would it be a host name in logs, or a user ID in a clickstream? 2. It looks like the data types for short-term memory are primitives and nested objects, correct? Do we support lists, sets , queues? 3. If we want to call back the history of the last 5 runs, how would we do that? 4. Flink state is persisted permanently and uploaded to blob storage after a checkpoint. What is the retention period for our short-term memory? How does the user define this retention? 5. Lastly but most importantly, conversation history is the most common form of short-term memory, but long conversations (in our case, which means large number of runs) are challenging for current LLMs. A full history may not fit within an LLM’s context window, leading to unrecoverable errors. Even if the LLM supports the full context length, most models still perform poorly over long contexts, becoming “distracted” by stale or off-topic content while also suffering from slower response times and higher costs. What mechanisms does our design provide to address this issue? GitHub link: https://github.com/apache/flink-agents/discussions/40#discussioncomment-13715117 ---- This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected]
