rosemarYuan opened a new pull request, #887: URL: https://github.com/apache/flink-agents/pull/887
<!-- * Thank you very much for contributing to Flink Agents. * Please add the relevant components in the PR title. E.g., [api], [runtime], [java], [python], [hotfix], etc. --> <!-- Please link the PR to the relevant issue(s). Hotfix doesn't need this. --> Linked issue: #886 ### Purpose of change Follow-up to #876, #886 .This change makes agent memory activity observable through the existing event infrastructure. **Emission model.** * Memory events are emitted at the action finish boundary, together with the action's user events. * The framework emits at most one event per memory scope and operation kind, for example, one short-term write event per action. * Each event's `value` is a dot-key flat map describing the operation's **net effect**. Within one action, the last read/write for the same path wins. * **Failed or unfinished actions emit no memory events.** * Actions triggered by memory events do not emit memory events themselves, preventing recursive subscriptions. **Recovery semantics.** Generation remains consistent with the framework's existing delivery and durability guarantees: * **At-least-once on replay.** After a failure, a replayed completed action may re-log its persisted memory events, and a replayed input may re-emit its run-begin event with restored short-term memory. Event Log consumers should tolerate duplicates. * **Durable execution.** A long-term operation wrapped in `durable_execute` is not recorded again when its result is replayed from state. Generated events describe operations that actually executed in the current attempt. ### Tests Coverage targets the design-critical paths: * **Cross-language wire-format conformance** — `MemoryEventWireFormatTest` in Java and `test_memory_event_wire_format.py` in Python pin the same attribute JSON, preventing SDK drift. * **Flush semantics** — `TestMemoryObservationFlush` covers emit-once-per-finished-action behavior, no emission on unfinished/failed drains, config gating, suppression awareness, and drain-then-discard semantics. A suppressed or disabled action still drains the long-term buffer, so records cannot leak into the next same-key action. * **Config fallback** — `TestMemoryEventSettings` covers all three resolution levels, including the case where the master switch is off but an operation-specific sub-key is explicitly on. * **Dot-key folding** — `TestMemoryEventBuilder` covers empty paths, single paths, nested paths, last-wins behavior, and long-term add/delete/get/search value shapes. * **Polymorphic serde** — `ActionStateSerdeTest` verifies that memory events round-trip through the `@class` mixin, so they can be persisted and replayed with the action's output events. * **Agent-run begin** — `AgentRunBeginEventEmissionTest`, `AgentRunBeginEventTest`, and `test_run_event` cover snapshot content, per-key isolation, and the guarantee that the snapshot is taken before the run's own writes. * **Long-term recording in Python** — `test_mem0_op_recording` and `test_mem0_recording_hook` cover thread-safe buffering and drain-by-key behavior, including re-buffering of non-matching keys. * **TTL default** — `ShortTermMemoryTTLIntegrationTest` pins the new `ON_CREATE_AND_WRITE` default. * **End-to-end logging** — `memory_event_logging_test.py` exercises the full emit → Event Log path. ### API Java: * `org.apache.flink.agents.api.event.MemoryEvent` * `org.apache.flink.agents.api.event.ShortTermWriteEvent` * `org.apache.flink.agents.api.event.ShortTermReadEvent` * `org.apache.flink.agents.api.event.SensoryWriteEvent` * `org.apache.flink.agents.api.event.SensoryReadEvent` * `org.apache.flink.agents.api.event.LongTermUpdateEvent` * `org.apache.flink.agents.api.event.LongTermGetEvent` * `org.apache.flink.agents.api.event.LongTermSearchEvent` * `org.apache.flink.agents.api.event.AgentRunBeginEvent` * The corresponding `EventType.*` constants * `org.apache.flink.agents.api.configuration.MemoryEventOptions` Python: * `flink_agents.api.events.memory_event` * `flink_agents.api.events.run_event` * `flink_agents.api.core_options.MemoryEventOptions` The new memory-event surface — event classes, `EventType.*` constants, and `MemoryEventOptions` — does not depend on any long-term-memory backend. The observation recording seam is a runtime-side default-method extension point: `InternalBaseLongTermMemory.drainObservationRecordsJson`, whose default implementation returns `[]`. A future pure-Java backend can therefore implement its own recording without another API change. **Configuration.** Memory event generation is controlled per operation kind. Configuration is resolved in the following order: 1. Operation-specific sub-key. 2. `memory.generate-event` master switch. 3. Built-in per-operation default. The built-in defaults are: * Writes: on. * Long-term operations: on. * Short-term/sensory reads: off. The master switch and operation-specific switches intentionally use `null` defaults, so the built-in defaults remain reachable. `agent-run.begin-event` has its own switch, defaults to on, and is independent of the `memory.generate-event` master switch. **Notable behavior change.** - The repo-wide default of `short-term-memory.state-ttl.update-type` changes from `ON_READ_AND_WRITE` to `ON_CREATE_AND_WRITE`. - The run-begin event is enabled by default and scans the key's full short-term memory on every input. Under `ON_READ_AND_WRITE`, that scan would refresh every entry's TTL and prevent expiry. Users who require read-refresh TTL semantics can set the option back explicitly, or disable `agent-run.begin-event`. See the documentation's Semantics & Caveats section for details. ### Documentation <!-- Do not remove this section. Check the proper box only. --> - [ ] `doc-needed` <!-- Your PR changes impact docs --> - [ ] `doc-not-needed` <!-- Your PR changes do not impact docs --> - [x] `doc-included` Documentation is included in `docs/content/docs/development/memory/memory_events.md This change also adds notes to the sensory/short-term memory page and the configuration reference. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
