wenjin272 opened a new pull request, #1429: URL: https://github.com/apache/flink-agents/pull/1429
Linked issue: #1428 (closes #1428) Design discussion: https://github.com/apache/flink-agents/discussions/1420 ### Purpose of change Actions can register delayed Events using native Flink timers, for approval polling, timeouts, and periodic checks without occupying a thread while waiting. Java and Python support processing time and event time. User documentation is included under **Actions → Delayed Execution**. The implementation is split into three feature commits (new-run processing timers, current-run processing timers, event-time timers), followed by one documentation commit. #### Runtime flow `RunnerContext` records timer operations during an Action. Successful completion resolves relative deadlines and journals operations in `ActionState`, then applies them with Memory updates. `ActionExecutionOperator` uses `InternalTimerService` for deadlines and keyed state for Event payloads. Expiration dispatches the Event through existing Action routing, either into its owning run or as a new input run. #### Key decisions Reuse registered Actions instead of adding a separate business callback API. Commit timer operations at Action completion so suspension does not expose partial effects. Event-time timers always start new runs; allowing them to hold their current run would conflict with waiting for earlier inputs before advancing event time. ### Behavioral Semantics #### Interaction decisions | Timer / mode | Context and same-key inputs | Output timestamp | | --- | --- | --- | | Processing / `TRIGGER_AFTER_CURRENT_RUN` | Fresh Sensory Memory; later inputs can proceed after the original run ends | Unset | | Processing / `CURRENT_RUN_WAIT_FOR_TRIGGER` | Retains context, Sensory Memory and attachments; later inputs wait | Original input timestamp | | Event time | New run only; no mode argument | Timer deadline | #### Behavioral contracts - Registration, replacement and cancellation take effect only when the Action completes successfully. Relative delays start then; replay retains the resolved deadline. - IDs are scoped to the operator and business key and shared across time domains. Same-ID registration replaces a pending timer; cancellation ignores missing IDs and cannot retract delivered Events. - Each registration fires once. Registering again from its Action supports periodic work. - Watermarks wait for earlier inputs, then for the runs triggered by due event-time timers and their subsequent Actions. Newly registered due timers are checked again at the same Watermark. Late registrations fire without requiring another Watermark and do not move downstream Watermarks backward. - Pending registrations, dispatched Events, and current-run waits survive checkpoint recovery and rescaling. #### Failure behavior Invalid IDs, negative delays, invalid timestamps/modes, nonserializable Events and `OutputEvent` targets raise errors. New-run Events reject attachments. Registration/cancellation inside `execute_async`/`executeAsync` tasks or internal subagent/chat invocations is rejected. Failed Actions do not apply their timer operations; triggered Actions use the existing failure and recovery path. Timers target streaming execution; future new-run processing timers do not keep bounded input open. ### Tests | Contract | Coverage | | --- | --- | | Completion-only effects, relative deadlines and replay | `NativeTimerTest`: await/commit, failed Action, journaled deadline | | Replacement, cancellation and shared IDs | `NativeTimerTest`, `EventTimeTimerTest`: same deadline, already queued Event, cross-domain replacement | | One-shot delivery and repeated checks | Java `NativeTimerE2ETest`; Python `native_timer_test.py` | | Current-run context, attachments and input ordering | `CurrentRunTimerTest`; current-run variants in Java/Python E2Es | | Watermark ordering, output timestamps and late registrations | `EventTimeTimerTest`; Java/Python event-time E2Es | | Recovery, replay and rescaling | All three operator timer suites; Python `timer_recovery_test.py` injects failure after a completed checkpoint | | Recovery does not corrupt another key's Memory | `CurrentRunTimerRecoveryConcurrencyTest` | | Validation and Java/Python API parity | `RunnerContextTimerModeTest`, `test_timers.py` | Validation after rebasing onto current main: - Focused Java API/runtime suites: JDK 11 — 204 passed, 2 skipped; JDK 21 — 182 passed, 27 skipped. Includes batch execution, in-flight checkpoint and runtime-identity coverage where supported. - Java Flink 2.3 E2E: 6 passed after a clean distribution rebuild. - Python timer unit tests: 32 passed; Ruff passed. - Python Flink 2.3 E2E on JDK 21: 4 passed, including failure after checkpoint and recovery of all three timer kinds. - Hugo 0.110.0 site build, documentation Python syntax, Spotless and `git diff --check` passed. Not verified: the full test suite and the complete supported Flink/JDK version matrix. Rescaling coverage uses operator harnesses rather than a live cluster resize. <details> <summary>Recovery and scheduling details</summary> Timer operations are replayed alongside Memory updates. Restored Action replay runs before timer delivery resumes, allowing replayed cancellation/replacement to win. Current-run timer ownership keeps the run active after its Action queue empties. Recovery of a timer-only wait takes the parallel execution lock before accessing keyed state. Event-time handling distinguishes the Watermark already processed by the timer service from the Watermark emitted downstream. </details> ### API Adds Java `registerTimer`, `registerTimerAt`, `registerEventTimeTimer`, `cancelTimer` and Python equivalents, plus `TimerMode`. Java methods are abstract: custom `RunnerContext` implementations must implement them. `Event.setSourceTimestamp(long)` becomes `setSourceTimestamp(Long)` to permit an unset timestamp; Java binaries using the old signature must be recompiled. YAML Agents can use timers through their Actions; no YAML schema change. Existing Actions that do not register timers retain their execution semantics. ### Documentation - [ ] `doc-needed` - [ ] `doc-not-needed` - [x] `doc-included` ### Was this patch authored or co-authored using generative AI tooling? - [x] Yes - [ ] No Generated-by: Codex CLI 0.153.4 (GPT-6) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
