wenjin272 opened a new pull request, #1429:
URL: https://github.com/apache/flink-agents/pull/1429

   Linked issue: #1428 (closes #1428)
   Design discussion: https://github.com/apache/flink-agents/discussions/1420
   
   ### Purpose of change
   
   Actions can register delayed Events using native Flink timers, for approval 
polling, timeouts, and periodic checks without occupying a thread while 
waiting. Java and Python support processing time and event time. User 
documentation is included under **Actions → Delayed Execution**.
   
   The implementation is split into three feature commits (new-run processing 
timers, current-run processing timers, event-time timers), followed by one 
documentation commit.
   
   #### Runtime flow
   
   `RunnerContext` records timer operations during an Action. Successful 
completion resolves relative deadlines and journals operations in 
`ActionState`, then applies them with Memory updates. `ActionExecutionOperator` 
uses `InternalTimerService` for deadlines and keyed state for Event payloads. 
Expiration dispatches the Event through existing Action routing, either into 
its owning run or as a new input run.
   
   #### Key decisions
   
   Reuse registered Actions instead of adding a separate business callback API. 
Commit timer operations at Action completion so suspension does not expose 
partial effects. Event-time timers always start new runs; allowing them to hold 
their current run would conflict with waiting for earlier inputs before 
advancing event time.
   
   ### Behavioral Semantics
   
   #### Interaction decisions
   
   | Timer / mode | Context and same-key inputs | Output timestamp |
   | --- | --- | --- |
   | Processing / `TRIGGER_AFTER_CURRENT_RUN` | Fresh Sensory Memory; later 
inputs can proceed after the original run ends | Unset |
   | Processing / `CURRENT_RUN_WAIT_FOR_TRIGGER` | Retains context, Sensory 
Memory and attachments; later inputs wait | Original input timestamp |
   | Event time | New run only; no mode argument | Timer deadline |
   
   #### Behavioral contracts
   
   - Registration, replacement and cancellation take effect only when the 
Action completes successfully. Relative delays start then; replay retains the 
resolved deadline.
   - IDs are scoped to the operator and business key and shared across time 
domains. Same-ID registration replaces a pending timer; cancellation ignores 
missing IDs and cannot retract delivered Events.
   - Each registration fires once. Registering again from its Action supports 
periodic work.
   - Watermarks wait for earlier inputs, then for the runs triggered by due 
event-time timers and their subsequent Actions. Newly registered due timers are 
checked again at the same Watermark. Late registrations fire without requiring 
another Watermark and do not move downstream Watermarks backward.
   - Pending registrations, dispatched Events, and current-run waits survive 
checkpoint recovery and rescaling.
   
   #### Failure behavior
   
   Invalid IDs, negative delays, invalid timestamps/modes, nonserializable 
Events and `OutputEvent` targets raise errors. New-run Events reject 
attachments. Registration/cancellation inside `execute_async`/`executeAsync` 
tasks or internal subagent/chat invocations is rejected. Failed Actions do not 
apply their timer operations; triggered Actions use the existing failure and 
recovery path. Timers target streaming execution; future new-run processing 
timers do not keep bounded input open.
   
   ### Tests
   
   | Contract | Coverage |
   | --- | --- |
   | Completion-only effects, relative deadlines and replay | 
`NativeTimerTest`: await/commit, failed Action, journaled deadline |
   | Replacement, cancellation and shared IDs | `NativeTimerTest`, 
`EventTimeTimerTest`: same deadline, already queued Event, cross-domain 
replacement |
   | One-shot delivery and repeated checks | Java `NativeTimerE2ETest`; Python 
`native_timer_test.py` |
   | Current-run context, attachments and input ordering | 
`CurrentRunTimerTest`; current-run variants in Java/Python E2Es |
   | Watermark ordering, output timestamps and late registrations | 
`EventTimeTimerTest`; Java/Python event-time E2Es |
   | Recovery, replay and rescaling | All three operator timer suites; Python 
`timer_recovery_test.py` injects failure after a completed checkpoint |
   | Recovery does not corrupt another key's Memory | 
`CurrentRunTimerRecoveryConcurrencyTest` |
   | Validation and Java/Python API parity | `RunnerContextTimerModeTest`, 
`test_timers.py` |
   
   Validation after rebasing onto current main:
   
   - Focused Java API/runtime suites: JDK 11 — 204 passed, 2 skipped; JDK 21 — 
182 passed, 27 skipped. Includes batch execution, in-flight checkpoint and 
runtime-identity coverage where supported.
   - Java Flink 2.3 E2E: 6 passed after a clean distribution rebuild.
   - Python timer unit tests: 32 passed; Ruff passed.
   - Python Flink 2.3 E2E on JDK 21: 4 passed, including failure after 
checkpoint and recovery of all three timer kinds.
   - Hugo 0.110.0 site build, documentation Python syntax, Spotless and `git 
diff --check` passed.
   
   Not verified: the full test suite and the complete supported Flink/JDK 
version matrix. Rescaling coverage uses operator harnesses rather than a live 
cluster resize.
   
   <details>
   <summary>Recovery and scheduling details</summary>
   
   Timer operations are replayed alongside Memory updates. Restored Action 
replay runs before timer delivery resumes, allowing replayed 
cancellation/replacement to win. Current-run timer ownership keeps the run 
active after its Action queue empties. Recovery of a timer-only wait takes the 
parallel execution lock before accessing keyed state. Event-time handling 
distinguishes the Watermark already processed by the timer service from the 
Watermark emitted downstream.
   
   </details>
   
   ### API
   
   Adds Java `registerTimer`, `registerTimerAt`, `registerEventTimeTimer`, 
`cancelTimer` and Python equivalents, plus `TimerMode`. Java methods are 
abstract: custom `RunnerContext` implementations must implement them. 
`Event.setSourceTimestamp(long)` becomes `setSourceTimestamp(Long)` to permit 
an unset timestamp; Java binaries using the old signature must be recompiled. 
YAML Agents can use timers through their Actions; no YAML schema change. 
Existing Actions that do not register timers retain their execution semantics.
   
   ### Documentation
   
   - [ ] `doc-needed`
   - [ ] `doc-not-needed`
   - [x] `doc-included`
   
   ### Was this patch authored or co-authored using generative AI tooling?
   
   - [x] Yes
   - [ ] No
   
   Generated-by: Codex CLI 0.153.4 (GPT-6)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to