wenjin272 opened a new pull request, #1149: URL: https://github.com/apache/flink-agents/pull/1149
Linked issue: Closes #1130. Related: #1086 (Event-backed call API; not implemented here). ### Purpose of change Chat consumers can now receive and inspect ordinary terminal failures through `ChatResponseEvent`, instead of losing the response under `IGNORE` or receiving only an upstream exception. This provides the terminal-result protocol needed by a future Event-backed call API while leaving failure handling to the consumer. #### Runtime flow `ChatRequestEvent` enters `ChatModelAction`; Java resolves any model router, then invokes each selected candidate through `ChatModelInvoker`. Python uses its existing direct-model path. Provider invocation returns a serializable success/failure outcome inside durable execution. The action applies response validation, structured-output parsing, retries, and Java candidate fallback. Tool requests continue the existing loop; only a final answer or exhausted ordinary failure emits a terminal response on the original request ID. #### Key decisions - Represent success/failure in the event, but make reading a failed `response` throw so existing consumers do not silently accept failure. - Store exception type/message as diagnostic text, not exception objects, stack traces, or a framework error-code taxonomy. - Remove `ErrorHandlingStrategy`; retry count and interval govern execution, independently of terminal outcome and router fallback. - Capture failures at owned invocation/strategy boundaries, not around the whole action, to preserve runtime recovery behavior. ### Behavioral Semantics #### Interaction decisions | Conditions | Result | |---|---| | Model attempt fails; retry budget remains | Retry with exponential backoff; no terminal response yet. | | Initial routed candidate exhausts retries; fallback enabled | Try remaining candidates; final failure reports the last candidate error. | | Tool round continues | Keep the serving model and original request ID; no intermediate Chat response. | | Judge call exhausts retries | Failed Chat response; do not invoke the default model as an error fallback. | | Judge returns an unrecognized verdict, or strategy normally abstains | Select the router's default model. | | Cancellation or failure in durable execution/event delivery | Propagate to the runner, not a normal Chat failure event. | #### Behavioral contracts - A terminal result is `SUCCESS` with a `ChatMessage`, or `FAILED` with nonempty error text; payloads are mutually exclusive in both languages. - Failed response access throws `ChatResponseException` / `ChatResponseError`; status and error remain inspectable. ReAct consequently propagates unhandled failures. - Retries, fallback, and tool rounds retain the initial request ID and accumulated retry statistics; they do not emit intermediate terminal responses. - Persisted invocation failures replay as data without calling the provider again. This is not an exactly-once physical-delivery or waiter-deduplication guarantee. #### Failure behavior Provider errors (including ordinary timeouts), rejected finish reasons, and structured-output errors consume the configured retry budget. Candidate resource-resolution failures can advance Java fallback. Router preparation/strategy failures and invalid runtime selections produce failed responses; declaration validation still rejects invalid plans. Ordinary tool failures keep their existing Tool-response feedback behavior. Cancellation/interruption and JVM `Error` propagate. Failures outside the owned conversion boundaries, including durable execution, required memory updates, and event delivery, propagate rather than being reported as model failures. This change does not promise a response when the runtime itself cannot complete or deliver one. ### Tests | Contract | Evidence | |---|---| | Success/failure payload validation and throwing response access | Java/Python `ChatResponseEvent` tests; ReAct consumer test | | Java/Python wire representation | Both cross-language event snapshot suites, including failed-response fixtures | | Retry exhaustion, backoff, rejected responses, default zero retries | Java/Python Chat action retry suites | | Router fallback, judge failure versus normal abstention, no default-model call on judge failure | `ChatModelActionRoutingTest`, routing executor tests | | Tool-loop correlation and retry totals | Java/Python tool-response and multi-round retry tests | | Runtime failures/cancellation escape conversion | Invoker/action/routing failure-boundary tests | | Durable failure replay without another provider invocation | `RunnerContextImplDurableExecuteTest.failedChatOutcomeReplaysThroughRuntimeState`; Python sync/async `test_chat_failure_outcome_replays_without_provider_call`; action-state serde tests | Final-head targeted verification: Java 267 passed, 12 skipped; Python 114 passed, 13 skipped. A separate Chat/routing/judge-validation run passed 144 Java tests. Skips are snapshot-generation cases. Spotless, changed-file Ruff checks, and `git diff --check` passed. Not verified: full repository suite, external-provider E2E, live Flink failover/event redelivery, and future call-waiter idempotency. Existing event/state compatibility is intentionally not provided. <details> <summary>Implementation invariants and verification commands</summary> Invocation and routing outcomes carry serializable data through durable storage. Terminal completion clears tool context/retry state; tool-context removal explicitly writes the changed map back to memory. Event serialization reads attributes rather than invoking the throwing response getter. Java: `mvn -B -ntp -pl runtime -am test '-Dtest=ChatModel*Test,*Routing*Test,ReActAgentTest,ChatResponseEventTest,CrossLanguageEventSnapshotTest,ActionStateSerdeTest,ConditionEvaluatorTest,RunnerContextImplDurableExecuteTest' -Dsurefire.failIfNoSpecifiedTests=false` (JDK 17, source target 11). Python: targeted Chat action/retry, event/snapshot, and `test_flink_runner_context_reconcilable.py` suites, with the active environment's purelib on `PYTHONPATH`. </details> ### API Breaking Java/Python event change: use `success(...)` / `failed(...)` factories and inspect status before optional response access. Old events without status are rejected; old durable Chat/routing payload compatibility is not supported. The original request ID and successful message semantics remain unchanged. `ErrorHandlingStrategy` and `error-handling-strategy` are removed from Java/Python/YAML configuration. Set `max-retries` explicitly to enable retries; its default changes from 3 to 0, preserving the previous effective default of no retries. Former `RETRY` users should set 3 to retain that budget. `retry-wait-interval` and candidate fallback remain available. Former judge `IGNORE` behavior becomes a failed response, not abstention. ### Documentation - [ ] `doc-needed` - [ ] `doc-not-needed` - [x] `doc-included` ### Was this patch authored or co-authored using generative AI tooling? - [x] Yes - [ ] No Generated-by: Codex CLI 0.153.4 (GPT-6) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
