wenjin272 opened a new issue, #1070: URL: https://github.com/apache/flink-agents/issues/1070
### Search before asking - [x] I searched in the [issues](https://github.com/apache/flink-agents/issues) and found nothing similar. ### Description When a Flink job cancellation interrupts an in-flight chat-model call, the resulting `InterruptedException` is handled as an ordinary model failure on the current `main` branch. There are two related problems: 1. `ChatModelInvoker.chatWithRetries()` catches every `Exception` and retries it when `ERROR_HANDLING_STRATEGY=RETRY`. Because throwing `InterruptedException` may clear the thread's interrupt status, retry backoff and additional model calls can continue after cancellation was requested, delaying task shutdown and issuing unnecessary external requests. 2. `RunnerContextImpl.durableExecuteCompletionOnly()` records every exception as a completed durable failure before rethrowing it. With an `ActionStateStore` configured, an interruption from the old execution attempt can therefore be replayed after recovery instead of re-executing the unfinished model call. Under `FAIL`, this can make the recovered action fail again; under `IGNORE`, the input can be skipped without producing a response; under `RETRY`, the replayed cancellation consumes retry budget. Relevant paths on `main`: - [`ChatModelInvoker.chatWithRetries()`](https://github.com/apache/flink-agents/blob/0417dad3177258e34af5d40507bae4d3f1e6b4bb/plan/src/main/java/org/apache/flink/agents/plan/actions/ChatModelInvoker.java#L155-L203) - [`RunnerContextImpl.durableExecuteCompletionOnly()`](https://github.com/apache/flink-agents/blob/0417dad3177258e34af5d40507bae4d3f1e6b4bb/runtime/src/main/java/org/apache/flink/agents/runtime/context/RunnerContextImpl.java#L558-L588) - [`ChatModelAction` IGNORE handling](https://github.com/apache/flink-agents/blob/0417dad3177258e34af5d40507bae4d3f1e6b4bb/plan/src/main/java/org/apache/flink/agents/plan/actions/ChatModelAction.java#L482-L516) Expected behavior: - Cancellation signals should propagate immediately regardless of `FAIL`, `RETRY`, or `IGNORE` and should never trigger another model attempt. - Cancellation should not be finalized as a durable success or failure. The call should remain unfinished, or its durable slot should be cleared, so recovery can re-execute or reconcile it. - Ordinary provider and network failures should continue to follow the configured error-handling strategy. ### How to reproduce #### Retry behavior 1. Register a test chat model whose `chat()` method blocks and throws `InterruptedException` when its execution thread is interrupted. 2. Configure `ERROR_HANDLING_STRATEGY=RETRY` with at least one retry. 3. Send a `ChatRequestEvent`, wait until the model call starts, and cancel/interrupt the executing task. 4. Observe that `chatWithRetries()` treats the interruption as a retryable model failure and may enter backoff or invoke the model again instead of propagating cancellation immediately. #### Durable recovery behavior 1. Enable an `ActionStateStore` and execute the same blocking chat model through the built-in `ChatModelAction`. 2. Interrupt the model call and verify that the durable `chat` call is recorded as `FAILED` with `InterruptedException`. 3. Recover from the preceding checkpoint and reprocess the same event/action. 4. Observe that the stored interruption is replayed without invoking the model again. With `FAIL` the recovered action fails again; with `IGNORE` the request is skipped. ### Version and environment - Apache Flink Agents `main` at `0417dad3177258e34af5d40507bae4d3f1e6b4bb` - Java execution path - The durable recovery case requires an `ActionStateStore` backend to be configured - Provider-independent when the chat-model call surfaces cancellation as `InterruptedException` ### Are you willing to submit a PR? - [ ] I'm willing to submit a PR! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
