wenjin272 opened a new issue, #1197: URL: https://github.com/apache/flink-agents/issues/1197
### Search before asking - [x] I searched existing issues for duplicates; related work is linked below where applicable. ### Description This is a child issue of #1055. #### Problem Runtime routing results have inconsistent handling when they name a model outside the declared candidate set. LlmJudgeRoutingExecutor converts an unparseable/non-candidate verdict to abstain, whereas ModelRoutingResolver.normalizeAndFinish throws RoutingFailure for a concrete non-candidate selection, including one returned by a custom strategy. Users therefore observe different failure/default behavior for the same invalid selection depending on the strategy implementation. #### Proposed direction Define a uniform runtime result contract: a valid candidate or abstain. The preferred direction is to normalize non-candidate results from custom/LLM strategies to abstain, retaining the reason in decision metadata/events/metrics or logs. If strict failure is preferred instead, apply it consistently across runtime strategy kinds and document the decision. Keep static declaration validation separate: invalid declared rule targets, candidate lists, and defaults should fail at build/plan time. Do not hide declaration errors through runtime abstention. Define how normalization interacts with durable decision persistence and replay, including candidate-set changes, without silently rerunning a completed routing decision. #### Acceptance criteria - A documented, consistent policy for runtime non-candidate results across custom and LLM strategies and the shared resolver. - Tests cover valid candidates, explicit abstain, non-candidate results, default selection, observable reasons, and durable replay. - Invalid static declarations still fail early; actual strategy execution exceptions retain the documented error-handling policy. Related: #1062 tracks routing overall; #1150 covers static candidate declaration validation. Relevant code: `plan/.../routing/ModelRoutingResolver.java`, `LlmJudgeRoutingExecutor.java`, and custom/rule strategy executors. ### Are you willing to submit a PR? - [ ] I'm willing to submit a PR! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
