purushah opened a new pull request, #1042:
URL: https://github.com/apache/flink-agents/pull/1042
## What
Implements the `Strategies.llm(...)` follow-up promised in discussion #897
and documented as the framework-managed follow-up in the #964 javadocs: **the
engine — not the strategy — executes the judge call**, on the same durable,
metered, observable chat path as any model call.
```java
env.addResource("judge", ResourceType.CHAT_MODEL, judgeModelDescriptor); //
any chat model
env.addResource("router", ResourceType.MODEL_ROUTER,
ModelRouter.of("small", "big")
.describe("small", "fast and cheap: chit-chat, lookups")
.describe("big", "expensive: code, SQL, multi-step reasoning")
.strategy(Strategies.llm("judge"))
.defaultModel("small")
.fallback(true)
.build());
```
## Design (per the #897 agreement)
- `LlmJudgeRoutingStrategy` carries configuration plus two pure functions
(prompt build, verdict parse); `route()` is never invoked — the resolver
detects the strategy and runs the judge chat under durable id
`"judge:<router>"` (engine retries, trace events, token attribution to the
judge model), then persists the decision — with its source and judge-inclusive
`decision_ms` — under the standard `"route:<router>"` id. Judge abstains
persist as abstains, so replay after a candidate-set change resolves to the
current default exactly like the strategy path.
- **Verdicts are constrained to candidate names**: the judge prompt is built
from the router's candidates + `describe(...)` lines; an unparseable, ambiguous
(multiple distinct candidates named), or non-candidate verdict abstains to the
default model with the cause recorded. A hijacked judge cannot steer routing
outside the declared candidates.
- **Failure policy**: a judge call that exhausts its retries honors the
request's error-handling strategy (`FAIL` surfaces the outage loudly, `IGNORE`
abstains to the default with the cause recorded); interrupts/cancellation
propagate and are never persisted as routing outcomes (including the
`InterruptedIOException` shapes HTTP stacks use — with `SocketTimeoutException`
explicitly treated as an ordinary failure, not cancellation).
- **Fail-fast validation**: `build()` constructs the strategy so its
argument rules apply at the registration call site, and `AgentPlan` validates
at plan construction — by instantiating the strategy exactly as the runtime
does — that the judge name resolves to a registered `CHAT_MODEL`. A judge setup
with a bound prompt, tools, or skills is rejected (it would silently corrupt
every verdict).
- **Python**: `add_resource(..., MODEL_ROUTER)` now raises an explicit
not-yet-supported error instead of dropping silently (flagged in the #964
review); a cross-language test proves Java plans carrying the new strategy args
still deserialize in Python.
## Compatibility impact
- Additive only: plain-chat agents and existing rule/custom routers are
untouched.
- Event consumers see a new `decision_source` value `llm_judge` and new
optional metadata keys (`judge_model`, `judge_prompt_tokens`,
`judge_completion_tokens`, `decision_source`).
- Deliberate trade-off: the judge path performs a second durable write
(`route:<router>` in addition to `judge:<router>`) to keep the route-record
schema uniform across router types and preserve replayed `decision_ms`.
## Test evidence
- 684 Java tests green (new: verdict routing incl. durable ids + metadata,
ambiguity/injection/parse table, judge retry budget, failure policy under FAIL
and IGNORE, prompt-bound judge rejection and IGNORE degradation, plan-time
validation incl. subclasses and null tolerance, prompt-template handling,
promptArgs-shaped requests); Python suites green incl. the cross-language
round-trip.
- End-to-end on a standalone Flink 2.3.0 cluster (dist jar + `flink run` of
the new `ModelRoutingJudgeExample`, local Ollama): judge verdicts routed
correctly with full metadata; fallback composition exercised live.
Generative AI was used for this change.
Generated-by: Claude Code 2.1.239 (Claude Fable 5)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
