weiqingy opened a new pull request, #965:
URL: https://github.com/apache/flink-agents/pull/965
Linked issue: #280
### Purpose of change
Neither side could fulfil a caller-supplied output schema. Java had no 4-arg
`chat` override, so the call reached the base default, which rejects a non-null
schema with `UnsupportedOperationException` so an unconstrained response is
never mistaken for a schema-conforming one. Python rejected it explicitly. The
capability was refused rather than silently degraded, and unavailable even on
models that support provider-enforced structure.
#### Runtime flow
`chat(messages, tools, modelParams, outputSchema)` calls `buildRequest`,
which resolves the effective model, then decides in order:
Whether native structured output applies. The schema must be a POJO `Class`
(Java) or a `BaseModel` wrapped in `OutputSchema` (Python),
`supportsNativeStructuredOutput(effectiveModel)` must return true, and the
caller must not have supplied its own `output_config`. If all hold, the schema
is translated and attached, and the decision is recorded.
Whether JSON prefill applies (Java only). On by default, off when tools are
present, off again when the schema was applied natively.
`buildRequest` returns the request together with that prefill decision.
`chat` passes the pair to `convertResponse`, which reconstructs the leading `{`
only when the request actually carried it.
#### Key decisions
Capability is a generational rule, not a per-snapshot list: structured
outputs are generally available for Claude 4.5 and later models, plus Mythos
Preview. Names from 4.6 onward are dateless and pinned, so they match exactly.
The three 4.5-generation names are aliases fronting a dated snapshot, so both
forms must match and those match by prefix. A prefix must retain the minor
version, since `claude-opus-4` would also capture `claude-opus-4-1-20250805`,
which predates the cutoff.
The Java translator extracts the config from a throwaway request. The Kotlin
facade `StructuredOutputsKt.outputFormatFromClass` would produce it directly
but is compiled `ACC_SYNTHETIC` and cannot be named from Java. The typed
`outputConfig(Class)` overload retypes the request and response as
`StructuredMessageCreateParams` and `StructuredMessage`, while the deserialized
POJO is discarded anyway.
A caller-supplied `output_config` wins rather than being overwritten. Unlike
parameters this connection always writes from typed setup fields, it is written
only when a schema arrives on a channel the caller does not control, so
overwriting would discard a deliberate choice.
The prefill decision was previously computed twice, with the response path
prepending `{` from one of them. It is computed once and carried, so a
desynchronized flag is not expressible.
### Implementation Description
#### Behavioral contracts
1. A POJO or `BaseModel` schema, a capable model, and no caller
`output_config` produce a request carrying the derived `output_config`.
2. A schema of any other shape produces no derived config.
3. A model name in neither allowlist, including null, reports not-capable
and produces no derived config.
4. A caller-supplied `output_config` is preserved unchanged, and no derived
config is written alongside it.
5. The capability predicate reads no instance state.
6. A natively applied schema suppresses `json_prefill` (Java).
7. When native is not applied, for any reason, the `json_prefill` decision
is unchanged (Java).
8. The response conversion uses the prefill decision the request was built
with (Java).
9. The 3-arg `chat` forwards all four arguments to the 4-arg form with a
null schema (Java).
10. Both languages carry the same twelve identifiers in the same order.
#### Failure behavior
An unsupported configuration never raises. An incapable model, an
untranslatable schema, or a caller-supplied `output_config` each leave the
request without a derived schema, and the prompt-engineering fallback governs.
A null effective model returns false rather than propagating the
`NullPointerException` that `Set.contains(null)` raises on an immutable
allowlist. If the SDK returns no `output_config` for a schema it accepted, Java
raises `IllegalStateException`, which is not reachable through the public API
since the same call sets the config two lines earlier. Provider errors are
unchanged: Java wraps them in `RuntimeException`, Python propagates the SDK
exception, a divergence predating this change.
A caller forcing the `NATIVE` strategy on a model the predicate rejects
degrades silently to the prompt fallback rather than raising, because the
requested strategy is not visible at this layer. Marked `TODO(#912)` in both
languages, matching the OpenAI and Azure connections.
### Tests
32 test methods, 64 cases. No network, and no mocking framework on the Java
side: response objects build offline from public SDK builders.
| Contract | Tests |
|---|---|
| 1 | `testNativeSchemaAppliedOnCapableModel`,
`test_native_output_config_applied_on_capable_model` |
| 2 | `testNonClassSchemaKeepsFallback`,
`test_native_output_config_not_applied_for_row_type_info` |
| 3 | `testNativeSchemaNotAppliedOnIncapableModel`,
`testIncapableModelsReportNotCapable`,
`test_capability_predicate_rejects_incapable_models` |
| 4 | `testCallerOutputConfigWinsOverSchema`,
`test_caller_output_config_wins_over_schema` |
| 5 | `testCapabilityReadsNoInstanceState`,
`test_capability_reads_no_instance_state` |
| 6 | `testJsonPrefillSuppressedWhenNativeApplies` |
| 7 | `testJsonPrefillAppliedWhenSchemaFallsBack`, plus five existing
prefill cases |
| 8 | `assertPrefillDecision` drives the real conversion on every row it
asserts |
| 9 | `testThreeArgChatForwardsNoSchema` |
| 10 | `testCapableModelsReportCapable`,
`testAliasPrefixMatchesDatedSnapshot`, and their Python twins. Both suites
hard-code the list as independent literals, so a typo in one production entry
fails |
Contracts 1, 3, 4, 6, 7 and 9 were each checked by mutation: inverting the
condition, truncating a prefix, dropping the guard, or altering a forwarded
argument fails the named test rather than passing quietly.
`testNativePathSendsNoBetaHeader` pins that the native path adds no
`anthropic-beta` header.
Not covered: no live request was made, so these pin what the connection
sends, never that the provider accepts it. The Python floor is unenforced by
any test, since the client is mocked, so floor correctness rests on
introspecting both versions.
### API
No new public API. Both connections override methods the foundation already
defines, and the Java 3-arg `chat` now delegates to the 4-arg form.
A caller doing nothing differently sees no change: schema-free requests
build exactly as before, `json_prefill` behaves as before, the response
conversion is unchanged. A caller passing a schema previously got an exception
on both sides, and now gets a provider-enforced response on a capable model, or
the prompt-engineering fallback otherwise.
Two dependency floors rise, each to the first release exposing the parameter
on the non-beta client: `com.anthropic:anthropic-java` 2.11.1 to 2.12.0, and
`anthropic` 0.64.0 to 0.77.0. The Java bump forces no other source change and
leaves transitive dependencies unchanged.
### Documentation
- [ ] `doc-needed`
- [x] `doc-not-needed`
- [ ] `doc-included`
No user-facing documentation changes. Which providers fulfil `output_schema`
natively rather than by prompt engineering is undocumented for every provider,
and belongs in the integration support matrix as one change rather than a third
of it landing here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
