weiqingy opened a new pull request, #1117: URL: https://github.com/apache/flink-agents/pull/1117
Linked issue: #280 ### Purpose of change The watsonx.ai connection could not apply a caller's output schema. Both languages sat on the foundation's rejecting default, so a caller passing an `output_schema` always fell back to prompt engineering, even though the chat endpoint accepts a `response_format` carrying a JSON Schema and enforces it during decoding. This adds the native path in Java and Python. The connection now reports the capability and sends the schema in the endpoint's `json_schema` envelope. One decision is worth calling out, because it differs from the other providers in this area. The OpenAI and Azure connections gate the capability on a model allowlist. This one reports it for every model instead. IBM publishes no per-model structured-output table and exposes no capability flag on the model specification, and no client I could find gates by model. What IBM does state is that Chat API support requires the vLLM runtime, so reaching this endpoint at all implies the runtime that applies the constraint. An allowlist here would encode a gate nobody documents, and it would answer "not supported" for models that in fact work. The Ollama connection already reports its capability unconditionally for a similar reason, and the override records the reasoning inline. That does diverge from the base contract, which describes the capability as model-dependent and asks an unrecognized model to report false. The divergence is deliberate and documented rather than silent. A caller has always been able to set `response_format` directly, and that still passes through untouched. Now that an output schema can also produce one, a request carrying both raises instead of silently choosing a winner. Reporting the capability while quietly not applying the caller's schema would make the framework's own contract untrue. Nothing changes at runtime yet. No production caller passes an output schema in either language, so this is the mechanism only. Wiring it to the policy layer belongs to #912. ### Tests Ten tests per language, offline. The Java side asserts the full serialized request body through a local stub server, which is new in this repository. That pins the wire shape, the placement and the envelope. The Python side stubs the vendor SDK, so it pins the parameters handed to it rather than the bytes the service receives. Both languages cover the capability answer, the envelope contents, the absence of a `response_format` when no schema is passed, the fallback for a schema form the connection does not translate, and the conflict with a caller-supplied value on both parameter channels. The Java tests also cover the schema derivation honoring Jackson's property annotations, since a schema naming a property differently from the mapper would produce a reply that satisfies the schema and still fails to deserialize. What the tests do not establish is that IBM accepts the document. An environment-gated live test ships for each language so that someone with a watsonx account can check that. **Neither has been run, because I hold no watsonx credentials.** They are skipped in CI and were skipped locally. Full suites pass in both languages, along with the cross-language guard that asserts a connection which cannot translate a schema rejects one. ### API No new public classes. The Java `buildPayload` test seam widens from three arguments to four and becomes an instance method, which is internal to the module. The user-visible change is that passing an `output_schema` to this connection now succeeds where it previously raised. Since no caller passes one today, no existing behavior changes. ### Documentation - [ ] `doc-needed` - [x] `doc-not-needed` - [ ] `doc-included` The capability is not reachable by a user until the policy layer is wired up in #912, so the user-facing documentation belongs with that change rather than this one. ### Was this patch authored or co-authored using generative AI tooling? - [x] Yes - [ ] No Generated-by: Claude Code 2.1.260 (Claude Opus 5) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
