Zhuoxi2000 opened a new pull request, #1186:
URL: https://github.com/apache/flink-agents/pull/1186
<!--
* Thank you very much for contributing to Flink Agents.
* Please add the relevant components in the PR title. E.g., [api],
[runtime], [java], [python], [hotfix], etc.
-->
<!-- Please link the PR to the relevant issue(s). Hotfix doesn't need this.
-->
Linked issue: #1059
### Purpose of change
<!-- What is the purpose of this change? -->
A chat message with media sent to a provider that cannot carry it now fails
with `UnsupportedContentBlockException` / `UnsupportedContentBlockError`
instead of being sent as its text alone. This is the last Phase 2 item of
#1059, "explicit unsupported-block errors for providers not yet migrated",
after #1164 (OpenAI Chat Completions, Azure OpenAI, vLLM) and #1178 (Ollama).
Covered: Java Anthropic, Gemini, Amazon Bedrock, IBM watsonx.ai and OpenAI
Responses; Python Anthropic, Tongyi and IBM watsonx.ai.
#### Runtime flow
1. Each covered connection's `chat()` first calls
`UnsupportedContentBlockException.rejectMedia(provider, messages)` (Java) or
`UnsupportedContentBlockError.reject_media(provider, messages)` (Python).
2. The first media block in any message, of any role, throws; text-only
messages go on to the unchanged conversion and request.
#### Key decisions
* Reject rather than send the text projection: dropping an image silently
changes the question the model answers, which is what #1059 asked to avoid.
* One shared helper next to the `forBlock` / `for_block` factory from #1164,
so every provider reports the same message: `{provider} cannot send an image
block (image/png, base64 source): this integration sends text only.`
* The check runs at the start of `chat()`, before any `try` that wraps
provider errors, so callers receive the documented type (the issue #1178 review
caught for Ollama).
* Native media support for these providers stays in the #1059 follow-ups;
this PR only makes the current gap explicit.
### Behavioral Semantics
<!-- For a non-trivial code change whose implementation is largely
AI-assisted: interaction decisions, behavioral contracts, and failure behavior.
See `contribution-guides/ai-assisted-pr.md`. Remove this heading and this
comment otherwise. -->
#### Interaction decisions
| Messages | Result |
|---|---|
| text only, any roles | unchanged request |
| any media block, any role | `UnsupportedContentBlockException` /
`UnsupportedContentBlockError`; nothing is sent |
#### Behavioral contracts
1. A covered connection throws for the first media block before sending
anything.
2. The error is the documented type itself when it leaves `chat()`, not a
wrapper.
3. The message names the provider, block type, media type and source type,
never the data or URL.
4. Requests without media are unchanged.
Java and Python behave identically.
#### Failure behavior
* Media in a message: thrown before any request is built or sent; the chat
action's error strategy applies, and RETRY fails the same way each attempt
since the error is deterministic.
* No other failure path changes.
### Tests
<!-- How is this change verified? -->
| Contract | Java | Python |
|---|---|---|
| 1, 2, 3 | `testMediaBlocksFailExplicitly` in
`AnthropicChatModelConnectionTest`, `GeminiChatModelConnectionTest`,
`BedrockChatModelConnectionTest`, `WatsonxChatModelConnectionTest`,
`OpenAIResponsesModelConnectionTest`, each through `chat()` and asserting the
exact message | `test_text_only_providers_reject_media.py`, parametrized over
Anthropic, Tongyi and watsonx, through `chat()` |
| 4 | the existing request-building tests of each connection | the existing
provider tests |
Not verified: a live provider; the check runs before any request, so none is
needed for these contracts.
<details>
<summary>Implementation invariants and supporting evidence</summary>
* `rejectMedia` / `reject_media` only read `getBlocks()` / `.blocks` and use
`forBlock` / `for_block`, so messages match #1164 and #1178.
* Verified locally: the five Java connection test classes and spotless;
Python Anthropic, Tongyi, watsonx and chat-message tests, with ruff.
</details>
### API
<!-- Does this change touches any public APIs? -->
New public helpers: `UnsupportedContentBlockException.rejectMedia(String,
List<ChatMessage>)` (Java) and
`UnsupportedContentBlockError.reject_media(provider, messages)` (Python).
Compatibility: text-only requests are unchanged. A message with media to one
of these providers used to be sent as its text only; it now fails.
### Documentation
<!-- Do not remove this section. Check the proper box only. -->
- [ ] `doc-needed` <!-- Your PR changes impact docs -->
- [ ] `doc-not-needed` <!-- Your PR changes do not impact docs -->
- [x] `doc-included` <!-- Your PR already contains the necessary
documentation updates -->
The provider-support note in `chat_models.md` names these integrations and
the error; the Bedrock limitation note says media raises.
### Was this patch authored or co-authored using generative AI tooling?
<!-- Do not remove this section. Check the proper box only. -->
- [x] Yes
- [ ] No
If yes, include a `Generated-by: <tool name and version> (<model name and
version>)` line, for example `Generated-by: Claude Code 2.1.226 (Claude Opus
4.6)`, in the commit message so it reaches Git history. Repeat the same line
here for reviewer visibility. See the [ASF generative tooling
guidance](https://www.apache.org/legal/generative-tooling.html).
Generated-by: Claude Code 2.1.259 (Claude Opus 5.5)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
