joeyutong commented on code in PR #1374:
URL: https://github.com/apache/flink-agents/pull/1374#discussion_r4545829560
##########
docs/content/docs/development/calling-llms/chat-models/providers.md:
##########
@@ -143,9 +145,136 @@ class MyAgent(Agent):
...
```
+#### Interface Selection
+
+Use the same setup options in Python, Java, and YAML.
+
+| Effective `api_type` | Operation | Setup options | Result content |
+|---|---|---|---|
+| `text` (default) | Text chat | `model="qwen-plus"` (default) | Text and,
when configured, tool calls |
+| `multimodal` | [Image](#image-input) or [video](#video-input) understanding
| `api_type="multimodal"` with a vision model such as `qwen3-vl-plus` | Text
and, when supported, tool calls |
+| `multimodal` (automatic) | [Speech
recognition](#audio-input-speech-recognition) |
`model="qwen-audio-3.1-asr-flash"`, `task="asr"` | Transcript text |
+| `multimodal` (automatic) | [Image generation and
editing](#synchronous-image-generation-and-editing) | `model="qwen-image-3.0"`,
`task="image_generation"` | Image blocks |
+
+The multimodal interface uses MultiModalConversation (MMC) in non-streaming
+mode. Setting either task selects MMC automatically, so you can omit
+`api_type`. For these tasks, leave `prompt`, `tools`, `skills`, and `subagents`
+unset on the setup and omit `output_schema` and `response_format`.
+
+Add media to user messages. Model responses use `ChatResult`, including token
Review Comment:
Updated in 78bf2aa1: removed the general result/usage paragraph.
User-message requirements and other input restrictions are now stated in each
capability section.
##########
docs/content/docs/development/calling-llms/chat-models/providers.md:
##########
@@ -143,9 +145,136 @@ class MyAgent(Agent):
...
```
+#### Interface Selection
+
+Use the same setup options in Python, Java, and YAML.
+
+| Effective `api_type` | Operation | Setup options | Result content |
+|---|---|---|---|
+| `text` (default) | Text chat | `model="qwen-plus"` (default) | Text and,
when configured, tool calls |
+| `multimodal` | [Image](#image-input) or [video](#video-input) understanding
| `api_type="multimodal"` with a vision model such as `qwen3-vl-plus` | Text
and, when supported, tool calls |
+| `multimodal` (automatic) | [Speech
recognition](#audio-input-speech-recognition) |
`model="qwen-audio-3.1-asr-flash"`, `task="asr"` | Transcript text |
+| `multimodal` (automatic) | [Image generation and
editing](#synchronous-image-generation-and-editing) | `model="qwen-image-3.0"`,
`task="image_generation"` | Image blocks |
+
+The multimodal interface uses MultiModalConversation (MMC) in non-streaming
+mode. Setting either task selects MMC automatically, so you can omit
+`api_type`. For these tasks, leave `prompt`, `tools`, `skills`, and `subagents`
+unset on the setup and omit `output_schema` and `response_format`.
+
+Add media to user messages. Model responses use `ChatResult`, including token
+usage and the finish reason when available. If the service reports only audio
+duration or image count for a task, token usage is absent.
+
+`api_type="auto"` recognizes the following MMC model families and exceptions:
+
+| Models | Matching rule |
+|--------|---------------|
+| `qwen-vl-plus`, `qwen-vl-max`, `qwen3-vl-plus`, `qwen3-vl-flash` | Exact
names and variants beginning with the name followed by `-` |
+| `qwen3.5-plus`, `qwen3.5-flash`, `qwen3.6-plus`, `qwen3.6-flash`,
`qwen3.7-plus`, `qwen3.7-flash`, `qwen3.8-max`, `qwen3.8-flash` | Exact names
and variants beginning with the name followed by `-` |
+| Qwen2-VL, Qwen2.5-VL, Qwen3-VL, Qwen3.5, Qwen3.6 and Qwen3.8 open-weight
models | A parameter size beginning with a digit after the family prefix, such
as `qwen3-vl-8b-instruct` or `qwen3.8-2.4t-a95b` |
+| QVQ | `qvq` or names beginning with `qvq-` |
+| `qwen3.7-max-2026-06-08`, `kimi-k2.5` | Exact names |
+
+Names not matching these rules use the text interface. In particular,
`qwen3.7-max`, its preview,
+and its `2026-05-17` and `2026-05-20` snapshots remain on text generation.
+Unlisted models and deployment aliases can select `multimodal` explicitly.
+VL embedding, rerank and OCR models are outside this chat model list.
+
+Routing identifies the API, not the model's availability or supported
invocation modes.
+QVQ requires streaming, which this connection does not support. Some
open-weight
+models also require streaming when thinking is enabled. See the
+[streaming
restrictions](https://help.aliyun.com/en/model-studio/deep-thinking).
+The connection does not change thinking parameters automatically.
+See the [text
generation](https://help.aliyun.com/en/model-studio/text-generation),
+[vision](https://help.aliyun.com/en/model-studio/vision), and
+[Kimi](https://help.aliyun.com/en/model-studio/kimi-api) documentation for
model
+and interface details.
+The presence of media in a message does not change the selected interface.
+
+For chat and media understanding in Python, `structured_output_strategy=AUTO`
Review Comment:
Updated in 78bf2aa1: added a dedicated Structured Output subsection, linking
the general guide and DashScope's supported-model documentation. It also
explains Java's prompt-based path and the cross-language restriction on
`NATIVE`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
