joeyutong commented on code in PR #1374:
URL: https://github.com/apache/flink-agents/pull/1374#discussion_r4545828003
##########
docs/content/docs/development/calling-llms/chat-models/providers.md:
##########
@@ -49,7 +49,7 @@ model, deployment, and server must also support the requested
input and tool cal
| Integration | Implementation language | Media input accepted by the
integration |
|---|---|---|
-| [Alibaba Cloud Model Studio
(DashScope)](#alibaba-cloud-model-studio-dashscope) | Python | No media input;
media blocks are rejected. |
+| [Alibaba Cloud Model Studio
(DashScope)](#alibaba-cloud-model-studio-dashscope) | Python | User images and
video files with `api_type="multimodal"` or an `auto` route to that interface.
Speech recognition and image editing select it through `task`. Media sources
can be HTTP(S) URLs or Base64. |
Review Comment:
Updated in 78bf2aa1: simplified the row to media inputs only. Configuration
and URL/Base64 details are now in the DashScope sections below.
##########
docs/content/docs/development/calling-llms/chat-models/providers.md:
##########
@@ -110,10 +110,12 @@ The DashScope integration currently has a Python
implementation only. To use it
| Parameter | Type | Default | Description |
|-----------|------|---------|-------------|
-| `model` | str | `"qwen-plus"` | Name of the chat model to use (an
`output_schema` is enforced natively only on `qwen3.7-max`,
`qwen3.7-max-preview`, `qwen3.7-max-2026-05-17` and `qwen3.7-max-2026-05-20`,
the schema-capable models DashScope serves on the text-generation interface
this connection calls; every other model, the `qwen-plus` default included,
falls back to prompting) |
+| `model` | str | `"qwen-plus"` | Name of the chat model to use. See
[Interface Selection](#interface-selection) for native `output_schema` support.
|
+| `api_type` | str | `"text"` | `text` uses Generation; `multimodal` uses
MultiModalConversation; `auto` selects the interface from the model name as
described below |
+| `task` | str | `None` | Omit for chat and media understanding. Set `asr` for
speech recognition or `image_generation` for image generation and editing. |
| `temperature` | float | `0.7` | Sampling temperature (0.0 to 2.0) |
| `extract_reasoning` | bool | `False` | Extract reasoning content from
response |
-| `additional_kwargs` | dict | `{}` | Additional DashScope API parameters. A
`response_format` supplied here is rejected only when it would be overwritten,
that is when an `output_schema` carrying a Pydantic model is set on a model
that applies it natively; otherwise it is passed through unchanged |
+| `additional_kwargs` | dict | `{}` | Additional DashScope API parameters. For
chat and media understanding, a supplied `response_format` is rejected when a
native `output_schema` would overwrite it. Otherwise it is passed through
unchanged. |
Review Comment:
Updated in 78bf2aa1: shortened this description and moved the
`response_format` conflict rule to the Structured Output subsection.
##########
docs/content/docs/development/calling-llms/chat-models/providers.md:
##########
@@ -143,9 +145,136 @@ class MyAgent(Agent):
...
```
+#### Interface Selection
+
+Use the same setup options in Python, Java, and YAML.
+
+| Effective `api_type` | Operation | Setup options | Result content |
+|---|---|---|---|
+| `text` (default) | Text chat | `model="qwen-plus"` (default) | Text and,
when configured, tool calls |
+| `multimodal` | [Image](#image-input) or [video](#video-input) understanding
| `api_type="multimodal"` with a vision model such as `qwen3-vl-plus` | Text
and, when supported, tool calls |
+| `multimodal` (automatic) | [Speech
recognition](#audio-input-speech-recognition) |
`model="qwen-audio-3.1-asr-flash"`, `task="asr"` | Transcript text |
Review Comment:
Updated in 78bf2aa1: reorganized the table by operation and configuration.
It now explicitly says that `asr` and `image_generation` select
`MultiModalConversation`, so `api_type` can be omitted. Model-name routing is
explained separately.
##########
docs/content/docs/development/calling-llms/chat-models/providers.md:
##########
@@ -143,9 +145,136 @@ class MyAgent(Agent):
...
```
+#### Interface Selection
+
+Use the same setup options in Python, Java, and YAML.
+
+| Effective `api_type` | Operation | Setup options | Result content |
+|---|---|---|---|
+| `text` (default) | Text chat | `model="qwen-plus"` (default) | Text and,
when configured, tool calls |
+| `multimodal` | [Image](#image-input) or [video](#video-input) understanding
| `api_type="multimodal"` with a vision model such as `qwen3-vl-plus` | Text
and, when supported, tool calls |
+| `multimodal` (automatic) | [Speech
recognition](#audio-input-speech-recognition) |
`model="qwen-audio-3.1-asr-flash"`, `task="asr"` | Transcript text |
+| `multimodal` (automatic) | [Image generation and
editing](#synchronous-image-generation-and-editing) | `model="qwen-image-3.0"`,
`task="image_generation"` | Image blocks |
+
+The multimodal interface uses MultiModalConversation (MMC) in non-streaming
+mode. Setting either task selects MMC automatically, so you can omit
+`api_type`. For these tasks, leave `prompt`, `tools`, `skills`, and `subagents`
+unset on the setup and omit `output_schema` and `response_format`.
+
+Add media to user messages. Model responses use `ChatResult`, including token
+usage and the finish reason when available. If the service reports only audio
+duration or image count for a task, token usage is absent.
+
+`api_type="auto"` recognizes the following MMC model families and exceptions:
Review Comment:
Updated in 78bf2aa1: clarified that `auto` selects `multimodal` for matching
model names and `text` otherwise. Replaced the detailed matching table with
examples and a link to the complete routing rules.
##########
docs/content/docs/development/calling-llms/chat-models/providers.md:
##########
@@ -143,9 +145,136 @@ class MyAgent(Agent):
...
```
+#### Interface Selection
+
+Use the same setup options in Python, Java, and YAML.
+
+| Effective `api_type` | Operation | Setup options | Result content |
+|---|---|---|---|
+| `text` (default) | Text chat | `model="qwen-plus"` (default) | Text and,
when configured, tool calls |
+| `multimodal` | [Image](#image-input) or [video](#video-input) understanding
| `api_type="multimodal"` with a vision model such as `qwen3-vl-plus` | Text
and, when supported, tool calls |
+| `multimodal` (automatic) | [Speech
recognition](#audio-input-speech-recognition) |
`model="qwen-audio-3.1-asr-flash"`, `task="asr"` | Transcript text |
+| `multimodal` (automatic) | [Image generation and
editing](#synchronous-image-generation-and-editing) | `model="qwen-image-3.0"`,
`task="image_generation"` | Image blocks |
+
+The multimodal interface uses MultiModalConversation (MMC) in non-streaming
+mode. Setting either task selects MMC automatically, so you can omit
+`api_type`. For these tasks, leave `prompt`, `tools`, `skills`, and `subagents`
+unset on the setup and omit `output_schema` and `response_format`.
+
+Add media to user messages. Model responses use `ChatResult`, including token
+usage and the finish reason when available. If the service reports only audio
+duration or image count for a task, token usage is absent.
+
+`api_type="auto"` recognizes the following MMC model families and exceptions:
+
+| Models | Matching rule |
+|--------|---------------|
+| `qwen-vl-plus`, `qwen-vl-max`, `qwen3-vl-plus`, `qwen3-vl-flash` | Exact
names and variants beginning with the name followed by `-` |
+| `qwen3.5-plus`, `qwen3.5-flash`, `qwen3.6-plus`, `qwen3.6-flash`,
`qwen3.7-plus`, `qwen3.7-flash`, `qwen3.8-max`, `qwen3.8-flash` | Exact names
and variants beginning with the name followed by `-` |
+| Qwen2-VL, Qwen2.5-VL, Qwen3-VL, Qwen3.5, Qwen3.6 and Qwen3.8 open-weight
models | A parameter size beginning with a digit after the family prefix, such
as `qwen3-vl-8b-instruct` or `qwen3.8-2.4t-a95b` |
+| QVQ | `qvq` or names beginning with `qvq-` |
+| `qwen3.7-max-2026-06-08`, `kimi-k2.5` | Exact names |
+
+Names not matching these rules use the text interface. In particular,
`qwen3.7-max`, its preview,
+and its `2026-05-17` and `2026-05-20` snapshots remain on text generation.
+Unlisted models and deployment aliases can select `multimodal` explicitly.
+VL embedding, rerank and OCR models are outside this chat model list.
+
+Routing identifies the API, not the model's availability or supported
invocation modes.
Review Comment:
Updated in 78bf2aa1: shortened this to the non-streaming restriction, with
links to DashScope's streaming and thinking requirements. The configuration
section separately states that media inputs do not select the interface
automatically.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
