wenjin272 commented on code in PR #1375:
URL: https://github.com/apache/flink-agents/pull/1375#discussion_r4522803028
##########
docs/content/docs/development/calling-llms/model_routing.md:
##########
@@ -117,6 +118,105 @@ environment.
Python uses the same builder with `default_model(name)` and
`fallback(enabled=...)`.
+<span id="yaml"></span>
+
+#### YAML
Review Comment:
Could we change this to `### YAML API` and add a matching `### Programmatic
API` heading before the programmatic example? Both belong directly under
`Register a Router`; the current heading skips a level.
##########
docs/content/docs/development/calling-llms/model_routing.md:
##########
@@ -117,6 +118,105 @@ environment.
Python uses the same builder with `default_model(name)` and
`fallback(enabled=...)`.
+<span id="yaml"></span>
+
+#### YAML
+
+Declare routers under `model_routers` on an agent or at the top level of the
file.
+A top-level router is shared by every agent loaded into the environment. The
+section uses the same descriptor fields as other implementation-class
resources:
+`name`, `clazz`, and optional `type`. The remaining fields are the descriptor
+arguments produced by the builder.
Review Comment:
Could we shorten this to: “Declare a router under `model_routers`, with
`clazz` set to `router`. The following fields configure its routing behavior.”
Shared versus agent-local placement and common descriptor fields are already
explained in the YAML API chapter. The explanation about arguments produced by
the builder is unnecessary here.
##########
python/flink_agents/api/yaml/aliases.py:
##########
@@ -163,6 +163,14 @@
"mcp": ResourceName.MCP_SERVER,
},
},
+ ResourceType.MODEL_ROUTER: {
+ "python": {
+ "model_router": ResourceName.ModelRouter.MODEL_ROUTER,
+ },
+ "java": {
+ "model_router": ResourceName.ModelRouter.Java.MODEL_ROUTER,
Review Comment:
Could we use `router` instead of `model_router` as the alias? The
`model_routers` section already identifies the resource type. Please update
both Java and Python alias tables, tests, and documentation consistently.
##########
docs/content/docs/development/calling-llms/model_routing.md:
##########
@@ -117,6 +118,105 @@ environment.
Python uses the same builder with `default_model(name)` and
`fallback(enabled=...)`.
+<span id="yaml"></span>
+
+#### YAML
+
+Declare routers under `model_routers` on an agent or at the top level of the
file.
+A top-level router is shared by every agent loaded into the environment. The
+section uses the same descriptor fields as other implementation-class
resources:
+`name`, `clazz`, and optional `type`. The remaining fields are the descriptor
+arguments produced by the builder.
+
+| Field | Required | Description |
+|-------|----------|-------------|
+| `candidates` | Required | Chat model setup names. Names must be non-empty
and unique. Order is the fallback order. |
+| `strategy_type` | Required | `rule_based`, `custom`, or, on Java,
`llm_judge` or `decision_model`. |
+| `strategy_args` | Depends on the strategy | Rule map, judge or
decision-model settings, or custom executor arguments. |
+| `strategy_executor_class` | Custom only | Fully qualified custom executor
class. |
+| `candidate_descriptions` | Optional | Map of candidate name to description. |
+| `default_model` | Optional | Candidate used when the strategy abstains. When
omitted, the first candidate is used. |
+| `fallback` | Optional | Try the remaining candidates after the selected
model fails. Defaults to `false`. |
+
+`clazz: model_router` selects the implementation for the loader's language. A
fully
+qualified class name works as well. Model routers do not cross languages:
Python
+YAML uses the default Python implementation, and Java YAML sets `type: java`.
+`strategy_args.rules` maps a candidate name to a regular expression.
`judge_model`
+and `decision_model` name registered chat-model and decision-model setups.
Review Comment:
Could we remove this paragraph? The introduction should already specify
`clazz: router`, and general class-name and language rules belong in the YAML
API chapter. Keep a short note beside the example saying that Java YAML
requires `type: java`. Strategy-specific fields such as `strategy_args.rules`,
`strategy_args.judge_model`, and `strategy_args.decision_model` belong in their
respective strategy sections.
##########
docs/content/docs/development/calling-llms/model_routing.md:
##########
@@ -117,6 +118,105 @@ environment.
Python uses the same builder with `default_model(name)` and
`fallback(enabled=...)`.
+<span id="yaml"></span>
+
+#### YAML
+
+Declare routers under `model_routers` on an agent or at the top level of the
file.
+A top-level router is shared by every agent loaded into the environment. The
+section uses the same descriptor fields as other implementation-class
resources:
+`name`, `clazz`, and optional `type`. The remaining fields are the descriptor
+arguments produced by the builder.
+
+| Field | Required | Description |
+|-------|----------|-------------|
+| `candidates` | Required | Chat model setup names. Names must be non-empty
and unique. Order is the fallback order. |
+| `strategy_type` | Required | `rule_based`, `custom`, or, on Java,
`llm_judge` or `decision_model`. |
+| `strategy_args` | Depends on the strategy | Rule map, judge or
decision-model settings, or custom executor arguments. |
+| `strategy_executor_class` | Custom only | Fully qualified custom executor
class. |
+| `candidate_descriptions` | Optional | Map of candidate name to description. |
+| `default_model` | Optional | Candidate used when the strategy abstains. When
omitted, the first candidate is used. |
+| `fallback` | Optional | Try the remaining candidates after the selected
model fails. Defaults to `false`. |
+
+`clazz: model_router` selects the implementation for the loader's language. A
fully
+qualified class name works as well. Model routers do not cross languages:
Python
+YAML uses the default Python implementation, and Java YAML sets `type: java`.
+`strategy_args.rules` maps a candidate name to a regular expression.
`judge_model`
+and `decision_model` name registered chat-model and decision-model setups.
+
+A declaration is checked when the agent plan is built. An invalid candidate
list,
+rule, default model, or strategy fails with the same error as the same
descriptor
+registered programmatically. A strategy the runtime does not support, such as
+`llm_judge` or `decision_model` on Python, fails at that same check.
+
+{{< tabs "YAML Router" >}}
+{{< tab "Python" >}}
+```yaml
+model_routers:
+ - name: router
+ clazz: model_router
+ candidates: [small, big]
+ candidate_descriptions:
+ small: Everyday requests
+ big: Code, SQL, and analysis
+ strategy_type: rule_based
+ strategy_args:
+ rules:
+ big: '\b(code|sql|program|analyze|prove)\b'
+ default_model: small
+ fallback: true
+
+agents:
+ - name: model_routing_agent
+ model_routers:
+ - name: local_router
+ clazz: model_router
+ candidates: [small, big]
+ strategy_type: rule_based
+ strategy_args:
+ rules:
+ big: '\b(code|sql)\b'
+ default_model: small
+```
+{{< /tab >}}
+{{< tab "Java" >}}
+```yaml
+model_routers:
+ - name: router
+ clazz: model_router
+ type: java
+ candidates: [small, big]
+ candidate_descriptions:
+ small: Everyday requests
+ big: Code, SQL, and analysis
+ strategy_type: rule_based
+ strategy_args:
+ rules:
+ big: '\b(code|sql|program|analyze|prove)\b'
+ default_model: small
+ fallback: true
+
+agents:
+ - name: model_routing_agent
+ model_routers:
+ - name: local_router
+ clazz: model_router
+ type: java
+ candidates: [small, big]
+ strategy_type: rule_based
+ strategy_args:
+ rules:
+ big: '\b(code|sql)\b'
+ default_model: small
+```
+{{< /tab >}}
+{{< /tabs >}}
+
+Reference the router name from a `ChatRequestEvent` the same way as a
programmatic
+router. Candidate names still refer to registered `CHAT_MODEL` resources. See
+[YAML]({{< ref "docs/development/create-agent/yaml" >}}) for where shared and
+agent-local sections are declared.
Review Comment:
Could we remove the first two sentences? `Call a Router` already explains
how to invoke a router, and the `candidates` field description already
identifies the referenced resources as chat model setups. A link to the YAML
API chapter can remain for the general declaration rules.
##########
docs/content/docs/development/calling-llms/model_routing.md:
##########
@@ -117,6 +118,105 @@ environment.
Python uses the same builder with `default_model(name)` and
`fallback(enabled=...)`.
+<span id="yaml"></span>
+
+#### YAML
+
+Declare routers under `model_routers` on an agent or at the top level of the
file.
+A top-level router is shared by every agent loaded into the environment. The
+section uses the same descriptor fields as other implementation-class
resources:
+`name`, `clazz`, and optional `type`. The remaining fields are the descriptor
+arguments produced by the builder.
+
+| Field | Required | Description |
+|-------|----------|-------------|
+| `candidates` | Required | Chat model setup names. Names must be non-empty
and unique. Order is the fallback order. |
+| `strategy_type` | Required | `rule_based`, `custom`, or, on Java,
`llm_judge` or `decision_model`. |
+| `strategy_args` | Depends on the strategy | Rule map, judge or
decision-model settings, or custom executor arguments. |
+| `strategy_executor_class` | Custom only | Fully qualified custom executor
class. |
+| `candidate_descriptions` | Optional | Map of candidate name to description. |
+| `default_model` | Optional | Candidate used when the strategy abstains. When
omitted, the first candidate is used. |
+| `fallback` | Optional | Try the remaining candidates after the selected
model fails. Defaults to `false`. |
+
+`clazz: model_router` selects the implementation for the loader's language. A
fully
+qualified class name works as well. Model routers do not cross languages:
Python
+YAML uses the default Python implementation, and Java YAML sets `type: java`.
+`strategy_args.rules` maps a candidate name to a regular expression.
`judge_model`
+and `decision_model` name registered chat-model and decision-model setups.
+
+A declaration is checked when the agent plan is built. An invalid candidate
list,
+rule, default model, or strategy fails with the same error as the same
descriptor
+registered programmatically. A strategy the runtime does not support, such as
+`llm_judge` or `decision_model` on Python, fails at that same check.
+
+{{< tabs "YAML Router" >}}
+{{< tab "Python" >}}
+```yaml
+model_routers:
Review Comment:
Could we keep a single agent-local YAML example instead of separate
Python/Java tabs containing both shared and local declarations? The language
variants differ only by `type: java`, which a short note can explain. Resource
scopes are already covered in the YAML API chapter. This would let the example
focus on routing configuration.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
