ambiguous-pointer opened a new pull request, #1585:
URL: https://github.com/apache/dubbo-admin/pull/1585
## Summary
Adds an explicit compatibility layer for the OpenAI-compatible providers the
AI agent talks to. Today every provider is registered through genkit's
`compat_oai` plugin with only an API key and a base URL, so the request-shaping
quirks of each endpoint are implicit and unconfigurable. This makes those
differences declared, validated and testable.
No behaviour change for the current default path — `dashscope/qwen3.5-plus`
produces the same request as before, plus a thinking-budget guard (see
*Findings*).
## What changed
**`component/models/compat.go` (new)** — a `Compat` block describing the
per-provider quirks that matter to an agent loop:
- the spelling of the thinking switch (`enable_thinking` / `thinking.type` /
`reasoning_effort` / `reasoning.effort` / `chat_template_kwargs`)
- the field thinking content arrives in, and how it is replayed on later
turns
- the thinking budget field name
- the length field (`max_tokens` vs `max_completion_tokens`)
- whether `system` or `developer` is accepted
- `strict` tool definitions, and forced tool choice
Three design choices worth calling out for review:
1. **Defaults are conservative, not OpenAI-like.** An unset field picks the
spelling the widest set of endpoints accepts, not the one real OpenAI uses.
2. **No base-URL sniffing.** pi-ai infers capabilities from the URL; that
works until a gateway sits in front of a provider, at which point it is
confidently wrong. Here `models.yaml` states the quirks outright.
3. **Booleans are pointers** so that "unset" stays distinct from "explicitly
false" when a provider-level block is merged with a per-model override. A plain
`bool` silently makes it impossible for one model to turn off something its
provider enables — a bug the tests caught during development.
**`component/agent/react/*`** — resolves the model's compat once at
construction and applies it while building the prompt. Adds `reasoning_effort`
to the agent spec, validated against the supported levels. Sampling parameters
are dropped when reasoning is on, matching the endpoint's documented behaviour.
**`schema/json/models.schema.json`** — adds the `compat` definition. This is
not optional: `provider` and `model_info` both set `additionalProperties:
false`, so adding the Go struct alone would have made every boot fail.
**Tests** — assert the *serialized request body* rather than the struct that
produced it, plus a suite that loads the shipped `models.yaml` through the real
loader and schema, and a fixture proving unknown compat keys are rejected
rather than silently ignored.
## Findings from measuring the endpoint
The DashScope entries were confirmed against the live `compatible-mode` API
(2026-10-04):
| Claim | Result |
|---|---|
| thinking switch for Qwen | top-level `enable_thinking` — confirmed |
| field carrying thinking | `reasoning_content` — confirmed |
| accepted roles | `system` only; `developer` returns 400 — confirmed |
| `tool_choice: "required"` | **silently ignored**, not rejected: asked a
question needing no tool, `required` and `auto` behaved identically |
Two findings worth flagging, because they are not visible from the code:
1. **`dashscope/qwen3.5-plus` reasons by default.** A request with no
`enable_thinking` still returns `reasoning_content`; only `enable_thinking:
false` suppresses it. Thinking draws from the same `max_tokens` pool as the
answer — an unprompted troubleshooting question burned 1648 reasoning tokens.
Since genkit does not read the field back (below), that was cost and budget
spent for nothing. Hence the budget guard being sent whenever one is
configured, rather than only when reasoning was explicitly requested.
2. **The stream-only thinking restriction does not apply here.** It applies
to open-weight Qwen3 deployments, not the hosted model, so the existing
non-streaming call path is unaffected. This closed what had been flagged as the
blocking design question.
## Not tested — please weigh in
**Reading thinking content back does not work yet.** genkit's `compat_oai`
only maps `Content` and `ToolCalls` out of the response:
```go
// genkit/plugins/compat_oai/generate.go:386-423
Message: &ai.Message{Role: ai.RoleModel},
...
if completion.Choices[0].Message.Content != "" { ... } // no
reasoning_content handling
```
openai-go does capture the raw field via `ExtraFields`, but genkit does not
surface it. So `ExtractThinking` has no call site and the replay/backfill path
stays dormant.
This is not a regression — it blocks nothing that worked before, and
DashScope was measured to not require the replay. It does mean the feature is
inert against self-hosted targets (DeepSeek, vLLM), where it would matter.
Three ways forward, none chosen here:
1. upstream the response mapping to genkit
2. build a Model in `component/models` instead of using `compat_oai`
3. reach the raw response from a different layer
Also unverified: the code sends `thinking_budget: -1` meaning "endpoint
default", but only an explicit positive value was measured.
`ignore_leading_whitespace` is set defensively on the DashScope block; the
behaviour it works around was observed with Ollama + Qwen3, not here.
## Testing
- `go vet ./...`
- `go test ./... -short` — 19 packages, including
`TestDashScopeMeasuredBehaviour` which pins the measured behaviours above
## Notes for the reviewer
- `Compat` currently reaches only the top level of the request body.
`strict`, the system role, and message ordering are decided by genkit's
`ai.WithTools` / `ai.WithSystem` / `ai.WithMessages` and are out of its reach —
the fields are resolved and documented but have no effect yet.
- The relationship to the design notes in
`docs/ai-context-management-design.md` is only the shared idea of a declared,
validated config block; the page-context work is untouched and its Phase 3 is
still missing.
- Two pre-existing P0s in this area are also untouched and are more serious
than this change: the backend silently drops the `context` field the frontend
sends, and the AI endpoints have no authentication.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
