ambiguous-pointer commented on PR #1585:
URL: https://github.com/apache/dubbo-admin/pull/1585#issuecomment-6720214558

   Thanks for the review — both of your approve conditions were fair, and the 
budget one had a real defect behind it.
   
   I checked your "15 declared, 3 applied" count independently before changing 
anything: only `thinking_format`, `max_tokens_field` and 
`thinking_budget_field` are read by `ApplyRequest`, and both `SupportsForced()` 
and `ExtractThinking()` have no caller outside the tests. Your count was right.
   
   ### 1. The reserved fields
   
   Rather than dropping them, I made the configuration unable to lie about them:
   
   - `Compat` now opens with an applied/reserved split, and each of the 12 
reserved fields says what it would do and why nothing applies it yet.
   - `Validate` **rejects** a reserved field that is configured, with a message 
naming the field, the reason it does not apply, and a pointer to the design 
document. The check runs last, so a mistake with a more specific explanation 
still gets that explanation.
   - `models.schema.json` marks every reserved key with a description saying it 
is reserved and configuring it is rejected.
   - `models.yaml` no longer carries the four reserved keys it used to 
(`supports_developer_role`, `supports_strict_mode`, 
`supports_forced_tool_choice`, `ignore_leading_whitespace`). What was measured 
about this endpoint is recorded in the new document instead — including the 
"required is silently ignored" finding, which is the one worth keeping.
   - `thinking_budget_field` went as well: on its own it sends nothing, so it 
was the same kind of "looks configured but is not".
   
   The fields stay in the struct, so the design is not lost and wiring one up 
later is a matter of reading it — the merge rules are already one-to-one with 
the config.
   
   ### 2. The budget guard
   
   You were right that it guarded nothing, and the risk was wider than 
DashScope: `-1` has no meaning for vLLM's `thinking_token_budget` or 
llama.cpp's `thinking_budget_tokens` at all, and may be rejected outright.
   
   - The cap is now a number: a new `thinking_budget` in tokens, sent only when 
it is positive, and sent as that number.
   - Unset means unset — no field is sent. A cap that is not a cap is not worth 
a field.
   - `Validate` requires `thinking_budget` to come with a 
`thinking_budget_field`.
   - The tests assert the **value** now: absent when no cap is configured, the 
configured number when it is, and one case per spelling.
   
   ### 3. `chat_template_kwargs`
   
   The inner key is configurable through a new `chat_template_key`. It is 
required for the `chat-template` format — vLLM serves model families that 
disagree on the name, so there is no safe default there — and defaults to 
`enable_thinking` for `qwen-chat-template`. There is a test that the inner key 
is selectable.
   
   ### 4. The `reasoning_effort` comment
   
   The comment above the level list now says that only the spelling is checked, 
and the field comment says the same. Which levels an endpoint accepts is a 
property of that endpoint; the two comments agree now.
   
   While looking at this I noticed something worth flagging: the level only 
survives on the formats that carry one. `openai` sends `reasoning_effort` and 
`openrouter` sends `reasoning.effort`, but a provider whose switch is a plain 
boolean — Qwen, DeepSeek, the chat-template formats — receives "thinking on" 
and nothing more. So `reasoning_effort: high` against a Qwen endpoint produces 
`enable_thinking: true` and the endpoint picks its own budget. That is now 
documented rather than left to be discovered from a request log.
   
   ### 5. The ADRs
   
   The conclusions were only ever in one line of Go comment each, so they are 
now in `docs/ai-provider-compat.md`: field naming by capability, conservative 
defaults, the two-level merge, the thinking-field probe order and the vLLM 
rename behind it, the separate capability bits, and the startup validation. The 
code comments point at the document's sections.
   
   ### 6. Naming
   
   `ResolvedCompat` mirrors the configuration field for field now: 
`SupportsForcedToolChoiceWithThinking` and `RequiresAssistantAfterToolResult`.
   
   ### 7. Rebase
   
   Rebased onto the current `ai` tip, `e6cef77`. `go build ./...`, `go vet 
./...` and `go test -count=1 ./...` are green — 22 packages.
   
   ### One behaviour change worth knowing
   
   The shipped configuration no longer sends a `thinking_budget` field at all. 
It used to send `-1`, which this endpoint reads as its own default, so the 
effect on the endpoint is the same — but the request body has one key less, and 
it no longer looks guarded when it is not.
   
   ### The gap you flagged first, restated
   
   Your first point was that the replay is not implemented either, and the 
description only said the read-back was. Both are now stated in the same place: 
the content never reaches this layer, so it can neither be read nor written 
back, and fixing the read side is what unblocks the replay. `thinking_field`, 
`send_back_thinking`, `thinking_tags` and `backfill_thinking_field` are 
reserved for that reason.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to