GeorgelPreput opened a new pull request, #13569:
URL: https://github.com/apache/apisix/pull/13569
### Description
`ai-proxy`/`ai-proxy-multi` can translate Anthropic-format clients to
OpenAI-compatible upstreams (via the `anthropic-messages-to-openai-chat`
converter), but not the reverse. There was no way to accept an OpenAI Chat
Completions request and forward it to an upstream that only speaks the native
Anthropic Messages API (`/v1/messages`) — e.g. **Azure AI Foundry Claude**,
whose only surface is
`https://<resource>.services.ai.azure.com/anthropic/v1/messages` (no
OpenAI-compatible `/chat/completions`). Today an OpenAI body sent through the
`anthropic` provider is passed through unchanged and rejected by the upstream
(`messages.0: use the top-level 'system' parameter ...`).
This PR adds the missing direction (non-streaming).
**Changes**
- **New `anthropic-compatible` provider** exposing *only* the
`anthropic-messages` capability. This is the crux: `ai-proxy/base.lua` selects
native passthrough whenever the provider declares a capability matching the
client protocol, *before* looking for a converter. The existing `anthropic`
provider declares **both** `openai-chat` and `anthropic-messages`, so OpenAI
clients always passthrough and never reach a converter. A provider exposing
only the Messages capability lets the converter run — mirroring how
`openai-compatible` is the vendor-neutral counterpart used as a conversion
*target*.
- **New `openai-chat` → `anthropic-messages` converter**:
- Request: `system` role → top-level `system`; `tools`/`tool_choice`;
`tool` messages → user `tool_result` blocks (coalesced); image parts;
`reasoning_effort` → `thinking`; `max_tokens`/`max_completion_tokens` →
`max_tokens` (defaulted since Anthropic requires it).
- Non-streaming response: content / `tool_use` / `thinking` →
`choices[].message`; stop-reason and usage mapping (incl. cache tokens) →
OpenAI shape.
- Headers: adds `anthropic-version`, strips OpenAI/stainless telemetry.
- **Streaming (`stream: true`) is rejected with a clear error** for now (see
follow-ups).
- Tests (`t/plugin/ai-proxy-openai-to-anthropic.t`, unit + e2e) and docs
(`ai-proxy.md`, `ai-proxy-multi.md`, en + zh).
Verified end-to-end against a live Azure AI Foundry Claude deployment via
`ai-proxy-multi` + `anthropic-compatible`: basic chat (with a `system` prompt),
tool-call requests (`tool_use` → `tool_calls`), and a multi-turn tool-result
round-trip all returned correct OpenAI-shaped responses; `stream: true`
returned a clean 400.
**Open questions for maintainers (documented as potential follow-ups rather
than decided here):**
1. **Streaming.** Supporting `stream: true` in this direction needs the
Anthropic Messages SSE adapter (`ai-protocols/anthropic-messages.lua`
`parse_sse_event`) to pass raw event data through to the converter (it
currently digests/drops `tool_use` and `finish_reason` detail). Happy to do
this in a follow-up — kept this PR focused to get directional feedback first.
2. **Upstream auth header handling.** The converter currently rewrites an
upstream `Authorization: Bearer <key>` to `x-api-key`. This works for
Azure/Anthropic **key-based** auth, but would break a **Microsoft Entra ID**
bearer token. Options: make the rewrite conditional, drop it and rely solely on
`auth.header`, or document it. Guidance welcome.
#### Which issue(s) this PR fixes:
Partially addresses #13566 (non-streaming; streaming tracked as a follow-up).
### Checklist
- [x] I have explained the need for this PR and the problem it solves
- [x] I have explained the changes or the new features added to this PR
- [x] I have added tests corresponding to this change
- [x] I have updated the documentation to reflect this change
- [x] I have verified that this change is backward compatible (purely
additive — new provider + new converter; no existing provider or the reverse
converter is modified)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
