BeiBaiNian opened a new pull request, #190:
URL: https://github.com/apache/casbin-gateway/pull/190
This PR is part of issue #179 "make a UI-first LLM API security gateway".
The unified proxy endpoint is the routing layer (milestone 1.2): it reads the
`model` field from the request body, matches an enabled channel from the 1.1
Channel table (ordered by Priority), forwards the request to the upstream with
the channel's API key injected, and relays the response back, including SSE
streaming.
## What changed
- **Added** **`controllers/proxy.go`**: OpenAI-compatible `POST
/v1/chat/completions` endpoint. It parses the body (only `model` and `stream`
are read; everything else is passed through), looks up the channel via
`GetChannelByModel`, injects `Authorization: Bearer ***` and forwards the raw
body. Non-stream responses are relayed header/status/body as-is; streaming
responses are flushed chunk-by-chunk via `c.Ctx.ResponseWriter.Write() +
Flush()`. Errors follow the OpenAI error format: 400 for invalid JSON / missing
model / no available channel, 502 for upstream connection failure or
misconfigured channel, 504 on upstream timeout. A shared package-level
`http.Client` is reused across requests for connection pooling. This endpoint
does NOT require Casdoor authentication (auth is deferred to milestone 1.3
tokens).
- **Modified** **`object/channel.go`**: Added `GetChannelByModel()` which
queries only `status=enabled` channels ordered by `priority` (ascending) and
returns the first channel supporting the requested model. Introduced the
`ErrNoChannelAvailable` sentinel error so callers can distinguish "no match"
(HTTP 400) from database failures (HTTP 502).
- **Modified** **`routers/filter.go`**: `TransparentStatic` now excludes the
`/v1/` prefix in addition to `/api/`, so the new endpoint is not swallowed by
static file handling and bypasses the Casdoor `ApiFilter`.
- **Modified** **`routers/router.go`**: Registered `POST
/v1/chat/completions`.
## Verification
- `go build ./...` passed.
- API tests against a local mock upstream all passed:
- Missing `model` / invalid JSON / unknown model → 400 with clear
OpenAI-format error
- Disabled channels are skipped; channels with empty base URL → 502
- Model routing honors Priority (lowest priority wins)
- Non-stream forwarding returns upstream status/body as-is (200, 429, 500
verified)
- SSE streaming (`stream=true`) relays all chunks with `text/event-stream`
+ flush
- Upstream connection failure → 502; slow upstream is waited for and
relayed
- Endpoint is reachable without any Casdoor cookie (contrast: `/api/*`
returns "please sign in first")
- `go test ./...` has 3 pre-existing failures unrelated to this PR (Alibaba
Cloud RDS client and local MySQL credentials in `run/` and `sync/` tests).
## Note
Per the plan, this endpoint is intentionally unauthenticated for now (auth
lands in milestone 1.3). Do not deploy this PR alone to production — it should
be merged and released back-to-back with 1.3.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
