BeiBaiNian opened a new pull request, #217:
URL: https://github.com/apache/casbin-gateway/pull/217
## Why
The gateway is the single choke point for /v1/chat/completions traffic, but
operators have no visibility into what models are actually being asked to do.
Debugging prompt injection, abuse, or unexpected model usage today means
tapping every upstream or grepping logs that never capture request bodies.
Prompt content is sensitive operational data, so any retention must be opt-in,
sanitized, and strictly bounded.
## What changed
Gateway can now record sanitized chat-completions requests in a new
llm\_request\_audit table, viewable by administrators in the management UI.
### Backend:
- object/llm\_request\_audit.go: new table (id, createdTime, model, channel,
clientIp, stream, payload, truncated, originalBytes), created via Sync2 on
startup.
- Best-effort async writer with a bounded queue: the proxy goroutine only
enqueues a metadata snapshot and never parses, sanitizes, or writes; a full
queue drops records and logs the drop count at most once a minute; graceful
shutdown drains accepted entries.
- Sanitization runs only on the writer goroutine — the sole body-retention
boundary. Headers (especially inbound authorization) are never copied into a
record. Credential-bearing fields and embedded secrets are redacted via
auditutil.SanitizeValue; bodies larger than 64 KiB are stored as safe metadata
(truncated, originalBytes) instead of raw bytes.
- Retention is bounded by age (llmRequestAuditRetentionDays) and count
(llmRequestAuditMaxRecords), pruned once a minute.
- controllers/proxy.go: forwardToChannel records an entry only after the
upstream responds, so each failover attempt keeps its real channel.
- New admin-only, paginated endpoint GET /api/get-llm-request-audits.
- conf/conf.go + conf/app.conf: llmRequestAuditEnabled (opt-in; disabled
installs retain nothing and pay no proxy-path cost),
llmRequestAuditQueueCapacity, llmRequestAuditRetentionDays,
llmRequestAuditMaxRecords.
Frontend:
New admin-only "LLM Request Audits" page
(web/src/pages/LlmRequestAuditsPage.tsx) with pagination and a table of channel
/ model / mode / client IP / stored / truncated; full i18n (en + zh); route and
sidebar entry added.
## Verification
- go test ./object -run TestLlmRequestAudit -v: covers sanitization,
credential redaction, oversized-body metadata, and invalid-JSON rejection.
- go test ./controllers ./object: passes, including the updated
forwardToChannel tests.
- go build ./...: clean.
- Manual: enable llmRequestAuditEnabled \= true, send requests through
/v1/chat/completions, confirm records appear in the admin page, embedded
credentials in messages/tools are redacted, oversized requests are truncated,
and entries beyond the retention window are pruned.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
