bobbai00 opened a new issue, #5604:
URL: https://github.com/apache/texera/issues/5604
### What happened?
Since #5421, the LiteLLM proxy endpoints in the access-control-service
(`/api/chat/{path}`, `/api/models`) are guarded by `@RolesAllowed({"REGULAR",
"ADMIN"})`: they only accept a Texera user JWT, and the proxy injects
`LITELLM_MASTER_KEY` into the downstream request to LiteLLM
(`AccessControlResource.scala`).
The agent-service was not updated for this. `createAgentInstance`
(`agent-service/src/server.ts`) still builds its OpenAI client with the static
`env.LLM_API_KEY`, which defaults to the literal string `"dummy"`
(`agent-service/src/config/env.ts`). The proxy rejects `Bearer dummy` with
**401 Unauthorized**, so on current `main` every agent generation call fails in
the default (proxied) deployment — agent creation succeeds, but the agent can
never produce a response.
```mermaid
sequenceDiagram
participant FE as Frontend (user JWT)
participant AS as agent-service
participant ACS as access-control-service (LiteLLM proxy)
participant LLM as LiteLLM
FE->>AS: create agent (userToken = user JWT) ✅
FE->>AS: send message
AS->>ACS: POST /api/chat/completions<br/>Authorization: Bearer dummy ❌
ACS-->>AS: 401 Unauthorized (REGULAR/ADMIN JWT required)
Note over ACS,LLM: never reached — proxy would
inject<br/>LITELLM_MASTER_KEY downstream
AS-->>FE: agent turn fails
```
Expected: the agent-service already holds the delegating user's JWT
(`userToken` from the create-agent request), so it should present that JWT to
the proxy. An explicitly configured `LLM_API_KEY` should still take precedence
for direct, unproxied LiteLLM deployments.
### How to reproduce?
1. Deploy current `main` with the standard topology: agent-service
`LLM_ENDPOINT` pointing at the access-control-service proxy (`:9096`, the
default in k8s/single-node).
2. Log in as a REGULAR user and create an agent (`POST /api/agents` with
`userToken`) — this succeeds.
3. Send a message to the agent over the websocket (`/api/agents/:id/react`).
4. The turn fails: the agent-service gets 401 from `POST
/api/chat/completions` because it sent `Authorization: Bearer dummy`.
Equivalent check without the agent: `curl -H "Authorization: Bearer dummy"
http://localhost:9096/api/models` → 401, while the same request with a valid
REGULAR-user JWT → 200.
### Version/Branch
1.3.0-incubating-SNAPSHOT (main)
### Commit Hash (Optional)
07ca5d4cd (regression introduced by 24e5b7018, #5421)
### What browsers are you seeing the problem on?
_No response_ (backend issue, browser-independent)
### Relevant log output
```shell
# agent-service (representative)
AI_APICallError: Unauthorized
statusCode: 401
url: http://access-control-service-svc:9096/api/chat/completions
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
