nic-6443 opened a new pull request, #13852:
URL: https://github.com/apache/apisix/pull/13852
Request-level fallback in `ai-proxy-multi` only covers 429 and 5xx. A
provider that answers 401 for an expired API key, or 402 for a drained quota,
returns that response straight to the client even when other instances with
working keys are configured. Active health checks can eventually take the bad
instance out of rotation, but they cannot rescue the request that is already in
flight — which is exactly the multi-key setup people use this plugin for.
This adds `fallback_http_statuses`, an explicit list of upstream statuses
that join the `http_429` / `http_5xx` entries of `fallback_strategy` in
triggering a fallback:
```json
{
"fallback_http_statuses": [401, 402],
"instances": [ ... ]
}
```
It is opt-in per status rather than a blanket "retry 4xx": most 4xx
responses are caused by the request itself, and retrying those on another
instance only burns quota on a request that will fail everywhere.
The decision lives in two places that have to agree. `ai-proxy/base.lua`
diverts only 429/5xx to the error path, where the response body is read and
handed to the retry callback; every other status is parsed and streamed to the
client and never reaches `retry_on_error` at all. So relaxing only the retry
condition would have had no effect — both now consult the same helper.
`max_retries` and `retry_on_failure_within_ms` bound these retries like any
other fallback, and with no matching status the behaviour is unchanged.
Tests cover the fallback on 401/402, the `max_retries` interaction, the
unchanged pass-through when the status is not configured, and the schema range
check.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
