janiussyafiq opened a new pull request, #13793: URL: https://github.com/apache/apisix/pull/13793
### Description The retry loop in `before_proxy` re-reads the parsed request body from the per-request cache (`ctx._request_body_table`), but `build_request` rewrites that shared table in place with per-instance options. A fallback attempt therefore starts from the previous instance's rewritten body: `$request_llm_model` is clobbered with the first instance's model, and the first instance's options (e.g. `temperature`) leak into the retry request sent to the fallback instance. This PR deep-copies the parsed body at the start of every attempt so each instance builds its request from the client's original body, and drops the redundant `request_llm_model` re-assignment in `before_proxy` (`detect_request_type` already captures it from the client body in the access phase, for both `ai-proxy` and `ai-proxy-multi`). The copy is O(body) and cheaper than the JSON decode of the same body already performed in the access phase. Keeping the shared cache pristine also fixes log-phase `post_arg.*` variable reads and the semantic balancer's re-pick on retry, both of which previously saw the instance-rewritten body instead of the client's request. #### Which issue(s) this PR fixes: Fixes #13769 ### Checklist - [x] I have explained the need for this PR and the problem it solves - [x] I have explained the changes or the new features added to this PR - [x] I have added tests corresponding to this change - [ ] I have updated the documentation to reflect this change - [x] I have verified that this change is backward compatible (If not, please discuss on the [APISIX mailing list](https://github.com/apache/apisix/tree/master#community) first) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
