sohurdc opened a new pull request, #8697:
URL: https://github.com/apache/hadoop/pull/8697
# YARN-11845. WebAppProxy: add `Connection: close` header to prevent
CLOSE_WAIT leaks
## Description of PR
This PR fixes an accumulation of sockets stuck in `CLOSE_WAIT` state on the
ResourceManager host that runs the WebAppProxy.
`WebAppProxyServlet.proxyLink()` creates a brand-new `HttpClient` for every
proxied request (`HttpClientBuilder.create()` → `build()`), sends the
request,
and only calls `base.releaseConnection()` afterward. It never closes the
`HttpClient` itself.
Apache HttpClient enables connection pooling / keep-alive by default, so the
backend (Application Master or History Server) keeps the connection open
after
returning the response, waiting to reuse it. But since the proxy throws away
the `HttpClient` right after each request, that pooled connection can never
be
reused. The corresponding socket on the proxy side therefore lingers in
`CLOSE_WAIT` until the `HttpClient` is finally garbage collected.
The fix is to explicitly ask the backend to close the connection once the
response has been sent, by setting a `Connection: close` request header on
the
outgoing request:
```java
base.setHeader("Connection", "close");
```
Because `Connection` is not in `PASS_THROUGH_HEADERS`, a client-supplied
`Connection: keep-alive` header is never forwarded, so the proxy always sends
`close` and the backend closes the connection as soon as the response is
written.
## How was this patch tested?
- Added `testWebAppProxyConnectionCloseHeader()` in
`TestWebAppProxyServlet`, which verifies that the proxied backend receives
a
`Connection: close` header in three cases:
1. a plain `GET` request,
2. a `GET` request where the client explicitly sends `Connection:
keep-alive`
(asserting the proxy overrides it with `close`),
3. a `PUT` request (asserting the header is also set on the PUT path).
- The existing `testWebAppProxyPassThroughHeaders` assertion (9 headers
received by the backend) is unaffected: the `Connection` header was already
among the counted headers, only its value changes from `Keep-Alive` to
`close`.
## For code changes
- [ ] The title of this PR accurately describes the issue.
- [ ] The code follows the project's style guidelines.
- [ ] New tests are added, and existing tests still pass.
- [ ] No user-facing API/behavior change is introduced (the proxy continues
to
return the same responses; it only instructs the backend to close the
underlying connection).
## Additional notes
- `Connection: close` does not change the response returned to the client; it
only affects the lifetime of the internal proxied connection, so it has no
user-visible impact.
- This is a one-line, low-risk change scoped to the web-proxy module.
## Commit message
```text
YARN-11845. WebAppProxy: add Connection: close header to prevent CLOSE_WAIT
socket buildup. Contributed by weishao <[email protected]>.
WebAppProxyServlet.proxyLink() creates a new HttpClient per request and never
closes it. With keep-alive enabled by default, the backend keeps the proxied
connection open and the proxy-side socket lingers in CLOSE_WAIT until GC.
Explicitly setting Connection: close makes the backend close the connection
after each response.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
