[
https://issues.apache.org/jira/browse/YARN-11845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jack Yang updated YARN-11845:
-----------------------------
Description:
In certain scenarios, we make API calls such as
[http://]{rm_address}/proxy/\{appid}/ws/v1/mapreduce/jobs to monitor the state
of running applications. During this process, we noticed a substantial number
of TCP connections accumulating in the CLOSE_WAIT state.
The key reason we think is that the connection is not closed in proxyLink() of
WebAppProxyServlet. The request link is: client -> rm -> appMaster, the rm
would proxy the request to appMaster, when the request done, rm keeps the
connection open but the appMaster side close the connection (FIN sent).
{code:java}
try {
HttpResponse httpResp = client.execute(base);
resp.setStatus(httpResp.getStatusLine().getStatusCode());
for (Header header : httpResp.getAllHeaders()) {
resp.setHeader(header.getName(), header.getValue());
}
if (c != null) {
resp.addCookie(c);
}
InputStream in = httpResp.getEntity().getContent();
if (in != null) {
IOUtils.copyBytes(in, out, 4096, true);
}
} finally {
// the connection is not closed properly
base.releaseConnection();
}{code}
We think there is no need to keep the connection open with upstream in this
scenarios. So maybe add Connection:close header is good choice.
{code:java}
base.setHeader("Connection", "close");{code}
was:
In certain scenarios, we make API calls such as
http://\{rm_address}/proxy/\{appid}/ws/v1/mapreduce/jobs to monitor the state
of running applications. During this process, we noticed a substantial number
of TCP connections accumulating in the CLOSE_WAIT state.
!image-2025-08-07-16-16-52-193.png!
The key reason we think is that the connection is not closed in proxyLink() of
WebAppProxyServlet. The request link is: client -> rm -> appMaster, the rm
would proxy the request to appMaster, when the request done, rm keeps the
connection open but the appMaster side close the connection (FIN sent).
{code:java}
try {
HttpResponse httpResp = client.execute(base);
resp.setStatus(httpResp.getStatusLine().getStatusCode());
for (Header header : httpResp.getAllHeaders()) {
resp.setHeader(header.getName(), header.getValue());
}
if (c != null) {
resp.addCookie(c);
}
InputStream in = httpResp.getEntity().getContent();
if (in != null) {
IOUtils.copyBytes(in, out, 4096, true);
}
} finally {
// the connection is not closed properly
base.releaseConnection();
}{code}
We think there is no need to keep the connection open with upstream in this
scenarios. So maybe add Connection:close header is good choice.
{code:java}
base.setHeader("Connection", "close");{code}
> CLOSE_WAIT leaks in the WebAppProxyServlet
> ------------------------------------------
>
> Key: YARN-11845
> URL: https://issues.apache.org/jira/browse/YARN-11845
> Project: Hadoop YARN
> Issue Type: Bug
> Components: webproxy
> Affects Versions: 3.3.6
> Reporter: Jack Yang
> Priority: Major
> Attachments: image-2025-08-07-16-16-52-193.png
>
>
> In certain scenarios, we make API calls such as
> [http://]{rm_address}/proxy/\{appid}/ws/v1/mapreduce/jobs to monitor the
> state of running applications. During this process, we noticed a substantial
> number of TCP connections accumulating in the CLOSE_WAIT state.
> The key reason we think is that the connection is not closed in proxyLink()
> of WebAppProxyServlet. The request link is: client -> rm -> appMaster, the rm
> would proxy the request to appMaster, when the request done, rm keeps the
> connection open but the appMaster side close the connection (FIN sent).
> {code:java}
> try {
> HttpResponse httpResp = client.execute(base);
> resp.setStatus(httpResp.getStatusLine().getStatusCode());
> for (Header header : httpResp.getAllHeaders()) {
> resp.setHeader(header.getName(), header.getValue());
> }
> if (c != null) {
> resp.addCookie(c);
> }
> InputStream in = httpResp.getEntity().getContent();
> if (in != null) {
> IOUtils.copyBytes(in, out, 4096, true);
> }
> } finally {
> // the connection is not closed properly
> base.releaseConnection();
> }{code}
> We think there is no need to keep the connection open with upstream in this
> scenarios. So maybe add Connection:close header is good choice.
> {code:java}
> base.setHeader("Connection", "close");{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
