Wang, Xinglong created YARN-9854:
------------------------------------
Summary: RM jetty hang due to WebAppProxyServlet lacks of timeout
while doing proxyLink
Key: YARN-9854
URL: https://issues.apache.org/jira/browse/YARN-9854
Project: Hadoop YARN
Issue Type: Improvement
Components: amrmproxy, resourcemanager, webapp
Reporter: Wang, Xinglong
Assignee: Wang, Xinglong
RM will proxy url request to [http://rm:port/proxy/application_xxxxx] to AM or
related history server.
Recently we met an issue https://issues.apache.org/jira/browse/SPARK-26961
which will cause Spark AM hang forever.
And we have a monitor tool to access [http://rm:port/proxy/application_xxxxx]
periodically. Thus all proxied connection to the hang spark AM will also hang
forever due to WebAppProxyServlet is lacking of socket connection timeout
setting while initialize httpclient towards this spark AM.
The jetty server holding RM servlets is with limited threads. In this case,
each time one such thread will hang due to waiting for Spark AM response.
Eventually all jetty threads serving http traffic hang and caused all RM web
links not responsive.
If we give timeout config to httpclient, we will be free of this issue.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
