[
https://issues.apache.org/jira/browse/HDFS-10789?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15478210#comment-15478210
]
Daryn Sharp commented on HDFS-10789:
------------------------------------
The current situation is all the jetty handler threads end up competing
unfairly with the ipc handlers. The unregulated competition easily causes
callq overflow and rpc calls are rejected. The jetty acceptor is still busy
accepting an unlimited number of connections even while all the jetty handlers
are blocked. A full GC is often inevitable when hit with an abusive surge.
With this patch, the callq can provide fairness. Also assuming there's at
least one jetty handler not blocked in the namesystem, new requests will be
rejected and their connections will be closed.
> Route webhdfs through the RPC call queue
> ----------------------------------------
>
> Key: HDFS-10789
> URL: https://issues.apache.org/jira/browse/HDFS-10789
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: ipc, webhdfs
> Reporter: Daryn Sharp
> Assignee: Daryn Sharp
> Attachments: HDFS-10789.patch
>
>
> Webhdfs is extremely expensive under load and is not subject to the QoS
> benefits of the RPC call queue. HADOOP-13537 provides the basis for routing
> webhdfs through the call queue to provide unified QoS.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
