[
https://issues.apache.org/jira/browse/HADOOP-14033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15860444#comment-15860444
]
Hudson commented on HADOOP-14033:
---------------------------------
SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #11229 (See
[https://builds.apache.org/job/Hadoop-trunk-Commit/11229/])
HADOOP-14033. Reduce fair call queue lock contention. Contributed by (kihwal:
rev 0c01cf57987bcc7a17154a3538960b67f625a9e5)
* (edit)
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/FairCallQueue.java
> Reduce fair call queue lock contention
> --------------------------------------
>
> Key: HADOOP-14033
> URL: https://issues.apache.org/jira/browse/HADOOP-14033
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: ipc
> Affects Versions: 2.7.0
> Reporter: Daryn Sharp
> Assignee: Daryn Sharp
> Fix For: 2.8.1, 3.0.0-alpha3
>
> Attachments: HADOOP-14033.patch
>
>
> Under heavy load the call queue may run dry yet clients experience high
> latency.
> The fcq requires producers and consumers to sync via a shared lock. Polling
> consumers hold the lock while scanning all sub-queues. Consumers are
> serialized despite the sub-queues being thread-safe blocking queues. The
> effect is to cause other producers/consumers to frequently park.
> The lock is unfair, so producers/consumers attempt to barge in on the lock.
> The outnumbered producers tend to remain blocked for an extended time. As
> load increases and the queues fill, the barging consumers drain the queues
> faster than the producers can fill it.
> Server metrics provide an illusion of healthy throughput, response time, and
> call queue length due to starvation on the ingress. Often as the load gets
> worse, the server looks better.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
