[
https://issues.apache.org/jira/browse/HADOOP-14033?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Akira Ajisaka updated HADOOP-14033:
-----------------------------------
Fix Version/s: 2.9.0
> Reduce fair call queue lock contention
> --------------------------------------
>
> Key: HADOOP-14033
> URL: https://issues.apache.org/jira/browse/HADOOP-14033
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: ipc
> Affects Versions: 2.7.0
> Reporter: Daryn Sharp
> Assignee: Daryn Sharp
> Fix For: 2.9.0, 2.8.1, 3.0.0-alpha4
>
> Attachments: HADOOP-14033.patch
>
>
> Under heavy load the call queue may run dry yet clients experience high
> latency.
> The fcq requires producers and consumers to sync via a shared lock. Polling
> consumers hold the lock while scanning all sub-queues. Consumers are
> serialized despite the sub-queues being thread-safe blocking queues. The
> effect is to cause other producers/consumers to frequently park.
> The lock is unfair, so producers/consumers attempt to barge in on the lock.
> The outnumbered producers tend to remain blocked for an extended time. As
> load increases and the queues fill, the barging consumers drain the queues
> faster than the producers can fill it.
> Server metrics provide an illusion of healthy throughput, response time, and
> call queue length due to starvation on the ingress. Often as the load gets
> worse, the server looks better.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
