[
https://issues.apache.org/jira/browse/HADOOP-10599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13996401#comment-13996401
]
Daryn Sharp commented on HADOOP-10599:
--------------------------------------
With the current global fair lock, I wonder if DNs could effectively starve
clients. The block updates may create a glut of write ops at the front of the
call queue. With no prioritization, clients create read bubbles in between the
write ops.
I think you've touched upon the angle we are using to attack the problem. Now
once I can finally get away from webhdfs hardening which derailed the efforts,
we're working on fine grain FSN locking + distinct BM locking + BM has its own
RPC service. This should effectively achieve prioritization of DNs with much
better performance characteristics overall.
> Support prioritization of DN RPCs over client RPCs
> --------------------------------------------------
>
> Key: HADOOP-10599
> URL: https://issues.apache.org/jira/browse/HADOOP-10599
> Project: Hadoop Common
> Issue Type: Sub-task
> Reporter: Ming Ma
>
> We might need to prioritize DN RPC over client RPC so that no matter what
> application do to NN RPC and FSNamesystem's global lock, DN's requests will
> be processed timely. After a cluster is configured to have service RPC server
> separated from client RPC server, it is mitigated to some degree with fair
> FSNamesystem's global lock. Also if the NN global lock can be made more fine
> grained; such need becomes less important. Still, it will be good to evaluate
> if this is a good option.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
