[
https://issues.apache.org/jira/browse/YARN-9927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16962987#comment-16962987
]
hcarrot edited comment on YARN-9927 at 10/30/19 12:39 PM:
----------------------------------------------------------
thank you [~epayne] [~adam.antal] for comment and concern.
The code has been finished. I think this mechanism can handle RMNodeStatusEvent
from different nodes concurrently. Because the lock object of different RMNode
state machine is different.
was (Author: hcarrot):
The code has been finished. I think this mechanism can handle RMNodeStatusEvent
from different nodes concurrently. Because the lock object of different RMNode
state machine is different.
[~adam.antal] [~epayne]
> RM multi-thread event processing mechanism
> ------------------------------------------
>
> Key: YARN-9927
> URL: https://issues.apache.org/jira/browse/YARN-9927
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: yarn
> Affects Versions: 3.0.0, 2.9.2
> Reporter: hcarrot
> Priority: Major
> Attachments: RM multi-thread event processing mechanism.pdf,
> YARN-9927-addMultiEventDispatcher.patch
>
>
> Recently, we have observed serious event blocking in RM event dispatcher
> queue. After analysis of RM event monitoring data and RM event processing
> logic, we found that
> 1) environment: a cluster with thousands of nodes
> 2) RMNodeStatusEvent dominates 90% time consumption of RM event scheduler
> 3) Meanwhile, RM event processing is in a single-thread mode, and It results
> in the low headroom of RM event scheduler, thus performance of RM.
> So we proposed a RM multi-thread event processing mechanism to improve RM
> performance.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
