[
https://issues.apache.org/jira/browse/YARN-9927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16956764#comment-16956764
]
Adam Antal commented on YARN-9927:
----------------------------------
Thanks for filing this [~hcarrot], interesting approach.
One question that came to my mind is that: are you certain the the dispatcher
is the real bottleneck here?
I mean if an event processing requires holding the lock the whole time, then we
just replace the time in the dispatcher queue with lock-holding time for each
event. We should dig down that for a certain event type how long the lock
should be hold.
> RM multi-thread event processing mechanism
> ------------------------------------------
>
> Key: YARN-9927
> URL: https://issues.apache.org/jira/browse/YARN-9927
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: yarn
> Affects Versions: 3.0.0, 2.9.2
> Reporter: hcarrot
> Priority: Minor
> Attachments: RM multi-thread event processing mechanism.pdf
>
>
> Recently, we have observed serious event blocking in RM event dispatcher
> queue. After analysis of RM event monitoring data and RM event processing
> logic, we found that the proportion of RMNodeStatusEvent is less than other
> events, but the overall processing time of it is more than other events.
> Meanwhile, RM event processing is in a single-thread mode, and It results in
> the decrease of RM's performance. So we proposed a RM multi-thread event
> processing mechanism to improve RM performance.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
