Prashant Golash created YARN-9916:
-------------------------------------
Summary: Improving Async Dispatcher
Key: YARN-9916
URL: https://issues.apache.org/jira/browse/YARN-9916
Project: Hadoop YARN
Issue Type: Improvement
Reporter: Prashant Golash
Currently, async dispatcher works in the single-threaded model.
There is another queue for the scheduler handler, but not all handlers are
non-blocking. In our cluster, this queue can go sometimes to 16M events, which
takes time to drain.
We should think of improving it:
# Either make multi-threads in the dispatcher which will pick queue events,
but this would require careful evaluation of the order of events.
# Or Make all downstream handlers similar to scheduler queue (this also needs
careful evaluation of out of order events).
Any other ideas are also welcome.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
