BELUGA BEHR created YARN-8789:
---------------------------------
Summary: Add BoundedQueue to AsyncDispatcher
Key: YARN-8789
URL: https://issues.apache.org/jira/browse/YARN-8789
Project: Hadoop YARN
Issue Type: Improvement
Components: applications
Affects Versions: 3.2.0
Reporter: BELUGA BEHR
I recently came across a scenario where an MR ApplicationMaster was failing
with an OOM exception. It had many thousands of Mappers and thousands of
Reducers. It was noted that in the logging that the event-queue of
{{AsyncDispatcher}} had a very large number of item in it and was seemingly
never decreasing.
I started looking at the code and thought it could use some clean up,
simplification, and the ability to specify a bounded queue so that any incoming
events are throttled until they can be processed. This will protect the
ApplicationMaster from a flood of events.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
