Wangda Tan commented on YARN-3508:

I tent to support [~jlowe] and [~jianhe]'s suggestion, make preemption events 
directly goes to scheduler event queue.

I think we cannot assume preemption events have higher priority than other 
events, in most cases, preemption events are just notify AM about something 
will happen. And manage two queues for scheduler can be complex, how to balance 
them, etc. To reduce complex, I suggest to only maintain one queue for 
scheduler until we have to.

> Preemption processing occuring on the main RM dispatcher
> --------------------------------------------------------
>                 Key: YARN-3508
>                 URL: https://issues.apache.org/jira/browse/YARN-3508
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: resourcemanager, scheduler
>    Affects Versions: 2.6.0
>            Reporter: Jason Lowe
>            Assignee: Varun Saxena
>         Attachments: YARN-3508.002.patch, YARN-3508.01.patch
> We recently saw the RM for a large cluster lag far behind on the 
> AsyncDispacher event queue.  The AsyncDispatcher thread was consistently 
> blocked on the highly-contended CapacityScheduler lock trying to dispatch 
> preemption-related events for RMContainerPreemptEventDispatcher.  Preemption 
> processing should occur on the scheduler event dispatcher thread or a 
> separate thread to avoid delaying the processing of other events in the 
> primary dispatcher queue.

This message was sent by Atlassian JIRA

Reply via email to