Wangda Tan updated YARN-4108:
    Attachment: YARN-4108-design-doc-v2.pdf

Had an offline discussion with [~curino]/[~chris.douglas]. Updated design doc 
(v2), highlight of changes:
- Added an example for people to better understand how lazy preemption works
- Added more explanations about architecture design / implementation details
- Added two items for discussion
1) Is it possible to combine batch preemption and surgical preemption.
2) Is it possible to kill preempted container processes at NM only when new 
container is ready to launch.

Thanks [~curino]/[~chris.douglas] for very insightful thoughts and suggestions!

> CapacityScheduler: Improve preemption to preempt only those containers that 
> would satisfy the incoming request
> --------------------------------------------------------------------------------------------------------------
>                 Key: YARN-4108
>                 URL: https://issues.apache.org/jira/browse/YARN-4108
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: capacity scheduler
>            Reporter: Wangda Tan
>            Assignee: Wangda Tan
>         Attachments: YARN-4108-design-doc-v1.pdf, 
> YARN-4108-design-doc-v2.pdf, YARN-4108.poc.1.patch
> This is sibling JIRA for YARN-2154. We should make sure container preemption 
> is more effective.
> *Requirements:*:
> 1) Can handle case of user-limit preemption
> 2) Can handle case of resource placement requirements, such as: hard-locality 
> (I only want to use rack-1) / node-constraints (YARN-3409) / black-list (I 
> don't want to use rack1 and host\[1-3\])
> 3) Can handle preemption within a queue: cross user preemption (YARN-2113), 
> cross applicaiton preemption (such as priority-based (YARN-1963) / 
> fairness-based (YARN-3319)).

This message was sent by Atlassian JIRA

Reply via email to