[
https://issues.apache.org/jira/browse/YARN-4945?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15478599#comment-15478599
]
Sunil G commented on YARN-4945:
-------------------------------
Thanks [~leftnoteasy] for the offline discussion. Updating the summary here.
1. All the comments other then IntraQueuePreemptableResourceCalculator &
IntraQueueCandidatesSelector: will be addressed in next patch.
IntraQueuePreemptableResourceCalculator & IntraQueueCandidatesSelector:
2.
bq.This is not correct for future policies, for example, fairness policy can
have a minimum resource allocated to each application, existing logic will
preempt all containers from the application with maximum fair sharing.
Current patch addresses most of the priority related handling from calculator.
This is need to refined and go with generalized preemption flow approach as
given in pseudo code.
3.
{{app.ideal = max(Q.unallocated, app.used + app.pending - app.selected)}}
Here it will be *min* instead of max.
4.psuedo code
- We will find each queue's unallocated share which is calculated as {{Q.used -
Q.selected;}} here Q.selected means selected containers from this queue by
earlier policies.
- Then will calculate app's ideal share iteratively starting from highest
priority app. Here Q.ideal will help to calculate app.ideal.
- {{intra_q_preemptable}} calculation is straight forward form the code.
- Last loop will try to find preemptable resource per app. Its DOES NOT select
candidates here.
{{app.preemptable = min(max(app.used - app.selected - app.ideal, 0),
intra_q_preemptable)}} will help to achieve the same.
> [Umbrella] Capacity Scheduler Preemption Within a queue
> -------------------------------------------------------
>
> Key: YARN-4945
> URL: https://issues.apache.org/jira/browse/YARN-4945
> Project: Hadoop YARN
> Issue Type: Bug
> Reporter: Wangda Tan
> Attachments: Intra-Queue Preemption Use Cases.pdf,
> IntraQueuepreemption-CapacityScheduler (Design).pdf, YARN-2009-wip.2.patch,
> YARN-2009-wip.patch, YARN-2009-wip.v3.patch, YARN-2009.v0.patch
>
>
> This is umbrella ticket to track efforts of preemption within a queue to
> support features like:
> YARN-2009. YARN-2113. YARN-4781.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
