[
https://issues.apache.org/jira/browse/YARN-4945?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15454965#comment-15454965
]
Sunil G commented on YARN-4945:
-------------------------------
Thanks [~leftnoteasy]
bq.What is intraQueuePreemptionCost? And what is AppPriorityComparator?
Yes, there are not used. I will do a clean up in next patch to remove
unnecessary code snippets.
bq.For the queue-level ideal allocation is it enough to trust result from
previous policies?
If we are planning for normalization (I want to introduce in next patch after
doing some more tests with SLS and real time), then we need to add up or
subtract resources which are preempted from previous run. And this is needed to
done per queue. Hence I kept the code. I can look into this and see how much
more i can reuse as possible.
bq.if (tq.intraQueuePreemptionCalculationDone == true), is this always false?
intraQueuePreemptionCalculationDone will be set as false during init. We
iterate over partitions, and there are chances that we may have multiple labels
in a queue. Since I am iterating over partitions and then look into queues
where this partition has some capacity, we may have same queue across
partition. I was calculating resource demand like partition -> queue level. To
avoid duplicate calculation, i use this variable.
bq.Should we lock the queue inside for (LeafQueue leafQUeue : queues)?
Make sense for me. I will handle this case.
bq.return value of getResourceDemandFromAppsPerQueue is not consumed by anyone.
I have cleaned up the code.
bq.TempAppPerQueue is not used by anyone
I am moving some common logic computation and some storage to here. I will
have this in next patch.
Thanks for sharing the flow. I also planned to do similar approach what you
suggested, but I am doing some tasks together. I feel I can rearrange the code
to fit to this code flow. However I feel
deductPreemptableResourcesBasedSelectedCandidates could be done while we loop
over each containers of an app also (we can deduct from resourceToObtain, but
do not add same container again. I handled this scenario already).
> [Umbrella] Capacity Scheduler Preemption Within a queue
> -------------------------------------------------------
>
> Key: YARN-4945
> URL: https://issues.apache.org/jira/browse/YARN-4945
> Project: Hadoop YARN
> Issue Type: Bug
> Reporter: Wangda Tan
> Attachments: Intra-Queue Preemption Use Cases.pdf,
> IntraQueuepreemption-CapacityScheduler (Design).pdf, YARN-2009-wip.2.patch,
> YARN-2009-wip.patch
>
>
> This is umbrella ticket to track efforts of preemption within a queue to
> support features like:
> YARN-2009. YARN-2113. YARN-4781.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
