[
https://issues.apache.org/jira/browse/YARN-5293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15356832#comment-15356832
]
Sunil G commented on YARN-5293:
-------------------------------
Thanks [~jhung] for a detailed idea about the configurations and its priorities.
I am thinking about the advantages of disabling preemption for
per-label-per-queue level. Since labels can give a physical categorization of
nodes based on certain types (upto the admin to decide), we can think that one
specific queue capacity can be splitted/viewed as few certain types of nods. So
my *highGPU* label nodes for the queue can be disabled for preemption when
compared with my *lowGPU*, thus keeping only some low priority resources for
preemption. This can be viewed from both angle. I am saving some of my critical
resources from getting preempted but still gives a certain percentage of
resource which are of low priority to preempt. But there is a fair level
balance needed, and it may be tough to maintain. Few points
- *labelA* for queueA is now disabled for preemption and consider this queue is
over-utilizing resources from *labelA*. *labelA* is configured in queueB also
which is under utilized. So ideally *queueA-labelA* has to be preempted to
satisfy queueB's need. This will be blocked now
- If only one label is configured per-queue, its same queue level disabling
- With label exclusivity disabled, same label can be used for NO_LABEL cases
too for a queue. So if this label is disabled from preemption, indirectly
NO_LABEL resources also will be saved. Such cases are hard to explain or
propagate to user/admin level for clarity.
I think more discussions are good here. cc/[~leftnoteasy], [~Naganarasimha
Garla], [~rohithsharma]
> Disable preemption per queue per partition
> ------------------------------------------
>
> Key: YARN-5293
> URL: https://issues.apache.org/jira/browse/YARN-5293
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: capacityscheduler
> Reporter: Jonathan Hung
>
> [YARN-2056|https://issues.apache.org/jira/browse/YARN-2056] added support for
> disabling preemption at queue level.
> [YARN-2498|https://issues.apache.org/jira/browse/YARN-2498] added support for
> per partition preemption. It would be useful to be able to disable preemption
> per partition per queue, so that if a queue Q has access to two labels A and
> B, we can disable preemption for Q in A but not in B.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
