[
https://issues.apache.org/jira/browse/YARN-4512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15091283#comment-15091283
]
Karthik Kambatla commented on YARN-4512:
----------------------------------------
[~elgoiri] - thanks for the quick review.
bq. Some places (e.g., YarnConfiguration) still use Oversubscription. I prefer
that term but according to the discussion it should be Overallocation everywhere
Should have updated the JIRA. As discussed on YARN-1011, there is a tension
between letting the OPPORTUNISTIC containers run vs preempting them to allow
running on a different node. To address this tension, I feel it would be useful
to have a preemption threshold. Given we have two thresholds, it seemed
reasonable to call them allocation and preemption.
{{overallocation.allocation}} seemed somewhat redundant. I am fine with either
name. And, we can always change the config before merging to trunk.
bq. Should we differentiate physical and virtual memory thresholds?
Since we are not necessarily oversubscribing virtual memory, I don't see the
need for virtual memory thresholds. Any particular reason this would help?
bq. What would be the use of OverAllocationInfoProto?
This is purely to allow more sophisticated policies on oversubscription instead
of just basic utilization thresholds. For instance, [~leftnoteasy] proposed
using history over the last several time periods.
bq. Should the RM know about the preemptionThreshold
The preemption happens on the NM. Theoretically, the RM might want to keep
track of this and aggressively make some decisions. No concrete ideas at the
moment, so left it out. Any suggestions on how we could use preemption
information on RM?
Will address rest of the comments in the next iteration.
> Provide a knob to turn on over-allocation
> -----------------------------------------
>
> Key: YARN-4512
> URL: https://issues.apache.org/jira/browse/YARN-4512
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: nodemanager
> Reporter: Karthik Kambatla
> Assignee: Karthik Kambatla
> Attachments: YARN-4512-YARN-1011.001.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
