[
https://issues.apache.org/jira/browse/YARN-8178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16448997#comment-16448997
]
Zian Chen commented on YARN-8178:
---------------------------------
Hi [~asuresh] , sure that would be great if I can be helpful on the code review
for your solution similar in YARN-6826. When we come up with the design for
over-commitment and aggressive preemption with the shorter timeout, we found
Opportunistic container is really suitable for the use case. Even better, we
could use the Opportunistic container in many more preemption scenarios as a
general way to increase the overall cluster resource utilization. We can
discuss more on how to use Opportunistic Container to make our solution more
reliable without bothering too much on Guaranteed container allocation. Thank
you for your help!
> [Umbrella] Resource Over-commitment Based on Opportunistic Container
> Preemption
> -------------------------------------------------------------------------------
>
> Key: YARN-8178
> URL: https://issues.apache.org/jira/browse/YARN-8178
> Project: Hadoop YARN
> Issue Type: New Feature
> Components: capacity scheduler
> Reporter: Zian Chen
> Priority: Major
>
> We want to provide an opportunistic container-based solution to achieve more
> aggressive preemption with shorter preemption monitoring interval.
> Instead of allowing applications to allocate resources with a mix of
> guaranteed and opportunistic containers, we allow newly submitted
> applications to only contain guaranteed containers. Meanwhile, we change the
> preemption logic to, instead of killing containers, demote guaranteed
> containers into opportunistic ones, so that when there are new applications
> submitted, we can ensure that these containers can be launched by preempting
> opportunistic containers.
> This approach is related to YARN-1011 but achieves over-commitment in a
> different way. However, we rely on opportunistic container part implemented
> in YARN-1011 to make our design work well.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
