[ https://issues.apache.org/jira/browse/YARN-8149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16436488#comment-16436488 ]
Wangda Tan commented on YARN-8149: ---------------------------------- [~tgraves], Preemption for large reserved container is already handled by existing code path, it won't guarantee all reserved container can be satisfied, but it can alleviate the problem a log: https://issues.apache.org/jira/browse/YARN-4390. I agree that we cannot remove this method in anytime soon (sadly), let's think more about how to better do reservation + preemption. I added moveReservedContainer (Swap) to CS part of YARN-5864. It is possible that we can consume that method to do better reservation. > Revisit behavior of Re-Reservation in Capacity Scheduler > -------------------------------------------------------- > > Key: YARN-8149 > URL: https://issues.apache.org/jira/browse/YARN-8149 > Project: Hadoop YARN > Issue Type: Bug > Reporter: Wangda Tan > Priority: Critical > > Frankly speaking, I'm not sure why we need the re-reservation. The formula is > not that easy to understand: > Inside: > {{org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.allocator.RegularContainerAllocator#shouldAllocOrReserveNewContainer}} > {code:java} > starvation = re-reservation / (#reserved-container * > (1 - min(requested-resource / max-alloc, > max-alloc - min-alloc / max-alloc)) > should_allocate = starvation + requiredContainers - reservedContainers > > 0{code} > I think we should be able to remove the starvation computation, just to check > requiredContainers > reservedContainers should be enough. > In a large cluster, we can easily overflow re-reservation to MAX_INT, see > YARN-7636. > -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org