[jira] [Commented] (YARN-10009) In Capacity Scheduler, DRC can treat minimum user limit percent as a max when custom resource is defined
[
https://issues.apache.org/jira/browse/YARN-10009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17001721#comment-17001721
]
Eric Payne commented on YARN-10009:
---
Thanks a lot for the help, [~ebadger] and [~leftnoteasy]!
> In Capacity Scheduler, DRC can treat minimum user limit percent as a max when
> custom resource is defined
>
>
> Key: YARN-10009
> URL: https://issues.apache.org/jira/browse/YARN-10009
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: capacity scheduler
>Affects Versions: 2.10.0, 3.3.0, 3.2.1, 3.1.3, 2.10.1
>Reporter: Eric Payne
>Assignee: Eric Payne
>Priority: Critical
> Fix For: 3.3.0, 3.2.2, 3.1.4, 2.10.1
>
> Attachments: YARN-10009.001.patch, YARN-10009.002.patch,
> YARN-10009.003.patch, YARN-10009.004.patch, YARN-10009.UT.patch,
> YARN-10009.branch-2.10.003.patch
>
>
> | |Memory|Vcores|res_1|
> |Queue1 Totals|20GB|100|80|
> |Resources requested by App1 in Queue1|8GB (40% of total)|8 (8% of total)|80
> (100% of total)|
> In the previous use case:
> - Queue1 has a value of 25 for {{miminum-user-limit-percent}}
> - User1 has requested 8 containers with {{}}
> each
> - {{res_1}} will be the dominant resource this case.
> All 8 containers should be assigned by the capacity scheduler, but with min
> user limit pct set to 25, only 2 containers are assigned.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10051) Throwing NoSuchElementException when even dispatcher handling NODE_UPDATE
[
https://issues.apache.org/jira/browse/YARN-10051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17001638#comment-17001638
]
Peter Bacsko commented on YARN-10051:
-
Hi [~yxing] can you explain why you think the root cause is different? To me it
looks the same as YARN-9552. Line numbers do not properly align because the
code has been changed since Hadoop 3.0, but even that difference is minimal.
Note that Hadoop 3.x branch has been declared obsolote, so fixes won't go there
anymore.
> Throwing NoSuchElementException when even dispatcher handling NODE_UPDATE
> -
>
> Key: YARN-10051
> URL: https://issues.apache.org/jira/browse/YARN-10051
> Project: Hadoop YARN
> Issue Type: Bug
> Components: resourcemanager
>Affects Versions: 3.0.0
>Reporter: Yong Xing
>Priority: Critical
> Fix For: 3.0.0
>
>
> Restarting a NM, I found the active RM crash. The Exception stack is as
> follows.
> {code:java}
> 2019-12-16 18:12:20,286 FATAL org.apache.hadoop.yarn.event.EventDispatcher:
> Error in handling event type NODE_UPDATE to the Event Dispatcher
> java.util.NoSuchElementException
> at
> java.util.concurrent.ConcurrentSkipListMap.firstKey(ConcurrentSkipListMap.java:2036)
> at
> java.util.concurrent.ConcurrentSkipListSet.first(ConcurrentSkipListSet.java:396)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.AppSchedulingInfo.getNextPendingAsk(AppSchedulingInfo.java:373)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSAppAttempt.isOverAMShareLimit(FSAppAttempt.java:941)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSAppAttempt.assignContainer(FSAppAttempt.java:1374)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSLeafQueue.assignContainer(FSLeafQueue.java:345)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSParentQueue.assignContainer(FSParentQueue.java:204)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.attemptScheduling(FairScheduler.java:1091)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.nodeUpdate(FairScheduler.java:958)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.handle(FairScheduler.java:1180)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.handle(FairScheduler.java:130)
> at
> org.apache.hadoop.yarn.event.EventDispatcher$EventProcessor.run(EventDispatcher.java:66)
> at java.lang.Thread.run(Thread.java:748)
> {code}
>
> This issue looks a bit same as
> [YARN-9552|https://issues.apache.org/jira/browse/YARN-9552]，[YARN-7382|https://issues.apache.org/jira/browse/YARN-7382].
> But the root cause is different.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
