[jira] [Commented] (YARN-10427) Duplicate Job IDs in SLS output
[
https://issues.apache.org/jira/browse/YARN-10427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17460347#comment-17460347
]
Gergely Pollák commented on YARN-10427:
---
[~snemeth] thank you for the patch I have only one minor observation:
The simulateFinishTimeMS is initialized as 0, and 0 value is supposed to mean
that simulateFinishTimeMS have not been calculated yet, however it's not
necessarily true, since simulateFinishTimeMS is calculated as the
currentTimestamp - baseline, it might be 0 (very unlikely, but theoretically
possible).
So I think -1 or Long.MIN_VALUE would be a better indicator for the "empty"
timestamp. Also we could introduce a boolean for the same purpose and then we
wouldn't have to rely on magic numbers, but it's acceptable here. (Also please
make a constant for the value, just to make clear what does this number mean)
> Duplicate Job IDs in SLS output
> ---
>
> Key: YARN-10427
> URL: https://issues.apache.org/jira/browse/YARN-10427
> Project: Hadoop YARN
> Issue Type: Bug
> Components: scheduler-load-simulator
>Affects Versions: 3.0.0, 3.3.0, 3.2.1, 3.4.0
> Environment: I ran the attached inputs on my MacBook Pro, using
> Hadoop compiled from the latest trunk (as of commit 139a43e98e). I also
> tested against 3.2.1 and 3.3.0 release branches.
>
>Reporter: Drew Merrill
>Assignee: Szilard Nemeth
>Priority: Major
> Attachments: YARN-10427-sls-scriptsandlogs.tar.gz,
> YARN-10427.001.patch, YARN-10427.002.patch, YARN-10427.003.patch,
> fair-scheduler.xml, inputsls.json, jobruntime.csv, jobruntime.csv,
> mapred-site.xml, sls-runner.xml, yarn-site.xml
>
>
> Hello, I'm hoping someone can help me resolve or understand some issues I've
> been having with the YARN Scheduler Load Simulator (SLS). I've been
> experimenting with SLS for several months now at work as we're trying to
> build a simulation model to characterize our enterprise Hadoop infrastructure
> for purposes of future capacity planning. In the process of attempting to
> verify and validate the SLS output, I've encountered a number of issues
> including runtime exceptions and bad output. The focus of this issue is the
> bad output. In all my simulation runs, the jobruntime.csv output seems to
> have one or more of the following problems: no output, duplicate job ids,
> and/or missing job ids.
>
> Because of where I work, I'm unable to provide the exact inputs I typically
> use, but I'm able to reproduce the problem of the duplicate Job IDS using
> some simplified inputs and configuration files, which I've attached, along
> with the output I obtained.
>
> The command I used to run the simulation:
> {{./runsls.sh --tracetype=SLS --tracelocation=./inputsls.json
> --output-dir=sls-run-1 --print-simulation
> --track-jobs=job_1,job_2,job_3,job_4,job_5,job_6,job_7,job_8,job_9,job_10}}
>
> Can anyone help me understand what would cause the duplicate Job IDs in the
> output? Is this a bug in Hadoop or a problem with my inputs? Thanks in
> advance.
>
> PS: This is my first issue I've ever opened so please be kind if I've missed
> something or am not understanding something obvious about the way Hadoop
> works. I'll gladly follow-up with more info as requested.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Resolved] (YARN-11030) ClassNotFoundException when aux service class is loaded from customized classpath
[
https://issues.apache.org/jira/browse/YARN-11030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hiroyuki Adachi resolved YARN-11030.
Resolution: Duplicate
Close as duplicate of YARN-9967.
> ClassNotFoundException when aux service class is loaded from customized
> classpath
> -
>
> Key: YARN-11030
> URL: https://issues.apache.org/jira/browse/YARN-11030
> Project: Hadoop YARN
> Issue Type: Bug
>Affects Versions: 3.3.0, 3.3.1
>Reporter: Hiroyuki Adachi
>Priority: Minor
>
> NodeManager failed to load the aux service with ClassNotFoundException while
> loading the class from the customized classpath.
> {noformat}
>
>
> value="org.apache.spark.network.yarn.YarnShuffleService"/>
> value="/tmp/spark-3.1.2-yarn-shuffle.jar"/>
>
> {noformat}
> {noformat}
> 2021-12-06 15:32:09,168 INFO org.apache.hadoop.util.ApplicationClassLoader:
> classpath: [file:/tmp/spark-3.1.2-yarn-shuffle.jar]
> 2021-12-06 15:32:09,168 INFO org.apache.hadoop.util.ApplicationClassLoader:
> system classes: [org.apache.spark.network.yarn.YarnShuffleService]
> 2021-12-06 15:32:09,169 INFO org.apache.hadoop.service.AbstractService:
> Service
> org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices failed
> in
> state INITED
> org.apache.hadoop.yarn.exceptions.YarnRuntimeException:
> java.lang.ClassNotFoundException:
> org.apache.spark.network.yarn.YarnShuffleService
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices.initAuxService(AuxServices.java:482)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices.serviceInit(AuxServices.java:761)
> at
> org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
> at
> org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:109)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl.serviceInit(ContainerManagerImpl.java:327)
> at
> org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
> at
> org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:109)
> at
> org.apache.hadoop.yarn.server.nodemanager.NodeManager.serviceInit(NodeManager.java:494)
> at
> org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
> at
> org.apache.hadoop.yarn.server.nodemanager.NodeManager.initAndStartNodeManager(NodeManager.java:962)
> at
> org.apache.hadoop.yarn.server.nodemanager.NodeManager.main(NodeManager.java:1042)
> Caused by: java.lang.ClassNotFoundException:
> org.apache.spark.network.yarn.YarnShuffleService
> at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
> at
> org.apache.hadoop.util.ApplicationClassLoader.loadClass(ApplicationClassLoader.java:189)
> at
> org.apache.hadoop.util.ApplicationClassLoader.loadClass(ApplicationClassLoader.java:157)
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:348)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxiliaryServiceWithCustomClassLoader.getInstance(AuxiliaryServiceWithCustomClassLoader.ja
> va:165)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices.createAuxServiceFromLocalClasspath(AuxServices.java:242)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices.createAuxService(AuxServices.java:271)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices.initAuxService(AuxServices.java:452)
> ... 10 more
> 2021-12-06 15:32:09,172 INFO org.apache.hadoop.service.AbstractService:
> Service
> org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl
>
> failed in state INITED{noformat}
>
> YARN-9075 may cause this problem. The default system classes were changed by
> this patch.
> Before YARN-9075: isSystemClass() returns false since the system classes does
> not contain the aux service class itself, and the class will be loaded from
> the customized classpath.
> [https://github.com/apache/hadoop/blob/rel/release-3.3.1/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/ApplicationClassLoader.java#L176]
> {noformat}
> 2021-12-06 15:50:21,332 INFO org.apache.hadoop.util.ApplicationClassLoader:
> classpath: [file:/tmp/spark-3.1.2-yarn-shuffle.jar]
> 2021-12-06 15:50:21,332 INFO org.apache.hadoop.util.ApplicationClassLoader:
> system classes: [java.,
[jira] [Updated] (YARN-11050) Typo in method name: RMWebServiceProtocol#removeFromCluserNodeLabels
[ https://issues.apache.org/jira/browse/YARN-11050?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated YARN-11050: -- Labels: newbie pull-request-available (was: newbie) > Typo in method name: RMWebServiceProtocol#removeFromCluserNodeLabels > > > Key: YARN-11050 > URL: https://issues.apache.org/jira/browse/YARN-11050 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Szilard Nemeth >Assignee: Szilard Nemeth >Priority: Trivial > Labels: newbie, pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-11050) Typo in method name: RMWebServiceProtocol#removeFromCluserNodeLabels
[ https://issues.apache.org/jira/browse/YARN-11050?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Szilard Nemeth reassigned YARN-11050: - Assignee: Szilard Nemeth > Typo in method name: RMWebServiceProtocol#removeFromCluserNodeLabels > > > Key: YARN-11050 > URL: https://issues.apache.org/jira/browse/YARN-11050 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Szilard Nemeth >Assignee: Szilard Nemeth >Priority: Trivial > -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-11050) Typo in method name: RMWebServiceProtocol#removeFromCluserNodeLabels
[ https://issues.apache.org/jira/browse/YARN-11050?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Szilard Nemeth updated YARN-11050: -- Labels: newbie (was: ) > Typo in method name: RMWebServiceProtocol#removeFromCluserNodeLabels > > > Key: YARN-11050 > URL: https://issues.apache.org/jira/browse/YARN-11050 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Szilard Nemeth >Assignee: Szilard Nemeth >Priority: Trivial > Labels: newbie > -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-11050) Typo in method name: RMWebServiceProtocol#removeFromCluserNodeLabels
Szilard Nemeth created YARN-11050: - Summary: Typo in method name: RMWebServiceProtocol#removeFromCluserNodeLabels Key: YARN-11050 URL: https://issues.apache.org/jira/browse/YARN-11050 Project: Hadoop YARN Issue Type: Improvement Reporter: Szilard Nemeth -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-11049) MutableConfScheduler is referred as plain String instead of class name
[ https://issues.apache.org/jira/browse/YARN-11049?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated YARN-11049: -- Labels: pull-request-available (was: ) > MutableConfScheduler is referred as plain String instead of class name > -- > > Key: YARN-11049 > URL: https://issues.apache.org/jira/browse/YARN-11049 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Szilard Nemeth >Assignee: Szilard Nemeth >Priority: Minor > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > In RMWebServices and TestRMWebServices, "MutableConfScheduler" is referred as > plain String instead of using Class.getSimpleName() -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-11049) MutableConfScheduler is referred as plain String instead of class name
Szilard Nemeth created YARN-11049: - Summary: MutableConfScheduler is referred as plain String instead of class name Key: YARN-11049 URL: https://issues.apache.org/jira/browse/YARN-11049 Project: Hadoop YARN Issue Type: Improvement Reporter: Szilard Nemeth Assignee: Szilard Nemeth In RMWebServices and TestRMWebServices, "MutableConfScheduler" is referred as plain String instead of using Class.getSimpleName() -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-11044) TestApplicationLimits.testLimitsComputation() has some ineffective asserts
[
https://issues.apache.org/jira/browse/YARN-11044?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Szilard Nemeth updated YARN-11044:
--
Summary: TestApplicationLimits.testLimitsComputation() has some ineffective
asserts (was: TestApplicationLimits.testLimitsComputation() has some
uneffective asserts)
> TestApplicationLimits.testLimitsComputation() has some ineffective asserts
> --
>
> Key: YARN-11044
> URL: https://issues.apache.org/jira/browse/YARN-11044
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Benjamin Teke
>Assignee: Benjamin Teke
>Priority: Major
> Labels: newbie, pull-request-available
> Time Spent: 40m
> Remaining Estimate: 0h
>
> TestApplicationLimits.testLimitsComputation() has the following two asserts:
> {code:java}
> // should default to global setting if per queue setting not set
> assertEquals((long)CapacitySchedulerConfiguration.DEFAULT_MAXIMUM_APPLICATIONMASTERS_RESOURCE_PERCENT,
> (long)csConf.getMaximumApplicationMasterResourcePerQueuePercent(
> queue.getQueuePath()));
> {code}
> and
> {code:java}
> assertEquals((long) 0.5,
> (long) csConf.getMaximumApplicationMasterResourcePerQueuePercent(
> queue.getQueuePath()));
> {code}
> In the current form neither of them make too much sense because
> getMaximumApplicationMasterResourcePerQueuePercent returns a float (between 0
> and 1.0), so the only way this will fail is when the configuration is below 0
> or above 1, but we're not testing invalid configurations here. This should be
> corrected.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-11044) TestApplicationLimits.testLimitsComputation() has some ineffective asserts
[
https://issues.apache.org/jira/browse/YARN-11044?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Szilard Nemeth updated YARN-11044:
--
Fix Version/s: 3.4.0
> TestApplicationLimits.testLimitsComputation() has some ineffective asserts
> --
>
> Key: YARN-11044
> URL: https://issues.apache.org/jira/browse/YARN-11044
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Benjamin Teke
>Assignee: Benjamin Teke
>Priority: Major
> Labels: newbie, pull-request-available
> Fix For: 3.4.0
>
> Time Spent: 40m
> Remaining Estimate: 0h
>
> TestApplicationLimits.testLimitsComputation() has the following two asserts:
> {code:java}
> // should default to global setting if per queue setting not set
> assertEquals((long)CapacitySchedulerConfiguration.DEFAULT_MAXIMUM_APPLICATIONMASTERS_RESOURCE_PERCENT,
> (long)csConf.getMaximumApplicationMasterResourcePerQueuePercent(
> queue.getQueuePath()));
> {code}
> and
> {code:java}
> assertEquals((long) 0.5,
> (long) csConf.getMaximumApplicationMasterResourcePerQueuePercent(
> queue.getQueuePath()));
> {code}
> In the current form neither of them make too much sense because
> getMaximumApplicationMasterResourcePerQueuePercent returns a float (between 0
> and 1.0), so the only way this will fail is when the configuration is below 0
> or above 1, but we're not testing invalid configurations here. This should be
> corrected.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10178) Global Scheduler async thread crash caused by 'Comparison method violates its general contract'
[
https://issues.apache.org/jira/browse/YARN-10178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17460280#comment-17460280
]
Hadoop QA commented on YARN-10178:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Logfile || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
39s{color} | {color:blue}{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} || ||
| {color:green}+1{color} | {color:green} dupname {color} | {color:green} 0m
0s{color} | {color:green}{color} | {color:green} No case conflicting files
found. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green}{color} | {color:green} The patch does not contain any
@author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red}{color} | {color:red} The patch doesn't appear to
include any new or modified tests. Please justify why no new tests are needed
for this patch. Also please list what manual steps were performed to verify
this patch. {color} |
|| || || || {color:brown} trunk Compile Tests {color} || ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 33m
23s{color} | {color:green}{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
4s{color} | {color:green}{color} | {color:green} trunk passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
58s{color} | {color:green}{color} | {color:green} trunk passed with JDK Private
Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
46s{color} | {color:green}{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
11s{color} | {color:green}{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
16m 42s{color} | {color:green}{color} | {color:green} branch has no errors when
building and testing our client artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
54s{color} | {color:green}{color} | {color:green} trunk passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
44s{color} | {color:green}{color} | {color:green} trunk passed with JDK Private
Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 {color} |
| {color:blue}0{color} | {color:blue} spotbugs {color} | {color:blue} 20m
20s{color} | {color:blue}{color} | {color:blue} Both FindBugs and SpotBugs are
enabled, using SpotBugs. {color} |
| {color:green}+1{color} | {color:green} spotbugs {color} | {color:green} 2m
0s{color} | {color:green}{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} || ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
55s{color} | {color:green}{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
0s{color} | {color:green}{color} | {color:green} the patch passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
0s{color} | {color:green}{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
49s{color} | {color:green}{color} | {color:green} the patch passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
49s{color} | {color:green}{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
38s{color} | {color:green}{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
53s{color} | {color:green}{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green}{color} | {color:green} The patch has no whitespace
issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
14m 6s{color} | {color:green}{color} | {color:green} patch has no errors when
building and testing our client artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
45s{color} | {color:green}{color} | {color:green} the patch passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 {color} |
|
[jira] [Commented] (YARN-10178) Global Scheduler async thread crash caused by 'Comparison method violates its general contract'
[
https://issues.apache.org/jira/browse/YARN-10178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17460183#comment-17460183
]
Andras Gyori commented on YARN-10178:
-
[~epayne] Uploaded a patch with the simplified logic of snapshot based sorting,
excluding the old logic and the unnecessary parts. As no objection was made in
terms of the Stream API, the latest patch is building on it.
> Global Scheduler async thread crash caused by 'Comparison method violates its
> general contract'
> ---
>
> Key: YARN-10178
> URL: https://issues.apache.org/jira/browse/YARN-10178
> Project: Hadoop YARN
> Issue Type: Bug
> Components: capacity scheduler
>Affects Versions: 3.2.1
>Reporter: tuyu
>Assignee: Qi Zhu
>Priority: Major
> Attachments: YARN-10178.001.patch, YARN-10178.002.patch,
> YARN-10178.003.patch, YARN-10178.004.patch, YARN-10178.005.patch,
> YARN-10178.006.patch
>
>
> Global Scheduler Async Thread crash stack
> {code:java}
> ERROR org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: Received
> RMFatalEvent of type CRITICAL_THREAD_CRASH, caused by a critical thread,
> Thread-6066574, that exited unexpectedly: java.lang.IllegalArgumentException:
> Comparison method violates its general contract!
>at
> java.util.TimSort.mergeHi(TimSort.java:899)
> at java.util.TimSort.mergeAt(TimSort.java:516)
> at java.util.TimSort.mergeForceCollapse(TimSort.java:457)
> at java.util.TimSort.sort(TimSort.java:254)
> at java.util.Arrays.sort(Arrays.java:1512)
> at java.util.ArrayList.sort(ArrayList.java:1462)
> at java.util.Collections.sort(Collections.java:177)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.policy.PriorityUtilizationQueueOrderingPolicy.getAssignmentIterator(PriorityUtilizationQueueOrderingPolicy.java:221)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.ParentQueue.sortAndGetChildrenAllocationIterator(ParentQueue.java:777)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.ParentQueue.assignContainersToChildQueues(ParentQueue.java:791)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.ParentQueue.assignContainers(ParentQueue.java:623)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.allocateOrReserveNewContainers(CapacityScheduler.java:1635)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.allocateContainerOnSingleNode(CapacityScheduler.java:1629)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.allocateContainersToNode(CapacityScheduler.java:1732)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.allocateContainersToNode(CapacityScheduler.java:1481)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.schedule(CapacityScheduler.java:569)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler$AsyncScheduleThread.run(CapacityScheduler.java:616)
> {code}
> JAVA 8 Arrays.sort default use timsort algo, and timsort has few require
> {code:java}
> 1.x.compareTo(y) != y.compareTo(x)
> 2.x>y,y>z --> x > z
> 3.x=y, x.compareTo(z) == y.compareTo(z)
> {code}
> if not Arrays paramters not satify this require,TimSort will throw
> 'java.lang.IllegalArgumentException'
> look at PriorityUtilizationQueueOrderingPolicy.compare function,we will know
> Capacity Scheduler use this these queue resource usage to compare
> {code:java}
> AbsoluteUsedCapacity
> UsedCapacity
> ConfiguredMinResource
> AbsoluteCapacity
> {code}
> In Capacity Scheduler Global Scheduler AsyncThread use
> PriorityUtilizationQueueOrderingPolicy function to choose queue to assign
> container,and construct a CSAssignment struct, and use
> submitResourceCommitRequest function add CSAssignment to backlogs
> ResourceCommitterService will tryCommit this CSAssignment,look tryCommit
> function,there will update queue resource usage
> {code:java}
> public boolean tryCommit(Resource cluster, ResourceCommitRequest r,
> boolean updatePending) {
> long commitStart = System.nanoTime();
> ResourceCommitRequest request =
> (ResourceCommitRequest) r;
>
> ...
> boolean isSuccess = false;
> if (attemptId != null) {
> FiCaSchedulerApp app = getApplicationAttempt(attemptId);
> // Required sanity check for attemptId - when async-scheduling enabled,
> // proposal might be outdated if AM failover just finished
> // and proposal
[jira] [Updated] (YARN-10178) Global Scheduler async thread crash caused by 'Comparison method violates its general contract'
[
https://issues.apache.org/jira/browse/YARN-10178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andras Gyori updated YARN-10178:
Attachment: YARN-10178.006.patch
> Global Scheduler async thread crash caused by 'Comparison method violates its
> general contract'
> ---
>
> Key: YARN-10178
> URL: https://issues.apache.org/jira/browse/YARN-10178
> Project: Hadoop YARN
> Issue Type: Bug
> Components: capacity scheduler
>Affects Versions: 3.2.1
>Reporter: tuyu
>Assignee: Qi Zhu
>Priority: Major
> Attachments: YARN-10178.001.patch, YARN-10178.002.patch,
> YARN-10178.003.patch, YARN-10178.004.patch, YARN-10178.005.patch,
> YARN-10178.006.patch
>
>
> Global Scheduler Async Thread crash stack
> {code:java}
> ERROR org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: Received
> RMFatalEvent of type CRITICAL_THREAD_CRASH, caused by a critical thread,
> Thread-6066574, that exited unexpectedly: java.lang.IllegalArgumentException:
> Comparison method violates its general contract!
>at
> java.util.TimSort.mergeHi(TimSort.java:899)
> at java.util.TimSort.mergeAt(TimSort.java:516)
> at java.util.TimSort.mergeForceCollapse(TimSort.java:457)
> at java.util.TimSort.sort(TimSort.java:254)
> at java.util.Arrays.sort(Arrays.java:1512)
> at java.util.ArrayList.sort(ArrayList.java:1462)
> at java.util.Collections.sort(Collections.java:177)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.policy.PriorityUtilizationQueueOrderingPolicy.getAssignmentIterator(PriorityUtilizationQueueOrderingPolicy.java:221)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.ParentQueue.sortAndGetChildrenAllocationIterator(ParentQueue.java:777)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.ParentQueue.assignContainersToChildQueues(ParentQueue.java:791)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.ParentQueue.assignContainers(ParentQueue.java:623)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.allocateOrReserveNewContainers(CapacityScheduler.java:1635)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.allocateContainerOnSingleNode(CapacityScheduler.java:1629)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.allocateContainersToNode(CapacityScheduler.java:1732)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.allocateContainersToNode(CapacityScheduler.java:1481)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.schedule(CapacityScheduler.java:569)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler$AsyncScheduleThread.run(CapacityScheduler.java:616)
> {code}
> JAVA 8 Arrays.sort default use timsort algo, and timsort has few require
> {code:java}
> 1.x.compareTo(y) != y.compareTo(x)
> 2.x>y,y>z --> x > z
> 3.x=y, x.compareTo(z) == y.compareTo(z)
> {code}
> if not Arrays paramters not satify this require,TimSort will throw
> 'java.lang.IllegalArgumentException'
> look at PriorityUtilizationQueueOrderingPolicy.compare function,we will know
> Capacity Scheduler use this these queue resource usage to compare
> {code:java}
> AbsoluteUsedCapacity
> UsedCapacity
> ConfiguredMinResource
> AbsoluteCapacity
> {code}
> In Capacity Scheduler Global Scheduler AsyncThread use
> PriorityUtilizationQueueOrderingPolicy function to choose queue to assign
> container,and construct a CSAssignment struct, and use
> submitResourceCommitRequest function add CSAssignment to backlogs
> ResourceCommitterService will tryCommit this CSAssignment,look tryCommit
> function,there will update queue resource usage
> {code:java}
> public boolean tryCommit(Resource cluster, ResourceCommitRequest r,
> boolean updatePending) {
> long commitStart = System.nanoTime();
> ResourceCommitRequest request =
> (ResourceCommitRequest) r;
>
> ...
> boolean isSuccess = false;
> if (attemptId != null) {
> FiCaSchedulerApp app = getApplicationAttempt(attemptId);
> // Required sanity check for attemptId - when async-scheduling enabled,
> // proposal might be outdated if AM failover just finished
> // and proposal queue was not be consumed in time
> if (app != null && attemptId.equals(app.getApplicationAttemptId())) {
> if (app.accept(cluster, request, updatePending)
> && app.apply(cluster, request, updatePending)) { //
[jira] [Commented] (YARN-10178) Global Scheduler async thread crash caused by 'Comparison method violates its general contract'
[
https://issues.apache.org/jira/browse/YARN-10178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17460145#comment-17460145
]
Eric Payne commented on YARN-10178:
---
[~gandras], if you do decide to pick this up, please post a new patch with the
following changes:
- Remove the unit test {{TestPriorityUtilizationQueueOrderingPolicy.java}}
- Remove the changes supporting the unit test in
{{PriorityUtilizationQueueOrderingPolicy.java}}:
{code:java}
+ // Just for test performance side effect/regression
+ public Iterator getOldAssignmentIterator(String partition) {
+// Since partitionToLookAt is a thread local variable, and every time we
+// copy and sort queues, so it's safe for multi-threading environment.
+PriorityUtilizationQueueOrderingPolicy.partitionToLookAt.set(partition);
List sortedQueue = new ArrayList<>(queues);
-Collections.sort(sortedQueue, new PriorityQueueComparator());
+Collections.sort(sortedQueue, new PriorityQueueComparatorOld());
return sortedQueue.iterator();
}
+ // Just for test performance side effect/regression
+ private class PriorityQueueComparatorOld implements Comparator {
+
+@Override
+public int compare(CSQueue q1, CSQueue q2) {
+ String p = partitionToLookAt.get();
... (etc)
{code}
> Global Scheduler async thread crash caused by 'Comparison method violates its
> general contract'
> ---
>
> Key: YARN-10178
> URL: https://issues.apache.org/jira/browse/YARN-10178
> Project: Hadoop YARN
> Issue Type: Bug
> Components: capacity scheduler
>Affects Versions: 3.2.1
>Reporter: tuyu
>Assignee: Qi Zhu
>Priority: Major
> Attachments: YARN-10178.001.patch, YARN-10178.002.patch,
> YARN-10178.003.patch, YARN-10178.004.patch, YARN-10178.005.patch
>
>
> Global Scheduler Async Thread crash stack
> {code:java}
> ERROR org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: Received
> RMFatalEvent of type CRITICAL_THREAD_CRASH, caused by a critical thread,
> Thread-6066574, that exited unexpectedly: java.lang.IllegalArgumentException:
> Comparison method violates its general contract!
>at
> java.util.TimSort.mergeHi(TimSort.java:899)
> at java.util.TimSort.mergeAt(TimSort.java:516)
> at java.util.TimSort.mergeForceCollapse(TimSort.java:457)
> at java.util.TimSort.sort(TimSort.java:254)
> at java.util.Arrays.sort(Arrays.java:1512)
> at java.util.ArrayList.sort(ArrayList.java:1462)
> at java.util.Collections.sort(Collections.java:177)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.policy.PriorityUtilizationQueueOrderingPolicy.getAssignmentIterator(PriorityUtilizationQueueOrderingPolicy.java:221)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.ParentQueue.sortAndGetChildrenAllocationIterator(ParentQueue.java:777)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.ParentQueue.assignContainersToChildQueues(ParentQueue.java:791)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.ParentQueue.assignContainers(ParentQueue.java:623)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.allocateOrReserveNewContainers(CapacityScheduler.java:1635)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.allocateContainerOnSingleNode(CapacityScheduler.java:1629)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.allocateContainersToNode(CapacityScheduler.java:1732)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.allocateContainersToNode(CapacityScheduler.java:1481)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.schedule(CapacityScheduler.java:569)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler$AsyncScheduleThread.run(CapacityScheduler.java:616)
> {code}
> JAVA 8 Arrays.sort default use timsort algo, and timsort has few require
> {code:java}
> 1.x.compareTo(y) != y.compareTo(x)
> 2.x>y,y>z --> x > z
> 3.x=y, x.compareTo(z) == y.compareTo(z)
> {code}
> if not Arrays paramters not satify this require,TimSort will throw
> 'java.lang.IllegalArgumentException'
> look at PriorityUtilizationQueueOrderingPolicy.compare function,we will know
> Capacity Scheduler use this these queue resource usage to compare
> {code:java}
> AbsoluteUsedCapacity
> UsedCapacity
> ConfiguredMinResource
> AbsoluteCapacity
> {code}
> In Capacity Scheduler Global Scheduler AsyncThread
[jira] [Updated] (YARN-11047) ResourceManager and NodeManager unable to connect to Hbase when ATSv2 is enabled
[ https://issues.apache.org/jira/browse/YARN-11047?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Viraj Jasani updated YARN-11047: Target Version/s: 3.4.0 > ResourceManager and NodeManager unable to connect to Hbase when ATSv2 is > enabled > > > Key: YARN-11047 > URL: https://issues.apache.org/jira/browse/YARN-11047 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Minni Mittal >Assignee: Viraj Jasani >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > yarn resourcemanager command has following issue: > 2021-12-14 19:26:33,345 WARN [pool-28-thread-1] > storage.TimelineStorageMonitor (TimelineStorageMonitor.java:run(95)) - Got > failure attempting to read from HBase, assuming Storage is down > java.lang.RuntimeException: org.apache.hadoop.hbase.DoNotRetryIOException: > java.lang.NoSuchMethodError: > [com.google.common.net|http://com.google.common.net/].HostAndPort.getHostText()Ljava/lang/String; > at > org.apache.hadoop.hbase.client.AbstractClientScanner$1.hasNext(AbstractClientScanner.java:95) > at > org.apache.hadoop.yarn.server.timelineservice.storage.reader.TimelineEntityReader.readEntities(TimelineEntityReader.java:283) > at > org.apache.hadoop.yarn.server.timelineservice.storage.HBaseStorageMonitor.healthCheck(HBaseStorageMonitor.java:77) > at > org.apache.hadoop.yarn.server.timelineservice.storage.TimelineStorageMonitor$MonitorThread.run(TimelineStorageMonitor.java:89) > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at java.lang.Thread.run(Thread.java:748) -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-11047) ResourceManager and NodeManager unable to connect to Hbase when ATSv2 is enabled
[ https://issues.apache.org/jira/browse/YARN-11047?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated YARN-11047: -- Labels: pull-request-available (was: ) > ResourceManager and NodeManager unable to connect to Hbase when ATSv2 is > enabled > > > Key: YARN-11047 > URL: https://issues.apache.org/jira/browse/YARN-11047 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Minni Mittal >Assignee: Viraj Jasani >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > yarn resourcemanager command has following issue: > 2021-12-14 19:26:33,345 WARN [pool-28-thread-1] > storage.TimelineStorageMonitor (TimelineStorageMonitor.java:run(95)) - Got > failure attempting to read from HBase, assuming Storage is down > java.lang.RuntimeException: org.apache.hadoop.hbase.DoNotRetryIOException: > java.lang.NoSuchMethodError: > [com.google.common.net|http://com.google.common.net/].HostAndPort.getHostText()Ljava/lang/String; > at > org.apache.hadoop.hbase.client.AbstractClientScanner$1.hasNext(AbstractClientScanner.java:95) > at > org.apache.hadoop.yarn.server.timelineservice.storage.reader.TimelineEntityReader.readEntities(TimelineEntityReader.java:283) > at > org.apache.hadoop.yarn.server.timelineservice.storage.HBaseStorageMonitor.healthCheck(HBaseStorageMonitor.java:77) > at > org.apache.hadoop.yarn.server.timelineservice.storage.TimelineStorageMonitor$MonitorThread.run(TimelineStorageMonitor.java:89) > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at java.lang.Thread.run(Thread.java:748) -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-11047) ResourceManager and NodeManager unable to connect to Hbase when ATSv2 is enabled
[ https://issues.apache.org/jira/browse/YARN-11047?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Viraj Jasani updated YARN-11047: Summary: ResourceManager and NodeManager unable to connect to Hbase when ATSv2 is enabled (was: yarn resourcemanager and nodemanager unable to connect with Hbase when ATSv2 is enabled) > ResourceManager and NodeManager unable to connect to Hbase when ATSv2 is > enabled > > > Key: YARN-11047 > URL: https://issues.apache.org/jira/browse/YARN-11047 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Minni Mittal >Assignee: Viraj Jasani >Priority: Major > > yarn resourcemanager command has following issue: > 2021-12-14 19:26:33,345 WARN [pool-28-thread-1] > storage.TimelineStorageMonitor (TimelineStorageMonitor.java:run(95)) - Got > failure attempting to read from HBase, assuming Storage is down > java.lang.RuntimeException: org.apache.hadoop.hbase.DoNotRetryIOException: > java.lang.NoSuchMethodError: > [com.google.common.net|http://com.google.common.net/].HostAndPort.getHostText()Ljava/lang/String; > at > org.apache.hadoop.hbase.client.AbstractClientScanner$1.hasNext(AbstractClientScanner.java:95) > at > org.apache.hadoop.yarn.server.timelineservice.storage.reader.TimelineEntityReader.readEntities(TimelineEntityReader.java:283) > at > org.apache.hadoop.yarn.server.timelineservice.storage.HBaseStorageMonitor.healthCheck(HBaseStorageMonitor.java:77) > at > org.apache.hadoop.yarn.server.timelineservice.storage.TimelineStorageMonitor$MonitorThread.run(TimelineStorageMonitor.java:89) > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at java.lang.Thread.run(Thread.java:748) -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
