[
https://issues.apache.org/jira/browse/YARN-9940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16969781#comment-16969781
]
kailiu_dev edited comment on YARN-9940 at 11/8/19 2:57 AM:
-----------------------------------------------------------
Dear， [~wilfreds] , {color:#172b4d}thanks for your replay!{color}
in YARN-8373{color:#172b4d}.{color}
{color:#000000}- synchronized (this) {
+ readLock.lock();
+ try \{ nodeIdList =
nodeTracker.sortedNodeList(nodeAvailableResourceComparator); + } finally \{
+ readLock.unlock(); }{color}
h3. *I think your meaning is that synchronized (this) is not same lock with the
writeLock/readLock, so while sort the nodes may get exception
'{color:#ff0000}Comparison method violates its general contract{color}' when
some nodes'Available resources change , so we shoule have one lock ,our may
same but my version is hadoop 2.7.2 and your hadoop version is above than 2.9
, the codes in hadoop 2.7.2 doesn't have nodeTracker, so my solution looks
is not same with YARN-8373{color:#172b4d}.{color}.*
was (Author: kailiu_dev):
Dear， [~wilfreds] , {color:#172b4d}thanks for your replay!{color}
in YARN-8373{color:#172b4d}.{color}
{color:#000000}- synchronized (this) {
+ readLock.lock();
+ try {
nodeIdList = nodeTracker.sortedNodeList(nodeAvailableResourceComparator);
+ } finally {
+ readLock.unlock();
}{color}
h3. *I think your meaning is that synchronized (this) is not same lock with
whith the writeLock/readLock, so while sort the nodes may get exception
'{color:#FF0000}Comparison method violates its general contract{color}' when
some nodes'Available resources change , so we shoule have one lock ,our may
same but my version is hadoop 2.7.2, the codes in hadoop 2.7.2 doesn't have
nodeTracker, so my solution looks is not same with
YARN-8373{color:#172b4d}.{color}.*
> avoid continuous scheduling thread crashes while sorting nodes get
> 'Comparison method violates its general contract'
> --------------------------------------------------------------------------------------------------------------------
>
> Key: YARN-9940
> URL: https://issues.apache.org/jira/browse/YARN-9940
> Project: Hadoop YARN
> Issue Type: Bug
> Components: fairscheduler
> Affects Versions: 2.7.2
> Reporter: kailiu_dev
> Assignee: kailiu_dev
> Priority: Major
> Attachments: YARN-9940-branch-2.7.2.001.patch
>
>
> 2019-10-16 09:14:51,215 ERROR
> org.apache.hadoop.yarn.YarnUncaughtExceptionHandler: Thread
> Thread[FairSchedulerContinuousScheduling,5,main] threw an Exception.
> java.lang.IllegalArgumentException: Comparison method violates its general
> contract!
> at java.util.TimSort.mergeHi(TimSort.java:868)
> at java.util.TimSort.mergeAt(TimSort.java:485)
> at java.util.TimSort.mergeForceCollapse(TimSort.java:426)
> at java.util.TimSort.sort(TimSort.java:223)
> at java.util.TimSort.sort(TimSort.java:173)
> at java.util.Arrays.sort(Arrays.java:659)
> at java.util.Collections.sort(Collections.java:217)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.continuousSchedulingAttempt(FairScheduler.java:1117)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler$ContinuousSchedulingThread.run(FairScheduler.java:296)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
