[
https://issues.apache.org/jira/browse/YARN-9940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16969831#comment-16969831
]
kailiu_dev edited comment on YARN-9940 at 11/8/19 6:15 AM:
-----------------------------------------------------------
[~wilfreds] , you say right that hadoop 2.7.2 has been
{color:#172b4d}EOL'ed{color}, but can you please help me review my my patch ,
check if can the code can {color:#172b4d}solving the issure, beause my company
need hadoop 2.7.2。{color}
*{color:#172b4d}in my code:{color}*
for example , befor my change in hadoop 2.7.2:
*sort node is:*
synchronized (this)
{ Collections.sort(nodeIdList, nodeAvailableResourceComparator); }
*but {color:#172b4d}completedContainer{color} is:*
writeLock.lock();
.....
node.releaseContainer(container);
writeLock.unlock();
*after change my code:*
{color:#172b4d}completedContainer{color} is:
writeLock.lock();
{color:#172b4d}if (continuousSchedulingEnabled) { {color}
{color:#172b4d} synchronized (this) { {color}
{color:#172b4d}node.releaseContainer(container); {color}
{color:#172b4d} }
}{color}
{color:#172b4d}writeLock.unlock();{color}
{color:#ff0000}so when releaseContainer , it should wait for geting schdule
lock when sorting node;{color}
{color:#ff0000}this is my simple example, the total code is in my
patch.{color}
{color:#172b4d}so above on you say "the method
{color}{{FairScheduler.completedContainer()}}{color:#172b4d} method is already
synchronised adding a synchronised block inside that will not help{color}", I
can think that your meaning is that it is not helpful in hadoop2.9 beause of
it has all used {color:#172b4d}writeLock/readLock,{color} but it is
helpful in hadoop2.7.2 ?
was (Author: kailiu_dev):
[~wilfreds] , you say right that hadoop 2.7.2 has been
{color:#172b4d}EOL'ed{color}, but can you please help me review my my patch ,
check if can the code can {color:#172b4d}solving the issure, beause my company
need hadoop 2.7.2。{color}
*{color:#172b4d}in my code:{color}*
for example , befor my change in hadoop 2.7.2:
*sort node is:*
synchronized (this)
{ Collections.sort(nodeIdList, nodeAvailableResourceComparator); }
*but {color:#172b4d}completedContainer{color} is:*
writeLock.lock();
.....
node.releaseContainer(container);
writeLock.unlock();
*after change my code:*
{color:#172b4d}completedContainer{color} is:
writeLock.lock();
{color:#172b4d}if (continuousSchedulingEnabled) { {color}
{color:#172b4d} synchronized (this) { {color}
{color:#172b4d}node.releaseContainer(container); {color}
{color:#172b4d} }
}{color}
{color:#172b4d}writeLock.unlock();{color}
{color:#ff0000}so when releaseContainer , it should wait for geting schdule
lock when sorting node;{color}
{color:#ff0000}this is my simple example, the total code is in my
patch.{color}
{color:#172b4d}so above on you say "the method
{color}{{FairScheduler.completedContainer()}}{color:#172b4d} method is already
synchronised adding a synchronised block inside that will not help{color}", I
can think that your meaning is that it is not helpful in hadoop2.9 beause of
it has all used {color:#172b4d}writeLock/readLock,{color} but it is
helpful in hadoop2.7.2 ?
> avoid continuous scheduling thread crashes while sorting nodes get
> 'Comparison method violates its general contract'
> --------------------------------------------------------------------------------------------------------------------
>
> Key: YARN-9940
> URL: https://issues.apache.org/jira/browse/YARN-9940
> Project: Hadoop YARN
> Issue Type: Bug
> Components: fairscheduler
> Affects Versions: 2.7.2
> Reporter: kailiu_dev
> Assignee: kailiu_dev
> Priority: Major
> Attachments: YARN-9940-branch-2.7.2.001.patch
>
>
> 2019-10-16 09:14:51,215 ERROR
> org.apache.hadoop.yarn.YarnUncaughtExceptionHandler: Thread
> Thread[FairSchedulerContinuousScheduling,5,main] threw an Exception.
> java.lang.IllegalArgumentException: Comparison method violates its general
> contract!
> at java.util.TimSort.mergeHi(TimSort.java:868)
> at java.util.TimSort.mergeAt(TimSort.java:485)
> at java.util.TimSort.mergeForceCollapse(TimSort.java:426)
> at java.util.TimSort.sort(TimSort.java:223)
> at java.util.TimSort.sort(TimSort.java:173)
> at java.util.Arrays.sort(Arrays.java:659)
> at java.util.Collections.sort(Collections.java:217)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.continuousSchedulingAttempt(FairScheduler.java:1117)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler$ContinuousSchedulingThread.run(FairScheduler.java:296)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
