[
https://issues.apache.org/jira/browse/YARN-9164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16733640#comment-16733640
]
Weiwei Yang commented on YARN-9164:
-----------------------------------
{color:#333333}I have committed the patch to trunk, cherry-picked to
branch-3.0, branch-3.1 and branch-3.2, thanks for the contribution
[~xiaoheipangzi].{color}
> Shutdown NM may cause NPE when opportunistic container scheduling is enabled
> ----------------------------------------------------------------------------
>
> Key: YARN-9164
> URL: https://issues.apache.org/jira/browse/YARN-9164
> Project: Hadoop YARN
> Issue Type: Bug
> Reporter: lujie
> Assignee: lujie
> Priority: Critical
> Fix For: 3.0.4, 3.1.2, 3.3.0, 3.2.1
>
> Attachments: YARN-9164-0.patch, YARN-9164-1.patch, YARN-9164-2.patch,
> hadoop-hires-resourcemanager-hadoop11.log
>
>
> We have meeted an NPE which can crash the whole cluster
> {code:java}
> 2018-12-31 22:18:11,924 FATAL org.apache.hadoop.yarn.event.EventDispatcher:
> Error in handling event type APP_ATTEMPT_REMOVED to the Event Dispatcher
> java.lang.NullPointerException
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.AbstractYarnScheduler.completedContainer(AbstractYarnScheduler.java:696)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.doneApplicationAttempt(CapacityScheduler.java:1123)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.handle(CapacityScheduler.java:1827)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.handle(CapacityScheduler.java:171)
> at
> org.apache.hadoop.yarn.event.EventDispatcher$EventProcessor.run(EventDispatcher.java:66)
> at java.lang.Thread.run(Thread.java:745)
> {code}
> this bug also happens in the latest trunk!
>
> workload is
> {code:java}
> $HADOOP_HOME/bin/hadoop jar
> $HADOOP_HOME/share/hadoop/mapreduce/hadoop-mapreduce-examples-$VERSION.jar pi
> -Dmapreduce.job.num-opportunistic-maps-percent="100" 50 100
> {code}
> while job is running , shutdown one NM
> also need inject sleep before AbstractYarnScheduler.getNode()
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
