[
https://issues.apache.org/jira/browse/YARN-2816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14213650#comment-14213650
]
Hudson commented on YARN-2816:
------------------------------
FAILURE: Integrated in Hadoop-Mapreduce-trunk #1958 (See
[https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1958/])
YARN-2816. NM fail to start with NPE during container recovery. Contributed by
Zhihai Xu (jlowe: rev 49c38898b0be64fc686d039ed2fb2dea1378df02)
* hadoop-yarn-project/CHANGES.txt
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/recovery/TestNMLeveldbStateStoreService.java
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/recovery/NMLeveldbStateStoreService.java
> NM fail to start with NPE during container recovery
> ---------------------------------------------------
>
> Key: YARN-2816
> URL: https://issues.apache.org/jira/browse/YARN-2816
> Project: Hadoop YARN
> Issue Type: Bug
> Components: nodemanager
> Affects Versions: 2.5.0
> Reporter: zhihai xu
> Assignee: zhihai xu
> Fix For: 2.7.0
>
> Attachments: YARN-2816.000.patch, YARN-2816.001.patch,
> YARN-2816.002.patch, leveldb_records.txt
>
>
> NM fail to start with NPE during container recovery.
> We saw the following crash happen:
> 2014-10-30 22:22:37,211 INFO org.apache.hadoop.service.AbstractService:
> Service
> org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl
> failed in state INITED; cause: java.lang.NullPointerException
> java.lang.NullPointerException
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl.recoverContainer(ContainerManagerImpl.java:289)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl.recover(ContainerManagerImpl.java:252)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl.serviceInit(ContainerManagerImpl.java:235)
> at
> org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
> at
> org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:107)
> at
> org.apache.hadoop.yarn.server.nodemanager.NodeManager.serviceInit(NodeManager.java:250)
> at
> org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
> at
> org.apache.hadoop.yarn.server.nodemanager.NodeManager.initAndStartNodeManager(NodeManager.java:445)
> at
> org.apache.hadoop.yarn.server.nodemanager.NodeManager.main(NodeManager.java:492)
> The reason is some DB files used in NMLeveldbStateStoreService are
> accidentally deleted to save disk space at
> /tmp/hadoop-yarn/yarn-nm-recovery/yarn-nm-state. This leaves some incomplete
> container record which don't have CONTAINER_REQUEST_KEY_SUFFIX(startRequest)
> entry in the DB. When container is recovered at
> ContainerManagerImpl#recoverContainer,
> The NullPointerException at the following code cause NM shutdown.
> {code}
> StartContainerRequest req = rcs.getStartRequest();
> ContainerLaunchContext launchContext = req.getContainerLaunchContext();
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
