Andrew Onischuk created AMBARI-10013:
----------------------------------------
Summary: Nodemanager fails to start with error
java.io.IOException: Linux container executor not configured properly (error=24)
Key: AMBARI-10013
URL: https://issues.apache.org/jira/browse/AMBARI-10013
Project: Ambari
Issue Type: Bug
Reporter: Andrew Onischuk
Assignee: Andrew Onischuk
Fix For: 2.0.0
In Ambari deployed clusters, Nodemanager fails to start with the following
exception for the test case where we write container_executor.cfg
**nodemanager log**
2015-03-05 20:16:19,431 INFO nodemanager.NodeManager
(SignalLogger.java:register(91)) - registered UNIX signal handlers for [TERM,
HUP, INT]
2015-03-05 20:16:21,363 WARN nodemanager.LinuxContainerExecutor
(LinuxContainerExecutor.java:init(184)) - Exit code from container executor
initialization is : 24
ExitCodeException exitCode=24: File /etc/hadoop/conf.empty must be owned by
root, but is owned by 1005
at org.apache.hadoop.util.Shell.runCommand(Shell.java:538)
at org.apache.hadoop.util.Shell.run(Shell.java:455)
at
org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:715)
at
org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor.init(LinuxContainerExecutor.java:181)
at
org.apache.hadoop.yarn.server.nodemanager.NodeManager.serviceInit(NodeManager.java:209)
at
org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
at
org.apache.hadoop.yarn.server.nodemanager.NodeManager.initAndStartNodeManager(NodeManager.java:462)
at
org.apache.hadoop.yarn.server.nodemanager.NodeManager.main(NodeManager.java:509)
2015-03-05 20:16:21,366 INFO nodemanager.ContainerExecutor
(ContainerExecutor.java:logOutput(283)) -
2015-03-05 20:16:21,367 INFO service.AbstractService
(AbstractService.java:noteFailure(272)) - Service NodeManager failed in state
INITED; cause: org.apache.hadoop.yarn.exceptions.YarnRuntimeException: Failed
to initialize container executor
org.apache.hadoop.yarn.exceptions.YarnRuntimeException: Failed to
initialize container executor
at
org.apache.hadoop.yarn.server.nodemanager.NodeManager.serviceInit(NodeManager.java:211)
at
org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
at
org.apache.hadoop.yarn.server.nodemanager.NodeManager.initAndStartNodeManager(NodeManager.java:462)
at
org.apache.hadoop.yarn.server.nodemanager.NodeManager.main(NodeManager.java:509)
Caused by: java.io.IOException: Linux container executor not configured
properly (error=24)
at
org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor.init(LinuxContainerExecutor.java:187)
at
org.apache.hadoop.yarn.server.nodemanager.NodeManager.serviceInit(NodeManager.java:209)
... 3 more
Caused by: ExitCodeException exitCode=24: File /etc/hadoop/conf.empty must
be owned by root, but is owned by 1005
at org.apache.hadoop.util.Shell.runCommand(Shell.java:538)
at org.apache.hadoop.util.Shell.run(Shell.java:455)
at
org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:715)
at
org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor.init(LinuxContainerExecutor.java:181)
... 4 more
2015-03-05 20:16:21,374 WARN service.AbstractService
(ServiceOperations.java:stopQuietly(82)) - When stopping the service
NodeManager : java.lang.NullPointerException
java.lang.NullPointerException
at
org.apache.hadoop.yarn.server.nodemanager.NodeManager.stopRecoveryStore(NodeManager.java:161)
at
org.apache.hadoop.yarn.server.nodemanager.NodeManager.serviceStop(NodeManager.java:273)
at
org.apache.hadoop.service.AbstractService.stop(AbstractService.java:221)
at
org.apache.hadoop.service.ServiceOperations.stop(ServiceOperations.java:52)
at
org.apache.hadoop.service.ServiceOperations.stopQuietly(ServiceOperations.java:80)
at
org.apache.hadoop.service.AbstractService.init(AbstractService.java:171)
at
org.apache.hadoop.yarn.server.nodemanager.NodeManager.initAndStartNodeManager(NodeManager.java:462)
at
org.apache.hadoop.yarn.server.nodemanager.NodeManager.main(NodeManager.java:509)
2015-03-05 20:16:21,375 FATAL nodemanager.NodeManager
(NodeManager.java:initAndStartNodeManager(465)) - Error starting NodeManager
org.apache.hadoop.yarn.exceptions.YarnRuntimeException: Failed to
initialize container executor
at
org.apache.hadoop.yarn.server.nodemanager.NodeManager.serviceInit(NodeManager.java:211)
at
org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
at
org.apache.hadoop.yarn.server.nodemanager.NodeManager.initAndStartNodeManager(NodeManager.java:462)
at
org.apache.hadoop.yarn.server.nodemanager.NodeManager.main(NodeManager.java:509)
Caused by: java.io.IOException: Linux container executor not configured
properly (error=24)
at
org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor.init(LinuxContainerExecutor.java:187)
at
org.apache.hadoop.yarn.server.nodemanager.NodeManager.serviceInit(NodeManager.java:209)
... 3 more
Caused by: ExitCodeException exitCode=24: File /etc/hadoop/conf.empty must
be owned by root, but is owned by 1005
at org.apache.hadoop.util.Shell.runCommand(Shell.java:538)
at org.apache.hadoop.util.Shell.run(Shell.java:455)
at
org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:715)
at
org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor.init(LinuxContainerExecutor.java:181)
... 4 more
**container_executor.cfg**
cat /etc/hadoop/conf/container-executor.cfg
yarn.nodemanager.local-dirs=/grid/0/hadoop/yarn/local
yarn.nodemanager.log-dirs=/grid/0/hadoop/yarn/log
yarn.nodemanager.linux-container-executor.group=hadoop
banned.users=yarn
min.user.id=1000
allowed.system.users=nobody
Please note that test code does not touch conf.empty folder
**conf.empty permissions**
ls -l /etc/hadoop/
total 4
lrwxrwxrwx 1 root root 22 Mar 4 21:01 conf -> /etc/hadoop/conf.empty
drwxr-xr-x 2 hdfs hadoop 4096 Mar 5 19:10 conf.empty
Attached are the conf directory, nodemanager logs, RM logs and console logs
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
