have you created the parent folder, such as /data1/yarn
---Original---
From: "??????"<[email protected]>
Date: 2017/2/7 00:43:47
To: "[email protected]"<[email protected]>;
Subject: Unable to localize container, when using LinuxContainerExecutor
Hi,
I tried to run secure containers in cluster, and followed the instructions
from
http://hadoop.apache.org/docs/r2.6.0/hadoop-project-dist/hadoop-common/SecureMode.html#LinuxContainerExecutor
After that, I ran the mapreduce example pi and got the following exception:
2017-02-06 17:46:24,018 INFO
org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor: main : command
provided 0
2017-02-06 17:46:24,018 INFO
org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor: main : run as user
is hadoop
2017-02-06 17:46:24,018 INFO
org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor: main : requested
yarn user is hadoop
2017-02-06 17:46:24,019 INFO
org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor: Failed to create
directory /data1/yarn/nm,/data2/yarn/nm,/data3/yarn/nm/usercache/hadoop - No
such file or directory
2017-02-06 17:46:24,019 INFO
org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService:
Localizer failed
java.io.IOException: Application application_1486374303689_0001 initialization
failed (exitCode=20) with output: main : command provided 0
main : run as user is hadoop
main : requested yarn user is hadoop
Failed to create directory
/data1/yarn/nm,/data2/yarn/nm,/data3/yarn/nm/usercache/hadoop - No such file or
directory
at
org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor.startLocalizer(LinuxContainerExecutor.java:241)
at
org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService$LocalizerRunner.run(ResourceLocalizationService.java:1088)
Caused by: ExitCodeException exitCode=20:
at org.apache.hadoop.util.Shell.runCommand(Shell.java:538)
at org.apache.hadoop.util.Shell.run(Shell.java:455)
at
org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:715)
at
org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor.startLocalizer(LinuxContainerExecutor.java:232)
... 1 more
Here is the container-executor.cfg under /etc/hadoop owned by root
(configurations are same to the yarn-site.xml):
yarn.nodemanager.local-dirs=/data1/yarn/nm,/data2/yarn/nm,/data3/yarn/nm
yarn.nodemanager.log-dirs=/data1/hadoop/logs/container
yarn.nodemanager.linux-container-executor.group=hadoop
banned.users=yarn
min.user.id=500
allowed.system.users=hadoop,dm
Hope to get some helps.
Regards,
Reid Chan.
