Oleksandr Shevchenko created YARN-8812:
------------------------------------------
Summary: Containers fail during creating a symlink which started
with hyphen for a resource file
Key: YARN-8812
URL: https://issues.apache.org/jira/browse/YARN-8812
Project: Hadoop YARN
Issue Type: Bug
Reporter: Oleksandr Shevchenko
Assignee: Oleksandr Shevchenko
When we run some job and add a file with alias started with hyphen then a
container fails during creating a symlink for a resource file:
{noformat}
yarn jar hadoop-mapreduce-examples.jar pi -files testfile#-symlink 1 1
{noformat}
or add a file to distributed cache in MR job by "job.addCacheFile"
Containers fail if resource file has a symlink started with hyphen with the
following error:
{noformat}
Stack trace: ExitCodeException exitCode=1:
/tmp/hadoop-yarn/nm-local-dir/usercache/yarn/appcache/application_1537449069809_0022/container_e01_1537449069809_0022_02_000001/launch_container.sh
ln: invalid option -- 'y'
Try 'ln --help' for more information.
at org.apache.hadoop.util.Shell.runCommand(Shell.java:572)
at org.apache.hadoop.util.Shell.run(Shell.java:466)
at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:768)
at
org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor.launchContainer(LinuxContainerExecutor.java:306)
{noformat}
The main cause of the problem is "launch_container.sh" script whitch contain
the following command for creating a symlink:
{noformat}
ln -sf "/tmp/hadoop-yarn/nm-local-dir/usercache/yarn/filecache/49/testfile"
"-symlink"
{noformat}
As the result "-symlink" parse as "-s" flag but not as a symlink name.
The same job successfully passed when running on MRv1 but not on YARN since
symlinks create in different ways. Unix systems support names which start with
the hyphen.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
