[
https://issues.apache.org/jira/browse/MAPREDUCE-3123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13122483#comment-13122483
]
Hitesh Shah commented on MAPREDUCE-3123:
----------------------------------------
Vinod/Thomas/Arun - a query on task.sh - the final command i.e. "exec setsid
/bin/bash -c ..." encapsulates the user provided command with "". The
assumption here seems to be they can use environment variables with the command
for expansion and that the onus of correctly escaping the command i.e escaping
any special chars is on the users themselves. Would that be correct?
For example, creating the symlink above is pretty simply by just quoting the
file name but if the symlink is something that will be executed then special
chars like "&" or even basic things like spaces can create problems if not
quoted and escaped properly.
> Symbolic links with special chars causing container/task.sh to fail
> -------------------------------------------------------------------
>
> Key: MAPREDUCE-3123
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3123
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Thomas Graves
> Assignee: Hitesh Shah
> Priority: Blocker
> Fix For: 0.23.0
>
>
> the following job throws an exception when you have the special characters in
> it.
> hadoop jar hadoop-streaming.jar -Dmapreduce.job.acl-view-job=*
> -Dmapreduce.job.queuename=queue1 -files
> file:///homes/user/hadoop/Streaming/data/streaming-980//InputDir#testlink!@$&*()-_+=
> -input Streaming/streaming-980/input.txt -mapper 'xargs cat' -reducer cat
> -output Streaming/streaming-980/Output -jobconf
> mapred.job.name=streamingTest-980 -jobconf mapreduce.job.acl-view-job=*
> Exception:
> 2011-09-27 20:58:48,903 INFO
> org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor:
> launchContainer:
> [container-executor, hadoopuser, 1, application_1317077272567_0239,
> container_1317077272567 0239_01_000001,
> tmp/mapred-local/usercache/hadoopuser/appcache/application_1317077272567_0239/container_1317077272567_0239_01_000001,
> tmp/mapred-local/nmPrivate/application_1317077272567_0239/container_1317077272567_0239_01
> 000001/task.sh,
> tmp/mapred-local/nmPrivate/container_1317077272567_0239_01_000001/container_1317077272567_0239_01_000001.tokens]1109221111-tests.jar:hadoop-mapreduce-p2011-09-27
> 20:58:48,944 WARN
> org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor: Exit code
> from container is : 2
>
> 2011-09-27
> 20:58:48,946 WARN
> org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor: Exception
> from container-launch :
>
>
> org.apache.hadoop.util.Shell$ExitCodeException:
> /tmp/mapred-local/usercache/hadoopuser/appcache/application_1317077272567_0239/container_1317077272567_0239_01_000001/task.sh:
> line 26: syntax error near unexpected token `-_+='
> /tmp/mapred-local/usercache/hadoopuser/appcache/application_1317077272567_0239/container_1317077272567_0239_01_000001/task.sh:
> line 26: `ln -sf
> /tmp/mapred-local/usercache/hadoopqa/filecache/-1888139433818483070/InputDir
> test
> ink!@$&*()-_+='kson-jaxrs-1.7.1.jar:/.m2/repository/org/codehaus/jackson/jackson-mapper-asl/1.7.3/jackson-mapper-asl-1.7.3.jar:.m2/repository/org/codehaus/jackson/jackson-xc/1.7.1/jackson-xc-1.7.1.jar:
> at org.apache.hadoop.util.Shell.runCommand(Shell.java:261)
>
>
>
> at org.apache.hadoop.util.Shell.run(Shell.java:188)
>
>
>
> at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:381)
>
>
>
> at
> org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor.launchContainer(LinuxContainerExecutor.java:174)
>
>
>
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:197)
>
>
>
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:62)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:619)
> 2011-09-27 20:58:48,951 INFO
> org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor:
> 2011-09-27 20:58:48,951 INFO
> org.apache.hadoop.yarn.server.nodemanager.containermanager.container.Container:
> Processing
> container_1317077272567_0239_01_000001 of type UPDATE_DIAGNOSTICS_MSG
> 2011-09-27 20:58:48,951 WARN
> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch:
> Container exited with a non-zero exit code 2
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
