[
https://issues.apache.org/jira/browse/MAPREDUCE-4847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13511031#comment-13511031
]
Peng Lei commented on MAPREDUCE-4847:
-------------------------------------
Thank you for your comment!
I have put the command in a script file as a workaround, it works. But in this
case, the command is not too complex to write a dedicate script file, and on
fly script generating is a bit tricky(at least for maintainer).
It seems hadoop can't run on windows without cygwin. Another solution may be:
add a new option to instruct streaming to use an alternative command invoker,
such as:
-command_invoker "sh -c"
This could solve the issue and didn't break the existing hadoop-streaming
application.
-Peng
> Command Parsing in Hadoop Streaming
> -----------------------------------
>
> Key: MAPREDUCE-4847
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4847
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: contrib/streaming
> Reporter: Peng Lei
> Labels: features
> Original Estimate: 4h
> Remaining Estimate: 4h
>
> Hadoop streaming parse the mapper and reducer commands by itself, this is not
> a good choice, when I write a complex mapper/reducer script inline, such as
> 'perl -ne ...', it don't work.
> An alternative way is to send the command to the shell, simply create new
> process(sh -c "command_and_args"), this not also simplize the streaming code,
> but also improve its capability!
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
