chmod is fine.
In some cases streaming job is a script that uses executables that are
shipped using -file option.
The executables loose the x permissions and the script fails. (I ran
into this and it took me a while to undersatnd the reason of failure).
This may be fixed or alleviated by
(a) adding an -executable option that is similar to -file but does
"chmod a+x"
(b) at least describing this in user documentation, so that the user
knows to do this on her own.
Of course, (a) is preferable...
-- ab
On Dec 7, 2006, at 7:42 AM, Sanjay Dahiya (JIRA) wrote:
[ http://issues.apache.org/jira/browse/HADOOP-476?page=all ]
Sanjay Dahiya updated HADOOP-476:
---------------------------------
Attachment: Hadoop-476.patch
This patch replaces commandline processing with apache CLI2.
Input validation is not done as part of this patch as thats moved to
JobClient now. It validates -mapper, -reducer, -combiner, -file
attributes to be files present and accessible on disk.
It doesn't validate a file to be an executable, that is not available
through java File API until Java 6. For now it runs "chmod +x" on the
file if its present and accessible.
comments?
Streaming should check for correctness of the task
--------------------------------------------------
Key: HADOOP-476
URL: http://issues.apache.org/jira/browse/HADOOP-476
Project: Hadoop
Issue Type: Bug
Components: contrib/streaming
Reporter: arkady borkovsky
Assigned To: Sanjay Dahiya
Attachments: Hadoop-476.patch
Currently, if anythin is wrong with streaming job, it dies without
any explanation.
Before creating and running actual MapReduce job, Streaming should
check if:
-- the executables (or scripts) for -mapper and -reducer are
available and have right permissions
-- the input fragments exist
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the
administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
