On Tue, Jul 7, 2009 at 10:26 PM, jason hadoop <[email protected]> wrote: > > The mapper has no control at the point where your mymapper.sh script is > running. > > You may wish to have the following commands in mymapper.sh before you > attempt your pipeline > (pwd > ls -l ./binary1 ./binary2 > echo $PATH) 1>&2
I tried this, and because I specify "-file mymapper.sh -file binary1 -file binary2" as arguments to the streaming command, the two binaries and the script end up in the same directory. > The 1>&2 redirects stdout to stderr. > > The commands output should end up in your job's userlog. > You could redirect it to a temporary file to make it available in /tmp. > > Chapter 8 of Pro Hadoop covers some fine details of streaming jobs. > > It may be that there is something going on in the environment that is > resulting in your permission denied error. > > > On Tue, Jul 7, 2009 at 9:04 PM, rvernica <[email protected]> wrote: > > > > > > > Ashish Venugopal wrote: > > > > > > I have a question regarding a mapper task that needs to call 2 binaries. > > > Ideally I would be able to do the following: > > > mymapper.sh > > > > > > ./binary1 | ./binary2 > > > > > > stream -mapper mymapper.sh ... -file binary1 -file binary2 -file > > > mymapper.sh > > > > > > But when this runs, I get "permission denied" executing binary1. I read > > on > > > the Wiki that Unix pipers are not > > > allowed, but it seemed to indicated that simply putting them on the > > > command > > > line was not allowed, but in a > > > script it was acceptable. Is this the case? > > > > > > Also, I have also tried to make binary1 call binary2 via forking, but I > > > get > > > the same permission denied > > > error. Is this a fundamental design decision (to allow only 1 binary in > > > the > > > mapper)? Or do I just need > > > to explicitly change the permissions somehow? > > > > > > Ashish > > > > > > > > > > Does anybody has any solution to this? > > > > I tried it in 17.2.1 and I still get Permission Denied. > > > > On the other hand, if I say: > > > > stream ... -reducer "binary1 | binary2" > > > > "|" and "binary2" are treated as arguments to "binary1" > > > > Thanks! > > -- > > View this message in context: > > http://www.nabble.com/permission-denied-on-additional-binaries-tp16551104p24384957.html > > Sent from the Hadoop core-user mailing list archive at Nabble.com. > > > > > > > -- > Pro Hadoop, a book to guide you from beginner to hadoop mastery, > http://www.amazon.com/dp/1430219424?tag=jewlerymall > www.prohadoopbook.com a community for Hadoop Professionals
