This is what I see when I stream a perl script: *stderr logs*
===== Task Information Header ===== Command: perl PigStreaming.pl (stdin-org.apache.pig.builtin.PigStreaming/stdout-org.apache.pig.builtin.PigStreaming) Start time: Mon Sep 26 15:27:47 UTC 2011 Input-split file: null Input-split start-offset: -1 Input-split length: -1 ===== * * * ===== PigStreaming.pl: starting processing PigStreaming.pl: Done ===== Task Information Footer ===== End time: Mon Sep 26 15:27:48 UTC 2011 Exit code: 0 Input records: 10000 Input bytes: 2044344 bytes (stdin using org.apache.pig.builtin.PigStreaming) Output records: 10000 Output bytes: 2044344 bytes (stdout using org.apache.pig.builtin.PigStreaming) ===== * * * ===== You don't have "perl" in the command. Not sure if this is the cause. Also can you print out some debug info in perl script? That will go to jobtracker UI as well. Daniel On Sun, Sep 25, 2011 at 9:40 PM, Deepak Reddy <[email protected]> wrote: > I have perl installed as /usr/bin/perl though I did try /usr/bin/env perl > and it didn't work in that case too > > DR > > -----Original Message----- > From: Dmitriy Ryaboy [mailto:[email protected]] > Sent: Sunday, September 25, 2011 9:12 PM > To: [email protected] > Subject: Re: Streaming error > > Are you sure you have perl installed in /usr/bin/perl ? What if you try > "/usr/bin/env perl"? > > D > > On Sun, Sep 25, 2011 at 12:01 PM, Deepak Reddy <[email protected] > >wrote: > > > These are the STDERR logs > > > > stderr logs > > > > ===== Task Information Header ===== > > Command: > > ./test.pl(stdin-org.apache.pig.builtin.PigStreaming/stdout-org.apache. > > pig.builtin.PigStreaming) Start time: Sun Sep 25 11:34:09 PDT 2011 > > Input-split file: null Input-split start-offset: -1 Input-split > > length: -1 > > ===== * * * ===== > > log4j:WARN No appenders could be found for logger > > (org.apache.hadoop.hdfs.DFSClient). > > log4j:WARN Please initialize the log4j system properly. > > > > -DR > > > > -----Original Message----- > > From: Deepak Reddy [mailto:[email protected]] > > Sent: Sunday, September 25, 2011 11:39 AM > > To: [email protected] > > Subject: RE: Streaming error > > > > Same error as before when I try with STREAM LOGS THROUGH CMD AS > > testword; > > > > 2011-09-25 11:36:50,182 [main] ERROR > > org.apache.pig.tools.grunt.GruntParser > > - ERROR 2997: Unable to recreate exception from backed error: > > org.apache.pig.backend.executionengine.ExecException: ERROR 2055: > > Received Error while processing the map plan. > > 2011-09-25 11:36:50,182 [main] ERROR > > org.apache.pig.tools.grunt.GruntParser > > - org.apache.pig.backend.executionengine.ExecException: ERROR 2997: > > Unable to recreate exception from backed error: > > org.apache.pig.backend.executionengine.ExecException: ERROR 2055: > > Received Error while processing the map plan. > > at > > > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.Launcher.getErrorMessages(Launcher.java:221) > > at > > > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.Launcher.getStats(Launcher.java:151) > > at > > > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.launchPig(MapReduceLauncher.java:337) > > at > > > org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.execute(HExecutionEngine.java:382) > > at > > org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:1209) > > at org.apache.pig.PigServer.execute(PigServer.java:1201) > > at org.apache.pig.PigServer.access$100(PigServer.java:129) > > at org.apache.pig.PigServer$Graph.execute(PigServer.java:1528) > > at org.apache.pig.PigServer.executeBatchEx(PigServer.java:373) > > at org.apache.pig.PigServer.executeBatch(PigServer.java:340) > > at > > org.apache.pig.tools.grunt.GruntParser.executeBatch(GruntParser.java:115) > > at > > > org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:172) > > at > > > org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:144) > > at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:90) > > at org.apache.pig.Main.run(Main.java:396) > > at org.apache.pig.Main.main(Main.java:107) > > > > thanks, > > DR > > > > -----Original Message----- > > From: Jonathan Coveney [mailto:[email protected]] > > Sent: Sunday, September 25, 2011 11:28 AM > > To: [email protected] > > Subject: Re: Streaming error > > > > I believe that > > > > STREAM LOGS THROUGH `./test.pl` AS testword; > > > > should be > > > > STREAM LOGS THROUGH CMD AS testword; > > > > 2011/9/25 Deepak Reddy <[email protected]> > > > > > Hi, > > > > > > I stripped out everything from my perl script and just printing a > > > hello world. > > > > > > #!/usr/bin/perl > > > print "hello world\n"; > > > > > > within my pig script I am running the script as > > > > > > DEFINE CMD `./test.pl` > > > SHIP ('/home/test/test.pl') > > > > > > STREAM LOGS THROUGH `./test.pl` AS testword; > > > > > > Now I get the following error (doesn't show any exit status code) > > > > > > 2011-09-25 10:11:45,822 WARN org.apache.hadoop.mapred.Child: Error > > > running child > > > org.apache.pig.backend.executionengine.ExecException: ERROR 2055: > > > Received Error while processing the map plan. > > > at > > > > > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBas > > e.runPipeline(PigMapBase.java:265) > > > at > > > > > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBas > > e.map(PigMapBase.java:232) > > > at > > > > > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBas > > e.map(PigMapBase.java:53) > > > at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144) > > > at > org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:647) > > > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:323) > > > at org.apache.hadoop.mapred.Child$4.run(Child.java:270) > > > at java.security.AccessController.doPrivileged(Native Method) > > > at javax.security.auth.Subject.doAs(Subject.java:396) > > > at > > > > > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformat > > ion.java:1127) > > > at org.apache.hadoop.mapred.Child.main(Child.java:264) > > > > > > any info is appreciated. > > > Thanks, > > > Deepak Reddy > > > > > > -----Original Message----- > > > From: Xiaomeng Wan [mailto:[email protected]] > > > Sent: Saturday, September 24, 2011 12:27 PM > > > To: [email protected] > > > Subject: Re: Streaming error > > > > > > also make sure it is executable before pack/ship it > > > > > > Shawn > > > > > > On Sat, Sep 24, 2011 at 5:57 AM, Norbert Burger > > > <[email protected]> > > > wrote: > > > > BASH exit status of 127 is a "command not found": > > > > > > > > http://tldp.org/LDP/abs/html/exitcodes.html > > > > > > > > Does geostream.pl shell out to other commands, or otherwise rely > > > > on > > > $PATH? > > > > Using fully-qualified paths may help. > > > > > > > > Norbert > > > > > > > > On Fri, Sep 23, 2011 at 6:59 PM, Deepak Reddy > > > > <[email protected]> > > > wrote: > > > > > > > >> This is from the job track error log > > > >> > > > >> org.apache.pig.backend.executionengine.ExecException: ERROR 2055: > > > >> Received Error while processing the map plan: > > 'geostream.plGeoLiteCity.dat ' > > > >> failed with exit status: 127 > > > >> at > > > >> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigM > > > >> ap > > > >> Ba > > > >> se.runPipeline(PigMapBase.java:265) > > > >> at > > > >> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigM > > > >> ap > > > >> Ba > > > >> se.map(PigMapBase.java:232) > > > >> at > > > >> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigM > > > >> ap > > > >> Ba > > > >> se.map(PigMapBase.java:53) > > > >> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144) > > > >> at > > > >> org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:647) > > > >> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:323) > > > >> at org.apache.hadoop.mapred.Child$4.run(Child.java:270) > > > >> at java.security.AccessController.doPrivileged(Native Method) > > > >> at javax.security.auth.Subject.doAs(Subject.java:396) > > > >> at > > > >> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInf > > > >> or > > > >> ma > > > >> tion.java:1127) > > > >> at org.apache.hadoop.mapred.Child.main(Child.java:264) > > > >> > > > >> Thanks, > > > >> Deepak Reddy > > > >> > > > >> -----Original Message----- > > > >> From: Daniel Dai [mailto:[email protected]] > > > >> Sent: Friday, September 23, 2011 3:56 PM > > > >> To: [email protected] > > > >> Subject: Re: Streaming error > > > >> > > > >> Go to JobtrackerUI to get more detailed error message. > > > >> > > > >> On Fri, Sep 23, 2011 at 3:54 PM, Deepak Reddy > > > >> <[email protected]> > > > >> wrote: > > > >> > > > >> > Hi, > > > >> > > > > >> > I am trying to run a pig streaming perl job using GeoLite DB > > > >> > and I am getting the following failure > > > >> > > > > >> > 2011-09-23 15:49:44,902 [main] ERROR > > > >> > org.apache.pig.tools.pigstats.PigStats > > > >> > - ERROR 2997: Unable to recreate exception from backed error: > > > >> > org.apache.pig.backend.executionengine.ExecException: ERROR 2055: > > > >> > Received Error while processing the map plan: '/usr/bin/perl > > > >> > geostream.plGeoLiteCity.dat ' failed with exit status: 2 > > > >> > > > > >> > I am shipping all the perl scripts(modules) and using the > > > >> > cached geolite DB on HDFS > > > >> > > > > >> > Can you please give me some pointers. > > > >> > > > > >> > Thank you, > > > >> > Deepak Reddy > > > >> > > > > >> > > > > > > > > > >
