This kind of sounds like https://issues.apache.org/jira/browse/PIG-1955 but I don't understand why it doesn't happen for anyone else..
On Mon, Apr 4, 2011 at 2:14 PM, Dimitris Iliopoulos < [email protected]> wrote: > So in terms of my Pig setup as I said I am using 0.8.0 and have the > following environment variables set: > > export PIG_INSTALL=/home/UNIXHOME/diliopoulos/Hadoop/lib/pig-0.8.0 > export PIG_CLASSPATH=$HADOOP_INSTALL/conf/ > export PATH=$PATH:$PIG_INSTALL/bin > > I have build Pig locally and have tried running the tutorial examples but I > am running into the same error messages when I run them on hadoop versus > locally. I have tried to move the above environment variables into > hadoop_env.sh but the errors persist. Have also tried explicitly passing > the > classpath by using "hadoop -cp" but still no luck. I'm pretty sure I am > missing something really obvious here but I can't seem to find it. > > - Dimitris > > On Fri, Apr 1, 2011 at 6:51 PM, <[email protected]> wrote: > > > Running the 0.8.0 release with Hadoop 0.20.2 and the rest are pretty > > standard. > > Sent via BlackBerry from T-Mobile > > > > -----Original Message----- > > From: Dmitriy Ryaboy <[email protected]> > > Date: Fri, 1 Apr 2011 18:26:34 > > To: <[email protected]> > > Reply-To: [email protected] > > Subject: Re: Unable to run Pig in MapReduce mode > > > > Are you running a released version of Pig or something from svn? > > Anything interesting about your log setup or classpath we should know > > about? > > > > D > > > > On Fri, Apr 1, 2011 at 6:19 PM, Dimitris Iliopoulos < > > [email protected]> wrote: > > > > > There is a "Fail" link and the error trace that I get from there is as > > > follows: > > > > > > java.io.IOException: Deserialization error: > > > org.apache.commons.logging.impl.Log4JLogger; > > > org.apache.commons.logging.impl.Log4JLogger; class invalid for > > > deserialization > > > > > > at > > > > > > org.apache.pig.impl.util.ObjectSerializer.deserialize(ObjectSerializer.java:55) > > > at > > > > > > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat.getStores(PigOutputFormat.java:217) > > > at > > > > > > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat.setupUdfEnvAndStores(PigOutputFormat.java:244) > > > at > > > > > > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat.getOutputCommitter(PigOutputFormat.java:274) > > > at org.apache.hadoop.mapred.Task.initialize(Task.java:415) > > > at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:354) > > > at org.apache.hadoop.mapred.Child.main(Child.java:170) > > > Caused by: java.io.InvalidClassException: > > > org.apache.commons.logging.impl.Log4JLogger; > > > org.apache.commons.logging.impl.Log4JLogger; class invalid for > > > deserialization > > > at > > > java.io.ObjectStreamClass.checkDeserialize(ObjectStreamClass.java:713) > > > at > > > > java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1732) > > > at > > > java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1328) > > > at > > > > java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1946) > > > at > > > java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1870) > > > at > > > > java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1752) > > > at > > > java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1328) > > > at > > java.io.ObjectInputStream.readObject(ObjectInputStream.java:350) > > > at java.util.LinkedList.readObject(LinkedList.java:964) > > > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > > > at > > > > > > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) > > > at > > > > > > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > > > at java.lang.reflect.Method.invoke(Method.java:597) > > > at > > > java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:974) > > > at > > > java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1848) > > > at > > > > java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1752) > > > at > > > java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1328) > > > at > > java.io.ObjectInputStream.readObject(ObjectInputStream.java:350) > > > at > > > > > > org.apache.pig.impl.util.ObjectSerializer.deserialize(ObjectSerializer.java:53) > > > ... 6 more > > > Caused by: java.io.InvalidClassException: > > > org.apache.commons.logging.impl.Log4JLogger; class invalid for > > > deserialization > > > at > > > java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:587) > > > at > > > java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1582) > > > at > > > java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1495) > > > at > > > > java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1731) > > > ... 23 more > > > > > > > > > On Fri, Apr 1, 2011 at 6:13 PM, Dmitriy Ryaboy <[email protected]> > > wrote: > > > > > > > Is there a line that says "Job Setup: Failed" at the top of the job > > page? > > > > Failed (or "Successful") should be a link, click through that to get > > the > > > > setup log. > > > > > > > > D > > > > > > > > On Fri, Apr 1, 2011 at 6:02 PM, Dimitris Iliopoulos < > > > > [email protected]> wrote: > > > > > > > > > Dmitriy, > > > > > > > > > > The jobtracker does indeed mark the job as failed but the problem > is > > > that > > > > > there are no log files. All map and reduce tasks are flagged as > > killed > > > > and > > > > > upon inspection produce no logs. The only logs are the ones I > > appended > > > > > before produced by Pig. > > > > > > > > > > - Dimitris > > > > > > > > > > On Fri, Apr 1, 2011 at 5:55 PM, Dmitriy Ryaboy <[email protected] > > > > > > wrote: > > > > > > > > > > > Dimitris, > > > > > > You should be able to look at the jobs submitted to your > jobtracker > > > and > > > > > > find > > > > > > out what happened to the pig job (the message " Job terminated > with > > > > > > anomalous status FAILED" leads me to believe there is some job > > setup > > > > > > problem). Do you see any pig job attempts on your jobtracker UI? > > > > > > > > > > > > D > > > > > > > > > > > > On Fri, Apr 1, 2011 at 2:16 PM, Dimitris Iliopoulos < > > > > > > [email protected]> wrote: > > > > > > > > > > > > > Currently using Pig 0.8 and Hadoop 0.20.2. I'm able to run > things > > > in > > > > > > local > > > > > > > mode as well as run fs -* commands from within Grunt running in > > > > > MapReduce > > > > > > > mode. I can't seem to be able to execute any Pig Latin > statement > > in > > > > > > > MapReduce mode. The fact that I can browse HDFS and access > files > > > > leads > > > > > me > > > > > > > to > > > > > > > believe that the setup is correct (also verified by the correct > > > > > namenode > > > > > > > and > > > > > > > jobtracker addresses connected to when firing off any pig > > command). > > > I > > > > > > don't > > > > > > > know why though I can't run anything else. A simple command > like > > > the > > > > > > > following: > > > > > > > > > > > > > > data = LOAD 'test.csv' USING PigStorage(','); > > > > > > > DUMP data; > > > > > > > > > > > > > > Produces the following stack trace: > > > > > > > > > > > > > > Pig Stack Trace > > > > > > > --------------- > > > > > > > ERROR 1066: Unable to open iterator for alias data > > > > > > > > > > > > > > org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1066: > > > > Unable > > > > > to > > > > > > > open iterator for alias data > > > > > > > at > > org.apache.pig.PigServer.openIterator(PigServer.java:754) > > > > > > > at > > > > > > > > > > > > > > > > org.apache.pig.tools.grunt.GruntParser.processDump(GruntParser.java:612) > > > > > > > at > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:303) > > > > > > > at > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:165) > > > > > > > at > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:141) > > > > > > > at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:76) > > > > > > > at org.apache.pig.Main.run(Main.java:465) > > > > > > > at org.apache.pig.Main.main(Main.java:107) > > > > > > > Caused by: java.io.IOException: Job terminated with anomalous > > > status > > > > > > FAILED > > > > > > > at > > org.apache.pig.PigServer.openIterator(PigServer.java:744) > > > > > > > ... 7 more > > > > > > > > > > > > > > Any other more elaborate collection of Pig statements will > > produce > > > > > this: > > > > > > > > > > > > > > Pig Stack Trace > > > > > > > --------------- > > > > > > > ERROR 2244: Job failed, hadoop does not return any error > message > > > > > > > > > > > > > > org.apache.pig.backend.executionengine.ExecException: ERROR > 2244: > > > Job > > > > > > > failed, hadoop does not return any error message > > > > > > > at > > > > > > > > > > > > > > > > org.apache.pig.tools.grunt.GruntParser.executeBatch(GruntParser.java:116) > > > > > > > at > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:169) > > > > > > > at > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:141) > > > > > > > at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:90) > > > > > > > at org.apache.pig.Main.run(Main.java:510) > > > > > > > at org.apache.pig.Main.main(Main.java:107) > > > > > > > > > > > > > > Where any input file is flagged as unreadable by the following > > > error > > > > > > > message > > > > > > > during execution: > > > > > > > > > > > > > > Input(s): > > > > > > > Failed to read data from "test.csv" > > > > > > > > > > > > > > All files are accessible in MapReduce mode via cat, ls etc. I > > might > > > > be > > > > > > > missing something very obvious here since this is the first > time > > > I've > > > > > > used > > > > > > > Pig so any help would be greatly appreciated. > > > > > > > > > > > > > > - Dimitris > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
