Running the 0.8.0 release with Hadoop 0.20.2 and the rest are pretty standard. Sent via BlackBerry from T-Mobile
-----Original Message----- From: Dmitriy Ryaboy <[email protected]> Date: Fri, 1 Apr 2011 18:26:34 To: <[email protected]> Reply-To: [email protected] Subject: Re: Unable to run Pig in MapReduce mode Are you running a released version of Pig or something from svn? Anything interesting about your log setup or classpath we should know about? D On Fri, Apr 1, 2011 at 6:19 PM, Dimitris Iliopoulos < [email protected]> wrote: > There is a "Fail" link and the error trace that I get from there is as > follows: > > java.io.IOException: Deserialization error: > org.apache.commons.logging.impl.Log4JLogger; > org.apache.commons.logging.impl.Log4JLogger; class invalid for > deserialization > > at > org.apache.pig.impl.util.ObjectSerializer.deserialize(ObjectSerializer.java:55) > at > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat.getStores(PigOutputFormat.java:217) > at > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat.setupUdfEnvAndStores(PigOutputFormat.java:244) > at > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat.getOutputCommitter(PigOutputFormat.java:274) > at org.apache.hadoop.mapred.Task.initialize(Task.java:415) > at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:354) > at org.apache.hadoop.mapred.Child.main(Child.java:170) > Caused by: java.io.InvalidClassException: > org.apache.commons.logging.impl.Log4JLogger; > org.apache.commons.logging.impl.Log4JLogger; class invalid for > deserialization > at > java.io.ObjectStreamClass.checkDeserialize(ObjectStreamClass.java:713) > at > java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1732) > at > java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1328) > at > java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1946) > at > java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1870) > at > java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1752) > at > java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1328) > at java.io.ObjectInputStream.readObject(ObjectInputStream.java:350) > at java.util.LinkedList.readObject(LinkedList.java:964) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > at java.lang.reflect.Method.invoke(Method.java:597) > at > java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:974) > at > java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1848) > at > java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1752) > at > java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1328) > at java.io.ObjectInputStream.readObject(ObjectInputStream.java:350) > at > org.apache.pig.impl.util.ObjectSerializer.deserialize(ObjectSerializer.java:53) > ... 6 more > Caused by: java.io.InvalidClassException: > org.apache.commons.logging.impl.Log4JLogger; class invalid for > deserialization > at > java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:587) > at > java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1582) > at > java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1495) > at > java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1731) > ... 23 more > > > On Fri, Apr 1, 2011 at 6:13 PM, Dmitriy Ryaboy <[email protected]> wrote: > > > Is there a line that says "Job Setup: Failed" at the top of the job page? > > Failed (or "Successful") should be a link, click through that to get the > > setup log. > > > > D > > > > On Fri, Apr 1, 2011 at 6:02 PM, Dimitris Iliopoulos < > > [email protected]> wrote: > > > > > Dmitriy, > > > > > > The jobtracker does indeed mark the job as failed but the problem is > that > > > there are no log files. All map and reduce tasks are flagged as killed > > and > > > upon inspection produce no logs. The only logs are the ones I appended > > > before produced by Pig. > > > > > > - Dimitris > > > > > > On Fri, Apr 1, 2011 at 5:55 PM, Dmitriy Ryaboy <[email protected]> > > wrote: > > > > > > > Dimitris, > > > > You should be able to look at the jobs submitted to your jobtracker > and > > > > find > > > > out what happened to the pig job (the message " Job terminated with > > > > anomalous status FAILED" leads me to believe there is some job setup > > > > problem). Do you see any pig job attempts on your jobtracker UI? > > > > > > > > D > > > > > > > > On Fri, Apr 1, 2011 at 2:16 PM, Dimitris Iliopoulos < > > > > [email protected]> wrote: > > > > > > > > > Currently using Pig 0.8 and Hadoop 0.20.2. I'm able to run things > in > > > > local > > > > > mode as well as run fs -* commands from within Grunt running in > > > MapReduce > > > > > mode. I can't seem to be able to execute any Pig Latin statement in > > > > > MapReduce mode. The fact that I can browse HDFS and access files > > leads > > > me > > > > > to > > > > > believe that the setup is correct (also verified by the correct > > > namenode > > > > > and > > > > > jobtracker addresses connected to when firing off any pig command). > I > > > > don't > > > > > know why though I can't run anything else. A simple command like > the > > > > > following: > > > > > > > > > > data = LOAD 'test.csv' USING PigStorage(','); > > > > > DUMP data; > > > > > > > > > > Produces the following stack trace: > > > > > > > > > > Pig Stack Trace > > > > > --------------- > > > > > ERROR 1066: Unable to open iterator for alias data > > > > > > > > > > org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1066: > > Unable > > > to > > > > > open iterator for alias data > > > > > at org.apache.pig.PigServer.openIterator(PigServer.java:754) > > > > > at > > > > > > > > > org.apache.pig.tools.grunt.GruntParser.processDump(GruntParser.java:612) > > > > > at > > > > > > > > > > > > > > > > > > > > org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:303) > > > > > at > > > > > > > > > > > > > > > > > > > > org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:165) > > > > > at > > > > > > > > > > > > > > > > > > > > org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:141) > > > > > at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:76) > > > > > at org.apache.pig.Main.run(Main.java:465) > > > > > at org.apache.pig.Main.main(Main.java:107) > > > > > Caused by: java.io.IOException: Job terminated with anomalous > status > > > > FAILED > > > > > at org.apache.pig.PigServer.openIterator(PigServer.java:744) > > > > > ... 7 more > > > > > > > > > > Any other more elaborate collection of Pig statements will produce > > > this: > > > > > > > > > > Pig Stack Trace > > > > > --------------- > > > > > ERROR 2244: Job failed, hadoop does not return any error message > > > > > > > > > > org.apache.pig.backend.executionengine.ExecException: ERROR 2244: > Job > > > > > failed, hadoop does not return any error message > > > > > at > > > > > > > > > org.apache.pig.tools.grunt.GruntParser.executeBatch(GruntParser.java:116) > > > > > at > > > > > > > > > > > > > > > > > > > > org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:169) > > > > > at > > > > > > > > > > > > > > > > > > > > org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:141) > > > > > at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:90) > > > > > at org.apache.pig.Main.run(Main.java:510) > > > > > at org.apache.pig.Main.main(Main.java:107) > > > > > > > > > > Where any input file is flagged as unreadable by the following > error > > > > > message > > > > > during execution: > > > > > > > > > > Input(s): > > > > > Failed to read data from "test.csv" > > > > > > > > > > All files are accessible in MapReduce mode via cat, ls etc. I might > > be > > > > > missing something very obvious here since this is the first time > I've > > > > used > > > > > Pig so any help would be greatly appreciated. > > > > > > > > > > - Dimitris > > > > > > > > > > > > > > >
