Hi, Ayon, Which version of Pig are you using? Can you share your script? Thanks, Daniel
On Mon, Dec 5, 2011 at 6:10 PM, Ayon Sinha <[email protected]> wrote: > Thanks Jonathan, that works but now I want to know why. I'm guessing I'm > loosing some optimization. The thing is the previous version of the script > was working fine without the -t ColumnMapKeyPrune. All I did was added a > new column to my tuple that needs to get flattened. > > Anyone know what's going on inside? > > -Ayon > See My Photos on Flickr > Also check out my Blog for answers to commonly asked questions. > > > > ________________________________ > From: Jonathan Coveney <[email protected]> > To: [email protected]; Ayon Sinha <[email protected]> > Sent: Monday, December 5, 2011 5:58 PM > Subject: Re: Pig 0.9.1's not so helpful error message > > There are some known issues with ColumnMapKeyPrune AFAIK, so instead of > doing: > > pig script.pig > > do > > pig -t ColumnMapKeyPrune script.pig > > does that work? > > 2011/12/5 Ayon Sinha <[email protected]> > > > This is all I got during the run. What does it mean? Now I have to debug > > line by line. Any hint is greatly appreciated. > > > > 2011-12-05 17:28:01,854 [main] ERROR org.apache.pig.tools.grunt.Grunt - > > ERROR 2000: Error processing rule ColumnMapKeyPrune. Try -t > > ColumnMapKeyPrune > > Details at logfile: /home/ayon/pig_1323134879161.log > > $ cat pig_1323134879161.log > > Pig Stack Trace > > --------------- > > ERROR 2000: Error processing rule ColumnMapKeyPrune. Try -t > > ColumnMapKeyPrune > > > > org.apache.pig.impl.logicalLayer.FrontendException: ERROR 2000: Error > > processing rule ColumnMapKeyPrune. Try -t ColumnMapKeyPrune > > at > > > org.apache.pig.newplan.optimizer.PlanOptimizer.optimize(PlanOptimizer.java:122) > > at > > > org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.compile(HExecutionEngine.java:281) > > at org.apache.pig.PigServer.compilePp(PigServer.java:1360) > > at > org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:1297) > > at org.apache.pig.PigServer.execute(PigServer.java:1286) > > at org.apache.pig.PigServer.executeBatch(PigServer.java:360) > > at > > org.apache.pig.tools.grunt.GruntParser.executeBatch(GruntParser.java:131) > > at > > > org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:192) > > at > > > org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:164) > > at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:81) > > at org.apache.pig.Main.run(Main.java:553) > > at org.apache.pig.Main.main(Main.java:108) > > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > > at > > > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) > > at > > > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > > at java.lang.reflect.Method.invoke(Method.java:597) > > at org.apache.hadoop.util.RunJar.main(RunJar.java:156) > > Caused by: java.util.ConcurrentModificationException > > at > java.util.AbstractList$Itr.checkForComodification(AbstractList.java:372) > > at java.util.AbstractList$Itr.next(AbstractList.java:343) > > at > > > org.apache.pig.newplan.logical.rules.ColumnPruneVisitor.removeSubTree(ColumnPruneVisitor.java:441) > > at > > > org.apache.pig.newplan.logical.rules.ColumnPruneVisitor.visit(ColumnPruneVisitor.java:421) > > at > > > org.apache.pig.newplan.logical.relational.LOForEach.accept(LOForEach.java:74) > > at > > > org.apache.pig.newplan.ReverseDependencyOrderWalker.walk(ReverseDependencyOrderWalker.java:70) > > at org.apache.pig.newplan.PlanVisitor.visit(PlanVisitor.java:50) > > at > > > org.apache.pig.newplan.logical.rules.ColumnMapKeyPrune$ColumnMapKeyPruneTransformer.transform(ColumnMapKeyPrune.java:141) > > at > > > org.apache.pig.newplan.optimizer.PlanOptimizer.optimize(PlanOptimizer.java:110) > > ... 16 more > > > > > ================================================================================ > > > > -Ayon > > See My Photos on Flickr > > Also check out my Blog for answers to commonly asked questions. > >
