[
https://issues.apache.org/jira/browse/PIG-4635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14907080#comment-14907080
]
Rohini Palaniswamy commented on PIG-4635:
-----------------------------------------
Daniel,
{code}
public boolean isVertexGroup() {
- return vertexGroupInfo != null;
+ return vertexGroupMembers != null;
}
{code}
vertexGroupMembers is always set for union operators. This change would not be
right as vertexGroupMembers will not be null even if it was not optimized as a
vertexgroup in UnionOptimizer or if UnionOptimizer was turned off. You should
not require the changes in TezPrinter.java as well. If needed for
PigGraceShuffleVertexManager we can maintain another boolean variable
isVertexGroup when setVertexGroupInfo is called which will be serialized.
- Can you also add assert for contents of the output in the test case? Would be
good to validate that considering there are very few test cases which will hit
PigGraceShuffleVertexManager code.
> NPE while running pig script in tez mode( pig 0.15 with tez 0.7)
> ----------------------------------------------------------------
>
> Key: PIG-4635
> URL: https://issues.apache.org/jira/browse/PIG-4635
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.15.0
> Environment: Pig: 0.15
> Tez: 0.7
> Reporter: Sachin Sabbarwal
> Assignee: Daniel Dai
> Fix For: 0.16.0, 0.15.1
>
> Attachments: PIG-4635-1.patch, debug.patch, main.pig, properties,
> tez-site.xml
>
>
> I was trying to run (attached) pig script on pig 0.15 in tez mode(with tez
> 0.7).
> I got following NullPointerException. I tried to send a mail to tez user
> mailing list and was asked to file a jira for this.
> Here is the NPE:
> 5 org.apache.pig.impl.plan.VisitorException: ERROR 0:
> java.lang.NullPointerException
> 6 at
> org.apache.pig.backend.hadoop.executionengine.tez.plan.optimizer.ParallelismSetter.visitTezOp(ParallelismSetter.java:201)
> 7 at
> org.apache.pig.backend.hadoop.executionengine.tez.plan.TezOperator.visit(TezOperator.java:246)
> 8 at
> org.apache.pig.backend.hadoop.executionengine.tez.plan.TezOperator.visit(TezOperator.java:53)
> 9 at
> org.apache.pig.impl.plan.DependencyOrderWalker.walk(DependencyOrderWalker.java:87)
> 10 at org.apache.pig.impl.plan.PlanVisitor.visit(PlanVisitor.java:46)
> 11 at
> org.apache.pig.backend.hadoop.executionengine.tez.TezLauncher.processLoadAndParallelism(TezLauncher.java:449)
> 12 at
> org.apache.pig.backend.hadoop.executionengine.tez.TezLauncher.launchPig(TezLauncher.java:163)
> 13 at
> org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.launchPig(HExecutionEngine.java:304)
> 14 at org.apache.pig.PigServer.launchPlan(PigServer.java:1390)
> 15 at
> org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:1375)
> 16 at org.apache.pig.PigServer.execute(PigServer.java:1364)
> 17 at org.apache.pig.PigServer.executeBatch(PigServer.java:415)
> 18 at org.apache.pig.PigServer.executeBatch(PigServer.java:398)
> 19 at
> org.apache.pig.tools.grunt.GruntParser.executeBatch(GruntParser.java:171)
> 20 at
> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:234)
> 21 at
> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:205)
> 22 at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:81)
> 23 at org.apache.pig.Main.run(Main.java:502)
> 24 at org.apache.pig.Main.main(Main.java:177)
> 25 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 26 at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> 27 at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> 28 at java.lang.reflect.Method.invoke(Method.java:606)
> 29 at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
> 30 Caused by: java.lang.NullPointerException
> 31 at
> org.apache.pig.backend.hadoop.executionengine.tez.plan.optimizer.TezOperDependencyParallelismEstimator.estimateParallelism(TezOperDependencyParallelismEstimator.java:114)
> 32 at
> org.apache.pig.backend.hadoop.executionengine.tez.plan.optimizer.ParallelismSetter.visitTezOp(ParallelismSetter.java:138)
> 33 ... 23 more
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
