[
https://issues.apache.org/jira/browse/PIG-443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12633958#action_12633958
]
Pradeep Kamath commented on PIG-443:
------------------------------------
- new patch is broken - does not contain the files under
src/org/apache/pig/backend/local
- The patch gives a Null POinter Exception when used with a hadoop cluster. I
wonder if this is breaking backward compatibility with illustrate in 1.4 - is
it worthwhile supporting reading data from dfs and using for illustrate. At the
least, we need to give an error message if this is not supported.
{noformat}
java -cp pig.jar:<location of hadoopsite.xml> org.apache.pig.Main
grunt> a = load 'st10k' as (name, age, gpa);
grunt> illustrate a;
2008-09-23 15:01:21,948 [main] ERROR org.apache.pig.pen.ExampleGenerator -
Error reading data
2008-09-23 15:01:21,949 [main] ERROR org.apache.pig.tools.grunt.GruntParser -
java.lang.NullPointerException
at
org.apache.pig.pen.EquivalenceClasses.GetEquivalenceClasses(EquivalenceClasses.java:73)
at
org.apache.pig.pen.DerivedDataVisitor.visit(DerivedDataVisitor.java:242)
at org.apache.pig.impl.logicalLayer.LOLoad.visit(LOLoad.java:145)
at org.apache.pig.impl.logicalLayer.LOLoad.visit(LOLoad.java:34)
at
org.apache.pig.impl.plan.DependencyOrderWalker.walk(DependencyOrderWalker.java:68)
at org.apache.pig.impl.plan.PlanVisitor.visit(PlanVisitor.java:51)
at
org.apache.pig.pen.LineageTrimmingVisitor.init(LineageTrimmingVisitor.java:98)
at
org.apache.pig.pen.LineageTrimmingVisitor.<init>(LineageTrimmingVisitor.java:90)
at
org.apache.pig.pen.ExampleGenerator.getExamples(ExampleGenerator.java:100)
at org.apache.pig.PigServer.getExamples(PigServer.java:627)
at
org.apache.pig.tools.grunt.GruntParser.processIllustrate(GruntParser.java:278)
at
org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:183)
at
org.apache.pig.tools.grunt.GruntParser.parseContOnError(GruntParser.java:93)
at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:58)
at org.apache.pig.Main.main(Main.java:282)
2008-09-23 15:01:21,949 [main] ERROR org.apache.pig.tools.grunt.GruntParser -
2008-09-23 15:01:21,949 [main] ERROR org.apache.pig.tools.grunt.GruntParser -
java.lang.NullPointerException
{noformat}
- If schema is not supplied like you said, it gives an exception asking for a
schema. Isn't this too restrictive? Is it possible to sample and generate data
based on real user input without considering the schema?
- In local mode, cogroup seems like cause an issue:
{noformat}
[/tmp/types]java -Xmx1024m -cp pig.jar org.apache.pig.Main
2008-09-23 15:29:38,477 [main] INFO
org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting to
hadoop file system at: file:///
2008-09-23 15:29:38,689 [main] INFO org.apache.hadoop.metrics.jvm.JvmMetrics -
Initializing JVM Metrics with processName=JobTracker, sessionId=
grunt> a = load '/home/pradeepk/work/pig/simpletest';
grunt> b = load '/home/pradeepk/work/pig/simpletest';
grunt> c = group a by $0, b by $0;
grunt> illustrate c;
2008-09-23 15:29:52,829 [main] ERROR org.apache.pig.pen.AugmentBaseDataVisitor
- Error visiting Cogroup during Augmentation phase of Example Generator! null
2008-09-23 15:29:52,833 [main] ERROR org.apache.pig.pen.ExampleGenerator -
Visitor exception while creating example data Error visiting Cogroup during
Augmentation phase of Example Generator! null
java.lang.NullPointerException
at
org.apache.pig.pen.util.DisplayExamples.MakeArray(DisplayExamples.java:184)
at
org.apache.pig.pen.util.DisplayExamples.PrintTabular(DisplayExamples.java:86)
at
org.apache.pig.pen.util.DisplayExamples.PrintTabular(DisplayExamples.java:80)
at
org.apache.pig.pen.util.DisplayExamples.PrintTabular(DisplayExamples.java:69)
at
org.apache.pig.pen.ExampleGenerator.getExamples(ExampleGenerator.java:138)
at org.apache.pig.PigServer.getExamples(PigServer.java:627)
at
org.apache.pig.tools.grunt.GruntParser.processIllustrate(GruntParser.java:278)
at
org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:183)
at
org.apache.pig.tools.grunt.GruntParser.parseContOnError(GruntParser.java:93)
at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:58)
at org.apache.pig.Main.main(Main.java:282)
java.lang.NullPointerException
at
org.apache.pig.pen.util.DisplayExamples.MakeArray(DisplayExamples.java:184)
at
org.apache.pig.pen.util.DisplayExamples.PrintTabular(DisplayExamples.java:86)
at
org.apache.pig.pen.util.DisplayExamples.PrintTabular(DisplayExamples.java:80)
at
org.apache.pig.pen.util.DisplayExamples.PrintTabular(DisplayExamples.java:69)
at
org.apache.pig.pen.ExampleGenerator.getExamples(ExampleGenerator.java:138)
at org.apache.pig.PigServer.getExamples(PigServer.java:627)
at
org.apache.pig.tools.grunt.GruntParser.processIllustrate(GruntParser.java:278)
at
org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:183)
at
org.apache.pig.tools.grunt.GruntParser.parseContOnError(GruntParser.java:93)
at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:58)
at org.apache.pig.Main.main(Main.java:282)
----------------------------------------------------
| c | group: bytearray | a: bag | b: bag |
----------------------------------------------------
| | a | {(a, 12)} | {(a, 12)} |
----------------------------------------------------
grunt>
{noformat}
> Illustrate for the Types branch
> -------------------------------
>
> Key: PIG-443
> URL: https://issues.apache.org/jira/browse/PIG-443
> Project: Pig
> Issue Type: New Feature
> Affects Versions: types_branch
> Reporter: Shubham Chopra
> Assignee: Shubham Chopra
> Fix For: types_branch
>
> Attachments: pig_types_22_09.patch, pig_types_23_09.patch
>
>
> This implementation of Illustrate uses visitors to traverse the plans.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
