[jira] Updated: (PIG-1153) [zebra] spliting columns at different levels in a complex record column into different column groups throws exception
[ https://issues.apache.org/jira/browse/PIG-1153?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yan Zhou updated PIG-1153: -- Status: Patch Available (was: Open) the single test failure appearts to be a test env issue; resubmitting now. > [zebra] spliting columns at different levels in a complex record column into > different column groups throws exception > - > > Key: PIG-1153 > URL: https://issues.apache.org/jira/browse/PIG-1153 > Project: Pig > Issue Type: Bug > Components: impl >Affects Versions: 0.7.0 >Reporter: Xuefu Zhang >Assignee: Yan Zhou > Attachments: PIG-1153.patch > > > The following code sample: > String strSch = "r1:record(f1:int, f2:int), r2:record(f5:int, > r3:record(f3:float, f4))"; > String strStorage = "[r1.f1, r2.r3.f3, r2.f5]; [r1.f2, r2.r3.f4]"; > Partition p = new Partition(schema.toString(), strStorage, null); > gives the following exception: > org.apache.hadoop.zebra.parser.ParseException: Different Split Types Set > on the same field: r2.f5 -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-1153) [zebra] spliting columns at different levels in a complex record column into different column groups throws exception
[ https://issues.apache.org/jira/browse/PIG-1153?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yan Zhou updated PIG-1153: -- Status: Open (was: Patch Available) > [zebra] spliting columns at different levels in a complex record column into > different column groups throws exception > - > > Key: PIG-1153 > URL: https://issues.apache.org/jira/browse/PIG-1153 > Project: Pig > Issue Type: Bug > Components: impl >Affects Versions: 0.7.0 >Reporter: Xuefu Zhang >Assignee: Yan Zhou > Attachments: PIG-1153.patch > > > The following code sample: > String strSch = "r1:record(f1:int, f2:int), r2:record(f5:int, > r3:record(f3:float, f4))"; > String strStorage = "[r1.f1, r2.r3.f3, r2.f5]; [r1.f2, r2.r3.f4]"; > Partition p = new Partition(schema.toString(), strStorage, null); > gives the following exception: > org.apache.hadoop.zebra.parser.ParseException: Different Split Types Set > on the same field: r2.f5 -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-1153) [zebra] spliting columns at different levels in a complex record column into different column groups throws exception
[ https://issues.apache.org/jira/browse/PIG-1153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12792883#action_12792883 ] Hadoop QA commented on PIG-1153: -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12428513/PIG-1153.patch against trunk revision 892416. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 2 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Pig-Patch-h8.grid.sp2.yahoo.net/147/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Pig-Patch-h8.grid.sp2.yahoo.net/147/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: http://hudson.zones.apache.org/hudson/job/Pig-Patch-h8.grid.sp2.yahoo.net/147/console This message is automatically generated. > [zebra] spliting columns at different levels in a complex record column into > different column groups throws exception > - > > Key: PIG-1153 > URL: https://issues.apache.org/jira/browse/PIG-1153 > Project: Pig > Issue Type: Bug > Components: impl >Affects Versions: 0.7.0 >Reporter: Xuefu Zhang >Assignee: Yan Zhou > Attachments: PIG-1153.patch > > > The following code sample: > String strSch = "r1:record(f1:int, f2:int), r2:record(f5:int, > r3:record(f3:float, f4))"; > String strStorage = "[r1.f1, r2.r3.f3, r2.f5]; [r1.f2, r2.r3.f4]"; > Partition p = new Partition(schema.toString(), strStorage, null); > gives the following exception: > org.apache.hadoop.zebra.parser.ParseException: Different Split Types Set > on the same field: r2.f5 -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-1146) Inconsistent column pruning in LOUnion
[
https://issues.apache.org/jira/browse/PIG-1146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12792842#action_12792842
]
Hadoop QA commented on PIG-1146:
+1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12428510/PIG-1146-1.patch
against trunk revision 892416.
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 3 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac
compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs warnings.
+1 release audit. The applied patch does not increase the total number of
release audit warnings.
+1 core tests. The patch passed core unit tests.
+1 contrib tests. The patch passed contrib unit tests.
Test results:
http://hudson.zones.apache.org/hudson/job/Pig-Patch-h8.grid.sp2.yahoo.net/146/testReport/
Findbugs warnings:
http://hudson.zones.apache.org/hudson/job/Pig-Patch-h8.grid.sp2.yahoo.net/146/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Console output:
http://hudson.zones.apache.org/hudson/job/Pig-Patch-h8.grid.sp2.yahoo.net/146/console
This message is automatically generated.
> Inconsistent column pruning in LOUnion
> --
>
> Key: PIG-1146
> URL: https://issues.apache.org/jira/browse/PIG-1146
> Project: Pig
> Issue Type: Bug
> Components: impl
>Affects Versions: 0.6.0
>Reporter: Daniel Dai
>Assignee: Daniel Dai
> Fix For: 0.7.0
>
> Attachments: PIG-1146-1.patch
>
>
> This happens when we do a union on two relations, if one column comes from a
> loader, the other matching column comes from a constant, and this column get
> pruned. We prune for the one from loader and did not prune the constant. Thus
> leaves union an inconsistent state. Here is a script:
> {code}
> a = load '1.txt' as (a0, a1:chararray, a2);
> b = load '2.txt' as (b0, b2);
> c = foreach b generate b0, 'hello', b2;
> d = union a, c;
> e = foreach d generate $0, $2;
> dump e;
> {code}
> 1.txt:
> {code}
> ulysses thompson64 1.90
> katie carson25 3.65
> {code}
> 2.txt:
> {code}
> luke king 0.73
> holly davidson 2.43
> {code}
> expected output:
> (ulysses thompson,1.90)
> (katie carson,3.65)
> (luke king,0.73)
> (holly davidson,2.43)
> real output:
> (ulysses thompson,)
> (katie carson,)
> (luke king,0.73)
> (holly davidson,2.43)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-1159) merge join right side table does not support comma seperated paths
[
https://issues.apache.org/jira/browse/PIG-1159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12792830#action_12792830
]
Hadoop QA commented on PIG-1159:
+1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12428500/PIG-1159.patch
against trunk revision 892416.
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 3 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac
compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs warnings.
+1 release audit. The applied patch does not increase the total number of
release audit warnings.
+1 core tests. The patch passed core unit tests.
+1 contrib tests. The patch passed contrib unit tests.
Test results:
http://hudson.zones.apache.org/hudson/job/Pig-Patch-h8.grid.sp2.yahoo.net/145/testReport/
Findbugs warnings:
http://hudson.zones.apache.org/hudson/job/Pig-Patch-h8.grid.sp2.yahoo.net/145/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Console output:
http://hudson.zones.apache.org/hudson/job/Pig-Patch-h8.grid.sp2.yahoo.net/145/console
This message is automatically generated.
> merge join right side table does not support comma seperated paths
> --
>
> Key: PIG-1159
> URL: https://issues.apache.org/jira/browse/PIG-1159
> Project: Pig
> Issue Type: Bug
>Affects Versions: 0.6.0
>Reporter: Jing Huang
>Assignee: Richard Ding
> Fix For: 0.7.0
>
> Attachments: PIG-1159.patch
>
>
> For example this is my script:(join_jira1.pig)
> register /grid/0/dev/hadoopqa/jars/zebra.jar;
> --a1 = load '1.txt' as (a:int,
> b:float,c:long,d:double,e:chararray,f:bytearray,r1(f1:chararray,f2:chararray),m1:map[]);
> --a2 = load '2.txt' as (a:int,
> b:float,c:long,d:double,e:chararray,f:bytearray,r1(f1:chararray,f2:chararray),m1:map[]);
> --sort1 = order a1 by a parallel 6;
> --sort2 = order a2 by a parallel 5;
> --store sort1 into 'asort1' using
> org.apache.hadoop.zebra.pig.TableStorer('[a,b,c,d]');
> --store sort2 into 'asort2' using
> org.apache.hadoop.zebra.pig.TableStorer('[a,b,c,d]');
> --store sort1 into 'asort3' using
> org.apache.hadoop.zebra.pig.TableStorer('[a,b,c,d]');
> --store sort2 into 'asort4' using
> org.apache.hadoop.zebra.pig.TableStorer('[a,b,c,d]');
> joinl = LOAD 'asort1,asort2' USING
> org.apache.hadoop.zebra.pig.TableLoader('a,b,c,d', 'sorted');
> joinr = LOAD 'asort3,asort4' USING
> org.apache.hadoop.zebra.pig.TableLoader('a,b,c,d', 'sorted');
> joina = join joinl by a, joinr by a using "merge" ;
> dump joina;
> ==
> here is the log:
> Backend error message
> -
> java.lang.IllegalArgumentException: Pathname
> /user/hadoopqa/asort3,hdfs:/gsbl90380.blue.ygrid.yahoo.com/user/hadoopqa/asort4
> from
> hdfs://gsbl90380.blue.ygrid.yahoo.com/user/hadoopqa/asort3,hdfs:/gsbl90380.blue.ygrid.yahoo.com/user/hadoopqa/asort4
> is not a valid DFS filename.
> at
> org.apache.hadoop.hdfs.DistributedFileSystem.getPathName(DistributedFileSystem.java:158)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:453)
> at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:648)
> at
> org.apache.pig.backend.hadoop.datastorage.HDataStorage.isContainer(HDataStorage.java:203)
> at
> org.apache.pig.backend.hadoop.datastorage.HDataStorage.asElement(HDataStorage.java:131)
> at
> org.apache.pig.backend.hadoop.datastorage.HDataStorage.asElement(HDataStorage.java:147)
> at
> org.apache.pig.impl.io.FileLocalizer.fullPath(FileLocalizer.java:534)
> at org.apache.pig.impl.io.FileLocalizer.open(FileLocalizer.java:338)
> at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POMergeJoin.seekInRightStream(POMergeJoin.java:398)
> at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POMergeJoin.getNext(POMergeJoin.java:184)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.runPipeline(PigMapBase.java:253)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:244)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapOnly$Map.map(PigMapOnly.java:65)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)
> at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:358)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
> at org.apache.hadoop.mapred.Child
