[ https://issues.apache.org/jira/browse/PIG-1526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12894087#action_12894087 ]
Hadoop QA commented on PIG-1526: -------------------------------- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12450900/PIG-1526.patch against trunk revision 980276. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 9 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. -1 contrib tests. The patch failed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Pig-Patch-h8.grid.sp2.yahoo.net/367/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Pig-Patch-h8.grid.sp2.yahoo.net/367/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: http://hudson.zones.apache.org/hudson/job/Pig-Patch-h8.grid.sp2.yahoo.net/367/console This message is automatically generated. > HiveColumnarLoader Partitioning Support > --------------------------------------- > > Key: PIG-1526 > URL: https://issues.apache.org/jira/browse/PIG-1526 > Project: Pig > Issue Type: Improvement > Affects Versions: 0.8.0 > Reporter: Gerrit Jansen van Vuuren > Assignee: Gerrit Jansen van Vuuren > Priority: Minor > Fix For: 0.8.0 > > Attachments: PIG-1526.patch > > > I've made allot improvements on the HiveColumnarLoader: > -> Added support for LoadMetadata and data path Partitioning > -> Improved and simplefied column loading > Data Path Partitioning: > Hive stores partitions as folders like to > /mytable/partition1=[value]/partition2=[value]. That is the table mytable > contains 2 partitions [partition1, partition2]. > The HiveColumnarLoader will scan the inputpath /mytable and add to the > PigSchema the columns partition2 and partition2. > These columns can then be used in filtering. > For example: We've got year,month,day,hour partitions in our data uploads. > So a table might look like mytable/year=2010/month=02/day=01. > Loading with the HiveColumnarLoader allows our pig scripts do filter by date > using the standard pig Filter operator. > I've added 2 classes for this: > -> PathPartitioner > -> PathPartitionHelper > These classes are not hive dependent and could be used by any other loader > that wants to support partitioning and helps with implementing the > LoadMetadata interface. > For this reason I though it best to put it into the package > org.apache.pig.piggybank.storage.partition. > What would be nice is in the future have the PigStorage also use these 2 > classes to provide automatic path partitioning support. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.