Dmitriy, Appreciate the help. We have tried against trunk and here's the output.
patching file src/org/apache/pig/backend/hadoop/hbase/HBaseTableInputFormat.java patching file src/org/apache/pig/backend/hadoop/hbase/HBaseStorage.java Hunk #6 FAILED at 80. Hunk #7 succeeded at 115 (offset 1 line). Hunk #9 succeeded at 167 (offset 1 line). Hunk #11 succeeded at 253 (offset 1 line). Hunk #13 succeeded at 288 (offset 1 line). Hunk #15 succeeded at 324 (offset 1 line). Hunk #17 succeeded at 377 (offset 1 line). Hunk #19 succeeded at 417 (offset 1 line). Hunk #21 succeeded at 451 (offset 1 line). 1 out of 21 hunks FAILED -- saving rejects to file src/org/apache/pig/backend/hadoop/hbase/HBaseStorage.java.rej patching file test/hbase-site.xml patching file test/org/apache/pig/test/TestHBaseStorage.java Hunk #3 FAILED at 42. 1 out of 3 hunks FAILED -- saving rejects to file test/org/apache/pig/test/TestHBaseStorage.java.rej patching file build.xml Hunk #1 succeeded at 51 (offset 1 line). Hunk #2 FAILED at 199. Hunk #3 succeeded at 587 (offset -83 lines). 1 out of 3 hunks FAILED -- saving rejects to file build.xml.rej patching file ivy.xml Hunk #1 succeeded at 84 (offset -5 lines). patching file ivy/libraries.properties Hunk #1 succeeded at 28 with fuzz 2 (offset -1 lines). patching file src/org/apache/pig/backend/hadoop/executionengine/mapReduceLayer/JobControlCompiler.java Hunk #1 succeeded at 111 (offset 1 line). Hunk #3 succeeded at 139 (offset 1 line). Hunk #4 succeeded at 175 with fuzz 1 (offset -3 lines). Hunk #5 succeeded at 201 (offset 1 line). Hunk #6 succeeded at 229 (offset -3 lines). Hunk #7 succeeded at 270 (offset 1 line). Hunk #8 succeeded at 276 (offset -3 lines). Hunk #9 succeeded at 298 (offset 1 line). Hunk #10 succeeded at 320 (offset -3 lines). Hunk #11 FAILED at 385. Hunk #12 succeeded at 394 with fuzz 1 (offset -2 lines). Hunk #13 succeeded at 402 (offset -4 lines). Hunk #14 succeeded at 418 (offset -2 lines). Hunk #15 succeeded at 430 (offset -4 lines). Hunk #16 succeeded at 457 (offset -2 lines). Hunk #17 FAILED at 484. Hunk #18 succeeded at 484 (offset -13 lines). Hunk #19 succeeded at 514 (offset -2 lines). Hunk #20 succeeded at 539 (offset -19 lines). Hunk #21 succeeded at 564 (offset -2 lines). Hunk #22 succeeded at 566 with fuzz 2 (offset -22 lines). Hunk #23 succeeded at 605 (offset -2 lines). Hunk #24 succeeded at 595 (offset -22 lines). Hunk #25 succeeded at 635 (offset -2 lines). Hunk #26 FAILED at 651. Hunk #27 FAILED at 680. Hunk #28 succeeded at 673 (offset -24 lines). Hunk #29 succeeded at 705 (offset -2 lines). Hunk #30 succeeded at 694 with fuzz 1 (offset -24 lines). Hunk #31 succeeded at 746 (offset -2 lines). Hunk #32 succeeded at 740 (offset -24 lines). Hunk #33 succeeded at 775 (offset -2 lines). Hunk #34 succeeded at 771 (offset -24 lines). Hunk #35 succeeded at 806 (offset -2 lines). Hunk #36 succeeded at 806 (offset -24 lines). Hunk #37 succeeded at 892 (offset -2 lines). Hunk #38 succeeded at 914 (offset -24 lines). Hunk #39 succeeded at 948 (offset -2 lines). Hunk #40 succeeded at 936 (offset -24 lines). Hunk #41 succeeded at 974 (offset -2 lines). Hunk #42 FAILED at 1072. Hunk #43 succeeded at 1093 (offset -31 lines). Hunk #44 succeeded at 1158 (offset -2 lines). Hunk #45 succeeded at 1141 (offset -31 lines). Hunk #46 succeeded at 1221 (offset -2 lines). Hunk #47 succeeded at 1212 (offset -31 lines). Hunk #48 succeeded at 1264 (offset -2 lines). Hunk #49 FAILED at 1290. Hunk #50 FAILED at 1302. Hunk #51 FAILED at 1322. 8 out of 51 hunks FAILED -- saving rejects to file src/org/apache/pig/backend/hadoop/executionengine/mapReduceLayer/JobControlCompiler.java.rej On Tue, Feb 15, 2011 at 3:56 PM, Dmitriy Ryaboy <[email protected]> wrote: > Thought it was against the trunk.. I did make it ignore whitespace changes, > maybe there's a problem there. I'll email you another patch off-list, let's > see if we can get it to work. > > D > > On Tue, Feb 15, 2011 at 2:45 PM, Matt Davies <[email protected]> wrote: > > > Dmitriy, > > > > I saw the patch posted to PIG-1680 from last night. Thanks! We have been > > trying to apply it, and find that the patch can't match some of the > hunks. > > Are you patching against trunk or a certain version? > > > > > > Thanks! > > -Matt > > > > On Mon, Feb 14, 2011 at 4:14 PM, Matt Davies <[email protected]> > wrote: > > > > > You are welcome! We did a filter before the store, and I confirmed that > > > there weren't any nulls. Just something undocumented feature ;) > > > > > > > > > On Mon, Feb 14, 2011 at 3:03 PM, jacob <[email protected]> > > wrote: > > > > > >> Thanks for the mention. Off the top of my head I got these sorts of > > >> errors when trying to store either null records or records with null > > >> fields. What happens if you FILTER out any null values you might have. > > >> Does the problem persist? > > >> > > >> --jacob > > >> @thedatachef > > >> > > >> On Mon, 2011-02-14 at 14:57 -0700, Matt Davies wrote: > > >> > Hey All, > > >> > > > >> > Running into a problem storing data from a pig script storing > results > > >> into > > >> > HBase. > > >> > > > >> > We are getting the following error: > > >> > > > >> > java.lang.NullPointerException > > >> > at > > >> > > > org.apache.hadoop.hbase.mapreduce.TableOutputFormat$TableRecordWriter.write(TableOutputFormat.java:126) > > >> > at > > >> > > > org.apache.hadoop.hbase.mapreduce.TableOutputFormat$TableRecordWriter.write(TableOutputFormat.java:81) > > >> > at > > >> > > > org.apache.pig.backend.hadoop.hbase.HBaseStorage.putNext(HBaseStorage.java:364) > > >> > at > > >> > > > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat$PigRecordWriter.write(PigOutputFormat.java:138) > > >> > at > > >> > > > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat$PigRecordWriter.write(PigOutputFormat.java:97) > > >> > at > > >> > > > org.apache.hadoop.mapred.MapTask$NewDirectOutputCollector.write(MapTask.java:523) > > >> > at > > >> > > > org.apache.hadoop.mapreduce.TaskInputOutputContext.write(TaskInputOutputContext.java:80) > > >> > at > > >> > > > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapOnly$Map.collect(PigMapOnly.java:48) > > >> > at > > >> > > > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.runPipeline(PigMapBase.java:238) > > >> > at > > >> > > > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:231) > > >> > at > > >> > > > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:53) > > >> > at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144) > > >> > at > > org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:639) > > >> > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:315) > > >> > at org.apache.hadoop.mapred.Child$4.run(Child.java:217) > > >> > at java.security.AccessController.doPrivileged(Native Method) > > >> > at javax.security.auth.Subject.doAs(Subject.java:396) > > >> > at > > >> > > > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1063) > > >> > at org.apache.hadoop.mapred.Child.main(Child.java:211) > > >> > > > >> > > > >> > We are using CDH3b3, and HBase 0.90.0 (from Apache direct). We've > > >> > followed the instructions to get pig 0.8.0 to work with CDH3 from > > >> > thedatachef(Thanks!) > > >> > > > >> > > > http://thedatachef.blogspot.com/2011/01/apache-pig-08-with-cloudera-cdh3.html > > >> . > > >> > > > >> > The relevant line from the pig script is below. We've applied the > > >> > patch to get the "-noWAL" working: > > >> > > > >> > STORE links INTO 'p' USING > > >> > org.apache.pig.backend.hadoop.hbase.HBaseStorage('a:t a:t2 > > >> > a:g','-noWAL'); > > >> > > > >> > > > >> > > > >> > Anyone know what could be causing this problem? > > >> > > > >> > > > >> > Thanks in advance, > > >> > > > >> > > > >> > Matt > > >> > > >> > > >> > > > > > >
