Awesome, thanks. We'll put that into 0.10.. and seems like we should backport it to 0.9 (.3), too.
D On Sun, Mar 25, 2012 at 7:36 PM, Nick Pinckernell <[email protected]> wrote: > Dmitriy, I followed up with the HBase user group which lead me to track > down the reason my region observer coprocessor was not firing. It was due > to a timestamp 'ts' in the method putNext(Tuple t) in HBaseStorage.java > which was a mismatch from the timestamp that was being compared in the > org.apache.hadoop.hbase.client.Put class on method has(byte [] family, > byte [] qualifier, long ts, byte [] value, boolean ignoreTS, boolean > ignoreValue) at: > > if (Arrays.equals(kv.getFamily(), family) && > Arrays.equals(kv.getQualifier(), qualifier) > && kv.getTimestamp() == ts) { > > The fix is simple, just removing the 'ts' long, and invoke the other > Put.add() method which does not use the versioning timestamp. > > Jira 2615 created with patch. > > On Sat, Mar 24, 2012 at 11:41 AM, Nick Pinckernell <[email protected]> wrote: > >> Sure, I will bring it up with the HBase folks and then open a Jira if >> necessary. >> Thanks! >> >> >> On Fri, Mar 23, 2012 at 6:59 PM, Dmitriy Ryaboy <[email protected]>wrote: >> >>> HBaseStorage was implemented for 0.90, which didn't have coprocessors, so >>> I >>> suppose this isn't terribly surprising. Do you mind opening a Jira, and >>> perhaps pointing it out on the HBase dev list? We aren't using any special >>> HBase apis in Pig, so it should just work.. maybe they deprecated >>> something? >>> >>> D >>> >>> On Thu, Mar 22, 2012 at 10:54 PM, Nick <[email protected]> wrote: >>> >>> > I'm having a possible issue with a simple pig load that writes to an >>> HBase >>> > table. The issue is that when I run the test pig script it does not >>> invoke >>> > the region observer coprocessor on the table. I have verified that my >>> > coprocessor executes when I use the HBase client API to do a simple >>> put(). >>> > >>> > Simple pig script is as follows (test.pig): >>> > register /dev/hbase-0.92.0/hbase-0.92.0.jar; >>> > register /dev/hbase-0.92.0/lib/zookeeper-3.4.2.jar; >>> > register /dev/hbase-0.92.0/lib/guava-r09.jar; >>> > A = load '/tmp/testdata.csv' using PigStorage(','); >>> > store A into 'hbase://test' using >>> > org.apache.pig.backend.hadoop.hbase.HBaseStorage ('f:t'); >>> > >>> > Using the following environment variables and command: >>> > export HADOOP_HOME=/dev/hadoop-1.0.0 >>> > export PIG_CLASSPATH=/dev/hadoop-1.0.0/conf >>> > export HBASE_HOME=/dev/hbase-0.92.0/ >>> > export PIG_CLASSPATH="`${HBASE_HOME}/bin/hbase >>> classpath`:$PIG_CLASSPATH" >>> > /dev/pig-0.9.2/bin/pig -x local -f test.pig >>> > >>> > I have also tried 'pig -x mapreduce' and it still does not seem to >>> invoke >>> > the coprocessor. After looking through the HBaseStorage class it >>> appears >>> > that the RecordWriter is getting HBase Put objects and that ultimately >>> > those are getting flushed so I'm not sure why the coprocessor is not >>> > executing. >>> > >>> > Is this by design, or am I missing something about how the output from >>> the >>> > pig job is being loaded into the HBase table? >>> > Thank you >>> > >>> >> >>
