[
https://issues.apache.org/jira/browse/HCATALOG-302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13228686#comment-13228686
]
[email protected] commented on HCATALOG-302:
--------------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4322/
-----------------------------------------------------------
Review request for hcatalog, Alan Gates, Francis Liu, and Vandana
Ayyalasomayajula.
Summary
-------
Fixes to get Hbase storagehandler working with Pig.
1)Includes changes to HCatStorer used by Pig. Modified HCatStorer to pass the
same OutputJobInfo in the multiple calls made in Pig to
HCatOutputFormat.setOutput. Else we ended up creating multiple orphaned
transactions which were never committed in zookeeper.
2) Moved stuff from getInputSplits and checkoutputSpecs to configure methods in
storagehandler.
3) Fixed a logic error in hbase record reader.
This addresses bug HCATALOG-302.
https://issues.apache.org/jira/browse/HCATALOG-302
Diffs
-----
http://svn.apache.org/repos/asf/incubator/hcatalog/trunk/storage-handlers/hbase/src/java/org/apache/hcatalog/hbase/HBaseDirectOutputFormat.java
1299921
http://svn.apache.org/repos/asf/incubator/hcatalog/trunk/storage-handlers/hbase/src/java/org/apache/hcatalog/hbase/HBaseHCatStorageHandler.java
1299921
http://svn.apache.org/repos/asf/incubator/hcatalog/trunk/src/java/org/apache/hcatalog/pig/HCatStorer.java
1299921
http://svn.apache.org/repos/asf/incubator/hcatalog/trunk/storage-handlers/hbase/src/java/org/apache/hcatalog/hbase/HBaseBulkOutputFormat.java
1299921
http://svn.apache.org/repos/asf/incubator/hcatalog/trunk/storage-handlers/hbase/src/java/org/apache/hcatalog/hbase/HBaseInputFormat.java
1299921
http://svn.apache.org/repos/asf/incubator/hcatalog/trunk/storage-handlers/hbase/src/java/org/apache/hcatalog/hbase/HbaseSnapshotRecordReader.java
1299921
http://svn.apache.org/repos/asf/incubator/hcatalog/trunk/storage-handlers/hbase/src/test/org/apache/hcatalog/hbase/TestHBaseBulkOutputFormat.java
1299921
http://svn.apache.org/repos/asf/incubator/hcatalog/trunk/storage-handlers/hbase/src/test/org/apache/hcatalog/hbase/TestHBaseDirectOutputFormat.java
1299921
http://svn.apache.org/repos/asf/incubator/hcatalog/trunk/storage-handlers/hbase/src/test/org/apache/hcatalog/hbase/TestHBaseInputFormat.java
1299921
Diff: https://reviews.apache.org/r/4322/diff
Testing
-------
Unit tests and Integration testing done.
Thanks,
Rohini
> unable to write to hbase channel. HBaseHCatStorageHandler class not found
> -------------------------------------------------------------------------
>
> Key: HCATALOG-302
> URL: https://issues.apache.org/jira/browse/HCATALOG-302
> Project: HCatalog
> Issue Type: Bug
> Components: hbase
> Affects Versions: 0.4
> Reporter: David Capwell
> Assignee: Rohini Palaniswamy
>
> This is the pig script:
> PigServer pigServer = PigServerBuilder.create(this.client.getConf());
> pigServer.registerQuery("A = LOAD '"+input+"' USING PigStorage()
> AS(key:chararray, value:chararray);");
> pigServer.registerQuery("STORE A INTO '"+this.getDb().getName() + "." +
> this.getTable().getName()+"' USING org.apache.hcatalog.pig.HCatStorer();");
> Error:
> 2012-03-09 03:04:12,105 WARN org.apache.hadoop.mapred.Child: Error running
> child
> java.io.IOException: Error in loading storage
> handler.org.apache.hcatalog.hbase.HBaseHCatStorageHandler
> at
> org.apache.hcatalog.common.HCatUtil.getStorageHandler(HCatUtil.java:518)
> at
> org.apache.hcatalog.common.HCatUtil.getStorageHandler(HCatUtil.java:474)
> at
> org.apache.hcatalog.mapreduce.HCatBaseOutputFormat.getOutputFormat(HCatBaseOutputFormat.java:77)
> at
> org.apache.hcatalog.mapreduce.HCatOutputFormat.getOutputCommitter(HCatOutputFormat.java:250)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputCommitter.getCommitters(PigOutputCommitter.java:89)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputCommitter.<init>(PigOutputCommitter.java:67)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat.getOutputCommitter(PigOutputFormat.java:278)
> at org.apache.hadoop.mapred.Task.initialize(Task.java:515)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:353)
> at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:396)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1082)
> at org.apache.hadoop.mapred.Child.main(Child.java:249)
> Caused by: java.lang.ClassNotFoundException:
> org.apache.hcatalog.hbase.HBaseHCatStorageHandler
> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:247)
> at
> org.apache.hcatalog.common.HCatUtil.getStorageHandler(HCatUtil.java:512)
> ... 13 more
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
