[
https://issues.apache.org/jira/browse/HCATALOG-302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13228912#comment-13228912
]
[email protected] commented on HCATALOG-302:
--------------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4322/#review5917
-----------------------------------------------------------
http://svn.apache.org/repos/asf/incubator/hcatalog/trunk/src/java/org/apache/hcatalog/pig/HCatStorer.java
<https://reviews.apache.org/r/4322/#comment12875>
nitpick: use getProperty isntead so you don't have to cast it to string.
http://svn.apache.org/repos/asf/incubator/hcatalog/trunk/storage-handlers/hbase/src/java/org/apache/hcatalog/hbase/HBaseHCatStorageHandler.java
<https://reviews.apache.org/r/4322/#comment12876>
nitpick: since you're changing this can you fix the formatting? :-)
http://svn.apache.org/repos/asf/incubator/hcatalog/trunk/storage-handlers/hbase/src/java/org/apache/hcatalog/hbase/HBaseHCatStorageHandler.java
<https://reviews.apache.org/r/4322/#comment12881>
I'd rather add it directly to JobConf. That way it's less of a hack and
less dependent on an unplanned behavior.
http://svn.apache.org/repos/asf/incubator/hcatalog/trunk/storage-handlers/hbase/src/java/org/apache/hcatalog/hbase/HBaseHCatStorageHandler.java
<https://reviews.apache.org/r/4322/#comment12880>
add a TODO to remove this once HCAT-308 is fixed.
http://svn.apache.org/repos/asf/incubator/hcatalog/trunk/storage-handlers/hbase/src/java/org/apache/hcatalog/hbase/HBaseHCatStorageHandler.java
<https://reviews.apache.org/r/4322/#comment12877>
Does this work without the foolproof code? Let's try to contain the bug
with pig in HCatStorer. It'll just add more complexity.
http://svn.apache.org/repos/asf/incubator/hcatalog/trunk/storage-handlers/hbase/src/java/org/apache/hcatalog/hbase/HBaseHCatStorageHandler.java
<https://reviews.apache.org/r/4322/#comment12879>
shouldn't you add it to copy? for the output you don't need the hack of
changing conf do you?
- Francis
On 2012-03-14 01:07:44, Rohini Palaniswamy wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/4322/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2012-03-14 01:07:44)
bq.
bq.
bq. Review request for hcatalog, Alan Gates, Francis Liu, and Vandana
Ayyalasomayajula.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. Fixes to get Hbase storagehandler working with Pig.
bq.
bq.
bq. 1)Includes changes to HCatStorer used by Pig. Modified HCatStorer to pass
the same OutputJobInfo in the multiple calls made in Pig to
HCatOutputFormat.setOutput. Else we ended up creating multiple orphaned
transactions which were never committed in zookeeper.
bq. 2) Moved stuff from getInputSplits and checkoutputSpecs to configure
methods in storagehandler.
bq. 3) Fixed a logic error in hbase record reader.
bq.
bq.
bq. This addresses bug HCATALOG-302.
bq. https://issues.apache.org/jira/browse/HCATALOG-302
bq.
bq.
bq. Diffs
bq. -----
bq.
bq.
http://svn.apache.org/repos/asf/incubator/hcatalog/trunk/src/java/org/apache/hcatalog/pig/HCatStorer.java
1299921
bq.
http://svn.apache.org/repos/asf/incubator/hcatalog/trunk/storage-handlers/hbase/src/java/org/apache/hcatalog/hbase/HBaseBulkOutputFormat.java
1299921
bq.
http://svn.apache.org/repos/asf/incubator/hcatalog/trunk/storage-handlers/hbase/src/java/org/apache/hcatalog/hbase/HBaseDirectOutputFormat.java
1299921
bq.
http://svn.apache.org/repos/asf/incubator/hcatalog/trunk/storage-handlers/hbase/src/java/org/apache/hcatalog/hbase/HBaseHCatStorageHandler.java
1299921
bq.
http://svn.apache.org/repos/asf/incubator/hcatalog/trunk/storage-handlers/hbase/src/java/org/apache/hcatalog/hbase/HBaseInputFormat.java
1299921
bq.
http://svn.apache.org/repos/asf/incubator/hcatalog/trunk/storage-handlers/hbase/src/java/org/apache/hcatalog/hbase/HbaseSnapshotRecordReader.java
1299921
bq.
http://svn.apache.org/repos/asf/incubator/hcatalog/trunk/storage-handlers/hbase/src/test/org/apache/hcatalog/hbase/TestHBaseBulkOutputFormat.java
1299921
bq.
http://svn.apache.org/repos/asf/incubator/hcatalog/trunk/storage-handlers/hbase/src/test/org/apache/hcatalog/hbase/TestHBaseDirectOutputFormat.java
1299921
bq.
http://svn.apache.org/repos/asf/incubator/hcatalog/trunk/storage-handlers/hbase/src/test/org/apache/hcatalog/hbase/TestHBaseInputFormat.java
1299921
bq.
bq. Diff: https://reviews.apache.org/r/4322/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq. Unit tests and Integration testing done.
bq.
bq.
bq. Thanks,
bq.
bq. Rohini
bq.
bq.
> unable to write to hbase channel. HBaseHCatStorageHandler class not found
> -------------------------------------------------------------------------
>
> Key: HCATALOG-302
> URL: https://issues.apache.org/jira/browse/HCATALOG-302
> Project: HCatalog
> Issue Type: Bug
> Components: hbase
> Affects Versions: 0.4
> Reporter: David Capwell
> Assignee: Rohini Palaniswamy
> Fix For: 0.4
>
> Attachments: HCATALOG-302-1.patch
>
>
> This is the pig script:
> PigServer pigServer = PigServerBuilder.create(this.client.getConf());
> pigServer.registerQuery("A = LOAD '"+input+"' USING PigStorage()
> AS(key:chararray, value:chararray);");
> pigServer.registerQuery("STORE A INTO '"+this.getDb().getName() + "." +
> this.getTable().getName()+"' USING org.apache.hcatalog.pig.HCatStorer();");
> Error:
> 2012-03-09 03:04:12,105 WARN org.apache.hadoop.mapred.Child: Error running
> child
> java.io.IOException: Error in loading storage
> handler.org.apache.hcatalog.hbase.HBaseHCatStorageHandler
> at
> org.apache.hcatalog.common.HCatUtil.getStorageHandler(HCatUtil.java:518)
> at
> org.apache.hcatalog.common.HCatUtil.getStorageHandler(HCatUtil.java:474)
> at
> org.apache.hcatalog.mapreduce.HCatBaseOutputFormat.getOutputFormat(HCatBaseOutputFormat.java:77)
> at
> org.apache.hcatalog.mapreduce.HCatOutputFormat.getOutputCommitter(HCatOutputFormat.java:250)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputCommitter.getCommitters(PigOutputCommitter.java:89)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputCommitter.<init>(PigOutputCommitter.java:67)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat.getOutputCommitter(PigOutputFormat.java:278)
> at org.apache.hadoop.mapred.Task.initialize(Task.java:515)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:353)
> at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:396)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1082)
> at org.apache.hadoop.mapred.Child.main(Child.java:249)
> Caused by: java.lang.ClassNotFoundException:
> org.apache.hcatalog.hbase.HBaseHCatStorageHandler
> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:247)
> at
> org.apache.hcatalog.common.HCatUtil.getStorageHandler(HCatUtil.java:512)
> ... 13 more
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
