[
https://issues.apache.org/jira/browse/HCATALOG-73?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13121573#comment-13121573
]
[email protected] commented on HCATALOG-73:
-------------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/2060/#review2371
-----------------------------------------------------------
storage-drivers/hbase/src/java/org/apache/hcatalog/hbase/HBaseDirectOutputFormat.java
<https://reviews.apache.org/r/2060/#comment5454>
Are you not extending TableOutputformat because of future changes like
disabling WAL and cacheing?
http://hbase.apache.org/book/perf.writing.html#def.log.flush
Deferred WAL writes seems to be table only and stuff like cacheing can be
controlled via the config object itself.
storage-drivers/hbase/src/test/org/apache/hcatalog/hbase/ManyMiniCluster.java
<https://reviews.apache.org/r/2060/#comment5455>
Almost every place that uses this variable turns it into a File object.
Can we switch this to a File?
storage-drivers/hbase/src/test/org/apache/hcatalog/hbase/ManyMiniCluster.java
<https://reviews.apache.org/r/2060/#comment5456>
Might also need to close the FileSystem. If you look at the HBase test
code itself, it closes it after it stops the DFS cluster.
FileSystem fs = this.dfsCluster.getFileSystem();
if (fs != null) {
fs.close();
}
FileSystem.closeAll();
storage-drivers/hbase/src/test/org/apache/hcatalog/hbase/ManyMiniCluster.java
<https://reviews.apache.org/r/2060/#comment5457>
Can we have this configurable via Configuration or via constructor?
storage-drivers/hbase/src/test/org/apache/hcatalog/hbase/ManyMiniCluster.java
<https://reviews.apache.org/r/2060/#comment5458>
Can you change the File to be
new File(workDir, "logs").getAbsolutePath()?
storage-drivers/hbase/src/test/org/apache/hcatalog/hbase/ManyMiniCluster.java
<https://reviews.apache.org/r/2060/#comment5459>
Why not have HBase use HDFS directly?
- David
On 2011-10-05 16:55:28, Francis Liu wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/2060/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2011-10-05 16:55:28)
bq.
bq.
bq. Review request for hcatalog and Vandana Ayyalasomayajula.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. hbase direct output storage driver as describe here:
bq.
bq.
https://cwiki.apache.org/confluence/display/HCATALOG/HBase+Output+Storage+Driver+-+Design
bq.
bq.
bq. This addresses bug hcatalog-73.
bq. https://issues.apache.org/jira/browse/hcatalog-73
bq.
bq.
bq. Diffs
bq. -----
bq.
bq.
storage-drivers/hbase/src/java/org/apache/hcatalog/hbase/HBaseDirectOutputFormat.java
PRE-CREATION
bq.
storage-drivers/hbase/src/java/org/apache/hcatalog/hbase/HBaseDirectOutputStorageDriver.java
PRE-CREATION
bq.
storage-drivers/hbase/src/test/org/apache/hcatalog/hbase/ManyMiniCluster.java
PRE-CREATION
bq.
storage-drivers/hbase/src/test/org/apache/hcatalog/hbase/SkeletonHBaseTest.java
PRE-CREATION
bq.
storage-drivers/hbase/src/test/org/apache/hcatalog/hbase/TestHBaseDirectOutputStorageDriver.java
PRE-CREATION
bq.
bq. Diff: https://reviews.apache.org/r/2060/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq. added a unit mini cluster test framework and unit tests.
bq.
bq. since this is a sub-module of hcatalog parent tests shouldn't be affected.
bq.
bq.
bq. Thanks,
bq.
bq. Francis
bq.
bq.
> Output Storage Driver for HBase (Direct PUTs)
> ---------------------------------------------
>
> Key: HCATALOG-73
> URL: https://issues.apache.org/jira/browse/HCATALOG-73
> Project: HCatalog
> Issue Type: Sub-task
> Affects Versions: 0.3
> Reporter: Francis Liu
> Assignee: Francis Liu
> Labels: HBASE
> Attachments: hbase_storage_driver.patch,
> hbase_storage_driver_rev2.patch, hbase_storage_driver_rev3.patch,
> hbase_storage_driver_rev4.patch
>
>
> This is the first variation for an HBase Output Storage Driver. Which will be
> doing individual PUTs for each row. The next iteration will involve the use
> of bulk imports.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
