[
https://issues.apache.org/jira/browse/HCATALOG-73?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13122946#comment-13122946
]
[email protected] commented on HCATALOG-73:
-------------------------------------------------------
bq. On 2011-10-05 23:03:10, David Capwell wrote:
bq. >
storage-drivers/hbase/src/java/org/apache/hcatalog/hbase/HBaseDirectOutputFormat.java,
line 40
bq. > <https://reviews.apache.org/r/2060/diff/3/?file=48147#file48147line40>
bq. >
bq. > Are you not extending TableOutputformat because of future changes
like disabling WAL and cacheing?
bq. >
bq. > http://hbase.apache.org/book/perf.writing.html#def.log.flush
bq. >
bq. > Deferred WAL writes seems to be table only and stuff like cacheing
can be controlled via the config object itself.
bq.
bq. Francis Liu wrote:
bq. Disabling WAL is what I'd like to try but deferred WAL sounds good
too. In general having our own OutputFormat keeps us flexible. Deferred is a
table config tho we need it per session since we will most likely be support
some sort of point write mechanism.
Disable WAL doesn't sound like a good idea cause it could lead to data loss.
If the user doesn't care (temp output maybe?) then I don't see an issue in
letting them opt-in.
"In general having our own OutputFormat keeps us flexible" Agreed.
- David
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/2060/#review2371
-----------------------------------------------------------
On 2011-10-07 10:22:10, Francis Liu wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/2060/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2011-10-07 10:22:10)
bq.
bq.
bq. Review request for hcatalog and Vandana Ayyalasomayajula.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. hbase direct output storage driver as describe here:
bq.
bq.
https://cwiki.apache.org/confluence/display/HCATALOG/HBase+Output+Storage+Driver+-+Design
bq.
bq.
bq. This addresses bug hcatalog-73.
bq. https://issues.apache.org/jira/browse/hcatalog-73
bq.
bq.
bq. Diffs
bq. -----
bq.
bq.
storage-drivers/hbase/src/java/org/apache/hcatalog/hbase/HBaseDirectOutputFormat.java
PRE-CREATION
bq.
storage-drivers/hbase/src/java/org/apache/hcatalog/hbase/HBaseDirectOutputStorageDriver.java
PRE-CREATION
bq.
storage-drivers/hbase/src/test/org/apache/hcatalog/hbase/ManyMiniCluster.java
PRE-CREATION
bq.
storage-drivers/hbase/src/test/org/apache/hcatalog/hbase/SkeletonHBaseTest.java
PRE-CREATION
bq.
storage-drivers/hbase/src/test/org/apache/hcatalog/hbase/TestHBaseDirectOutputStorageDriver.java
PRE-CREATION
bq.
bq. Diff: https://reviews.apache.org/r/2060/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq. added a unit mini cluster test framework and unit tests.
bq.
bq. since this is a sub-module of hcatalog parent tests shouldn't be affected.
bq.
bq.
bq. Thanks,
bq.
bq. Francis
bq.
bq.
> Output Storage Driver for HBase (Direct PUTs)
> ---------------------------------------------
>
> Key: HCATALOG-73
> URL: https://issues.apache.org/jira/browse/HCATALOG-73
> Project: HCatalog
> Issue Type: Sub-task
> Affects Versions: 0.3
> Reporter: Francis Liu
> Assignee: Francis Liu
> Labels: HBASE
> Attachments: hbase_storage_driver.patch,
> hbase_storage_driver_rev2.patch, hbase_storage_driver_rev3.patch,
> hbase_storage_driver_rev4.patch, hbase_storage_driver_rev5.patch
>
>
> This is the first variation for an HBase Output Storage Driver. Which will be
> doing individual PUTs for each row. The next iteration will involve the use
> of bulk imports.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
