[
https://issues.apache.org/jira/browse/HBASE-2315?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13400170#comment-13400170
]
Jonathan Hsieh commented on HBASE-2315:
---------------------------------------
@flavio
Li's been inactive for a while so I'd suggest taking HBASE-5937 and making the
WAL an interface. This would allow for experimentation here and with other
alternate wal implementation related JIRA's
HBASE-6116 is about a potentially improving the latency when writing to a
single hdfs file. Normally writes to hdfs get pipelined to 2 other replicas on
2 different data nodes and this incurs some overhead. The intuition behind
that is that for small writes the overhead may cause more latency than the data
transfer and the hope is that by having the client send in a parallel/broadcast
we'd reduce the overhead latency.
HBASE-5699 is about improving throughput and improving latency by writing to
multiple hdfs files. This attacks the scenarios where a data node may be
blocked (gc/swapping/etc) causing all other writes to be blocked. Currently a
region server serves many regions but all of a region server's writes go to a
single WAL files with writes to different regions are intermingled.
Let's say these normally take 10ms but in a gc case it ends up taking 1000ms.
In one scenario with a wal per region, writes to another region could go a
different hdfs file and not get blocked.
In another, we'd have two wal files for the region server and we could detect
that one write was blocking (let's say after 20ms) then the retry against
another set of data nodes. Ideally we've improved our worst case from 1000ms
to 30-40ms.
> BookKeeper for write-ahead logging
> ----------------------------------
>
> Key: HBASE-2315
> URL: https://issues.apache.org/jira/browse/HBASE-2315
> Project: HBase
> Issue Type: New Feature
> Components: regionserver
> Reporter: Flavio Junqueira
> Attachments: HBASE-2315.patch, bookkeeperOverview.pdf,
> zookeeper-dev-bookkeeper.jar
>
>
> BookKeeper, a contrib of the ZooKeeper project, is a fault tolerant and high
> throughput write-ahead logging service. This issue provides an implementation
> of write-ahead logging for hbase using BookKeeper. Apart from expected
> throughput improvements, BookKeeper also has stronger durability guarantees
> compared to the implementation currently used by hbase.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
