[
https://issues.apache.org/jira/browse/HBASE-10278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13868551#comment-13868551
]
Himanshu Vashishtha commented on HBASE-10278:
---------------------------------------------
Thanks for reviewing the doc Liang (and sorry about this delay in replying).
True, to handle longer outages (rack down, for e.g.), we could tune the
switching policy to avoid tiny log files (for e.g., take number of append ops
since last switched, etc).
Yes, 300ms is the avg time (total time for 1k ops was about 30sec). I didn't
really dig into it to know the why it is better than as compared to 1 file
scenario, but for me the interesting bit was about 568/1000 ops took more than
a sec.
Yes, The replication needs to handle two opened files. To get minimal impact on
Replication, I am thinking of adding a separate ReplicationSource thread for
the second WAL. But, I still need to look into it more if there is a better way
to achieve this.
> Provide better write predictability
> -----------------------------------
>
> Key: HBASE-10278
> URL: https://issues.apache.org/jira/browse/HBASE-10278
> Project: HBase
> Issue Type: New Feature
> Reporter: Himanshu Vashishtha
> Assignee: Himanshu Vashishtha
> Attachments: Multiwaldesigndoc.pdf
>
>
> Currently, HBase has one WAL per region server.
> Whenever there is any latency in the write pipeline (due to whatever reasons
> such as n/w blip, a node in the pipeline having a bad disk, etc), the overall
> write latency suffers.
> Jonathan Hsieh and I analyzed various approaches to tackle this issue. We
> also looked at HBASE-5699, which talks about adding concurrent multi WALs.
> Along with performance numbers, we also focussed on design simplicity,
> minimum impact on MTTR & Replication, and compatibility with 0.96 and 0.98.
> Considering all these parameters, we propose a new HLog implementation with
> WAL Switching functionality.
> Please find attached the design doc for the same. It introduces the WAL
> Switching feature, and experiments/results of a prototype implementation,
> showing the benefits of this feature.
> The second goal of this work is to serve as a building block for concurrent
> multiple WALs feature.
> Please review the doc.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
