[
https://issues.apache.org/jira/browse/HBASE-10278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14249181#comment-14249181
]
stack commented on HBASE-10278:
-------------------------------
[~busbey] Could do the 0.89fb tack first, before this.
How would you implement this in new regime [~busbey]? It changes FSHLog. You'd
do a derivative or decorated FSHLog, SwitchingFSHLog? Can get rid of stuff
like the 'enabled' flag and checks.
Looking at last patch, pity we couldn't switch to the other writer when rolling
log on current writer (given rolling takes a while). Looks like this was a
consideration: "2241 * NOTE: Don't switch if there is a ongoing log
roll. Most likely, this could be a redundant
2242 * step."
Patch is worth a study. Pity has to be a syncmonitor but not sure how else
you'd do it.
On keeping around edits, one implementation, rather than append the WAL
directly as we do now, instead, kept the edits in a single list and then did
bulk appends (IIRC, no advantage doing bulk append over single appends). Edits
stayed in List until syncs came back to say it was ok let them go. IIRC, it
was not that much slower. It was a little more involved (was easier just doing
the WAL append immediately since then we were done) but it might be worth
considering having a single list of all outstanding edits on other side of the
ring buffer as store for edits in flight (downside would be extra thread
coordination)
> Provide better write predictability
> -----------------------------------
>
> Key: HBASE-10278
> URL: https://issues.apache.org/jira/browse/HBASE-10278
> Project: HBase
> Issue Type: New Feature
> Components: wal
> Reporter: Himanshu Vashishtha
> Assignee: Himanshu Vashishtha
> Attachments: 10278-trunk-v2.1.patch, 10278-trunk-v2.1.patch,
> 10278-wip-1.1.patch, Multiwaldesigndoc.pdf, SwitchWriterFlow.pptx
>
>
> Currently, HBase has one WAL per region server.
> Whenever there is any latency in the write pipeline (due to whatever reasons
> such as n/w blip, a node in the pipeline having a bad disk, etc), the overall
> write latency suffers.
> Jonathan Hsieh and I analyzed various approaches to tackle this issue. We
> also looked at HBASE-5699, which talks about adding concurrent multi WALs.
> Along with performance numbers, we also focussed on design simplicity,
> minimum impact on MTTR & Replication, and compatibility with 0.96 and 0.98.
> Considering all these parameters, we propose a new HLog implementation with
> WAL Switching functionality.
> Please find attached the design doc for the same. It introduces the WAL
> Switching feature, and experiments/results of a prototype implementation,
> showing the benefits of this feature.
> The second goal of this work is to serve as a building block for concurrent
> multiple WALs feature.
> Please review the doc.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
