[
https://issues.apache.org/jira/browse/HBASE-12719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey updated HBASE-12719:
--------------------------------
Attachment: HBASE-12719.1.patch.txt
HBASE-12719_comparison_chart.tiff
attaching an implementation as well as a chart showing the perf change going
from disabled WAL to various amounts of HDFS interactions.
if you look at "HBASE-12719_comparison_chart" you can see the disabled WAL
(which is all noops) in red at the top. The cyan line is FSHLog but w/o any
HDFS interactions. The default FSHLog and the new test driver but with all
interactions enabled are at the bottom; they've drawn over each other so only
one is apparent.
> Add test WAL provider to quantify FSHLog overhead in the absence of HDFS.
> -------------------------------------------------------------------------
>
> Key: HBASE-12719
> URL: https://issues.apache.org/jira/browse/HBASE-12719
> Project: HBase
> Issue Type: Improvement
> Components: test, wal
> Reporter: Sean Busbey
> Assignee: Sean Busbey
> Priority: Minor
> Fix For: 2.0.0, 1.1.0
>
> Attachments: HBASE-12719.1.patch.txt,
> HBASE-12719_comparison_chart.tiff
>
>
> Discussion in HBASE-5699 included showing the max throughput for our WAL
> benchmark (using the DisabledWALProvider) compared to when we are actually
> dealing with synchronization and talking with HDFS.
> What we can't see in that comparison is how much of the (sizable) gap is due
> to the coordination done in FSHLog to deal with multi-threading and how much
> of it is due to talking to HDFS.
> Make a test-only provider that we can use to isolate the cost for HDFS
> appends, HDFS flush, and file rolling.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
