[
https://issues.apache.org/jira/browse/HBASE-5699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14247132#comment-14247132
]
Sean Busbey commented on HBASE-5699:
------------------------------------
These are nodes with 5 data drives. There are 6 physical disks in the machines
and one is set aside for OS.
I was also surprised about the boost, since my initial thinking was that we'd
be constrained in the single wal case by writing to one pipeline. However, once
I think through things more it makes a little more sense. For one thing we only
use hflush and not hsync, so even for network flushes we're still largely in
memory. That also means those datanodes can keep handling the write to disk as
we get the next pipeline. On the far end of this chart for 128MB blocks, that
should be happening every ~1.5 seconds for the single wal. That allows us to
keep more than just 3 disks busy even in the single wal case.
It's possible the gain shown by the perf eval will be bigger once I get
HBASE-12339 in place. As is, we're paying the overhead of new block allocation
instead of the overhead of new file allocation. I don't have enough info to
know if that delta matters though.
> Run with > 1 WAL in HRegionServer
> ---------------------------------
>
> Key: HBASE-5699
> URL: https://issues.apache.org/jira/browse/HBASE-5699
> Project: HBase
> Issue Type: Improvement
> Components: Performance, wal
> Reporter: binlijin
> Assignee: Sean Busbey
> Priority: Critical
> Attachments: HBASE-5699.3.patch.txt, HBASE-5699.4.patch.txt,
> HBASE-5699_#workers_vs_MiB_per_s_1x1col_512Bval_wal_count_1,2,4.tiff,
> HBASE-5699_write_iops_multiwal-1_1_to_200_threads.tiff,
> HBASE-5699_write_iops_multiwal-2_10,50,120,190,260,330,400_threads.tiff,
> HBASE-5699_write_iops_multiwal-4_10,50,120,190,260,330,400_threads.tiff,
> HBASE-5699_write_iops_multiwal-6_10,50,120,190,260,330,400_threads.tiff,
> HBASE-5699_write_iops_upstream_1_to_200_threads.tiff, PerfHbase.txt,
> hbase-5699_multiwal_400-threads_stats_sync_heavy.txt,
> hbase-5699_total_throughput_sync_heavy.txt,
> results-hbase5699-upstream.txt.bz2, results-hbase5699-wals-1.txt.bz2,
> results-updated-hbase5699-wals-2.txt.bz2,
> results-updated-hbase5699-wals-4.txt.bz2,
> results-updated-hbase5699-wals-6.txt.bz2
>
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
