[
https://issues.apache.org/jira/browse/HBASE-5699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14247173#comment-14247173
]
Sean Busbey commented on HBASE-5699:
------------------------------------
bq. I was also surprised about the boost, since my initial thinking was that
we'd be constrained in the single wal case by writing to one pipeline. However,
once I think through things more it makes a little more sense. For one thing we
only use hflush and not hsync, so even for network flushes we're still largely
in memory. That also means those datanodes can keep handling the write to disk
as we get the next pipeline. On the far end of this chart for 128MB blocks,
that should be happening every ~1.5 seconds for the single wal. That allows us
to keep more than just 3 disks busy even in the single wal case.
One other note about the rate at which we are rolling pipelines already. This
had me thinking about the improvements in HBASE-10278. If we're rolling that
often under load, I wonder if we'd be better off just forcing a roll at
whatever the "pipeline sync is slow" threshold is rather than maintain the
state to do a switch. A question better exploded on that ticket, I suppose.
> Run with > 1 WAL in HRegionServer
> ---------------------------------
>
> Key: HBASE-5699
> URL: https://issues.apache.org/jira/browse/HBASE-5699
> Project: HBase
> Issue Type: Improvement
> Components: Performance, wal
> Reporter: binlijin
> Assignee: Sean Busbey
> Priority: Critical
> Attachments: HBASE-5699.3.patch.txt, HBASE-5699.4.patch.txt,
> HBASE-5699_#workers_vs_MiB_per_s_1x1col_512Bval_wal_count_1,2,4.tiff,
> HBASE-5699_write_iops_multiwal-1_1_to_200_threads.tiff,
> HBASE-5699_write_iops_multiwal-2_10,50,120,190,260,330,400_threads.tiff,
> HBASE-5699_write_iops_multiwal-4_10,50,120,190,260,330,400_threads.tiff,
> HBASE-5699_write_iops_multiwal-6_10,50,120,190,260,330,400_threads.tiff,
> HBASE-5699_write_iops_upstream_1_to_200_threads.tiff, PerfHbase.txt,
> hbase-5699_multiwal_400-threads_stats_sync_heavy.txt,
> hbase-5699_total_throughput_sync_heavy.txt,
> results-hbase5699-upstream.txt.bz2, results-hbase5699-wals-1.txt.bz2,
> results-updated-hbase5699-wals-2.txt.bz2,
> results-updated-hbase5699-wals-4.txt.bz2,
> results-updated-hbase5699-wals-6.txt.bz2
>
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
