[
https://issues.apache.org/jira/browse/HBASE-15131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15107918#comment-15107918
]
Yu Li commented on HBASE-15131:
-------------------------------
Thanks for bring this up [~enis].
bq. Using bounded provider with default 4 (assuming 12 disks). Should we also
look at the number of disks from datanode dirs and auto-configure?
FWIW, in our production cluster (for Alibaba search engine) each machine has 3
SATA-SSD plus 9 SATA disks, and we did a comparison test with storage type of
the WALs directory set to ALL_SSD, the result is (in brief):
||WAL number||AverageLatency||
|2|4.4ms|
|4|4.2ms|
|8|4.4ms|
So we chose 4 as the best suit
As for pure SATA environment, we didn't do the comparison test against WAL
number. However, [~busbey] has done a great work on comparing perf result with
different WAL number in HBASE-5699 and I'd like to quote some here:
{quote}
INCREASING NUMBER OF PIPELINES
If you look at each of the
{{HBASE-5699_write_iops_multiwal-X_10,50,120,190,260,330,400_threads.tiff}}
charts, as we ramp up the number of writers we manage to push more overall
activity through the cluster.
It's not a linear gain because splitting out the pipelines means that we do
more overall syncs since fewer of them get obviated by our sync grouping.
In this test, expanding from 2 to 4 or 6 pipelines didn't provide much benefit
because at up to 400 concurrent sync-heavy writers we just get to maxing out
the number of iops that can be done with 2 pipelines.
{quote}
Since we are still using hflush and pipeline write, I think Sean's result still
applies.
> Turn on multi-WAL by default
> ----------------------------
>
> Key: HBASE-15131
> URL: https://issues.apache.org/jira/browse/HBASE-15131
> Project: HBase
> Issue Type: New Feature
> Components: wal
> Reporter: Enis Soztutar
> Fix For: 2.0.0, 1.3.0
>
>
> Something to discuss for 2.0 or even 1.3 or 1.4. Should we turn on multi-WAL
> by default now that it has seen some production use.
> Most of the known issues has been fixed I believe for replication, metrics
> etc. See HBASE-14457.
> Using bounded provider with default 4 (assuming 12 disks). Should we also
> look at the number of disks from datanode dirs and auto-configure?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
