[
https://issues.apache.org/jira/browse/HBASE-6116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13404536#comment-13404536
]
Lars Hofhansl commented on HBASE-6116:
--------------------------------------
This was with a 6 DN/RS cluster with real HW. :)
Whatever scenario I tried the parallel write path was never faster.
I tried with PerformanceEvaluation and defaults and a presplit table (i.e. 1000
byte values, no autoflush, etc). In that case I basically just saturated the
client's network (~ 92mb/s in a 1gig link). I tested with --nomapred.
I then tried with a single region, to see if that one RegionServer would seen
an advantage.
When that did not show any gains, I hacked PerformanceEvaluation to let me use
smaller - 100 byte - value and to optionally also enable autoflush.
Now the network on the client is no longer saturated.
In that case parallel writes were actually slower, which really surprised me,
as I had assumed that many individual puts that are all written to the WAL
would show a big post for parallel writes.
It's possible test my testing methodology is flawed.
> Allow parallel HDFS writes for HLogs.
> -------------------------------------
>
> Key: HBASE-6116
> URL: https://issues.apache.org/jira/browse/HBASE-6116
> Project: HBase
> Issue Type: Bug
> Reporter: Lars Hofhansl
> Assignee: Lars Hofhansl
> Attachments: 6116-v1.txt, apurtell-patches.zip,
> pipelined-vs-parallel-comparison.zip
>
>
> In HDFS-1783 I adapted Dhrubas changes to be used in Hadoop trunk.
> This issue will include the necessary reflection changes to optionally enable
> this for the WALs in HBase.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
