[
https://issues.apache.org/jira/browse/HBASE-14540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14941309#comment-14941309
]
John Leach commented on HBASE-14540:
------------------------------------
Elliott, that is what I intuitively thought as well for a long time. A few
implementations have changed my mind on this...
FYI, Here is a nice article on smart batching and why it is important even in
low latency systems.
http://mechanical-sympathy.blogspot.com/2011/10/smart-batching.html
Stack, let me know if I can help on the testing front. I know you put a ton of
work in on the disruptor piece.
> Write Ahead Log Batching Optimization
> -------------------------------------
>
> Key: HBASE-14540
> URL: https://issues.apache.org/jira/browse/HBASE-14540
> Project: HBase
> Issue Type: Improvement
> Reporter: John Leach
> Attachments: HBaseWALBlockingWaitStrategy.java
>
>
> The new write ahead log mechanism seems to batch too few mutations when
> running inside the disruptor. As we scaled our load up (many threads with
> small writes), we saw the number of hdfs sync operations grow in concert with
> the number of writes. Generally, one would expect the size of the batches to
> grow but the number of actual sync operations to settle.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
