[
https://issues.apache.org/jira/browse/HBASE-15536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15585269#comment-15585269
]
ramkrishna.s.vasudevan commented on HBASE-15536:
------------------------------------------------
As said, we are doing some experiments to check how we can run with AsyncWAL
when we have the write path in offheap structures. This JIRA creates an
outputStream which allows to work with ByteBuffers.
But overall we see a significant degrade just in using the current trunk with
ASYNC_WAL and default_WAL.
With PE tool with 10 cols and 50 threads
With Default WAL
============
2185.113s
With Async WAL
============
3758.702s
Since this is a single node system the load on the DN and NN is heavy and so
the msg
bq.Slow sync cost: 166 ms, current pipeline:
[DatanodeInfoWithStorage[10.224.54.65:18223,DS-bc84f329-2043-4891-b878-f368c774b246,DISK]]
The thread model between the default case and the ASYNC WALbasically tries to
be the same. We have some contentions with the ArrayDeque that is in AsyncWAL
now.
But the major difference I could see is that in the default WAL case the
checksum is being done in the append() flow because append() adds the data to
the FSDataOS.
In the AsyncWAL we add the data to the new BAOS but it is the sync call that
actually does the check summing.
Interesting thing is that when we run the WALPE tool with 50 threads with key
size 50 : val : 200 , AsyncWAL is performing better in terms of ops/sec.
Default WAL
{code}
Summary: threads=50, iterations=1000000, syncInterval=0 took 1475.368s
33889.848ops/s
{code}
AsyncWAL
{code}
Summary: threads=50, iterations=1000000, syncInterval=0 took 1368.501s
36536.328ops/s
{code}
> Make AsyncFSWAL as our default WAL
> ----------------------------------
>
> Key: HBASE-15536
> URL: https://issues.apache.org/jira/browse/HBASE-15536
> Project: HBase
> Issue Type: Sub-task
> Components: wal
> Affects Versions: 2.0.0
> Reporter: Duo Zhang
> Assignee: Duo Zhang
> Priority: Critical
> Fix For: 2.0.0
>
> Attachments: HBASE-15536-v1.patch, HBASE-15536-v2.patch,
> HBASE-15536-v3.patch, HBASE-15536-v4.patch, HBASE-15536-v5.patch,
> HBASE-15536.patch
>
>
> As it should be predicated on passing basic cluster ITBLL
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
