ramkrishna.s.vasudevan commented on HBASE-15536:

As said, we are doing some experiments to check how we can run with AsyncWAL 
when we have the write path in offheap structures. This JIRA creates an 
outputStream which allows to work with ByteBuffers. 
But overall we see a significant degrade just in using the current trunk with 
ASYNC_WAL and default_WAL. 
With PE tool with 10 cols and 50 threads
With Default WAL

With Async WAL

Since this is a single node system the load on the DN and NN is heavy and so 
the msg
bq.Slow sync cost: 166 ms, current pipeline: 

The thread model between the default case and the ASYNC WALbasically tries to 
be the same. We have some contentions with the ArrayDeque that is in AsyncWAL 
But the major difference I could see is that in the default WAL case the 
checksum is being done in the append() flow because append() adds the data to 
the FSDataOS.
In the AsyncWAL we add the data to the new BAOS but it is the sync call that 
actually does the check summing. 

Interesting thing is that when we run the WALPE tool with 50 threads with key 
size 50 : val : 200 , AsyncWAL is performing better in terms of ops/sec.
Default WAL
Summary: threads=50, iterations=1000000, syncInterval=0 took 1475.368s 

Summary: threads=50, iterations=1000000, syncInterval=0 took 1368.501s 

> Make AsyncFSWAL as our default WAL
> ----------------------------------
>                 Key: HBASE-15536
>                 URL: https://issues.apache.org/jira/browse/HBASE-15536
>             Project: HBase
>          Issue Type: Sub-task
>          Components: wal
>    Affects Versions: 2.0.0
>            Reporter: Duo Zhang
>            Assignee: Duo Zhang
>            Priority: Critical
>             Fix For: 2.0.0
>         Attachments: HBASE-15536-v1.patch, HBASE-15536-v2.patch, 
> HBASE-15536-v3.patch, HBASE-15536-v4.patch, HBASE-15536-v5.patch, 
> HBASE-15536.patch
> As it should be predicated on passing basic cluster ITBLL

This message was sent by Atlassian JIRA

Reply via email to