[
https://issues.apache.org/jira/browse/HBASE-17035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15643365#comment-15643365
]
ramkrishna.s.vasudevan commented on HBASE-17035:
------------------------------------------------
While loading 50G of data with 50 threads. AsyncFSWAL is more than twice the
time it takes for FShLOg.
There are 126 log roll requests with fSHLog but with AsyncFSWAL we have 215 log
roll requests. And we have lot of log rolls with 10MB to 13MB sizes and that
what I got is not due to memory flush.
Interesting part is this
{code}
2016-11-07 18:09:13,688 INFO
[regionserver/stobdtserver5/10.224.54.65:16041.logRoller] wal.AbstractFSWAL:
Rolled WAL
/hbase1/WALs/stobdtserver5,16041,1478521203795/stobdtserver5%2C16041%2C1478521203795.1478522344435
with entries=1566632, filesize=518.92 MB; new WAL
/hbase1/WALs/stobdtserver5,16041,1478521203795/stobdtserver5%2C16041%2C1478521203795.1478522353615
2016-11-07 18:09:13,853 INFO
[regionserver/stobdtserver5/10.224.54.65:16041.logRoller] wal.AbstractFSWAL:
Rolled WAL
/hbase1/WALs/stobdtserver5,16041,1478521203795/stobdtserver5%2C16041%2C1478521203795.1478522353615
with entries=1566992, filesize=13.88 MB; new WAL
/hbase1/WALs/stobdtserver5,16041,1478521203795/stobdtserver5%2C16041%2C1478521203795.1478522353689
{code}
The new WAL file generated is immediately rolled to another file of size
13.88MB but the total number of entries is almost the same but the size is so
small. This pattern repeats quite frequently.
> Check why we roll a wal writer at 10MB when the configured roll size is 120M+
> with AsyncFSWAL
> ---------------------------------------------------------------------------------------------
>
> Key: HBASE-17035
> URL: https://issues.apache.org/jira/browse/HBASE-17035
> Project: HBase
> Issue Type: Sub-task
> Components: wal
> Affects Versions: 2.0.0
> Reporter: Duo Zhang
> Assignee: Duo Zhang
> Fix For: 2.0.0
>
>
> Found this when addressing HBASE-16890. It is one of the possible reason that
> why AsyncFSWAL performs worse than FSHLog when running PE tool.
> https://issues.apache.org/jira/browse/HBASE-16890?focusedCommentId=15636688&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15636688
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
