[
https://issues.apache.org/jira/browse/HBASE-4044?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
mingjian updated HBASE-4044:
----------------------------
Description:
We found that the hlog flush to disk would block other write threads. When
one thread exec "doWrite(info, logKey, edit);", the others wait for
"updateLock" in HLog.java.
Why not the others add their edits into a list and wait. When sync's time,
the whole list sync to disk once. I think it will decrease the IO calls.
So Maybe we will make two lists for edits. Each thread write to the
"waledits" and wait for "updateLock". Each thread can copy the "waledits" to
"flushedits" and flush the "flushedits" to disk once it gets "updateLock".
In my test, it can increase the write speed of 100% when I set
"hbase.regionserver.handler.count"=100.
was:
We found that the hlog flush to disk would block other write threads. When
one thread exec "doWrite(info, logKey, edit);", the others wait for
"updateLock" in HLog.java.
Why not the others add their edits into a list and wait. When sync's time,
the whole list sync to disk once. I think it will decrease the IO calls.
So Maybe we will make two lists for edits. Each thread write to the
"waledits" and wait for "updateLock". Each thread can copy the "waledits" to
"flushedits" and flush the "flushedits" to
disk once it gets "updateLock".
In my test, it can increase the write speed of 100% when I set
"hbase.regionserver.handler.count"=100.
> [performance]decrease IO calls when flush hlog
> ----------------------------------------------
>
> Key: HBASE-4044
> URL: https://issues.apache.org/jira/browse/HBASE-4044
> Project: HBase
> Issue Type: Improvement
> Components: performance, regionserver
> Reporter: mingjian
> Assignee: mingjian
> Fix For: 0.90.4
>
> Attachments: HLog-trunk-v1.patch
>
>
> We found that the hlog flush to disk would block other write threads.
> When one thread exec "doWrite(info, logKey, edit);", the others wait for
> "updateLock" in HLog.java.
> Why not the others add their edits into a list and wait. When sync's
> time, the whole list sync to disk once. I think it will decrease the IO
> calls.
> So Maybe we will make two lists for edits. Each thread write to the
> "waledits" and wait for "updateLock". Each thread can copy the "waledits" to
> "flushedits" and flush the "flushedits" to disk once it gets "updateLock".
> In my test, it can increase the write speed of 100% when I set
> "hbase.regionserver.handler.count"=100.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
