[
https://issues.apache.org/jira/browse/HBASE-2922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jean-Daniel Cryans updated HBASE-2922:
--------------------------------------
Attachment: HBASE-2922-v2.patch
Patch that was +1ed and that I'm committing.
> HLog cleanup is done under the updateLock, major slowdown
> ---------------------------------------------------------
>
> Key: HBASE-2922
> URL: https://issues.apache.org/jira/browse/HBASE-2922
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.20.6, 0.89.20100621
> Reporter: Jean-Daniel Cryans
> Attachments: HBASE-2922-v2.patch
>
>
> Something I've seen quite often in our production environment:
> {quote}
> 2010-08-16 16:17:27,104 INFO org.apache.hadoop.hbase.regionserver.HLog:
> removing old hlog file
> /hbase/.logs/rs22,60020,1280909840873/hlog.dat.1282000385321 whose highest
> sequence/edit id is 64837079950
> 2010-08-16 16:17:27,286 INFO org.apache.hadoop.hbase.regionserver.HLog:
> removing old hlog file
> /hbase/.logs/rs22,60020,1280909840873/hlog.dat.1282000392770 whose highest
> sequence/edit id is 64837088260
> 2010-08-16 16:17:27,452 INFO org.apache.hadoop.hbase.regionserver.HLog:
> removing old hlog file
> /hbase/.logs/rs22,60020,1280909840873/hlog.dat.1282000399300 whose highest
> sequence/edit id is 64837096566
> 2010-08-16 16:17:27,635 INFO org.apache.hadoop.hbase.regionserver.HLog:
> removing old hlog file
> /hbase/.logs/rs22,60020,1280909840873/hlog.dat.1282000406997 whose highest
> sequence/edit id is 64837104865
> 2010-08-16 16:17:27,827 INFO org.apache.hadoop.hbase.regionserver.HLog:
> removing old hlog file
> /hbase/.logs/rs22,60020,1280909840873/hlog.dat.1282000413803 whose highest
> sequence/edit id is 64837113153
> 2010-08-16 16:17:27,993 INFO org.apache.hadoop.hbase.regionserver.HLog:
> removing old hlog file
> /hbase/.logs/rs22,60020,1280909840873/hlog.dat.1282000421709 whose highest
> sequence/edit id is 64837121467
> 2010-08-16 16:17:28,160 INFO org.apache.hadoop.hbase.regionserver.HLog:
> removing old hlog file
> /hbase/.logs/rs22,60020,1280909840873/hlog.dat.1282000427333 whose highest
> sequence/edit id is 64837129775
> 2010-08-16 16:17:28,432 INFO org.apache.hadoop.hbase.regionserver.HLog:
> removing old hlog file
> /hbase/.logs/rs22,60020,1280909840873/hlog.dat.1282000434365 whose highest
> sequence/edit id is 64837138074
> 2010-08-16 16:17:28,518 INFO org.apache.hadoop.hbase.regionserver.HLog:
> removing old hlog file
> /hbase/.logs/rs22,60020,1280909840873/hlog.dat.1282000440347 whose highest
> sequence/edit id is 64837146376
> 2010-08-16 16:17:28,612 WARN org.apache.hadoop.hbase.regionserver.HLog: IPC
> Server handler 39 on 60020 took 1801ms appending an edit to hlog; editcount=0
> 2010-08-16 16:17:28,615 WARN org.apache.hadoop.hbase.regionserver.HLog: IPC
> Server handler 37 on 60020 took 1804ms appending an edit to hlog; editcount=1
> 2010-08-16 16:17:28,615 WARN org.apache.hadoop.hbase.regionserver.HLog: IPC
> Server handler 25 on 60020 took 1805ms appending an edit to hlog; editcount=2
> ...
> 2010-08-16 16:17:28,619 WARN org.apache.hadoop.hbase.regionserver.HLog: IPC
> Server handler 41 on 60020 took 1875ms appending an edit to hlog; editcount=50
> 2010-08-16 16:17:28,619 WARN org.apache.hadoop.hbase.regionserver.HLog: IPC
> Server handler 24 on 60020 took 1876ms appending an edit to hlog; editcount=51
> 2010-08-16 16:17:28,619 WARN org.apache.hadoop.hbase.regionserver.HLog: IPC
> Server handler 48 on 60020 took 1881ms appending an edit to hlog; editcount=54
> {quote}
> And looking at HLog.rollWriter, we roll then cleanup those unused hlog files
> under updateLock, which blocks all the appenders (as shown). We should only
> do the first part under that lock
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
