[
https://issues.apache.org/jira/browse/HBASE-2922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12900918#action_12900918
]
HBase Review Board commented on HBASE-2922:
-------------------------------------------
Message from: "Jean-Daniel Cryans" <[email protected]>
bq. On 2010-08-20 16:08:22, stack wrote:
bq. > /trunk/src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLog.java,
line 449
bq. > <http://review.cloudera.org/r/694/diff/2/?file=7677#file7677line449>
bq. >
bq. > This is an invariant? Will multiple threads arriving here at same
time pickup same filenum?
Can't have multiple threads there, it's under this.cacheFlushLock.lock();
bq. On 2010-08-20 16:08:22, stack wrote:
bq. > /trunk/src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLog.java,
line 452
bq. > <http://review.cloudera.org/r/694/diff/2/?file=7677#file7677line452>
bq. >
bq. > Should this be inside the update lock? If many threads, each will
create a writer? Or is thought that this is mostly single-threaded and that if
multiple threads arrive here at same time, its unusual and just let each roll?
Still under this.cacheFlushLock.lock();
bq. On 2010-08-20 16:08:22, stack wrote:
bq. > /trunk/src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLog.java,
line 457
bq. > <http://review.cloudera.org/r/694/diff/2/?file=7677#file7677line457>
bq. >
bq. > Yeah, should this be inside the update lock so many concurrent
threads don't all create instances of these streams?
Still under this.cacheFlushLock.lock();
bq. On 2010-08-20 16:08:22, stack wrote:
bq. > /trunk/src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLog.java,
line 682
bq. > <http://review.cloudera.org/r/694/diff/2/?file=7677#file7677line682>
bq. >
bq. > Yeah, is this safe? If two threads come in here at same time, they
proceed in series but when the second gets to here he's going to claim he
rolled the wrong old file?
Still under this.cacheFlushLock.lock();
- Jean-Daniel
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
http://review.cloudera.org/r/694/#review976
-----------------------------------------------------------
> HLog cleanup is done under the updateLock, major slowdown
> ---------------------------------------------------------
>
> Key: HBASE-2922
> URL: https://issues.apache.org/jira/browse/HBASE-2922
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.20.6, 0.89.20100621
> Reporter: Jean-Daniel Cryans
>
> Something I've seen quite often in our production environment:
> {quote}
> 2010-08-16 16:17:27,104 INFO org.apache.hadoop.hbase.regionserver.HLog:
> removing old hlog file
> /hbase/.logs/rs22,60020,1280909840873/hlog.dat.1282000385321 whose highest
> sequence/edit id is 64837079950
> 2010-08-16 16:17:27,286 INFO org.apache.hadoop.hbase.regionserver.HLog:
> removing old hlog file
> /hbase/.logs/rs22,60020,1280909840873/hlog.dat.1282000392770 whose highest
> sequence/edit id is 64837088260
> 2010-08-16 16:17:27,452 INFO org.apache.hadoop.hbase.regionserver.HLog:
> removing old hlog file
> /hbase/.logs/rs22,60020,1280909840873/hlog.dat.1282000399300 whose highest
> sequence/edit id is 64837096566
> 2010-08-16 16:17:27,635 INFO org.apache.hadoop.hbase.regionserver.HLog:
> removing old hlog file
> /hbase/.logs/rs22,60020,1280909840873/hlog.dat.1282000406997 whose highest
> sequence/edit id is 64837104865
> 2010-08-16 16:17:27,827 INFO org.apache.hadoop.hbase.regionserver.HLog:
> removing old hlog file
> /hbase/.logs/rs22,60020,1280909840873/hlog.dat.1282000413803 whose highest
> sequence/edit id is 64837113153
> 2010-08-16 16:17:27,993 INFO org.apache.hadoop.hbase.regionserver.HLog:
> removing old hlog file
> /hbase/.logs/rs22,60020,1280909840873/hlog.dat.1282000421709 whose highest
> sequence/edit id is 64837121467
> 2010-08-16 16:17:28,160 INFO org.apache.hadoop.hbase.regionserver.HLog:
> removing old hlog file
> /hbase/.logs/rs22,60020,1280909840873/hlog.dat.1282000427333 whose highest
> sequence/edit id is 64837129775
> 2010-08-16 16:17:28,432 INFO org.apache.hadoop.hbase.regionserver.HLog:
> removing old hlog file
> /hbase/.logs/rs22,60020,1280909840873/hlog.dat.1282000434365 whose highest
> sequence/edit id is 64837138074
> 2010-08-16 16:17:28,518 INFO org.apache.hadoop.hbase.regionserver.HLog:
> removing old hlog file
> /hbase/.logs/rs22,60020,1280909840873/hlog.dat.1282000440347 whose highest
> sequence/edit id is 64837146376
> 2010-08-16 16:17:28,612 WARN org.apache.hadoop.hbase.regionserver.HLog: IPC
> Server handler 39 on 60020 took 1801ms appending an edit to hlog; editcount=0
> 2010-08-16 16:17:28,615 WARN org.apache.hadoop.hbase.regionserver.HLog: IPC
> Server handler 37 on 60020 took 1804ms appending an edit to hlog; editcount=1
> 2010-08-16 16:17:28,615 WARN org.apache.hadoop.hbase.regionserver.HLog: IPC
> Server handler 25 on 60020 took 1805ms appending an edit to hlog; editcount=2
> ...
> 2010-08-16 16:17:28,619 WARN org.apache.hadoop.hbase.regionserver.HLog: IPC
> Server handler 41 on 60020 took 1875ms appending an edit to hlog; editcount=50
> 2010-08-16 16:17:28,619 WARN org.apache.hadoop.hbase.regionserver.HLog: IPC
> Server handler 24 on 60020 took 1876ms appending an edit to hlog; editcount=51
> 2010-08-16 16:17:28,619 WARN org.apache.hadoop.hbase.regionserver.HLog: IPC
> Server handler 48 on 60020 took 1881ms appending an edit to hlog; editcount=54
> {quote}
> And looking at HLog.rollWriter, we roll then cleanup those unused hlog files
> under updateLock, which blocks all the appenders (as shown). We should only
> do the first part under that lock
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
