[
https://issues.apache.org/jira/browse/HBASE-1207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12719378#action_12719378
]
Jonathan Gray commented on HBASE-1207:
--------------------------------------
Later on in the compaction process, this method is called:
{noformat}
private int updateStorefiles(final long logCacheFlushId,
final StoreFile sf, final NavigableSet<KeyValue> cache)
throws IOException {
int count = 0;
this.lock.writeLock().lock();
try {
this.storefiles.put(Long.valueOf(logCacheFlushId), sf);
count = this.storefiles.size();
// Tell listeners of the change in readers.
notifyChangedReadersObservers();
this.memcache.clearSnapshot(cache);
return count;
} finally {
this.lock.writeLock().unlock();
}
}
{noformat}
Do we need this lock given the implementation of changed readers observers now?
I think that we do... this might also means we can drop synchronization in
StoreScanner.
> Fix locking in memcache flush
> -----------------------------
>
> Key: HBASE-1207
> URL: https://issues.apache.org/jira/browse/HBASE-1207
> Project: Hadoop HBase
> Issue Type: Bug
> Affects Versions: 0.19.0
> Reporter: Ben Maurer
> Assignee: stack
> Fix For: 0.20.0
>
>
> memcache flushing holds a write lock while it reopens StoreFileScanners. I
> had a case where this process timed out and caused an exception to be thrown,
> which made the region server believe it had been unable to flush it's cache
> and shut itself down.
> Stack trace is:
> #
> "regionserver/0:0:0:0:0:0:0:0:60020.cacheFlusher" daemon prio=10
> tid=0x00000000562df400 nid=0x15d1 runnable
> [0x000000004108b000..0x000000004108bd90]
> #
> java.lang.Thread.State: RUNNABLE
> #
> at java.util.zip.CRC32.updateBytes(Native Method)
> #
> at java.util.zip.CRC32.update(CRC32.java:45)
> #
> at org.apache.hadoop.util.DataChecksum.update(DataChecksum.java:223)
> #
> at
> org.apache.hadoop.fs.FSInputChecker.readChecksumChunk(FSInputChecker.java:241)
> #
> at org.apache.hadoop.fs.FSInputChecker.fill(FSInputChecker.java:177)
> #
> at org.apache.hadoop.fs.FSInputChecker.read1(FSInputChecker.java:194)
> #
> at org.apache.hadoop.fs.FSInputChecker.read(FSInputChecker.java:159)
> #
> - locked <0x00002aaaec1bd2d8> (a
> org.apache.hadoop.hdfs.DFSClient$BlockReader)
> #
> at
> org.apache.hadoop.hdfs.DFSClient$BlockReader.read(DFSClient.java:1061)
> #
> - locked <0x00002aaaec1bd2d8> (a
> org.apache.hadoop.hdfs.DFSClient$BlockReader)
> #
> at
> org.apache.hadoop.hdfs.DFSClient$DFSInputStream.readBuffer(DFSClient.java:1616)
> #
> - locked <0x00002aaad1239000> (a
> org.apache.hadoop.hdfs.DFSClient$DFSInputStream)
> #
> at
> org.apache.hadoop.hdfs.DFSClient$DFSInputStream.read(DFSClient.java:1666)
> #
> - locked <0x00002aaad1239000> (a
> org.apache.hadoop.hdfs.DFSClient$DFSInputStream)
> #
> at
> org.apache.hadoop.hdfs.DFSClient$DFSInputStream.read(DFSClient.java:1593)
> #
> - locked <0x00002aaad1239000> (a
> org.apache.hadoop.hdfs.DFSClient$DFSInputStream)
> #
> at java.io.DataInputStream.readInt(DataInputStream.java:371)
> #
> at
> org.apache.hadoop.hbase.io.SequenceFile$Reader.next(SequenceFile.java:1943)
> #
> - locked <0x00002aaad1238c38> (a
> org.apache.hadoop.hbase.io.SequenceFile$Reader)
> #
> at
> org.apache.hadoop.hbase.io.SequenceFile$Reader.next(SequenceFile.java:1844)
> #
> - locked <0x00002aaad1238c38> (a
> org.apache.hadoop.hbase.io.SequenceFile$Reader)
> #
> at
> org.apache.hadoop.hbase.io.SequenceFile$Reader.next(SequenceFile.java:1890)
> #
> - locked <0x00002aaad1238c38> (a
> org.apache.hadoop.hbase.io.SequenceFile$Reader)
> #
> at org.apache.hadoop.hbase.io.MapFile$Reader.next(MapFile.java:525)
> #
> - locked <0x00002aaad1238b80> (a
> org.apache.hadoop.hbase.io.HalfMapFileReader)
> #
> at
> org.apache.hadoop.hbase.io.HalfMapFileReader.next(HalfMapFileReader.java:192)
> #
> - locked <0x00002aaad1238b80> (a
> org.apache.hadoop.hbase.io.HalfMapFileReader)
> #
> at
> org.apache.hadoop.hbase.regionserver.StoreFileScanner.getNext(StoreFileScanner.java:312)
> #
> at
> org.apache.hadoop.hbase.regionserver.StoreFileScanner.openReaders(StoreFileScanner.java:110)
> #
> at
> org.apache.hadoop.hbase.regionserver.StoreFileScanner.updateReaders(StoreFileScanner.java:378)
> #
> at
> org.apache.hadoop.hbase.regionserver.HStore.notifyChangedReadersObservers(HStore.java:737)
> #
> at
> org.apache.hadoop.hbase.regionserver.HStore.updateReaders(HStore.java:725)
> #
> at
> org.apache.hadoop.hbase.regionserver.HStore.internalFlushCache(HStore.java:694)
> #
> - locked <0x00002aaab7b41d30> (a java.lang.Integer)
> #
> at
> org.apache.hadoop.hbase.regionserver.HStore.flushCache(HStore.java:630)
> #
> at
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:881)
> #
> at
> org.apache.hadoop.hbase.regionserver.HRegion.flushcache(HRegion.java:789)
> #
> at
> org.apache.hadoop.hbase.regionserver.MemcacheFlusher.flushRegion(MemcacheFlusher.java:227)
> #
> at
> org.apache.hadoop.hbase.regionserver.MemcacheFlusher.run(MemcacheFlusher.java:137)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
