[
https://issues.apache.org/jira/browse/HBASE-26281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
chenglei reassigned HBASE-26281:
--------------------------------
Assignee: chenglei
> DBB got from BucketCache would be freed unexpectedly before RPC completed
> -------------------------------------------------------------------------
>
> Key: HBASE-26281
> URL: https://issues.apache.org/jira/browse/HBASE-26281
> Project: HBase
> Issue Type: Bug
> Components: BucketCache
> Affects Versions: 3.0.0-alpha-1, 2.4.6
> Reporter: chenglei
> Assignee: chenglei
> Priority: Critical
>
> {{DBB}} got from BucketCache would be freed unexpectedly before RPC completed
> for following two cases:
> # Because of splitting or stream read，two or more read threads may try to
> cache the same {{Block}} concurrently and one thread read all of the next
> block header could replace the another one didn't(see HBASE-20447). When
> replacing {{Block}}，the following
> {{BucketCache.WriterThread.putIntoBackingMap}} method invokes
> {{BucketCache.blockEvicted}} in line 916,which free the {{BucketEntry}}
> directly in line 544, not through the {{RefCount}}.
> {code:java}
> 912 private void putIntoBackingMap(BlockCacheKey key, BucketEntry
> bucketEntry) {
> 913 BucketEntry previousEntry = backingMap.put(key, bucketEntry);
> 914 if (previousEntry != null && previousEntry != bucketEntry) {
> 915 previousEntry.withWriteLock(offsetLock, () -> {
> 916 blockEvicted(key, previousEntry, false);
> 917 return null;
> 918 });
> 919 }
> 920 }
> 543 void blockEvicted(BlockCacheKey cacheKey, BucketEntry bucketEntry,
> boolean decrementBlockNumber) {
> 544 bucketAllocator.freeBlock(bucketEntry.offset());
> 545 realCacheSize.add(-1 * bucketEntry.getLength());
> 546 blocksByHFile.remove(cacheKey);
> 547 if (decrementBlockNumber) {
> 548 this.blockNumber.decrement();
> 549 }
> 550 }
>
> {code}
> Freeing the {{BucketEntry}} directly may cause {{HFileBlock}} returned from
> {{BucketCache.getBlock}} be freed unexpectedly before RPC
> completed.HBASE-26155 fixed this problem to some extent, but not completely.
> The {{BucketCache.getBlock}} might be invoked just before
> {{BucketCache.cacheBlockWithWaitInternal}} and after the
> {{BucketEntry.isRpcRef}} checking, considering the following sequence:
> 1. Block1 was cached successfully,the {{RefCnt}} of Block1 is 1.
> 2.Thread1 caching the same {{BlockCacheKey}} with Block2 satisfied
> {{BlockCacheUtil.shouldReplaceExistingCacheBlock}}, so Block2 would replace
> Block1, but thread1 stopped before {{BucketCache.cacheBlockWithWaitInternal}}
> 3. Thread2 invoking {{BucketCache.getBlock}} with the same
> {{BlockCacheKey}}, which returned Block1, the {{RefCnt}} of Block1 is 2.
> 4. Thread1 continues caching Block2, in
> {{BucketCache.WriterThread.putIntoBackingMap}} , the old Block1 is freed
> directly which {{RefCnt}} is 2. The Block1 is still used by Thread2 and the
> content of Block1 would be overwitten after it is freed, which may cause a
> serious error.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
