[
https://issues.apache.org/jira/browse/HBASE-30246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on HBASE-30246 started by Xiao Liu.
----------------------------------------
> MOB compaction does not close MobCell after resolving reference cells
> ---------------------------------------------------------------------
>
> Key: HBASE-30246
> URL: https://issues.apache.org/jira/browse/HBASE-30246
> Project: HBase
> Issue Type: Bug
> Components: Compaction, mob
> Reporter: Xiao Liu
> Assignee: Xiao Liu
> Priority: Major
> Fix For: 2.7.0, 3.0.0-beta-2, 2.5.16, 2.6.7
>
>
> During MOB compaction, \{{DefaultMobStoreCompactor}} resolves each MOB
> reference cell via:
> {code:java}
> mobCell = mobStore.resolve(c, cacheMobBlocksOnCompaction, false).getCell();
> {code}
> {\{HMobStore#resolve(...)}} returns a \{{Closeable}} \{{MobCell}} that owns
> the
> {\{StoreFileScanner}} opened against the MOB file (and the off-heap
> ByteBuffers backing
> the cell). The code calls \{{getCell()}} and drops the \{{MobCell}}, so the
> scanner/buffers
> are never released — once per reference cell, for the whole compaction. On
> the off-heap
> read path this pressures the \{{ByteBuffAllocator}} pool and can lead to
> buffer exhaustion
> during large MOB compactions. The bug is long-standing (it predates
> HBASE-30177, which only
> changed the cacheBlocks argument) and was spotted during the HBASE-30177
> review.
> h3. Fix
> Close the \{{MobCell}} via try-with-resources right after resolving. The cell
> must be copied
> to the heap *before* close, because closing releases/recycles the buffers
> backing it, while
> HFile writers/encoders retain references to appended cells (lastCell,
> firstCellInBlock,
> encoder prevCell) until \{{beforeShipped()}}. Closing without copying would
> turn the leak into
> silent HFile corruption.
> {code:java}
> protected ExtendedCell resolveMobCell(ExtendedCell reference) throws
> IOException {
> try (MobCell mobCell = mobStore.resolve(reference,
> cacheMobBlocksOnCompaction, false)) {
> return KeyValueUtil.copyToNewKeyValue(mobCell.getCell());
> }
> }
> {code}
> h3. Testing
> * New
> \{{TestDefaultMobStoreCompactor#testResolveMobCellClosesMobCellAndReturnsIndependentCopy}}
> asserts \{{close()}} is called and an independent heap copy is returned.
> * \{{TestHMobStore}} updated to close resolved \{{MobCell}}s.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
