[
https://issues.apache.org/jira/browse/HBASE-30246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated HBASE-30246:
-----------------------------------
Labels: pull-request-available (was: )
> MOB compaction does not close MobCell after resolving reference cells
> ---------------------------------------------------------------------
>
> Key: HBASE-30246
> URL: https://issues.apache.org/jira/browse/HBASE-30246
> Project: HBase
> Issue Type: Bug
> Components: Compaction, mob
> Reporter: Xiao Liu
> Assignee: Xiao Liu
> Priority: Major
> Labels: pull-request-available
> Fix For: 2.7.0, 3.0.0-beta-2, 2.5.16, 2.6.7
>
>
> During MOB compaction, {{DefaultMobStoreCompactor}} resolves each MOB
> reference cell via:
> {code:java}
> mobCell = mobStore.resolve(c, cacheMobBlocksOnCompaction, false).getCell();
> {code}
> {{HMobStore#resolve(...)}} returns a {{Closeable}} {{MobCell}} that owns the
> {{StoreFileScanner}} opened against the MOB file (and the off-heap
> ByteBuffers backing
> the cell). The code calls {{getCell()}} and drops the {{MobCell}}, so the
> scanner/buffers
> are never released — once per reference cell, for the whole compaction. On
> the off-heap
> read path this pressures the {{ByteBuffAllocator}} pool and can lead to
> buffer exhaustion
> during large MOB compactions. The bug is long-standing (it predates
> HBASE-30177, which only
> changed the cacheBlocks argument) and was spotted during the HBASE-30177
> review.
> h3. Fix
> Close the {{MobCell}} via try-with-resources right after resolving. The cell
> must be copied
> to the heap *before* close, because closing releases/recycles the buffers
> backing it, while
> HFile writers/encoders retain references to appended cells (lastCell,
> firstCellInBlock,
> encoder prevCell) until {{beforeShipped()}}. Closing without copying would
> turn the leak into
> silent HFile corruption.
> {code:java}
> protected ExtendedCell resolveMobCell(ExtendedCell reference) throws
> IOException {
> try (MobCell mobCell = mobStore.resolve(reference,
> cacheMobBlocksOnCompaction, false)) {
> return KeyValueUtil.copyToNewKeyValue(mobCell.getCell());
> }
> }
> {code}
> h3. Testing
> * New
> {{TestDefaultMobStoreCompactor#testResolveMobCellClosesMobCellAndReturnsIndependentCopy}}
> asserts {{close()}} is called and an independent heap copy is returned.
> * {{TestHMobStore}} updated to close resolved {{MobCell}}s.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
