[
https://issues.apache.org/jira/browse/HBASE-30392?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
huginn updated HBASE-30392:
---------------------------
Description:
When an HFile block is cached on write, the cache buffer contains the block
header and on-disk data, but HFileBlock's onDiskDataSizeWithHeader metadata
also includes trailing checksum bytes that are not present in the cache buffer.
This occurs in HFileBlock.Writer#getBlockForCaching().
cloneOnDiskBufferWithHeader() copies only the header and on-disk data, while
the metadata is set to onDiskBlockBytesWithHeader.size() +
onDiskChecksum.length. The resulting size mismatch can make cache validation or
later block unpacking use an incorrect length, causing cache mismatches or
read/decompression failures.
was:
When an HFile block is cached on write, the cache buffer contains the block
header and on-disk data, but HFileBlock's onDiskDataSizeWithHeader metadata
also includes trailing checksum bytes that are not present in the cache buffer.
This occurs in HFileBlock.Writer#getBlockForCaching().
cloneOnDiskBufferWithHeader() copies only the header and on-disk data, while
the metadata is set to onDiskBlockBytesWithHeader.size() +
onDiskChecksum.length. The resulting size mismatch can make cache validation or
later block unpacking use an incorrect length, causing cache mismatches or
read/decompression failures.
The fix is to set onDiskDataSizeWithHeader to
onDiskBlockBytesWithHeader.size(), while leaving onDiskSizeWithoutHeader
unchanged because the physical HFile block size still includes checksum bytes.
> Exclude checksum bytes from cache-on-write HFile block data size
> ----------------------------------------------------------------
>
> Key: HBASE-30392
> URL: https://issues.apache.org/jira/browse/HBASE-30392
> Project: HBase
> Issue Type: Bug
> Components: BlockCache
> Affects Versions: 2.4.11
> Reporter: huginn
> Priority: Major
> Labels: pull-request-available
>
> When an HFile block is cached on write, the cache buffer contains the block
> header and on-disk data, but HFileBlock's onDiskDataSizeWithHeader metadata
> also includes trailing checksum bytes that are not present in the cache
> buffer.
> This occurs in HFileBlock.Writer#getBlockForCaching().
> cloneOnDiskBufferWithHeader() copies only the header and on-disk data, while
> the metadata is set to onDiskBlockBytesWithHeader.size() +
> onDiskChecksum.length. The resulting size mismatch can make cache validation
> or later block unpacking use an incorrect length, causing cache mismatches or
> read/decompression failures.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
