Wei-Chiu Chuang created HDDS-16834:
--------------------------------------
Summary: COMPOSITE_CRC key checksum fails or is wrong when a block
contributes no composite CRC bytes
Key: HDDS-16834
URL: https://issues.apache.org/jira/browse/HDDS-16834
Project: Apache Ozone
Issue Type: Bug
Components: Ozone Client
Reporter: Wei-Chiu Chuang
h3. Summary
Ozone client file checksum logic was copied from HDFS. With
{{ozone.client.checksum.combine.mode=COMPOSITE_CRC}} (the default),
{{BaseFileChecksumHelper#makeCompositeCrcResult}} assumes every located block
contributes exactly one 4-byte CRC at offset {{i * 4}} in {{blockChecksumBuf}}.
That is incorrect when a block's composite block checksum is empty (zero chunk
CRCs composed), so the client appends 0 bytes for that block.
Same failure class as
[HDFS-17992|https://issues.apache.org/jira/browse/HDFS-17992].
h3. Affected code
*
{{hadoop-ozone/client/src/main/java/org/apache/hadoop/ozone/client/checksum/BaseFileChecksumHelper.java}}
— {{makeCompositeCrcResult}}, {{populateBlockChecksumBuf}}, {{checksumBlock}}
* Block computers: {{ReplicatedBlockChecksumComputer#computeCompositeCrc}},
{{ECBlockChecksumComputer#computeCompositeCrc}} (empty digest when nothing is
composed)
h3. Symptoms
# Zero-length key with block location(s) and checksum requested with {{length >
0}}: may throw {{ArrayIndexOutOfBoundsException}} / {{readInt out of bounds}}
from {{CrcUtil.readInt}} when {{blockChecksumBuf}} is empty or misaligned; or
{{PathIOException}} if {{chunkInfos}} is empty (Ozone-specific early failure).
# Empty block in the middle of a non-empty key: following block CRCs are read
at wrong offsets → incorrect COMPOSITE_CRC returned without error.
# Trailing empty block: may read past logical CRC data (sometimes masked by
zero padding).
{{MD5MD5CRC}} is unaffected (fixed-width MD5 per block).
h3. User impact
* {{ozone sh key checksum}} / OFS {{getFileChecksum}} with default COMPOSITE_CRC
* Cross-cluster copy or tooling that compares COMPOSITE_CRC checksums
* Leftover zero-byte keys with allocated but empty blocks (failed writers,
lease recovery), analogous to HDFS DistCp + {{-pc}} failures
h3. Proposed fix
Port the approach described in HDFS-17992:
* Walk block checksum bytes with a cursor; skip blocks that contribute no bytes.
* If no bytes were composed, return the same checksum as a key with no blocks
(for COMPOSITE_CRC, not only the MD5MD5CRC magic shortcut).
* Add regression tests for replicated and EC layouts.
h3. Related
* [HDFS-17992|https://issues.apache.org/jira/browse/HDFS-17992]
* Optional follow-up: empty-key shortcut in {{compute()}} returns
{{MD5MD5CRC32GzipFileChecksum}} even when combine mode is COMPOSITE_CRC (when
{{length == 0}} and no block list is fetched).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
