qijiale76 opened a new pull request, #8809: URL: https://github.com/apache/hadoop/pull/8809
### Description of PR JIRA: https://issues.apache.org/jira/browse/HDFS-17992 FileChecksumHelper#makeCompositeCrcResult assumed one 4-byte CRC per located block, but DataNodes return an empty block checksum for empty blocks. This made getFileChecksum fail with "readInt out of bounds" for zero-length files that have an empty block, and silently return a wrong checksum when an empty block is followed by data. - Read block CRCs with a cursor and skip blocks that contribute no bytes. - If no bytes are composed, return the same checksum as a file without blocks (extracted into makeEmptyFileChecksum()). - Don't fail building the debug string for an empty block checksum. ### How was this patch tested? New unit test TestFileChecksumHelper (hadoop-hdfs-client) feeds block checksums the way DataNodes return them. Without the fix it fails for the zero-length file with an empty block (ArrayIndexOutOfBoundsException), the empty block in the middle (wrong CRC) and the debug logging case (IllegalArgumentException). Expected values are compared with java.util.zip.CRC32C of the file content. Existing TestFileChecksum (MD5MD5CRC and COMPOSITE_CRC, replicated and striped) and TestGetFileChecksum pass. ### For code changes: - [x] Does the title of this PR start with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')? - [ ] Object storage: Have the integration tests been executed and the endpoint declared according to the connector-specific documentation? *Note: Automated CI testing doesn't cover all cases so manual testing with cloud storage is still required.* - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the `LICENSE`, `LICENSE-binary`, `NOTICE-binary` files? ### AI Tooling If an AI tool was used: - [x] The PR includes the phrase "Contains content generated by <tool>" where <tool> is the name of the AI tool used. - [x] My use of AI contributions follows the ASF legal policy https://www.apache.org/legal/generative-tooling.html Contains content generated by Claude Code (Anthropic). 🤖 Generated with [Claude Code](https://claude.com/claude-code) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
