yihua opened a new pull request, #20319: URL: https://github.com/apache/hudi/pull/20319
### Describe the issue this Pull Request addresses Related to #20308. The HFile format doc lists the width of each field but never says what those widths imply: file offsets are 64-bit, so an HFile can grow past 2 GiB and 4 GiB, while block sizes and key/value lengths are 32-bit, so only a single block is bounded. The native reader bug fixed in #20309 came from storing an absolute file position in an `int`, and the doc should make that hard to repeat. The doc also states that intermediate and leaf index blocks share the root index entry layout, which is not the case. ### Summary and Changelog Adds an "Offsets and size limits" section to `hudi-io/hfile_format.md` that tabulates which fields are 8-byte offsets and which are 4-byte sizes, states the resulting bounds (file unbounded, block, key and value under 2 GiB), and spells out the rule for implementations: absolute positions in a long, block-relative positions in an int. Corrects the multi-level index paragraph to describe the non-root block layout (entry count, secondary index of relative offsets, entries with an unprefixed key), matching what `HFileLeafIndexBlock` parses. ### Impact Documentation only. ### Risk Level none ### Documentation Update This PR is the documentation update. ### Contributor's checklist - [x] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute) - [x] Enough context is provided in the sections above - [ ] Adequate tests were added if applicable -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
