yihua opened a new pull request, #894:
URL: https://github.com/apache/hudi-rs/pull/894
## Description
closes #893
`HFileWriter::write` cloned every `(String, Vec<u8>)` entry to sort them and
grew its data block buffer entry by entry. It now sorts references to the
entries (the same stable sort, so duplicate keys keep their order) and reserves
the data block once: each entry takes its key and value plus 21 bytes of
framing. The written bytes are unchanged.
Bulk appends that build the record-level index got faster locally: about
20.3 s to 16.9 s for 10M rows and 2.4 s to 1.9 s for 2.5M rows.
## How are the changes test-covered
- [ ] N/A
- [x] Automated tests (unit and/or integration tests)
- [ ] Manual tests
- [ ] Details are described below
Behavior-preserving; the HFile round-trip test and the record index write
and read suites pass unchanged.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
