yihua opened a new pull request, #906:
URL: https://github.com/apache/hudi-rs/pull/906
## Description
closes #905
Stacked on #869; review that first.
`DataBlock::entries` walks an HFile data block's entries as borrowed key and
value slices, parsing the same layout `KeyValue::parse` does, and fails on an
entry that does not fit the block rather than ending the walk early, since a
lookup that stopped quietly would report its keys as absent.
`HFileReader::records_in_block_admitted` builds records only for the keys a
predicate admits, and the record index log-block scan uses it, so a lookup of a
few thousand keys among millions of entries pays key comparisons instead of a
record allocation per entry.
On a 10M-row table, a 10,000-key upsert spent about 90% of its time in this
decode before the change.
## How are the changes test-covered
- [ ] N/A
- [x] Automated tests (unit and/or integration tests)
- [ ] Manual tests
- [ ] Details are described below
`entries_match_key_values_and_reject_a_truncated_entry` checks the walk
against the copying iterator on a hand-built block and that a truncated tail
entry is an error; `test_records_in_block_admitted_matches_the_full_block`
checks the admitted records against the filtered full block on the HBase
fixture. The record index and HFile suites pass unchanged.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
