yihua opened a new pull request, #903:
URL: https://github.com/apache/hudi-rs/pull/903
## Description
closes #902
`take_rows` gathered every chunk through Arrow's `take` kernel, so an append
copied its whole input once more on the preparation thread. When the indices
form a contiguous ascending run, which is how `append` splits its input, it now
returns `RecordBatch::slice`, a view with no copy; any other index list still
goes through `take`. Every consumer of the chunk works on a sliced batch.
## How are the changes test-covered
- [ ] N/A
- [x] Automated tests (unit and/or integration tests)
- [ ] Manual tests
- [ ] Details are described below
`test_take_rows_slices_a_contiguous_run_and_gathers_the_rest` covers a run,
a single row, an unordered and a repeated index list, a gap, and an empty list.
The write suites pass unchanged.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
