yihua opened a new pull request, #896:
URL: https://github.com/apache/hudi-rs/pull/896
## Description
closes #895
`_hoodie_commit_seqno` was built with a `format!` per row into a
`Vec<String>` before becoming an Arrow array. A small `commit_seqnos` helper
now writes each `{instant}_{partition_id}-{row}-0` value straight into a
`StringBuilder`, so the column is one buffer with no per-row allocation. The
values, type and nullability are unchanged.
Locally this cut append CPU time by about 0.25 s for 2.5M rows and 0.5 s for
10M rows.
## How are the changes test-covered
- [ ] N/A
- [x] Automated tests (unit and/or integration tests)
- [ ] Manual tests
- [ ] Details are described below
`test_commit_seqnos_number_rows_from_the_offset` pins the exact values from
a non-zero offset across a digit boundary, and the empty case. The existing
write suites pass unchanged.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
