gripleaf opened a new pull request, #242:
URL: https://github.com/apache/paimon-cpp/pull/242
### Purpose
Linked issue: close #240
Reduce avoidable CPU and memory-management overhead in the Avro and
manifest read paths:
- Skip the redundant recursive Arrow array validation in release builds
while retaining it in debug builds.
- Reuse a single `ColumnarRow` when scanning rows in the same manifest
batch, updating only its row index.
- Reserve Avro Builder capacity based on the configured batch size.
- Reserve List and Map child Builder capacity using decoded Avro block
sizes, including nested Struct builders.
These changes reduce repeated type fingerprint calculation, row-view
construction and destruction, memory reallocations, and data copying.
### Tests
- `paimon-avro-format-test`: 68 tests passed.
- `ManifestFileTest.*`: 8 tests passed.
- `ColumnarRowTest.TestSimple`: passed.
- Added coverage for:
- Reusing `ColumnarRow` across different row indexes.
- Recursive capacity reservation for Struct builders.
- Correct capacity handling for nested List builders.
- `pre-commit run --files <changed-files>`: passed.
- `git diff --check`: passed.
### API and Format
No public API changes under `include/paimon/`.
No storage format or protocol changes.
### Documentation
No documentation changes. This is an internal performance optimization and
does not introduce new user-facing functionality.
### Generative AI tooling
Generated-by: OpenAI Codex (GPT-5)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
