siddubakka commented on PR #12322: URL: https://github.com/apache/arrow-rs/pull/12322#issuecomment-7005508554
@etseidl All three are in the latest push. Chunks extending to record boundaries is now called out in the first paragraph, so a large repeated value can overshoot by more than a batch. The byte budget split gets its own paragraph with the INT64 example, about 1,250 rows per page at a 10,000 byte limit. The last paragraph now says the exception is a write path thing: the Arrow writer can pass compact levels, while level slices supplied to the column writer stay materialized, so write_batch_size behaves as documented there. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
