westonpace opened a new pull request, #34435: URL: https://github.com/apache/arrow/pull/34435
### Rationale for this change We changed the default chunk size from 64Mi rows to 1Mi rows. However, it turns out that this property was being treated not just as the default but also as the absolute max. So it was no longer possible to specify chunk sizes larger than 1Mi rows. This change separates those two things and restores the max to 64Mi rows. ### What changes are included in this PR? Pyarrow will now set the `ParquetWriterProperties::max_row_group_length` to 64Mi when constructing a parquet writer. ### Are these changes tested? Yes. Unit tests are added. ### Are there any user-facing changes? No. The previous change #34281 changed two defaults (absolute max and default). This PR restores the absolute max back to what it was before. So it is removing a user-facing change. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
