guillaume-rochette-oxb opened a new issue, #48962: URL: https://github.com/apache/arrow/issues/48962
### Describe the enhancement requested Hi everyone, I would like to add a functionality enabling to dynamically restack/resize a stream of pa.RecordBatch w.r.t. to minimums and maximums of rows and bytes. That would means that too large batches would be chunked in smaller ones, and conversely small batches would be concatenated into bigger ones. That way we would have more predictable resource usage for parallel processing tasks. Cheers, Guillaume ### Component(s) Python -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
