thisisnic commented on issue #31705: URL: https://github.com/apache/arrow/issues/31705#issuecomment-6419670396
Closing this as it dates from arrow 7.0.0 and the dataset writer has changed substantially since then (writer backpressure in 16.0.0 via #40224, and the remaining parquet pre-buffer memory retention is tracked in #39808). If memory still grows without bound during `write_dataset()` on a current release please open a new issue with a reprex. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
