wiedld commented on code in PR #17943:
URL: https://github.com/apache/datafusion/pull/17943#discussion_r2413065243
##########
datafusion/datasource-parquet/src/file_format.rs:
##########
@@ -1301,7 +1303,9 @@ impl FileSink for ParquetSink {
let props = parquet_props.clone();
let skip_arrow_metadata =
self.parquet_options.global.skip_arrow_metadata;
let parallel_options_clone = parallel_options.clone();
- let pool = Arc::clone(context.memory_pool());
+ // Create a reservation for the parallel parquet writing
+ let reservation =
MemoryConsumer::new("ParquetSink(ParallelWriter)")
Review Comment:
Do you mean the `MemoryConsumer::new(format("ParquetSink[path={path}]"))` on
line 1273? That one is for the non-parallel writing.
Whereas this reservation here is the root of the parallel path writing.
Perhaps I should change naming?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
