findepi commented on code in PR #16816:
URL: https://github.com/apache/datafusion/pull/16816#discussion_r2223260832
##########
datafusion/functions-aggregate/src/array_agg.rs:
##########
@@ -315,11 +313,7 @@ impl Accumulator for ArrayAggAccumulator {
};
if !val.is_empty() {
- // The ArrayRef might be holding a reference to its original input
buffer, so
- // storing it here directly copied/compacted avoids over
accounting memory
- // not used here.
- self.values
- .push(make_array(copy_array_data(&val.to_data())));
Review Comment:
Compacting has two aspects
- accurate memory accounting
- reducing memory usage
i understand the PR improves accounting.
I just want to understand whether this PR increases memory usage and by how
much.
For example, does the memory usage now depend on how array_agg input is
calculated? **_E.g._** if it is coming from highly filtered parquet files, will
we use more memory now?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org
For additional commands, e-mail: github-h...@datafusion.apache.org
