kosiew opened a new pull request, #24099: URL: https://github.com/apache/datafusion/pull/24099
## Which issue does this PR close? * Part of #23393 ## Rationale for this change Two aggregate accumulators underreport owned memory in their `size()` implementations: * `CountGroupsAccumulator` calculates vector backing storage using the wrong element type. * `SlidingDistinctSumAccumulator` reports only the accumulator struct size and omits its owned hash map allocation. This change corrects those capacity-based estimates while preserving the different `Accumulator::size()` and `GroupsAccumulator::size()` self-inclusion contracts. ## What changes are included in this PR? * Add `vec_capacity_bytes` to calculate a vector’s backing allocation using its actual element type. * Update `CountGroupsAccumulator::size()` to report the capacity-based allocation of its counts vector while remaining self-excluded. * Update `SlidingDistinctSumAccumulator::size()` to include `size_of_val(self)` and an estimate of its hash map’s allocated entry capacity. * Document that the hash map estimate excludes implementation-specific control bytes. ## Are these changes tested? Yes. This PR adds the following unit tests: * `count_groups_size_includes_vec_capacity` * `vec_capacity_bytes_uses_element_type` * `sliding_distinct_sum_size_includes_hash_map_capacity` The tests verify that: * grouped count size grows when vector backing storage is allocated; * vector capacity accounting uses the vector’s element type; * sliding distinct-sum size includes hash map capacity, grows after the map expands, and continues to reflect retained capacity after entries are retracted. ## Are there any user-facing changes? No user-facing API or aggregate result changes are intended. This PR only corrects internal memory-size estimates used for aggregate accounting. ## LLM-generated code disclosure This PR includes LLM-generated code and comments. All LLM-generated content has been manually reviewed. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
