ctsk commented on issue #17897:
URL: https://github.com/apache/datafusion/issues/17897#issuecomment-3365859172
Hey, this is an issue that is specific to this implementation of
MinMaxBytesAccumulator: The issue is not the `self.min_max.resize`, but the
next line: The allocation of the `location` vec:
```rust
let mut locations = vec![MinMaxLocation::ExistingMinMax;
total_num_groups];
```
(I believe the scrollable code snippet above hid this)
This is where the quadratic behaviour occurs: If say each new batch adds 8K
new groups, then the location vector during the first update_batch would be 8K
elements in size, in the second update_batch 16K elements then 24K
elements....., that is an asympototically quadratic vector.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
