dragonknightk opened a new pull request, #27115: URL: https://github.com/apache/datafusion/pull/27115
## Which issue does this PR close? - Closes #26891. ## Rationale for this change Grouped `max`/`min` over structs with ordered aggregation can fail with Resources exhausted even when the data is small. When groups are emitted, the memory accounting subtracts the row count instead of the byte size, so emitted groups keep being counted. Specifically, `MinMaxStructState` tracks the memory of stored values in `total_data_bytes` via `get_array_memory_size()`, but `EmitTo::First(n)` subtracted the `len()` of the emitted values, which is only 1 per group. Ordered aggregation calls `First(n)` after every batch, so the reported size grows with the number of groups seen. ## What changes are included in this PR? Subtract `get_array_memory_size()` of the emitted values instead. ## What is the testing strategy for this PR? Added a sqllogictest in `ordered_aggregate_spill.slt` that fails with Resources exhausted before the fix and returns the correct result after it, plus a unit test. ## Are there any user-facing changes? No API changes. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
