kosiew opened a new issue, #26099: URL: https://github.com/apache/datafusion/issues/26099
Parent: #23393 Context: [PR comment](https://github.com/apache/datafusion/pull/26074#discussion_r4198007007) ## Goal Make partial group-ordering memory accounting include the retained allocation for its `Vec<ScalarValue>` sort key. ## Invariant / contract `GroupOrderingPartial::heap_size()` reports the retained heap allocations owned by its state. For an in-progress state, this includes the sort-key vector's allocated capacity and heap allocations owned by its initialized scalar elements, each exactly once. It does not add an inline descriptor already charged by `GroupOrdering::size()`. ## Tasks - [ ] Replace the live-element-only sort-key accounting in `State::heap_size()` with capacity-aware `DFHeapSize` accounting, following the grouped `COUNT` accumulator's established approach. - [ ] Add a deterministic test with spare `Vec<ScalarValue>` capacity and a heap-owning scalar, proving spare slots and initialized scalar payloads are accounted for without relying on allocator measurements. - [ ] Preserve the existing ownership boundary: do not charge inline descriptors or allocations owned by other ordering state again. ## Acceptance criteria - [ ] An in-progress sort key with spare vector capacity reports that capacity even when the vector's live length is smaller. - [ ] Heap allocations owned by initialized `ScalarValue`s remain included. - [ ] Existing start/complete and ordering-state accounting behavior remains unchanged. ## Tests / verification - Add or update the focused `GroupOrderingPartial` tests in `datafusion/physical-plan/src/aggregates/order/partial.rs`. - Run `cargo test -p datafusion-physical-plan`. - Run `./ci/scripts/doc_prettier_check.sh --write --allow-dirty` for this Markdown change. ## Dependencies / blockers Keep the descriptor accounting established by the `GroupOrdering` owner separate from this retained-capacity fix. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
