peterxcli opened a new pull request, #5461: URL: https://github.com/apache/datafusion-comet/pull/5461
## Which issue does this PR close? Part of #5212 (finding 13). ## Rationale for this change `MultiPartitionShuffleRepartitioner::spill` takes ownership of its buffered batches and partition indices before calling `PartitionWriter::write`. If a write failed, the early return skipped reservation release and pinned/repeated-spill cleanup even though those batches had already been dropped. The deterministic reproducer left 96 bytes reserved after the injected failure. It also retained one pinned buffer and 32 repeated-spill bytes while the buffered batches and indices were already empty. ## What changes are included in this PR? - Preserve the first partition-writer error while running the existing spill cleanup unconditionally. - Drop the taken partitioned batches before releasing their reservation. - Add a fake failing `PartitionWriter` test covering exact reservation, pinned/repeated state, and spill-metric deltas on both success and failure paths. ## How are these changes tested? - `cargo test -p datafusion-comet-shuffle --lib` (35 tests) - `cargo clippy -p datafusion-comet-shuffle --lib --tests -- -D warnings` - `cargo fmt --all -- --check` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
