Jbansal2 opened a new pull request, #7097:
URL: https://github.com/apache/datafusion-comet/pull/7097
## Which issue does this PR close?
Closes #7072.
## Rationale for this change
`MultiPartitionShuffleRepartitioner::buffer_partitioned_batch_may_spill`
pushes each row index into the partition's `indices` vector one at a time with
`Vec::push`, which re-checks capacity on every iteration even though the exact
capacity was just reserved via `indices.reserve(row_indices.len())` immediately
before the loop.
## What changes are included in this PR?
Replace the per-element `for ... { indices.push(...) }` loop with a single
`indices.extend(row_indices.iter().map(...))` call, which takes the
`TrustedLen` specialization and writes the tuples in one pass instead of
re-checking capacity on every push. Behavior and the resulting `indices`
contents are unchanged.
## How are these changes tested?
This is a pure internal refactor of an already-covered code path
(`buffer_partitioned_batch_may_spill` is exercised by the existing unit tests
in `native/shuffle/src/partitioners/multi_partition.rs`, e.g.
`test_buffer_partitioned_batch_may_spill_*`), and does not change observable
behavior, so no new tests were added. I was not able to run `cargo bench
--bench shuffle_writer` in this environment (toolchain install exceeded the
sandbox time limit); per the issue, the change is small enough that it's
reasonable to land without a measured baseline either way.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
