bharadwaj-pendyala opened a new pull request, #2683: URL: https://github.com/apache/datafusion-ballista/pull/2683
# Which issue does this PR close? Closes #2599. # Rationale for this change `ShuffleWriterExec`, `RangeShuffleWriterExec` and `SortShuffleWriterExec` each carried the same ~60 line `execute` body for the oneshot handoff. A protocol change had to land three times. This keeps the scope agreed on the issue: the handoff only. The two `run_coordinator`s and the stats/`ShuffleWriter` impls stay separate. # What changes are included in this PR? - Adds `take_handoff` and `handoff_stream` to `shuffle_writer.rs` for oneshot setup, receiver lookup, `summaries_to_batch`, and `RecordBatchStreamAdapter` creation. - Keeps `initialized` and `handoffs` as separate helper arguments because `SortShuffleWriterExec` has its own state with `column_null_counts`. - Reduces each `execute` to the mutex lock and the two helper calls. - Passes the exec name into the helpers, so every error string stays word for word, including the existing "called twice or out of range" error. - Adds `execute_same_partition_twice_or_out_of_range_errors`. Nothing covered that error before, and the test passes on both main and this branch. - Makes `result_schema` and `summaries_to_batch` private, since only `shuffle_writer.rs` calls them now. # Are there any user-facing changes? No. The behaviour and error messages don't change. # How was this tested? On base 98d724b3: - `cargo test -p ballista-core --lib`: 415 passed, 0 failed - `cargo clippy --all-targets -p ballista-core --all-features -- -D warnings`: clean - `cargo fmt --all -- --check`: clean -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
