ulysses-you commented on PR #58339: URL: https://github.com/apache/spark/pull/58339#issuecomment-5474284642
Round 2 is addressed in 9a450ca; per-finding replies are on the four inline threads. **3 (description regressed):** rewritten. The coarsening-sites bullet now names all four -- key-dropping projection, union merge, `GroupPartitionsExec` reducers/join-key projection, and the join-key projection in `KeyedPartitioning.createShuffleSpec` -- plus a separate bullet for the spurious-marker scoping (`ShuffledJoin`'s `InnerLike` arm is the only merge site; broadcast joins pass through or expand one side only). The test list separates the wrong-results and regression repros (each measured failing pre-fix: 2/4, 3/4, 0/4 rows, planning throw, 2-vs-1 shuffles) from the invariant pins. No shapes are knowingly left broken. **Coordination with #58351:** acknowledged. This PR's narrowing refusals test the projection arity directly (`projectablePositions.length < numPositions`, `joinKeyPositions.length < expressions.length`) and deliberately never route through `isNarrowed`/`isCollapsed`, per your finding-10 note; the marker scoping is per-member semantics, not an OR-normalized layout property, so it should not gain the require-agreement either. Whatever lands second rebases; I have both the `partitioning.scala` and `GroupPartitionsExec` conflict shapes memorized by now. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
