ulysses-you commented on PR #58681: URL: https://github.com/apache/spark/pull/58681#issuecomment-5747101741
Thank you @peter-toth, @szehon-ho, @cloud-fan, @dongjoon-hyun and @sunchao for the thorough reviews! All comments have been addressed and fixed. The prerequisites have landed: SPARK-59272 (#58814, with @peter-toth's #58858) and SPARK-59289 (#58659), so the landing-order condition from the earlier reviews is satisfied, and the branch is rebased onto master (`d39cc1784c0`). Two new commits answer the latest review round: - `3533901e52a` (docs): the partition.filter row and migration entry now say key groups *may* be skipped and name the skip -- filtering does not run when either side's partitioning may contain unknown partition keys (`EnsureRequirements.filtersKeys`); the preserveKeyOrderingOnCoalesce row and entry now note that a join that reduced the partition keys reports no order (`outputOrdering` also gates on `reducers.isEmpty`); the sorted-merge row describes the feasibility condition directly instead of naming the internal `SafeForKWayMerge` trait. - `a3ad406dadc` (tests): SPARK-55992 is parameterized over both filter settings (`ExpectedPartitionKeys` 2 vs 0), removing the last pinned case; the SPARK-59256 pairing test keeps covering the unset default and the pinned-off rollback as two direct calls to a local helper. Note: the branch was force-pushed for the rebase; earlier replies referencing `8c5c30e0807` map to `e223f6a8ad1` after it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
