peter-toth commented on PR #59306: URL: https://github.com/apache/spark/pull/59306#issuecomment-6190744721
Thanks for the approval, @dongjoon-hyun. The two points outside the diff: - 9: Done in [`34f2672de31`](https://github.com/apache/spark/pull/59306/commits/34f2672de316aad274f3f555af3c04b9cc745a44). `checkKWayMergeBeforeTransform` now checks the merging node's `outputOrdering`. - 10: I measured it. With AQE and DPP, turning `spark.sql.sources.v2.bucketing.enabled` off after planning fails the query with `Can't zip RDDs with unequal numbers of partitions`. When the runtime filter prunes nothing, it returns wrong rows instead. On 4.0 and 4.1 it happens without AQE too. I filed SPARK-60260 for it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
