wangyum commented on code in PR #37519:
URL: https://github.com/apache/spark/pull/37519#discussion_r947496531
##########
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala:
##########
@@ -86,6 +86,7 @@ abstract class Optimizer(catalogManager: CatalogManager)
PushDownPredicates,
PushDownLeftSemiAntiJoin,
PushLeftSemiLeftAntiThroughJoin,
+ EliminateSorts,
Review Comment:
Eliminate the sorts between `Join` and `Sort` may affect the results, for
example:
https://github.com/apache/spark/blob/e9cc1024df4d587a0f456842d495db91984ed9db/sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/EliminateSortsSuite.scala#L366-L376
```
== FAIL: Plans do not match ===
'GlobalLimit 10 'GlobalLimit 10
+- 'LocalLimit 10 +- 'LocalLimit 10
+- 'Join LeftOuter +- 'Join LeftOuter
:- LocalLimit 10 :- LocalLimit 10
! : +- Project [a#0, b#0] : +- Sort [a#0 ASC
NULLS FIRST, b#0 DESC NULLS LAST], true
! : +- LocalRelation [a#0, b#0, c#0] : +- Project [a#0,
b#0]
! +- Project [d#0] : +-
LocalRelation [a#0, b#0, c#0]
! +- LocalRelation [d#0] +- Project [d#0]
! +- LocalRelation
[d#0]
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
