maytasm commented on PR #37330: URL: https://github.com/apache/spark/pull/37330#issuecomment-1867364494
@cloud-fan Can you expand on why you think the `Sort [20231221 ASC NULLS FIRST], true` looks wrong to remove? It is a sort that has no reference and should be fine to remove. `Sort [20231221 ASC NULLS FIRST], true` would look something like:  Since Literal is foldable, `newOrders` would be empty. As `newOrders` is empty, the sort is then removed. This is explained in https://github.com/apache/spark/pull/11840 I believe this PR prevents the optimization added in https://github.com/apache/spark/pull/11840 to work as intended since it convert the no-op sort into RepartitionByExpression, which can no longer be removed -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
