Github user henryr commented on the issue:
https://github.com/apache/spark/pull/21049
@dilipbiswal thanks for the clarification.
I agree that this particular case - where the alias is the root of a
logical plan - might need special handling. Is there any reason to actually use
an alias at the root of a plan like this (outside of composing with other
plans, where this optimization would apply)? My suggestion would be, since
there are no references to the name the alias introduces, to consider just
dropping the alias node during optimization (and then the sort would not get
dropped).
It does seem to be an edge case though - no matter how we handle
unreferred-to aliases, the optimization seems to be appropriate for the general
case where aliases do correspond to subqueries. What do you think?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
