maropu commented on issue #27077: [SPARK-30408][SQL] Should not remove orderBy in sortBy clause in Optimizer URL: https://github.com/apache/spark/pull/27077#issuecomment-571891836 > The content of sortData should be same as groupData. I might miss your point, but why do you think so? It seems the plans below looks ok to me; ``` scala> groupData.explain(true) == Parsed Logical Plan == 'Sort ['b ASC NULLS FIRST], true +- Project [_1#3 AS a#10, _2#4 AS b#11, _3#5 AS c#12] +- LocalRelation [_1#3, _2#4, _3#5] == Analyzed Logical Plan == a: string, b: int, c: int Sort [b#11 ASC NULLS FIRST], true +- Project [_1#3 AS a#10, _2#4 AS b#11, _3#5 AS c#12] +- LocalRelation [_1#3, _2#4, _3#5] == Optimized Logical Plan == Sort [b#11 ASC NULLS FIRST], true +- LocalRelation [a#10, b#11, c#12] == Physical Plan == *(1) Sort [b#11 ASC NULLS FIRST], true, 0 /// <==== global sort +- Exchange rangepartitioning(b#11 ASC NULLS FIRST, 200), true, [id=#7] +- LocalTableScan [a#10, b#11, c#12] scala> sortData.explain(true) == Parsed Logical Plan == 'Sort ['c ASC NULLS FIRST], false +- Sort [b#11 ASC NULLS FIRST], true +- Project [_1#3 AS a#10, _2#4 AS b#11, _3#5 AS c#12] +- LocalRelation [_1#3, _2#4, _3#5] == Analyzed Logical Plan == a: string, b: int, c: int Sort [c#12 ASC NULLS FIRST], false +- Sort [b#11 ASC NULLS FIRST], true +- Project [_1#3 AS a#10, _2#4 AS b#11, _3#5 AS c#12] +- LocalRelation [_1#3, _2#4, _3#5] == Optimized Logical Plan == Sort [c#12 ASC NULLS FIRST], false +- LocalRelation [a#10, b#11, c#12] == Physical Plan == *(1) Sort [c#12 ASC NULLS FIRST], false, 0 /// <==== local sort +- *(1) LocalTableScan [a#10, b#11, c#12] ```
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
