Github user dilipbiswal commented on a diff in the pull request:
https://github.com/apache/spark/pull/9844#discussion_r45715703
--- Diff:
sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/analysis/AnalysisSuite.scala
---
@@ -217,5 +217,23 @@ class AnalysisSuite extends AnalysisTest {
nullResult,
udf4)
// checkUDF(udf4, expected4)
+ }
+
+ test("SPARK-11863 mixture of aliases and real columns in orderby clause
- tpcds 19,55,71") {
+ val a = testRelation2.output(0)
+ val c = testRelation2.output(2)
+ val alias1 = a.as("a1")
+ val alias2 = c.as("a2")
+ val alias3 = count(a).as("a3")
+
+ val plan = testRelation2.
+ groupBy('a, 'c) ('a.as("a1"), 'c.as("a2"), count('a).as("a3")).
+ orderBy('a1.asc, 'c.asc)
--- End diff --
@cloud-fan Will do. Wenchen, there are a few test failures. I am still
looking at it. So i think our idea to NOT consider the already resolved
attribute for pushdown decision is causing the issue.
Here are the tests
1) SELECT count(*) FROM orderByData GROUP BY a ORDER BY count(*)
In this case we want the sort attribute representing the count(*) to be
replace by the
group by alias.
2) SELECT a FROM orderByData GROUP BY a ORDER BY a, count(*), sum(b)
In this case we want the count(*) to be pushed down to aggregate
In both these case, we are skipping pushdown processing because its a
resolved attribute.
Given this wenchen, may i request you to look at the original fix. After
learning more about
different conditions, it seems like that may be a safer fix. Let me know
what you think.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
