This is an automated email from the ASF dual-hosted git repository.
wenchen pushed a commit to branch branch-3.0
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-3.0 by this push:
new 4868b4d [SPARK-31225][SQL] Override sql method of OuterReference
4868b4d is described below
commit 4868b4d119e5fd7099d9f33f23a4164007360712
Author: Kent Yao <[email protected]>
AuthorDate: Fri Mar 27 15:21:19 2020 +0800
[SPARK-31225][SQL] Override sql method of OuterReference
### What changes were proposed in this pull request?
OuterReference is one LeafExpression, so it's children is Nil, which makes
its SQL representation always be outer(). This makes our explain-command and
error msg unclear when OuterReference exists.
e.g.
```scala
org.apache.spark.sql.AnalysisException:
Aggregate/Window/Generate expressions are not valid in where clause of the
query.
Expression in where clause: [(in.`value` = max(outer()))]
Invalid expressions: [max(outer())];;
```
This PR override its `sql` method with its `prettyName` and single argment
`e`'s `sql` methond
### Why are the changes needed?
improve err message
### Does this PR introduce any user-facing change?
yes, the err msg caused by OuterReference has changed
### How was this patch tested?
modified ut results
Closes #27985 from yaooqinn/SPARK-31225.
Authored-by: Kent Yao <[email protected]>
Signed-off-by: Wenchen Fan <[email protected]>
(cherry picked from commit 5945d46c11a86fd85f9e65f24c2e88f368eee01f)
Signed-off-by: Wenchen Fan <[email protected]>
---
.../org/apache/spark/sql/catalyst/expressions/namedExpressions.scala | 1 +
.../resources/sql-tests/results/postgreSQL/aggregates_part1.sql.out | 4 ++--
.../results/subquery/negative-cases/invalid-correlation.sql.out | 2 +-
.../sql-tests/results/udf/postgreSQL/udf-aggregates_part1.sql.out | 4 ++--
4 files changed, 6 insertions(+), 5 deletions(-)
diff --git
a/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/namedExpressions.scala
b/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/namedExpressions.scala
index 02e90f8..77b4cec 100644
---
a/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/namedExpressions.scala
+++
b/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/namedExpressions.scala
@@ -382,6 +382,7 @@ case class OuterReference(e: NamedExpression)
override def nullable: Boolean = e.nullable
override def prettyName: String = "outer"
+ override def sql: String = s"$prettyName(${e.sql})"
override def name: String = e.name
override def qualifier: Seq[String] = e.qualifier
override def exprId: ExprId = e.exprId
diff --git
a/sql/core/src/test/resources/sql-tests/results/postgreSQL/aggregates_part1.sql.out
b/sql/core/src/test/resources/sql-tests/results/postgreSQL/aggregates_part1.sql.out
index 5efb58c..f7bba96 100644
---
a/sql/core/src/test/resources/sql-tests/results/postgreSQL/aggregates_part1.sql.out
+++
b/sql/core/src/test/resources/sql-tests/results/postgreSQL/aggregates_part1.sql.out
@@ -381,8 +381,8 @@ struct<>
org.apache.spark.sql.AnalysisException
Aggregate/Window/Generate expressions are not valid in where clause of the
query.
-Expression in where clause: [(sum(DISTINCT CAST((outer() + b.`four`) AS
BIGINT)) = CAST(b.`four` AS BIGINT))]
-Invalid expressions: [sum(DISTINCT CAST((outer() + b.`four`) AS BIGINT))];
+Expression in where clause: [(sum(DISTINCT CAST((outer(a.`four`) + b.`four`)
AS BIGINT)) = CAST(b.`four` AS BIGINT))]
+Invalid expressions: [sum(DISTINCT CAST((outer(a.`four`) + b.`four`) AS
BIGINT))];
-- !query
diff --git
a/sql/core/src/test/resources/sql-tests/results/subquery/negative-cases/invalid-correlation.sql.out
b/sql/core/src/test/resources/sql-tests/results/subquery/negative-cases/invalid-correlation.sql.out
index ec7ecf2..d703d4e 100644
---
a/sql/core/src/test/resources/sql-tests/results/subquery/negative-cases/invalid-correlation.sql.out
+++
b/sql/core/src/test/resources/sql-tests/results/subquery/negative-cases/invalid-correlation.sql.out
@@ -109,7 +109,7 @@ struct<>
-- !query output
org.apache.spark.sql.AnalysisException
Expressions referencing the outer query are not supported outside of
WHERE/HAVING clauses:
-Aggregate [min(outer(t2a#x)) AS min(outer())#x]
+Aggregate [min(outer(t2a#x)) AS min(outer(t2.`t2a`))#x]
+- SubqueryAlias t3
+- Project [t3a#x, t3b#x, t3c#x]
+- SubqueryAlias t3
diff --git
a/sql/core/src/test/resources/sql-tests/results/udf/postgreSQL/udf-aggregates_part1.sql.out
b/sql/core/src/test/resources/sql-tests/results/udf/postgreSQL/udf-aggregates_part1.sql.out
index adf434b..76637bf 100644
---
a/sql/core/src/test/resources/sql-tests/results/udf/postgreSQL/udf-aggregates_part1.sql.out
+++
b/sql/core/src/test/resources/sql-tests/results/udf/postgreSQL/udf-aggregates_part1.sql.out
@@ -372,8 +372,8 @@ struct<>
org.apache.spark.sql.AnalysisException
Aggregate/Window/Generate expressions are not valid in where clause of the
query.
-Expression in where clause: [(sum(DISTINCT CAST((outer() + b.`four`) AS
BIGINT)) = CAST(CAST(udf(ansi_cast(four as string)) AS INT) AS BIGINT))]
-Invalid expressions: [sum(DISTINCT CAST((outer() + b.`four`) AS BIGINT))];
+Expression in where clause: [(sum(DISTINCT CAST((outer(a.`four`) + b.`four`)
AS BIGINT)) = CAST(CAST(udf(ansi_cast(four as string)) AS INT) AS BIGINT))]
+Invalid expressions: [sum(DISTINCT CAST((outer(a.`four`) + b.`four`) AS
BIGINT))];
-- !query
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
