This is an automated email from the ASF dual-hosted git repository.
cloud-fan pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new 12226c492163 [SPARK-58171][SQL] Alias bare top-level OuterReferences
in Project and Aggregate outputs
12226c492163 is described below
commit 12226c4921638b41057a359b71263938a07e9d4a
Author: Mihailo Timotic <[email protected]>
AuthorDate: Fri Jul 17 22:09:02 2026 +0800
[SPARK-58171][SQL] Alias bare top-level OuterReferences in Project and
Aggregate outputs
### What changes were proposed in this pull request?
This is a follow-up to
[SPARK-55794](https://issues.apache.org/jira/browse/SPARK-55794) ("Always alias
`OuterReference`s").
SPARK-55794 (and its follow-up) wrap `OuterReference`s produced by
outer-scope star expansion in an `Alias` so they get a fresh `ExprId` in the
subquery's scope. This PR extends that handling to also alias a **bare
top-level `OuterReference`** that appears directly as a
`Project`/`Aggregate` output of a subquery.
Concretely:
- `aliasIfOuterReference` is moved out of `ResolveReferences` (where it
was `private`) into `ColumnResolutionHelper` (as `protected`) so it can
be shared.
- It is now applied at the final output slot of both `ResolveReferences`
(the `Project` project list) and `ResolveReferencesInAggregate` (the
aggregate expression list), in addition to the existing star-expansion
call sites.
- A bare top-level `OuterReference` is wrapped as `Alias(outerReference,
toPrettySQL(outerReference.e))`; anything that is already a
`NamedExpression` (an `Alias` from struct star expansion, or an explicit
alias) is left unchanged. `OuterReference`s nested inside an
expression (e.g. `COALESCE(t.*)`, `least(outer(a), ...)`) stay bare --
only top-level project/aggregate outputs are aliased.
For example, `SELECT (SELECT c1) FROM t` now resolves the inner project
as:
Project [outer(c1#x) AS c1#x] -- was: Project [outer(c1#x)]
+- OneRowRelation
### Why are the changes needed?
Without the `Alias`, a bare top-level `OuterReference` leaks the outer
attribute's `ExprId` through the operator's output, because
`OuterReference.toAttribute` strips the wrapper. Downstream operators
that reference that leaked `ExprId` can end up with incorrect
expression-id tracking, and the leaked id can collide with an outer
attribute of the same query block (for example, the left side of a `LATERAL
JOIN`). Aliasing the reference gives it a fresh `ExprId` and removes the
collision, keeping expression-id tracking correct and consistent with
how star-expanded outer references are already handled.
### Does this PR introduce _any_ user-facing change?
No. There is no change to query results or output schemas. The only
visible effect is on the resolved plan's string representation: a bare
top-level outer reference now renders as `outer(x) AS x` instead of
`outer(x)` (including inside error-message `treeNode` parameters).
### How was this patch tested?
- Updated `ResolveSubquerySuite` (the SPARK-36028 expected plan now
carries the aliased outer reference).
- Regenerated the affected golden files under
`sql-tests/analyzer-results` and `sql-tests/results`. Every diff is exactly the
`outer(x)` ->
`outer(x) AS x` rewrite; no result values or output schemas change.
- Ran the affected `SQLQueryTestSuite` cases with golden-file generation
disabled to confirm the goldens are consistent with the code (all
pass).
### Was this patch authored or co-authored using generative AI tooling?
No
Closes #57301 from
mihailotim-db/mihailo-timotic_data/port-always-alias-outer-references.
Authored-by: Mihailo Timotic <[email protected]>
Signed-off-by: Wenchen Fan <[email protected]>
---
.../spark/sql/catalyst/analysis/Analyzer.scala | 16 ++-------------
.../catalyst/analysis/ColumnResolutionHelper.scala | 16 +++++++++++++++
.../analysis/ResolveReferencesInAggregate.scala | 2 +-
.../catalyst/analysis/ResolveSubquerySuite.scala | 24 +++++++++++++++++++++-
.../sql-tests/analyzer-results/cte-legacy.sql.out | 2 +-
.../sql-tests/analyzer-results/cte-nested.sql.out | 4 ++--
.../analyzer-results/cte-nonlegacy.sql.out | 4 ++--
.../sql-tests/analyzer-results/cte.sql.out | 2 +-
.../analyzer-results/join-lateral.sql.out | 2 +-
.../sql-tests/analyzer-results/order-by.sql.out | 2 +-
.../analyzer-results/postgreSQL/join.sql.out | 8 ++++----
.../analyzer-results/sql-session-variables.sql.out | 2 +-
.../sql-udf-name-precedence-legacy.sql.out | 4 ++--
.../sql-udf-name-precedence.sql.out | 4 ++--
.../sql-tests/analyzer-results/sql-udf.sql.out | 8 ++++----
.../exists-subquery/exists-aggregate.sql.out | 4 ++--
.../scalar-subquery/scalar-subquery-select.sql.out | 4 ++--
.../udf/postgreSQL/udf-join.sql.out | 8 ++++----
.../resources/sql-tests/results/order-by.sql.out | 2 +-
.../resources/sql-tests/results/sql-udf.sql.out | 2 +-
20 files changed, 73 insertions(+), 47 deletions(-)
diff --git
a/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala
b/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala
index 7d22ab263059..14574916d1ac 100644
---
a/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala
+++
b/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala
@@ -1753,7 +1753,8 @@ class Analyzer(
// Lateral column alias has higher priority than outer reference.
val resolvedWithLCA = resolveLateralColumnAlias(resolvedBasic)
val resolvedFinal = resolvedWithLCA.map(resolveColsLastResort)
- p.copy(projectList =
resolvedFinal.map(_.asInstanceOf[NamedExpression]))
+ p.copy(projectList =
+ resolvedFinal.map(e =>
aliasIfOuterReference(e.asInstanceOf[NamedExpression])))
case o: OverwriteByExpression if o.table.resolved =>
// The delete condition of `OverwriteByExpression` will be passed to
the table
@@ -2040,19 +2041,6 @@ class Analyzer(
}.map(_.asInstanceOf[NamedExpression])
}
- /**
- * Wrap an outer-scope star expansion result in [[Alias]] so that the
[[OuterReference]]
- * attribute gets a fresh ExprId in the subquery's scope. This prevents
the outer ExprId from
- * leaking through [[Project.output]] when the expansion goes through a
derived table.
- * Struct star expansion already produces [[Alias]] nodes, so those are
left unchanged.
- */
- private def aliasIfOuterReference(e: NamedExpression): NamedExpression = e
match {
- case _: Alias => e
- case outerReference: OuterReference =>
- Alias(outerReference, toPrettySQL(outerReference.e))()
- case _ => e
- }
-
/**
* Returns true if `exprs` contains a [[Star]].
*/
diff --git
a/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/ColumnResolutionHelper.scala
b/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/ColumnResolutionHelper.scala
index 55488e8eed95..08bfcc446029 100644
---
a/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/ColumnResolutionHelper.scala
+++
b/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/ColumnResolutionHelper.scala
@@ -509,6 +509,22 @@ trait ColumnResolutionHelper extends Logging with
DataTypeErrorsBase {
resolveVariables(resolveOuterRef(e))
}
+ /**
+ * Wrap a top-level [[OuterReference]] of a subquery's Project/Aggregate
output in an [[Alias]]
+ * so that it gets a fresh ExprId in the subquery's scope. Without the
[[Alias]] the outer
+ * attribute's ExprId leaks through the operator's output (since
[[OuterReference.toAttribute]]
+ * strips the wrapper), and downstream operators referencing that leaked
ExprId can produce
+ * incorrect expression-id tracking. Expressions that are already a
[[NamedExpression]] other
+ * than a bare [[OuterReference]] (e.g. an [[Alias]] from struct star
expansion or an explicit
+ * alias) are left unchanged.
+ */
+ protected def aliasIfOuterReference(e: NamedExpression): NamedExpression = e
match {
+ case _: Alias => e
+ case outerReference: OuterReference =>
+ Alias(outerReference, toPrettySQL(outerReference.e))()
+ case _ => e
+ }
+
def resolveExprInAssignment(
expr: Expression,
hostPlan: LogicalPlan,
diff --git
a/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveReferencesInAggregate.scala
b/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveReferencesInAggregate.scala
index f01c00f9fa75..0fd19f4f0518 100644
---
a/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveReferencesInAggregate.scala
+++
b/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveReferencesInAggregate.scala
@@ -65,7 +65,7 @@ class ResolveReferencesInAggregate(val catalogManager:
CatalogManager) extends S
resolveExpressionByPlanChildren(_, planForResolve))
val resolvedAggExprsWithLCA =
resolveLateralColumnAlias(resolvedAggExprsBasic)
val resolvedAggExprsFinal =
resolvedAggExprsWithLCA.map(resolveColsLastResort)
- .map(_.asInstanceOf[NamedExpression])
+ .map(e => aliasIfOuterReference(e.asInstanceOf[NamedExpression]))
// `groupingExpressions` may rely on `aggregateExpressions`, due to
features like GROUP BY alias
// and GROUP BY ALL. We only do basic resolution for
`groupingExpressions`, and will further
// resolve it after `aggregateExpressions` are all resolved. Note: the
basic resolution is
diff --git
a/sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/analysis/ResolveSubquerySuite.scala
b/sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/analysis/ResolveSubquerySuite.scala
index 30ef95329fa1..48286359ee80 100644
---
a/sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/analysis/ResolveSubquerySuite.scala
+++
b/sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/analysis/ResolveSubquerySuite.scala
@@ -254,7 +254,8 @@ class ResolveSubquerySuite extends AnalysisTest {
// SELECT (SELECT a) FROM t1
checkAnalysis(
Project(ScalarSubquery(t0.select($"a")).as("sub") :: Nil, t1),
- Project(ScalarSubquery(Project(OuterReference(a) :: Nil, t0),
Seq(a)).as("sub") :: Nil, t1)
+ Project(ScalarSubquery(
+ Project(OuterReference(a).as(a.name) :: Nil, t0), Seq(a)).as("sub") ::
Nil, t1)
)
// SELECT (SELECT a + b + c AS r FROM t2) FROM t1
checkAnalysis(
@@ -274,6 +275,27 @@ class ResolveSubquerySuite extends AnalysisTest {
)
}
+ test("SPARK-58171: alias a bare outer reference that is an Aggregate
output") {
+ // SELECT * FROM t1, LATERAL (SELECT a, count(b) AS cnt FROM t2 GROUP BY a)
+ // The bare outer reference `a` in the Aggregate output must be aliased so
it gets a fresh
+ // ExprId, instead of leaking the outer attribute's ExprId through the
Aggregate output.
+ // The query is only resolved (not passed through CheckAnalysis) because a
correlated
+ // aggregate output is otherwise rejected as an unsupported correlated
reference.
+ val plan = lateralJoin(t1, t2.groupBy($"a")($"a", Count($"b").as("cnt")))
+ val resolved = getAnalyzer.execute(plan)
+ val aggregate = resolved.collectWithSubqueries { case a: Aggregate => a
}.head
+ // The first output is the aliased outer reference; the outer attribute's
ExprId is not
+ // leaked through the alias (the alias carries a fresh ExprId).
+ aggregate.aggregateExpressions.head match {
+ case alias @ Alias(o: OuterReference, name) =>
+ assert(name == a.name)
+ assert(o.exprId == a.exprId)
+ assert(alias.exprId != a.exprId)
+ case other =>
+ fail(s"Expected the outer reference to be wrapped in an Alias, but
got: $other")
+ }
+ }
+
test("SPARK-47509: Incorrect results for subquery expressions in
LambdaFunctions") {
val data = LocalRelation(Seq(
$"key".int,
diff --git
a/sql/core/src/test/resources/sql-tests/analyzer-results/cte-legacy.sql.out
b/sql/core/src/test/resources/sql-tests/analyzer-results/cte-legacy.sql.out
index b48a2ac31665..3d8281b88e34 100644
--- a/sql/core/src/test/resources/sql-tests/analyzer-results/cte-legacy.sql.out
+++ b/sql/core/src/test/resources/sql-tests/analyzer-results/cte-legacy.sql.out
@@ -62,7 +62,7 @@ SELECT (
) FROM range(1)
-- !query analysis
Project [scalar-subquery#x [id#xL] AS scalarsubquery(id)#xL]
-: +- Project [outer(id#xL)]
+: +- Project [outer(id#xL) AS id#xL]
: +- OneRowRelation
+- Range (0, 1, step=1)
diff --git
a/sql/core/src/test/resources/sql-tests/analyzer-results/cte-nested.sql.out
b/sql/core/src/test/resources/sql-tests/analyzer-results/cte-nested.sql.out
index e13c4758df45..ab241f58f85c 100644
--- a/sql/core/src/test/resources/sql-tests/analyzer-results/cte-nested.sql.out
+++ b/sql/core/src/test/resources/sql-tests/analyzer-results/cte-nested.sql.out
@@ -68,7 +68,7 @@ Project [scalar-subquery#x [id#xL] AS scalarsubquery(id)#x]
: +- WithCTE
: :- CTERelationDef xxxx, false
: : +- SubqueryAlias unreferenced
-: : +- Project [outer(id#xL)]
+: : +- Project [outer(id#xL) AS id#xL]
: : +- OneRowRelation
: +- Project [1 AS 1#x]
: +- OneRowRelation
@@ -87,7 +87,7 @@ Project [scalar-subquery#x [id#xL] AS scalarsubquery(id)#xL]
: : +- SubqueryAlias unreferenced
: : +- Project [1 AS 1#x]
: : +- OneRowRelation
-: +- Project [outer(id#xL)]
+: +- Project [outer(id#xL) AS id#xL]
: +- OneRowRelation
+- Range (0, 1, step=1)
diff --git
a/sql/core/src/test/resources/sql-tests/analyzer-results/cte-nonlegacy.sql.out
b/sql/core/src/test/resources/sql-tests/analyzer-results/cte-nonlegacy.sql.out
index 65b7abbec236..6c5110111154 100644
---
a/sql/core/src/test/resources/sql-tests/analyzer-results/cte-nonlegacy.sql.out
+++
b/sql/core/src/test/resources/sql-tests/analyzer-results/cte-nonlegacy.sql.out
@@ -68,7 +68,7 @@ Project [scalar-subquery#x [id#xL] AS scalarsubquery(id)#x]
: +- WithCTE
: :- CTERelationDef xxxx, false
: : +- SubqueryAlias unreferenced
-: : +- Project [outer(id#xL)]
+: : +- Project [outer(id#xL) AS id#xL]
: : +- OneRowRelation
: +- Project [1 AS 1#x]
: +- OneRowRelation
@@ -87,7 +87,7 @@ Project [scalar-subquery#x [id#xL] AS scalarsubquery(id)#xL]
: : +- SubqueryAlias unreferenced
: : +- Project [1 AS 1#x]
: : +- OneRowRelation
-: +- Project [outer(id#xL)]
+: +- Project [outer(id#xL) AS id#xL]
: +- OneRowRelation
+- Range (0, 1, step=1)
diff --git a/sql/core/src/test/resources/sql-tests/analyzer-results/cte.sql.out
b/sql/core/src/test/resources/sql-tests/analyzer-results/cte.sql.out
index d808b246ba39..9531d353baec 100644
--- a/sql/core/src/test/resources/sql-tests/analyzer-results/cte.sql.out
+++ b/sql/core/src/test/resources/sql-tests/analyzer-results/cte.sql.out
@@ -657,7 +657,7 @@ Project [scalar-subquery#x [x#x] AS scalarsubquery(x)#x]
: +- WithCTE
: :- CTERelationDef xxxx, false
: : +- SubqueryAlias q
-: : +- Project [outer(x#x)]
+: : +- Project [outer(x#x) AS x#x]
: : +- OneRowRelation
: +- Project [x#x]
: +- SubqueryAlias q
diff --git
a/sql/core/src/test/resources/sql-tests/analyzer-results/join-lateral.sql.out
b/sql/core/src/test/resources/sql-tests/analyzer-results/join-lateral.sql.out
index cb46c265512f..ea6feb9667ff 100644
---
a/sql/core/src/test/resources/sql-tests/analyzer-results/join-lateral.sql.out
+++
b/sql/core/src/test/resources/sql-tests/analyzer-results/join-lateral.sql.out
@@ -347,7 +347,7 @@ Project [a#x, b#x]
+- LateralJoin lateral-subquery#x [c1#x && c2#x], Inner
: +- SubqueryAlias s
: +- Project [c1#x AS a#x, c2#x AS b#x]
- : +- Project [outer(c1#x), outer(c2#x)]
+ : +- Project [outer(c1#x) AS c1#x, outer(c2#x) AS c2#x]
: +- OneRowRelation
+- SubqueryAlias spark_catalog.default.t1
+- View (`spark_catalog`.`default`.`t1`, [c1#x, c2#x])
diff --git
a/sql/core/src/test/resources/sql-tests/analyzer-results/order-by.sql.out
b/sql/core/src/test/resources/sql-tests/analyzer-results/order-by.sql.out
index d137bc29759e..abaaf9c9fb8c 100644
--- a/sql/core/src/test/resources/sql-tests/analyzer-results/order-by.sql.out
+++ b/sql/core/src/test/resources/sql-tests/analyzer-results/order-by.sql.out
@@ -172,7 +172,7 @@ org.apache.spark.sql.catalyst.ExtendedAnalysisException
"errorClass" :
"UNSUPPORTED_SUBQUERY_EXPRESSION_CATEGORY.UNSUPPORTED_CORRELATED_SCALAR_SUBQUERY",
"sqlState" : "0A000",
"messageParameters" : {
- "treeNode" : "Sort [scalar-subquery#x [a#x] ASC NULLS FIRST], true\n: +-
Project [outer(a#x)]\n: +- LocalRelation [col1#x]\n+- Project [a#x, b#x]\n
+- SubqueryAlias testdata\n +- View (`testData`, [a#x, b#x])\n +-
Project [cast(a#x as int) AS a#x, cast(b#x as int) AS b#x]\n +-
Project [a#x, b#x]\n +- SubqueryAlias testData\n
+- LocalRelation [a#x, b#x]\n"
+ "treeNode" : "Sort [scalar-subquery#x [a#x] ASC NULLS FIRST], true\n: +-
Project [outer(a#x) AS a#x]\n: +- LocalRelation [col1#x]\n+- Project [a#x,
b#x]\n +- SubqueryAlias testdata\n +- View (`testData`, [a#x, b#x])\n
+- Project [cast(a#x as int) AS a#x, cast(b#x as int) AS b#x]\n
+- Project [a#x, b#x]\n +- SubqueryAlias testData\n
+- LocalRelation [a#x, b#x]\n"
},
"queryContext" : [ {
"objectType" : "",
diff --git
a/sql/core/src/test/resources/sql-tests/analyzer-results/postgreSQL/join.sql.out
b/sql/core/src/test/resources/sql-tests/analyzer-results/postgreSQL/join.sql.out
index 439094c11286..311a67539294 100644
---
a/sql/core/src/test/resources/sql-tests/analyzer-results/postgreSQL/join.sql.out
+++
b/sql/core/src/test/resources/sql-tests/analyzer-results/postgreSQL/join.sql.out
@@ -3517,7 +3517,7 @@ Project [x#x, f1#x, y#x]
+- LateralJoin lateral-subquery#x [x#x], Inner
: +- SubqueryAlias ss2
: +- Project [x#x AS y#x]
- : +- Project [outer(x#x)]
+ : +- Project [outer(x#x) AS x#x]
: +- OneRowRelation
+- Join Inner, (x#x = cast(f1#x as double))
:- SubqueryAlias ss1
@@ -3547,7 +3547,7 @@ Project [x#x, f1#x, y#x]
+- LateralJoin lateral-subquery#x [x#x], Inner
: +- SubqueryAlias ss2
: +- Project [x#x AS y#x]
- : +- Project [outer(x#x)]
+ : +- Project [outer(x#x) AS x#x]
: +- OneRowRelation
+- SubqueryAlias j
+- Join Inner, (x#x = cast(f1#x as double))
@@ -3579,7 +3579,7 @@ Project [q1#xL, q2#xL, q1#xL, q2#xL, xq1#xL, yq1#xL,
yq2#xL]
+- LateralJoin lateral-subquery#x [q1#xL && q1#xL && q2#xL], Inner
: +- SubqueryAlias v
: +- Project [q1#xL AS xq1#xL, q1#xL AS yq1#xL, q2#xL AS yq2#xL]
- : +- Project [outer(q1#xL), outer(q1#xL), outer(q2#xL)]
+ : +- Project [outer(q1#xL) AS q1#xL, outer(q1#xL) AS q1#xL,
outer(q2#xL) AS q2#xL]
: +- OneRowRelation
+- Join LeftOuter, (q2#xL = q1#xL)
:- SubqueryAlias x
@@ -3610,7 +3610,7 @@ Project [q1#xL, q2#xL]
+- LateralJoin lateral-subquery#x [q1#xL && q1#xL && q2#xL], Inner
: +- SubqueryAlias v
: +- Project [q1#xL AS xq1#xL, q1#xL AS yq1#xL, q2#xL AS yq2#xL]
- : +- Project [outer(q1#xL), outer(q1#xL), outer(q2#xL)]
+ : +- Project [outer(q1#xL) AS q1#xL, outer(q1#xL) AS q1#xL,
outer(q2#xL) AS q2#xL]
: +- OneRowRelation
+- Join LeftOuter, (q2#xL = q1#xL)
:- SubqueryAlias x
diff --git
a/sql/core/src/test/resources/sql-tests/analyzer-results/sql-session-variables.sql.out
b/sql/core/src/test/resources/sql-tests/analyzer-results/sql-session-variables.sql.out
index 4b6fc5d45014..a2f4b52173e3 100644
---
a/sql/core/src/test/resources/sql-tests/analyzer-results/sql-session-variables.sql.out
+++
b/sql/core/src/test/resources/sql-tests/analyzer-results/sql-session-variables.sql.out
@@ -2155,7 +2155,7 @@ Project [c1#x AS 2#x]
+- LateralJoin lateral-subquery#x [var1#x], Inner
: +- SubqueryAlias TT
: +- Project [var1#x AS c1#x]
- : +- Project [outer(var1#x)]
+ : +- Project [outer(var1#x) AS var1#x]
: +- OneRowRelation
+- SubqueryAlias T
+- LocalRelation [var1#x]
diff --git
a/sql/core/src/test/resources/sql-tests/analyzer-results/sql-udf-name-precedence-legacy.sql.out
b/sql/core/src/test/resources/sql-tests/analyzer-results/sql-udf-name-precedence-legacy.sql.out
index 04fb7715961d..375fe778a71b 100644
---
a/sql/core/src/test/resources/sql-tests/analyzer-results/sql-udf-name-precedence-legacy.sql.out
+++
b/sql/core/src/test/resources/sql-tests/analyzer-results/sql-udf-name-precedence-legacy.sql.out
@@ -131,7 +131,7 @@ SELECT outer_vs_param(42)
-- !query analysis
Project [outer_vs_param(x#x) AS outer_vs_param(42)#x]
: +- Project [scalar-subquery#x [x#x] AS scalarsubquery(x)#x]
-: : +- Project [outer(x#x)]
+: : +- Project [outer(x#x) AS x#x]
: : +- OneRowRelation
: +- SubqueryAlias v1
: +- View (`v1`, [x#x])
@@ -153,7 +153,7 @@ CreateSQLFunctionCommand outer_param_pure, x INT, INT,
(SELECT (SELECT x)), fals
SELECT outer_param_pure(42)
-- !query analysis
Project [outer_param_pure(x#x) AS outer_param_pure(42)#x]
-: +- Project [outer(x#x)]
+: +- Project [outer(x#x) AS x#x]
: +- OneRowRelation
+- Project [cast(42 as int) AS x#x]
+- OneRowRelation
diff --git
a/sql/core/src/test/resources/sql-tests/analyzer-results/sql-udf-name-precedence.sql.out
b/sql/core/src/test/resources/sql-tests/analyzer-results/sql-udf-name-precedence.sql.out
index 03e2b3c6d8cb..1bfab2fc03a4 100644
---
a/sql/core/src/test/resources/sql-tests/analyzer-results/sql-udf-name-precedence.sql.out
+++
b/sql/core/src/test/resources/sql-tests/analyzer-results/sql-udf-name-precedence.sql.out
@@ -151,7 +151,7 @@ SELECT outer_vs_param(42)
-- !query analysis
Project [outer_vs_param(x#x) AS outer_vs_param(42)#x]
: +- Project [scalar-subquery#x [x#x] AS scalarsubquery(x)#x]
-: : +- Project [outer(x#x)]
+: : +- Project [outer(x#x) AS x#x]
: : +- OneRowRelation
: +- SubqueryAlias v1
: +- View (`v1`, [x#x])
@@ -173,7 +173,7 @@ CreateSQLFunctionCommand outer_param_pure, x INT, INT,
(SELECT (SELECT x)), fals
SELECT outer_param_pure(42)
-- !query analysis
Project [outer_param_pure(x#x) AS outer_param_pure(42)#x]
-: +- Project [outer(x#x)]
+: +- Project [outer(x#x) AS x#x]
: +- OneRowRelation
+- Project [cast(42 as int) AS x#x]
+- OneRowRelation
diff --git
a/sql/core/src/test/resources/sql-tests/analyzer-results/sql-udf.sql.out
b/sql/core/src/test/resources/sql-tests/analyzer-results/sql-udf.sql.out
index 294dfbe85321..e6e933bdeb33 100644
--- a/sql/core/src/test/resources/sql-tests/analyzer-results/sql-udf.sql.out
+++ b/sql/core/src/test/resources/sql-tests/analyzer-results/sql-udf.sql.out
@@ -1718,7 +1718,7 @@ CreateSQLFunctionCommand spark_catalog.default.foo2_2b, a
INT, INT, 1 + (SELECT
SELECT foo2_2b(5)
-- !query analysis
Project [spark_catalog.default.foo2_2b(a#x) AS
spark_catalog.default.foo2_2b(5)#x]
-: +- Project [outer(a#x)]
+: +- Project [outer(a#x) AS a#x]
: +- OneRowRelation
+- Project [cast(5 as int) AS a#x]
+- OneRowRelation
@@ -2543,12 +2543,12 @@ Project [c1#x, c2#x, spark_catalog.default.foo3_1a(c1,
c2)#x]
SELECT c1, (SELECT c1), (SELECT foo3_1b(c1)), SUM(c2) FROM t1 GROUP BY 1, 2, 3
-- !query analysis
Aggregate [c1#x, scalar-subquery#x [c1#x], scalar-subquery#x [c1#x]], [c1#x,
scalar-subquery#x [c1#x] AS scalarsubquery(c1)#x, scalar-subquery#x [c1#x] AS
scalarsubquery(c1)#x, sum(c2#x) AS sum(c2)#xL]
-: :- Project [outer(c1#x)]
+: :- Project [outer(c1#x) AS c1#x]
: : +- OneRowRelation
: :- Project [spark_catalog.default.foo3_1b(x#x) AS
spark_catalog.default.foo3_1b(outer(spark_catalog.default.t1.c1))#x]
: : +- Project [cast(outer(c1#x) as int) AS x#x]
: : +- OneRowRelation
-: :- Project [outer(c1#x)]
+: :- Project [outer(c1#x) AS c1#x]
: : +- OneRowRelation
: +- Project [spark_catalog.default.foo3_1b(x#x) AS
spark_catalog.default.foo3_1b(outer(spark_catalog.default.t1.c1))#x]
: +- Project [cast(outer(c1#x) as int) AS x#x]
@@ -2961,7 +2961,7 @@ org.apache.spark.sql.catalyst.ExtendedAnalysisException
"errorClass" :
"UNSUPPORTED_SUBQUERY_EXPRESSION_CATEGORY.UNSUPPORTED_CORRELATED_SCALAR_SUBQUERY",
"sqlState" : "0A000",
"messageParameters" : {
- "treeNode" : "Join Inner, (spark_catalog.default.foo3_1g(x#x) = 2)\n: +-
Project [outer(x#x)]\n: +- OneRowRelation\n:- SubqueryAlias
spark_catalog.default.t1\n: +- View (`spark_catalog`.`default`.`t1`, [c1#x,
c2#x])\n: +- Project [cast(col1#x as int) AS c1#x, cast(col2#x as int) AS
c2#x]\n: +- LocalRelation [col1#x, col2#x]\n+- SubqueryAlias
spark_catalog.default.t2\n +- View (`spark_catalog`.`default`.`t2`, [c1#x,
c2#x])\n +- Project [cast(col1#x as int) AS c [...]
+ "treeNode" : "Join Inner, (spark_catalog.default.foo3_1g(x#x) = 2)\n: +-
Project [outer(x#x) AS x#x]\n: +- OneRowRelation\n:- SubqueryAlias
spark_catalog.default.t1\n: +- View (`spark_catalog`.`default`.`t1`, [c1#x,
c2#x])\n: +- Project [cast(col1#x as int) AS c1#x, cast(col2#x as int) AS
c2#x]\n: +- LocalRelation [col1#x, col2#x]\n+- SubqueryAlias
spark_catalog.default.t2\n +- View (`spark_catalog`.`default`.`t2`, [c1#x,
c2#x])\n +- Project [cast(col1#x as in [...]
}
}
diff --git
a/sql/core/src/test/resources/sql-tests/analyzer-results/subquery/exists-subquery/exists-aggregate.sql.out
b/sql/core/src/test/resources/sql-tests/analyzer-results/subquery/exists-subquery/exists-aggregate.sql.out
index 4da75ae2764f..63785264348d 100644
---
a/sql/core/src/test/resources/sql-tests/analyzer-results/subquery/exists-subquery/exists-aggregate.sql.out
+++
b/sql/core/src/test/resources/sql-tests/analyzer-results/subquery/exists-subquery/exists-aggregate.sql.out
@@ -483,7 +483,7 @@ GROUP BY
cast(EXISTS (SELECT id FROM dept where dept.dept_id = emp.dept_id) AS int)
-- !query analysis
Aggregate [cast(exists#x [id#x && dept_id#x] as int)], [cast(exists#x [id#x &&
dept_id#x] as int) AS CAST(exists(id, dept_id) AS INT)#x]
-: :- Project [outer(id#x)]
+: :- Project [outer(id#x) AS id#x]
: : +- Filter (dept_id#x = outer(dept_id#x))
: : +- SubqueryAlias dept
: : +- View (`DEPT`, [dept_id#x, dept_name#x, state#x])
@@ -491,7 +491,7 @@ Aggregate [cast(exists#x [id#x && dept_id#x] as int)],
[cast(exists#x [id#x && d
: : +- Project [dept_id#x, dept_name#x, state#x]
: : +- SubqueryAlias DEPT
: : +- LocalRelation [dept_id#x, dept_name#x, state#x]
-: +- Project [outer(id#x)]
+: +- Project [outer(id#x) AS id#x]
: +- Filter (dept_id#x = outer(dept_id#x))
: +- SubqueryAlias dept
: +- View (`DEPT`, [dept_id#x, dept_name#x, state#x])
diff --git
a/sql/core/src/test/resources/sql-tests/analyzer-results/subquery/scalar-subquery/scalar-subquery-select.sql.out
b/sql/core/src/test/resources/sql-tests/analyzer-results/subquery/scalar-subquery/scalar-subquery-select.sql.out
index 9ca0517a60e8..b1ec23cf4a30 100644
---
a/sql/core/src/test/resources/sql-tests/analyzer-results/subquery/scalar-subquery/scalar-subquery-select.sql.out
+++
b/sql/core/src/test/resources/sql-tests/analyzer-results/subquery/scalar-subquery/scalar-subquery-select.sql.out
@@ -516,7 +516,7 @@ Project [t1a#x, scalar-subquery#x [t1a#x] AS count_t2#xL,
scalar-subquery#x [t1a
SELECT t1c, (SELECT t1c) FROM t1
-- !query analysis
Project [t1c#x, scalar-subquery#x [t1c#x] AS scalarsubquery(t1c)#x]
-: +- Project [outer(t1c#x)]
+: +- Project [outer(t1c#x) AS t1c#x]
: +- OneRowRelation
+- SubqueryAlias t1
+- View (`t1`, [t1a#x, t1b#x, t1c#x, t1d#xL, t1e#x, t1f#x, t1g#x, t1h#x,
t1i#x])
@@ -530,7 +530,7 @@ Project [t1c#x, scalar-subquery#x [t1c#x] AS
scalarsubquery(t1c)#x]
SELECT t1c, (SELECT t1c WHERE t1c = 8) FROM t1
-- !query analysis
Project [t1c#x, scalar-subquery#x [t1c#x && t1c#x] AS scalarsubquery(t1c,
t1c)#x]
-: +- Project [outer(t1c#x)]
+: +- Project [outer(t1c#x) AS t1c#x]
: +- Filter (outer(t1c#x) = 8)
: +- OneRowRelation
+- SubqueryAlias t1
diff --git
a/sql/core/src/test/resources/sql-tests/analyzer-results/udf/postgreSQL/udf-join.sql.out
b/sql/core/src/test/resources/sql-tests/analyzer-results/udf/postgreSQL/udf-join.sql.out
index 09ef51fcf231..9edacd126afe 100644
---
a/sql/core/src/test/resources/sql-tests/analyzer-results/udf/postgreSQL/udf-join.sql.out
+++
b/sql/core/src/test/resources/sql-tests/analyzer-results/udf/postgreSQL/udf-join.sql.out
@@ -3199,7 +3199,7 @@ Project [x#x, f1#x, y#x]
+- LateralJoin lateral-subquery#x [x#x], Inner
: +- SubqueryAlias ss2
: +- Project [x#x AS y#x]
- : +- Project [outer(x#x)]
+ : +- Project [outer(x#x) AS x#x]
: +- OneRowRelation
+- Join Inner, (x#x = cast(f1#x as double))
:- SubqueryAlias ss1
@@ -3229,7 +3229,7 @@ Project [x#x, f1#x, y#x]
+- LateralJoin lateral-subquery#x [x#x], Inner
: +- SubqueryAlias ss2
: +- Project [x#x AS y#x]
- : +- Project [outer(x#x)]
+ : +- Project [outer(x#x) AS x#x]
: +- OneRowRelation
+- SubqueryAlias j
+- Join Inner, (x#x = cast(f1#x as double))
@@ -3261,7 +3261,7 @@ Project [q1#xL, q2#xL, q1#xL, q2#xL, xq1#xL, yq1#xL,
yq2#xL]
+- LateralJoin lateral-subquery#x [q1#xL && q1#xL && q2#xL], Inner
: +- SubqueryAlias v
: +- Project [q1#xL AS xq1#xL, q1#xL AS yq1#xL, q2#xL AS yq2#xL]
- : +- Project [outer(q1#xL), outer(q1#xL), outer(q2#xL)]
+ : +- Project [outer(q1#xL) AS q1#xL, outer(q1#xL) AS q1#xL,
outer(q2#xL) AS q2#xL]
: +- OneRowRelation
+- Join LeftOuter, (q2#xL = q1#xL)
:- SubqueryAlias x
@@ -3292,7 +3292,7 @@ Project [q1#xL, q2#xL]
+- LateralJoin lateral-subquery#x [q1#xL && q1#xL && q2#xL], Inner
: +- SubqueryAlias v
: +- Project [q1#xL AS xq1#xL, q1#xL AS yq1#xL, q2#xL AS yq2#xL]
- : +- Project [outer(q1#xL), outer(q1#xL), outer(q2#xL)]
+ : +- Project [outer(q1#xL) AS q1#xL, outer(q1#xL) AS q1#xL,
outer(q2#xL) AS q2#xL]
: +- OneRowRelation
+- Join LeftOuter, (q2#xL = q1#xL)
:- SubqueryAlias x
diff --git a/sql/core/src/test/resources/sql-tests/results/order-by.sql.out
b/sql/core/src/test/resources/sql-tests/results/order-by.sql.out
index 0fb46cc28646..b9b1ca301c80 100644
--- a/sql/core/src/test/resources/sql-tests/results/order-by.sql.out
+++ b/sql/core/src/test/resources/sql-tests/results/order-by.sql.out
@@ -174,7 +174,7 @@ org.apache.spark.sql.catalyst.ExtendedAnalysisException
"errorClass" :
"UNSUPPORTED_SUBQUERY_EXPRESSION_CATEGORY.UNSUPPORTED_CORRELATED_SCALAR_SUBQUERY",
"sqlState" : "0A000",
"messageParameters" : {
- "treeNode" : "Sort [scalar-subquery#x [a#x] ASC NULLS FIRST], true\n: +-
Project [outer(a#x)]\n: +- LocalRelation [col1#x]\n+- Project [a#x, b#x]\n
+- SubqueryAlias testdata\n +- View (`testData`, [a#x, b#x])\n +-
Project [cast(a#x as int) AS a#x, cast(b#x as int) AS b#x]\n +-
Project [a#x, b#x]\n +- SubqueryAlias testData\n
+- LocalRelation [a#x, b#x]\n"
+ "treeNode" : "Sort [scalar-subquery#x [a#x] ASC NULLS FIRST], true\n: +-
Project [outer(a#x) AS a#x]\n: +- LocalRelation [col1#x]\n+- Project [a#x,
b#x]\n +- SubqueryAlias testdata\n +- View (`testData`, [a#x, b#x])\n
+- Project [cast(a#x as int) AS a#x, cast(b#x as int) AS b#x]\n
+- Project [a#x, b#x]\n +- SubqueryAlias testData\n
+- LocalRelation [a#x, b#x]\n"
},
"queryContext" : [ {
"objectType" : "",
diff --git a/sql/core/src/test/resources/sql-tests/results/sql-udf.sql.out
b/sql/core/src/test/resources/sql-tests/results/sql-udf.sql.out
index b227fd3c9475..3b522f6e47a6 100644
--- a/sql/core/src/test/resources/sql-tests/results/sql-udf.sql.out
+++ b/sql/core/src/test/resources/sql-tests/results/sql-udf.sql.out
@@ -3172,7 +3172,7 @@ org.apache.spark.sql.catalyst.ExtendedAnalysisException
"errorClass" :
"UNSUPPORTED_SUBQUERY_EXPRESSION_CATEGORY.UNSUPPORTED_CORRELATED_SCALAR_SUBQUERY",
"sqlState" : "0A000",
"messageParameters" : {
- "treeNode" : "Join Inner, (spark_catalog.default.foo3_1g(x#x) = 2)\n: +-
Project [outer(x#x)]\n: +- OneRowRelation\n:- SubqueryAlias
spark_catalog.default.t1\n: +- View (`spark_catalog`.`default`.`t1`, [c1#x,
c2#x])\n: +- Project [cast(col1#x as int) AS c1#x, cast(col2#x as int) AS
c2#x]\n: +- LocalRelation [col1#x, col2#x]\n+- SubqueryAlias
spark_catalog.default.t2\n +- View (`spark_catalog`.`default`.`t2`, [c1#x,
c2#x])\n +- Project [cast(col1#x as int) AS c [...]
+ "treeNode" : "Join Inner, (spark_catalog.default.foo3_1g(x#x) = 2)\n: +-
Project [outer(x#x) AS x#x]\n: +- OneRowRelation\n:- SubqueryAlias
spark_catalog.default.t1\n: +- View (`spark_catalog`.`default`.`t1`, [c1#x,
c2#x])\n: +- Project [cast(col1#x as int) AS c1#x, cast(col2#x as int) AS
c2#x]\n: +- LocalRelation [col1#x, col2#x]\n+- SubqueryAlias
spark_catalog.default.t2\n +- View (`spark_catalog`.`default`.`t2`, [c1#x,
c2#x])\n +- Project [cast(col1#x as in [...]
}
}
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
