[
https://issues.apache.org/jira/browse/SPARK-61966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated SPARK-61966:
-----------------------------------
Labels: pull-request-available (was: )
> QUALIFY resolves a SELECT alias before a same-named input column
> ----------------------------------------------------------------
>
> Key: SPARK-61966
> URL: https://issues.apache.org/jira/browse/SPARK-61966
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 5.0.0
> Environment: Spark 5.0.0-SNAPSHOT, upstream master at
> \{{dada2a6db5d73b152d6ba16e4fe21406f86abfe3}} (2026-11-15), local mode,
> default SQL settings
> Reporter: Yibo Dong
> Priority: Major
> Labels: pull-request-available
>
> h2. What happened
> Spark resolves an unqualified name in {{QUALIFY}} to a SELECT window alias
> instead of a same-named input column, contrary to the documented input-column
> precedence. This changes which row survives the filter.
> h2. How to reproduce
> Run the following queries in a fresh Spark SQL session:
> {code:sql}
> SELECT id, ROW_NUMBER() OVER (ORDER BY score DESC) AS rn
> FROM VALUES
> (1, 30, 0),
> (2, 20, 0),
> (3, 10, 1)
> AS t(id, score, rn)
> QUALIFY rn = 1
> ORDER BY id;
> -- Control: explicitly reference the input column.
> SELECT id, ROW_NUMBER() OVER (ORDER BY score DESC) AS rn
> FROM VALUES
> (1, 30, 0),
> (2, 20, 0),
> (3, 10, 1)
> AS t(id, score, rn)
> QUALIFY t.rn = 1
> ORDER BY id;
> {code}
> h2. Expected result
> Both queries should return {{(3, 3)}}. The input column {{rn}} takes
> precedence over the SELECT alias, and {{QUALIFY}} filters after window
> evaluation.
> h2. Actual result
> The first query returns {{(1, 1)}}, while the qualified-column control
> returns {{(3, 3)}}.
> The analyzed plan for the first query binds {{rn}} to the window alias
> instead of the input column.
> h2. Additional information
> Spark's QUALIFY documentation explicitly specifies that input columns take
> precedence over same-named SELECT aliases:
> https://spark.apache.org/docs/latest/sql-ref-syntax-qry-select-qualify.html
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]