Yibo Dong created SPARK-61966:
---------------------------------

             Summary: QUALIFY resolves a SELECT alias before a same-named input 
column
                 Key: SPARK-61966
                 URL: https://issues.apache.org/jira/browse/SPARK-61966
             Project: Spark
          Issue Type: Bug
          Components: SQL
    Affects Versions: 5.0.0
         Environment: Spark 5.0.0-SNAPSHOT, upstream master at 
\{{dada2a6db5d73b152d6ba16e4fe21406f86abfe3}} (2026-11-15), local mode, default 
SQL settings
            Reporter: Yibo Dong


h2. What happened

Spark resolves an unqualified name in {{QUALIFY}} to a SELECT window alias 
instead of a same-named input column, contrary to the documented input-column 
precedence. This changes which row survives the filter.

h2. How to reproduce

Run the following queries in a fresh Spark SQL session:

{code:sql}
SELECT id, ROW_NUMBER() OVER (ORDER BY score DESC) AS rn
FROM VALUES
(1, 30, 0),
(2, 20, 0),
(3, 10, 1)
AS t(id, score, rn)
QUALIFY rn = 1
ORDER BY id;

-- Control: explicitly reference the input column.
SELECT id, ROW_NUMBER() OVER (ORDER BY score DESC) AS rn
FROM VALUES
(1, 30, 0),
(2, 20, 0),
(3, 10, 1)
AS t(id, score, rn)
QUALIFY t.rn = 1
ORDER BY id;
{code}

h2. Expected result

Both queries should return {{(3, 3)}}. The input column {{rn}} takes precedence 
over the SELECT alias, and {{QUALIFY}} filters after window evaluation.

h2. Actual result

The first query returns {{(1, 1)}}, while the qualified-column control returns 
{{(3, 3)}}.

The analyzed plan for the first query binds {{rn}} to the window alias instead 
of the input column.

h2. Additional information

Spark's QUALIFY documentation explicitly specifies that input columns take 
precedence over same-named SELECT aliases:

https://spark.apache.org/docs/latest/sql-ref-syntax-qry-select-qualify.html



--
This message was sent by Atlassian Jira
(v10.3.26#10030026)

---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to