Steve Zhang created SPARK-62102:
-----------------------------------
Summary: Fail pipe |> SET with AMBIGUOUS_REFERENCE when the name
matches two copies of a column
Key: SPARK-62102
URL: https://issues.apache.org/jira/browse/SPARK-62102
Project: Spark
Issue Type: Bug
Components: SQL
Affects Versions: 4.0.0
Reporter: Steve Zhang
A pipe {{|> SET}} assignment to a name that matches two copies of the same
column fails with an internal error instead of {{AMBIGUOUS_REFERENCE}}:
{code:sql}
SELECT x, x FROM t |> SET x = 1;
-- java.lang.IndexOutOfBoundsException: 1
{code}
Both copies are the same attribute, so resolving {{x}} finds one column and
doesn't raise {{AMBIGUOUS_REFERENCE}}, as it does for two different columns
({{SELECT 1 AS a, 2 AS a |> SET a = 3}}). The star expansion in
{{UnresolvedStarExceptOrReplace}} then replaces both copies, and the second
replacement reads past the end of the one-element replacement list.
The assignment should fail with {{AMBIGUOUS_REFERENCE}}, as it does for two
different columns with the name. Pipe {{SET}} has failed this way since it was
added in Spark 4.0.0 (commit 68be1da5ba, under SPARK-49566).
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]