[
https://issues.apache.org/jira/browse/SPARK-48101?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Holden Karau updated SPARK-48101:
---------------------------------
Priority: Minor (was: Major)
> When using INSERT OVERWRITE with Spark CTEs they may not be fully resolved
> --------------------------------------------------------------------------
>
> Key: SPARK-48101
> URL: https://issues.apache.org/jira/browse/SPARK-48101
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 3.3.0, 3.4.0, 3.5.1
> Reporter: Holden Karau
> Priority: Minor
>
> Repro:
> ```sql
> DROP TABLE IF EXISTS local.cte1;
> DROP TABLE IF EXISTS local.cte2;
> DROP TABLE IF EXISTS local.cte3;
> CREATE TABLE local.cte1 (id INT, fname STRING);
> CREATE TABLE local.cte2 (id2 INT);
> CREATE TABLE local.cte3 (id INT);
> WITH test_fake AS (SELECT * FROM local.cte1 WHERE id = 1 AND id2 = 1),
> test_fake2 AS (SELECT * FROM local.cte2 WHERE id2 = 1) INSERT OVERWRITE TABLE
> local.cte3 SELECT id2 as id FROM test_fake2;
> WITH test_fake AS (SELECT * FROM local.cte1 WHERE id = 1 AND id2 = 1),
> test_fake2 AS (SELECT * FROM local.cte2 WHERE id2 = 1) SELECT id2 as id FROM
> test_fake2;
> ```
>
> Here we would expect both of the last two SQL expressions to fail, but
> instead only the first one does.
>
> There are more complicated cases, and in those cases, the invalid CTE is
> treated as a null table, but this is the simplest repro I've been able to
> come up with so far.
>
> This occurs using both local w/Iceberg catalog or the SparkSession catalog.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
