Holden Karau created SPARK-48101:
------------------------------------
Summary: When using INSERT OVERWRITE with Spark CTEs they may not
be fully resolved
Key: SPARK-48101
URL: https://issues.apache.org/jira/browse/SPARK-48101
Project: Spark
Issue Type: Bug
Components: SQL
Affects Versions: 3.5.1, 3.4.0, 3.3.0
Reporter: Holden Karau
Repro:
```sql
DROP TABLE IF EXISTS local.cte1;
DROP TABLE IF EXISTS local.cte2;
DROP TABLE IF EXISTS local.cte3;
CREATE TABLE local.cte1 (id INT, fname STRING);
CREATE TABLE local.cte2 (id2 INT);
CREATE TABLE local.cte3 (id INT);
WITH test_fake AS (SELECT * FROM local.cte1 WHERE id = 1 AND id2 = 1),
test_fake2 AS (SELECT * FROM local.cte2 WHERE id2 = 1) INSERT OVERWRITE TABLE
local.cte3 SELECT id2 as id FROM test_fake2;
WITH test_fake AS (SELECT * FROM local.cte1 WHERE id = 1 AND id2 = 1),
test_fake2 AS (SELECT * FROM local.cte2 WHERE id2 = 1) SELECT id2 as id FROM
test_fake2;
```
Here we would expect both of the last two SQL expressions to fail, but instead
only the first one does.
There are more complicated cases, and in those cases, the invalid CTE is
treated as a null table, but this is the simplest repro I've been able to come
up with so far.
This occurs using both local w/Iceberg catalog or the SparkSession catalog.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
