Hans-Martin Keller created SPARK-56921:
------------------------------------------
Summary: Optimizer failure: NormalizeCTEIds brakes CTE references
for queries with nested CTEs
Key: SPARK-56921
URL: https://issues.apache.org/jira/browse/SPARK-56921
Project: Spark
Issue Type: Bug
Components: Optimizer
Affects Versions: 4.1.1, 4.2.0
Environment: Tested with Java API (JUnit test), but also crashed on
Claudera cluster.
Reporter: Hans-Martin Keller
Optimizer crashs in InlineCTE or other rule due to a broken CTERelationRefs.
This only happens for plans with nested WithCTE nodes. The broken CTE relation
is introduced by NormalizeCTEIds.
A simple test case is described in https://github.com/apache/spark/issues/55799.
In short, you have to execute a SQL query with CTE and define the result as
temp view. Then use this temp view in different SQL queries and union the
result. Any operation on the union will fail.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
