[
https://issues.apache.org/jira/browse/SPARK-39551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dongjoon Hyun updated SPARK-39551:
----------------------------------
Fix Version/s: 3.2.2
> Add AQE invalid plan check
> --------------------------
>
> Key: SPARK-39551
> URL: https://issues.apache.org/jira/browse/SPARK-39551
> Project: Spark
> Issue Type: Improvement
> Components: SQL
> Affects Versions: 3.3.0
> Reporter: Wei Xue
> Assignee: Wei Xue
> Priority: Minor
> Fix For: 3.2.2, 3.4.0, 3.3.1
>
>
> AQE logical optimization rules can lead to invalid physical plans as certain
> physical plan nodes are not compatible with others. E.g.,
> `BroadcastExchangeExec` can only work as a direct child of broadcast join
> nodes.
> Logical optimizations, on the other hand, are not (and should not be) aware
> of such restrictions. So a general solution here is to check for invalid
> plans and throw exceptions, which can be caught by AQE replanning process.
> And if such an exception is captured, AQE can void the current replanning
> result and keep using the latest valid plan.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
