[
https://issues.apache.org/jira/browse/SPARK-39551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17557199#comment-17557199
]
Apache Spark commented on SPARK-39551:
--------------------------------------
User 'maryannxue' has created a pull request for this issue:
https://github.com/apache/spark/pull/36953
> Add AQE invalid plan check
> --------------------------
>
> Key: SPARK-39551
> URL: https://issues.apache.org/jira/browse/SPARK-39551
> Project: Spark
> Issue Type: Improvement
> Components: SQL
> Affects Versions: 3.3.0
> Reporter: Wei Xue
> Priority: Minor
>
> AQE logical optimization rules can lead to invalid physical plans as certain
> physical plan nodes are not compatible with others. E.g.,
> `BroadcastExchangeExec` can only work as a direct child of broadcast join
> nodes.
> Logical optimizations, on the other hand, are not (and should not be) aware
> of such restrictions. So a general solution here is to check for invalid
> plans and throw exceptions, which can be caught by AQE replanning process.
> And if such an exception is captured, AQE can void the current replanning
> result and keep using the latest valid plan.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
