[
https://issues.apache.org/jira/browse/SPARK-32430?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-32430:
------------------------------------
Assignee: (was: Apache Spark)
> Allow plugins to inject rules into AQE query stage preparation
> --------------------------------------------------------------
>
> Key: SPARK-32430
> URL: https://issues.apache.org/jira/browse/SPARK-32430
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 3.0.0
> Reporter: Thomas Graves
> Priority: Major
>
> Provide a generic mechanism for plugins to inject rules into the AQE "query
> prep" stage that happens before query stage creation.
> this goes along with https://issues.apache.org/jira/browse/SPARK-32332 where
> the current AQE implementation doesn't allow for users to properly extend it
> for columnar processing.
> The issue here is that we create new query stages but we do not have access
> to the parent plan of the new query stage so certain things can not be
> determined because you have to know what the parent did. With this change it
> would allow you to add TAGs to be able to figure out what is going on.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
