[ https://issues.apache.org/jira/browse/SPARK-32430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17164481#comment-17164481 ]
Apache Spark commented on SPARK-32430: -------------------------------------- User 'andygrove' has created a pull request for this issue: https://github.com/apache/spark/pull/29224 > Allow plugins to inject rules into AQE query stage preparation > -------------------------------------------------------------- > > Key: SPARK-32430 > URL: https://issues.apache.org/jira/browse/SPARK-32430 > Project: Spark > Issue Type: Bug > Components: SQL > Affects Versions: 3.0.0 > Reporter: Thomas Graves > Priority: Major > > Provide a generic mechanism for plugins to inject rules into the AQE "query > prep" stage that happens before query stage creation. > this goes along with https://issues.apache.org/jira/browse/SPARK-32332 where > the current AQE implementation doesn't allow for users to properly extend it > for columnar processing. > The issue here is that we create new query stages but we do not have access > to the parent plan of the new query stage so certain things can not be > determined because you have to know what the parent did. With this change it > would allow you to add TAGs to be able to figure out what is going on. > -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org