[ https://issues.apache.org/jira/browse/SPARK-35093?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Thomas Graves resolved SPARK-35093. ----------------------------------- Fix Version/s: 3.2.0 3.1.2 3.0.3 Assignee: Andy Grove Resolution: Fixed > AQE columnar mismatch on exchange reuse > --------------------------------------- > > Key: SPARK-35093 > URL: https://issues.apache.org/jira/browse/SPARK-35093 > Project: Spark > Issue Type: Sub-task > Components: SQL > Affects Versions: 3.0.2, 3.1.1 > Reporter: Andy Grove > Assignee: Andy Grove > Priority: Major > Fix For: 3.0.3, 3.1.2, 3.2.0 > > > With AQE enabled, AdaptiveSparkPlanExec will attempt to reuse exchanges that > are semantically equal. > This is done by comparing the canonicalized plan for two Exchange nodes to > see if they are the same. > Unfortunately this does not take into account the fact that two exchanges > with the same canonical plan might be replaced by a plugin in a way that > makes them not compatible. For example, a plugin could create one version > with supportsColumnar=true and another with supportsColumnar=false. It is not > valid to re-use exchanges if there is a supportsColumnar mismatch. > I have tested a fix for this and will put up a PR once I figure out how to > write the tests. > > -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org