maryannxue commented on a change in pull request #28275:
URL: https://github.com/apache/spark/pull/28275#discussion_r411726964
##########
File path:
sql/core/src/main/scala/org/apache/spark/sql/execution/QueryExecution.scala
##########
@@ -274,18 +275,17 @@ object QueryExecution {
* are correct, insert whole stage code gen, and try to reduce the work done
by reusing exchanges
* and subqueries.
*/
- private[execution] def preparations(sparkSession: SparkSession):
Seq[Rule[SparkPlan]] = {
-
- val sparkSessionWithAqeOff = getOrCloneSessionWithAqeOff(sparkSession)
-
+ private[execution] def preparations(
+ sparkSession: SparkSession,
+ adaptiveExecutionRule: Option[InsertAdaptiveSparkPlan] = None):
Seq[Rule[SparkPlan]] = {
+ // `AdaptiveSparkPlanExec` is a leaf node. If inserted, all the following
rules will be no-op
+ // as the original plan is hidden behind `AdaptiveSparkPlanExec`.
+ adaptiveExecutionRule.toSeq ++
Seq(
- // `AdaptiveSparkPlanExec` is a leaf node. If inserted, all the
following rules will be no-op
- // as the original plan is hidden behind `AdaptiveSparkPlanExec`.
- InsertAdaptiveSparkPlan(AdaptiveExecutionContext(sparkSession)),
// If the following rules apply, it means the main query is not AQE-ed,
so we make sure the
// subqueries are not AQE-ed either.
- PlanDynamicPruningFilters(sparkSessionWithAqeOff),
Review comment:
We do not need to turn off AQE here, as those rules will call
`prepareExecutedPlan` without an `adaptiveExecutionRule`.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
