ulysses-you commented on code in PR #39624:
URL: https://github.com/apache/spark/pull/39624#discussion_r1130968837
##########
sql/core/src/main/scala/org/apache/spark/sql/execution/adaptive/AdaptiveSparkPlanExec.scala:
##########
@@ -220,10 +221,28 @@ case class AdaptiveSparkPlanExec(
}
private def getExecutionId: Option[Long] = {
- // If the `QueryExecution` does not match the current execution ID, it
means the execution ID
- // belongs to another (parent) query, and we should not call update UI in
this query.
Option(context.session.sparkContext.getLocalProperty(SQLExecution.EXECUTION_ID_KEY))
- .map(_.toLong).filter(SQLExecution.getQueryExecution(_) eq context.qe)
+ .map(_.toLong)
+ }
+
+ private lazy val shouldUpdatePlan: Boolean = {
+ // If the `QueryExecution` does not match the current execution ID, it
means the execution ID
+ // belongs to another (parent) query, and we should call update metrics
instead of plan in
+ // this query. For example:
+ //
+ // ...
+ // |
+ // AdaptiveSparkPlanExec (query execution 0, no execution id)
+ // |
+ // InMemoryTableScanExec
+ // |
+ // ...
+ // |
+ // AdaptiveSparkPlanExec (query execution 1, execution id 0)
Review Comment:
They are two query executions. How can the outer AQE work through IMR if we
replace the nested AQE with `CacheTableQueryStageExec`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
