ulysses-you commented on code in PR #39624:
URL: https://github.com/apache/spark/pull/39624#discussion_r1122539330
##########
sql/core/src/main/scala/org/apache/spark/sql/execution/adaptive/AdaptiveSparkPlanExec.scala:
##########
@@ -220,10 +220,15 @@ case class AdaptiveSparkPlanExec(
}
private def getExecutionId: Option[Long] = {
- // If the `QueryExecution` does not match the current execution ID, it
means the execution ID
- // belongs to another (parent) query, and we should not call update UI in
this query.
Option(context.session.sparkContext.getLocalProperty(SQLExecution.EXECUTION_ID_KEY))
- .map(_.toLong).filter(SQLExecution.getQueryExecution(_) eq context.qe)
+ .map(_.toLong)
+ }
+
+ private lazy val shouldUpdatePlan: Boolean = {
+ // If the `QueryExecution` does not match the current execution ID, it
means the execution ID
+ // belongs to another (parent) query, and we should call update metrics
instead of plan in
+ // this query.
Review Comment:
A cached plan in AdaptiveSparkPlan is not a subquery but we can not update
plan which would overwrite the whole query plan.
e.g. we can not update plan for query execution 0. Instead, we should update
metrics for it.
```
...
|
AdaptiveSparkPlanExec (query execution 0, no execution id)
|
InMemoryTableScanExec
|
...
|
AdaptiveSparkPlanExec (query execution 1, execution id 0)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
