xuanyuanking commented on a change in pull request #23371: [SPARK-26223][SQL]
Track metastore operation time in scan node
URL: https://github.com/apache/spark/pull/23371#discussion_r243725187
##########
File path:
sql/core/src/main/scala/org/apache/spark/sql/execution/ui/SQLAppStatusListener.scala
##########
@@ -273,6 +281,12 @@ class SQLAppStatusListener(
exec.metricsValues = aggregateMetrics(exec)
exec.completionTime = Some(new Date(time))
exec.endEvents += 1
+ // Update both physicalPlanDescription and planGraph while the
description changed.
Review comment:
All the changes in `SQLAppStatusListener` and `SQLExecution` in e29ad45 is
to support metastore operation phase happened in
`HiveTableScanExec.rawPartitions`. Different from other phases, `rawPartitions`
was initialized in execution but the UI and plan description generated before
execution. If we think it's not worth to change the execution end event, the
changes and metrics for rawPartitions will revert together.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
