cloud-fan commented on a change in pull request #23396: [SPARK-26397][SQL]
Driver side only metrics support
URL: https://github.com/apache/spark/pull/23396#discussion_r246699968
##########
File path:
sql/core/src/main/scala/org/apache/spark/sql/execution/SparkPlan.scala
##########
@@ -74,10 +74,22 @@ abstract class SparkPlan extends QueryPlan[SparkPlan] with
Logging with Serializ
}
/**
- * @return All metrics containing metrics of this SparkPlan.
+ * @return Executor side metrics of this SparkPlan, which needs to be
serialized and sent
+ * to executors.
*/
def metrics: Map[String, SQLMetric] = Map.empty
+ /**
+ * @return Driver side metrics of this SparkPlan, which only create, update
and display
+ * on driver side, no need to be serialized and sent to executors.
+ */
+ def driverMetrics: Map[String, SQLMetric] = Map.empty
+
+ /**
+ * @return All metrics of this SparkPlan.
+ */
+ @inline def allMetrics: Map[String, SQLMetric] = metrics ++ driverMetrics
Review comment:
when I say inline I mean remove this method and put the method body to where
we use it.
`(plan.metrics ++ plan.driverMetrics).toSeq.map ...`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
