heyihong commented on code in PR #53801:
URL: https://github.com/apache/spark/pull/53801#discussion_r2696334809
##########
sql/api/src/main/scala/org/apache/spark/sql/Observation.scala:
##########
@@ -58,12 +57,15 @@ class Observation(val name: String) {
private val isRegistered = new AtomicBoolean()
- private val promise = Promise[Row]()
+ private val promise = Promise[() => Row]()
+
+ private val lazyMetricsFuture: Future[() => Row] = promise.future
/**
- * Future holding the (yet to be completed) observation.
+ * Future holding the (yet to be completed) observation. Lazy to avoid
collecting the metrics
+ * until it is needed.
*/
- val future: Future[Row] = promise.future
+ lazy val future: Future[Row] =
lazyMetricsFuture.map(_())(ExecutionContext.global)
Review Comment:
@cloud-fan Per @hvanhovell's comment, I let Observation.tryComplete handle
the metric collection instead, so no thread pool is needed, though
SQLExecution.withNewExecutionId() may take longer to finish
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
