skonto edited a comment on pull request #34039: URL: https://github.com/apache/spark/pull/34039#issuecomment-926435483
@BOOTMGR imho triggerExecution time looks a good candidate to be part of the metrics that should be exposed to the metrics system if needed automatically (what else is available from the query reporting info?) without having to depend on the listener. >I do not want to push it at every batch end (expensive sink), rather I would like to have them flushed along with other metrics via MetricsSystem. Makes sense if the sink is too expensive. However, in a pull model things are not much different, metrics will be sampled anyway at some interval eg. Prometheus, via some metrics exporter, independently of flushing. So the equivalent in terms of costs, for a push model would be to push at some appropriate rate. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
