squito commented on issue #25156: [SPARK-28394][SCHEDULER]: Performance Metrics for Driver URL: https://github.com/apache/spark/pull/25156#issuecomment-511557418 I have only very very briefly looked at this -- but I will be honest, while I do think it would be good expose more metrics on the driver, this approach seems very complicated for what its trying to do. I don't think this info is well suited to the spark UI -- it seems too tightly connected to internal details, I think it would be overwhelming for 99% of users. As an alternative, what about exposing this info in the metrics System? You could change `EventLoop` to expose the queue depth as a metric, and perhaps even add metrics on processing time etc. as part of that?
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
