LucaCanali opened a new pull request #28528: URL: https://github.com/apache/spark/pull/28528
### What changes were proposed in this pull request? The Apache Spark metrics system provides many useful insights on the Spark workload. In particular, the [executor source metrics](https://github.com/apache/spark/blob/master/docs/monitoring.md#component-instance--executor) provide detailed info, including the number of active tasks, some I/O metrics, and task metrics details. The executor source metrics, contrary to other sources (for example ExecutorMetrics source), is not available when running in local mode. This PR proposes to register the executor source with the Spark metrics system when running in local mode. ### Why are the changes needed? Having executor metrics in local mode can be very useful when testing and troubleshooting Spark workloads in a development environment. ### Does this PR introduce _any_ user-facing change? - Adds executor source metrics when running in local mode - Adds a configuration parameter to switch on/off the executor metrics: `spark.metrics.executorSource.enabled` (default is true) ### How was this patch tested? Manually tested + added tests in `SourceConfigSuite` ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
