srowen opened a new pull request #26427: [SPARK-29795][CORE] Explicitly clear registered metrics on MetricSystem shutdown URL: https://github.com/apache/spark/pull/26427 ### What changes were proposed in this pull request? Explicitly clear registered metrics when `MetricsSystem` shuts down. ### Why are the changes needed? See https://issues.apache.org/jira/browse/SPARK-29795 for a complete explanation. The TL;DR is there is some evidence this could leak resources after Spark is shut down, and that may be a minor issue in Spark 3+ for apps or tests that re-start SparkContexts in the same JVM. ### Does this PR introduce any user-facing change? The possible difference here is that, after Spark is stopped, metrics are no longer available. It's unclear to me whether this is intended behavior anyway. ### How was this patch tested? See https://issues.apache.org/jira/browse/SPARK-29795 for more context: - Spark 3 already passes tests without this change - Spark 2.4 does too, as exists in branch-2.4 now - Spark 2.4 fails tests if metrics 4.x is used, without this change The last point is not directly relevant, as Spark 2.4 will not use metrics 4.x. It's evidence that it addresses some potential issue, however.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
