szhem commented on a change in pull request #24556: [SPARK-27641][CORE] Fix MetricsSystem to remove unregistered source correctly URL: https://github.com/apache/spark/pull/24556#discussion_r284139036
########## File path: core/src/main/scala/org/apache/spark/metrics/source/AccumulatorSource.scala ########## @@ -64,10 +64,11 @@ class DoubleAccumulatorSource extends AccumulatorSource */ @Experimental object LongAccumulatorSource { - def register(sc: SparkContext, accumulators: Map[String, LongAccumulator]): Unit = { + def register(sc: SparkContext, accumulators: Map[String, LongAccumulator]): Source = { Review comment: `AccumulatorSource` is package private as well. So to extend it, it will be necessary to place your code within the same package. So there is no need to expose package private `Source` and make it a part of the public contract. To implement tests without changing the API you could use something like the following ```scala val source = new LongAccumulatorSource source.register(accumulators) sc.env.metricsSystem.registerSource(source) ``` ... instead of ```scala LongAccumulatorSource.register(mockContext, Map("laF" -> laFirst, "laS" -> laSecond)) ``` So the necessary source to delete during the tests will be already stored in the local variable. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org