xkrogen commented on pull request #33116: URL: https://github.com/apache/spark/pull/33116#issuecomment-878521045
Just put up a new diff which implements a custom `Timer` subclass, `TimerWithMillisecondSnapshots`, which acts the same as a normal `Timer` and stores nanoseconds internally, but exposes all values as milliseconds. It works pretty cleanly. For `YarnShuffleServiceMetrics`, I removed the `nanos` suffix that the reporter was adding, so it is now unit-agnostic. Note that this suffix only appeared in 3.2.0 (via SPARK-25358 / #32388), so it hasn't been released yet and we can safely remove it if we get this PR into `branch-3.2`. My only concern with this approach is if some other metrics reporter (besides `YarnShuffleService`) may try to use these custom timers as if they still had nanosecond units. I'm not familiar enough with how the metrics are used in the non-YARN context to know if this will be an issue, so input is welcomed here. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
