xkrogen opened a new pull request #33523:
URL: https://github.com/apache/spark/pull/33523
`ExternalBlockHandler` exposes 3 metrics which are Dropwizard `Timer`
metrics, and are named with a `millis` suffix:
```
private final Timer openBlockRequestLatencyMillis = new Timer();
private final Timer registerExecutorRequestLatencyMillis = new Timer();
private final Timer finalizeShuffleMergeLatencyMillis = new Timer();
```
However these Dropwizard Timers by default use nanoseconds
([documentation](https://metrics.dropwizard.io/3.2.3/getting-started.html#timers)).
This PR adds a new `Timer` subclass, `TimerWithCustomTimeUnit`, which
accepts a `TimeUnit` at creation time and exposes timing information using this
time unit when values are read. Internally, values are still stored with
nanosecond-level precision. The `Timer` metrics within `ExternalBlockHandler`
are updated to use the new class with milliseconds as the unit.
This introduces a user-facing change as the values of the metrics themselves
have changed, to expose milliseconds instead of nanoseconds. Note that this
does not affect metrics such as `openBlockRequestLatencyMillis_count` or
`openBlockRequestLatencyMillis_rate1`, only the `Snapshot`-related metrics
(`max`, `median`, percentiles, etc.). For the YARN case, these metrics are not
exposed prior to 3.2, so there is no change. It was possible for the nanosecond
values to be consumed by some other metrics reporter reading the Dropwizard
metrics directly, but I'm not aware of any such usages, so this PR is primarily
code cleanup and to make branch 3.1 match with 3.2/master.
Note that this differs from the master/3.2 version (#33116) primarily
because there are no changes needed in `YarnShuffleServiceMetrics`.
Unit tests have been updated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
