zentol commented on a change in pull request #10082: [FLINK-14164][runtime] Add
a meter ‘numberOfRestarts’ to show number of restarts as well as its rate
URL: https://github.com/apache/flink/pull/10082#discussion_r343059023
##########
File path:
flink-runtime/src/main/java/org/apache/flink/runtime/scheduler/SchedulerBase.java
##########
@@ -193,6 +197,11 @@ public SchedulerBase(
this.failoverTopology = executionGraph.getFailoverTopology();
this.inputsLocationsRetriever = new
ExecutionGraphToInputsLocationsRetrieverAdapter(executionGraph);
+
+ // Use the counter from execution graph to avoid modifying
execution graph interfaces
+ // Can be a new SimpleCounter created here after the legacy
scheduler is removed.
+ this.numberOfRestartsCounter =
executionGraph.getNumberOfRestartsCounter();
+ jobManagerJobMetricGroup.meter(NUMBER_OF_RESTARTS, new
MeterView(numberOfRestartsCounter));
Review comment:
I agree that the meter doesn't make sense; the volume of events is simply to
low.
You'd want something it to work on hours I suppose, which is possible to
implement (it's just a slightly modified MeterView, see below for the
differences), but we'd technically violate the meter interface as it explicitly
says that rates are per second.
And I am wondering whether it wouldn't be perfectly fine to push this
responsibility to the user, given that all time-series databases support
queries that can derive the rate in whatever granularity they desire.
That's my overall stance on metrics anyway; expose the basic raw data, and
have users build more sophisticated things on top, with the only exception
being things we want in the WebUI.
How to implement a HourMeter:
- take the `MeterView`
- pin the values array to 60 values
- don't divide by the timeSpan when calculating the rate
- only act on every 12th update() call
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
