Thanks everyone for reviewing the FLIP I’ll open the voting thread today, as all feedback seems to be addressed. Efrat
On Wed, 12 Aug 2026 at 14:17 Piotr Nowojski <[email protected]> wrote: > Thanks Efrat. > > +1 for this. > > Best, > Piotrek > > wt., 11 sie 2026 o 16:45 Efrat Levitan <[email protected]> napisał(a): > > > Thanks for the feedback Piotr > > Based on offline discussions I updated FLIP-595. > > With an additional MonotonicCounter interface we will migrate system > > counters to, > > while declaring Counter#dec an optional operation (in addition to its > > deprecation) > > Monotonicity by contract is orthogonal to the removal of Counter#dec due > in > > the next major release. > > Regards, > > Efrat > > > > On Thu, 23 Jul 2026 at 12:25, Piotr Nowojski <[email protected]> > wrote: > > > > > Hi, > > > > > > Yes, it also sounds generally speaking good to me. > > > > > > I'm just not sure whether we should make the existing `Counter` > > monotonic, > > > or introduce the new `MonotonicCounter`. > > > If we do the former, how would you propose Efrat to do that? Just > > > deprecating `Counter#dec` in let's say Flink 2.4, > > > will not make `Counter` monotonic. That will happen only once we drop > > > `Counter#dec` (in? 2.5? 2.6? 3.0?). Or am I > > > missing something? > > > > > > Best, > > > Piotrek > > > > > > czw., 2 lip 2026 o 23:26 Ryan van Huuksloot via dev < > > [email protected]> > > > napisał(a): > > > > > > > Hi Efrat, > > > > > > > > Sorry for the delay. > > > > > > > > Overall this sounds good to me. We haven't encountered this too often > > > but I > > > > do know what you are referring to; it's annoying to know if the > counter > > > was > > > > decremented versus reset. > > > > > > > > I'm not well versed in all of the touchpoints to give the best > guidance > > > but > > > > it does make sense and is a really nice quality of life improvement > for > > > > observability. > > > > > > > > Ryan van Huuksloot > > > > Staff Engineer, Infrastructure | Streaming Platform > > > > [image: Shopify] > > > > < > > https://www.shopify.com/?utm_medium=salessignatures&utm_source=hs_email > > > > > > > > > > > > > > > > On Thu, Jun 25, 2026 at 1:24 PM Efrat Levitan <[email protected] > > > > > > wrote: > > > > > > > > > Hey Ryan, > > > > > Thank you for reading through the description. > > > > > I published the FLIP in hope it will provides a better > understanding > > > > > of the proposal [1] > > > > > > > > > > > Is the issue you are trying to resolve that we don't know if a > > > Counter > > > > > is monotonic? > > > > > Yes this is the main concern of the FLIP, distinguishing > monotonicity > > > > > so backend systems can tell that there was integer overflow or > system > > > > > restart during a given time and are able to smooth it out in > graphs. > > > > > If Counter.dec was widely used in flink we were having a different > > > > > discussion, but since it was only historically introduced > (Dropwizard > > > > > compatibility?) and never used within our codebase, I believe > > > > > declaring monotonicity is worth deprecating Counter.dec (+ > providing > > > > > the UpDownCounter alternative) > > > > > > > > > > > Prometheus casts it to a Gauge because that code path hasn't been > > > > > upgraded in years > > > > > Prometheus' latest java client still throws[2] if you try to > decrease > > > > > it. This is by design, and the reason flink counters are mapped to > > > > > prom gauges. > > > > > You didn't share more about the internal implementation but if it > > uses > > > > > prometheus counters, I'd assume negative deltas are ignored as in > > OTel > > > > > reporter today > > > > > This FLIP can actually pave the way for it to be contributed- > > > > > with true monotonicity, Flink counter will correctly map to > > prometheus > > > > > Counter (please check out the "proposed change" section of the > FLIP) > > > > > > > > > > Let me know what you think. > > > > > Efrat > > > > > > > > > > [1] https://cwiki.apache.org/confluence/x/twDuGQ > > > > > [2] > > > > > > > > > > > > > > > https://github.com/prometheus/client_java/blob/main/prometheus-metrics-core/src/main/java/io/prometheus/metrics/core/metrics/Counter.java#L189 > > > > > > > > > > On Mon, 22 Jun 2026 at 17:23, Ryan van Huuksloot via dev > > > > > <[email protected]> wrote: > > > > > > > > > > > > Hi Efrat, > > > > > > > > > > > > I'd like to clarify the intention of the FLIP, as it seems like > we > > > are > > > > > > introducing multiple issues. > > > > > > > > > > > > Is the issue you are trying to resolve that we don't know if a > > > Counter > > > > is > > > > > > monotonic? > > > > > > > > > > > > After reading the links and the FLIP description, it sounds like > we > > > > need > > > > > to > > > > > > fix the OTEL reporter. I don't immediately see any major reason > to > > > > > overhaul > > > > > > the entire counter system. The current Counter can go up and > down, > > > but > > > > > the > > > > > > OTEL reporter didn't respect that. > > > > > > > > > > > > I also wanted to point out that Prometheus casts it to a Gauge > > > because > > > > > that > > > > > > code path hasn't been upgraded in years. It should use the Native > > > > > Counters > > > > > > that now exist in Prometheus, we just haven't done the work to > > > > > > upgrade/migrate. We have an internal implementation to move Flink > > > > > Counters > > > > > > to Prometheus Native Counters. > > > > > > > > > > > > Is there something I am missing? > > > > > > > > > > > > Ryan van Huuksloot > > > > > > Staff Engineer, Infrastructure | Streaming Platform > > > > > > [image: Shopify] > > > > > > < > > > > > > https://www.shopify.com/?utm_medium=salessignatures&utm_source=hs_email > > > > > > > > > > > > > > > > > > > > > > > > On Mon, Jun 15, 2026 at 2:07 PM Efrat Levitan < > > [email protected] > > > > > > > > > wrote: > > > > > > > > > > > > > Hi all, > > > > > > > I would like to start a discussion about how to differentiate > > > > > > > monotonic and non monotonic counters in flink metrics. > > > > > > > > > > > > > > Monotonic (ever-increasing) Counters can benefit from automatic > > > reset > > > > > > > detection on the monitoring system side (when the value drops > we > > > can > > > > > > > safely assume the process was reset and synthetically adjust > the > > > > > > > value) > > > > > > > > > > > > > > Historically, flink Counter can be decremented / incremented > by a > > > > > > > non-positive value, but this has almost never been used > > > intentionally > > > > > > > across the flink codebase, i.e if a counter got decremented > this > > is > > > > > > > usually a bug [1]. > > > > > > > So though system-emitted counters are effectively monotonic, > > > > exporters > > > > > > > must respect org.apache.flink.metrics.Counter contract and > assume > > > > > > > non-monotonicity. > > > > > > > > > > > > > > I'd like to propose deprecating > > > org.apache.flink.metrics.Counter#dec > > > > > > > in favor of a new UpDownCounter implementation. This matches > > modern > > > > > > > metric APIs like OTel, where the regular Counter is > monotonic[2] > > > and > > > > > > > an additional UpDownCounter supports[3] non-positive additions. > > > > > > > > > > > > > > While it seems to be the cleanest approach, we could still > avoid > > > the > > > > > > > deprecation by introducing a MonotonicCounter and have all > flink > > > > > > > counters migrated, or expand the Counter interface to declare > > > > > > > monotonicity (based on the implementation). > > > > > > > > > > > > > > Recognising monotonicity will also align counters reporting > > across > > > > > > > monitoring systems. Today, for instance, Otel reporter drops[4] > > > > > > > non-incremental data points with a warning, while Prometheus > > > reporter > > > > > > > casts[5] them as Gauges. > > > > > > > > > > > > > > I'm looking forward to your feedback > > > > > > > Efrat > > > > > > > > > > > > > > [1] https://issues.apache.org/jira/browse/FLINK-39892 > > > > > > > [2] > > > > > > > > > > > > > > > > > > > > > > https://github.com/open-telemetry/opentelemetry-java/blob/main/api/all/src/main/java/io/opentelemetry/api/metrics/LongCounter.java#L40 > > > > > > > [3] > > > > > > > > > > > > > > > > > > > > > > https://github.com/open-telemetry/opentelemetry-java/blob/main/api/all/src/main/java/io/opentelemetry/api/metrics/LongUpDownCounter.java > > > > > > > [4] https://issues.apache.org/jira/browse/FLINK-39893 > > > > > > > [5] > > > > > > > > > > > > > > > > > > > > > > https://github.com/apache/flink/blob/master/flink-metrics/flink-metrics-prometheus/src/main/java/org/apache/flink/metrics/prometheus/AbstractPrometheusReporter.java#L177-L184 > > > > > > > > > > > > > > > > > > > > > >
