Hi Brian, Can you please correct the summary of my understanding?!
1. With Prometheus as a metrics storage and PromQL provider, the exact values of the Max and Min stats of a histogram can't be fetched as of now. This is not possible with Prometheus Classic Histograms, Prometheus Native Histograms, or collecting OTEL instrumented metrics via OTEL Collector in Prometheus (as a Metrics Store/Backend and PromQL Provider). 2. Esp with Classic Histograms, Min/Max are not provided in a standard way. If these values are needed, separate gauges to capture these values are required. Even such an implementation is not ideal when there is high cardinality and dimensioning involved with the request. 3. Event-based logs and traces may give such exact information compared to Prometheus Histograms. 4. With cumulative histograms, it is possible to get Max & Min values for a specific duration. 5. With Delta histograms, it is possible to secure these values. But these are not supported by Prometheus and OTEL today. Thanks, Teja On Monday, June 23, 2025 at 11:25:38 PM UTC+2 Brian Candler wrote: > See also > https://prometheus.io/docs/specs/native_histograms/#opentelemetry-interoperability > > I note that native histograms have exponentially-varying sizes. In > principle, if you set the factors appropriately (e.g. say scaling of 1.1 = > each bucket is 10% larger than the next) then the answers you get for min > and max should be within 10% of the actual answer. But you won't get the > *exact* minimum or maximum which is what you were asking for. > > On Monday, 23 June 2025 at 22:13:56 UTC+1 Brian Candler wrote: > >> On Monday, 23 June 2025 at 14:31:34 UTC+1 tejaswini vadlamudi wrote: >> >> but with histograms, mainly with Native Histograms, is there a >> possibility to get Max and Min values for a period of time? >> >> >> No. If that's not clear from my explanations so far, then I have not >> done a very good job. >> >> >> >> With OTEL-based metrics instrumentation, it is possible to record max and >> min values. See >> https://opentelemetry.io/docs/specs/otel/metrics/data-model/#histogram >> >> >> Please read the link I posted before: >> >> https://github.com/open-telemetry/opentelemetry-collector-contrib/issues/33645 >> especially the last comment: >> >> https://github.com/open-telemetry/opentelemetry-collector-contrib/issues/33645#issuecomment-2514894327 >> >> The OTEL histogram's way of reporting max/min values are not compatible >> with Prometheus. Prometheus histograms are cumulative: that is, they never >> reset. They are just counters which keep incrementing forever. If they had >> a min/max then it would be the min/max over all time, which is not useful. >> >> You can create a separate timeseries for min/max, but it would not be >> part of the histogram. It would either have to be min/max over successive >> timeslots (e.g. 1 minute intervals), or min/max over a sliding window >> (which is expensive, as you've have to buffer all the samples for that time >> window). >> >> > I think histogram_quantile(0, >> rate(http_request_duration_seconds[1m])) will give me min value. Is it >> correct? >> >> No, it's not. Again, it seems I have not been explaining this well. >> >> Let's say your histogram buckets are: >> >> 0-10ms (le="0.01") >> 10-20ms (le="0.02") >> 20-30ms (le="0.03") >> 30-40ms (le="0.04") >> 40-50ms (le="0.05") >> 50ms+ (le="inf") >> >> A sample comes in with request time of 12.3ms, and this is the fastest >> over the time period of interest. >> >> All this does is add 1 to the 10-20ms bucket (counter) in the histogram. >> The actual value of 12.3ms is LOST. The only thing you will know from >> looking at the histogram is that the 10-20ms bucket has incremented, but >> the 0-10ms bucket has not, and therefore the minimum value is somewhere >> between 10ms and 20ms. If you use the histogram_quantile(0...) formula you >> showed, the answer will be "0.01", which I also explained previously with a >> worked example. This tells you "the minimum value is at least 10ms" - and >> implicitly not more than 20ms, as that's where the next bucket starts. >> >> I think I had better hand over to someone else now. >> > -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-users+unsubscr...@googlegroups.com. To view this discussion visit https://groups.google.com/d/msgid/prometheus-users/e9200734-db00-4ab8-9bf4-c2745504357fn%40googlegroups.com.
