Also if you want to take this further, you can attach an "exemplar" to each bucket - one specific example request which generated a latency point within that range. This makes it very easy to jump to a specific log or trace with a latency within that range.
Google for "prometheus exemplar" for more info. Note that it's currently an experimental feature which needs to be enabled. https://prometheus.io/docs/prometheus/latest/configuration/configuration/#exemplars https://prometheus.io/docs/prometheus/latest/feature_flags/#exemplars-storage On Tuesday, 21 June 2022 at 10:18:27 UTC+1 [email protected] wrote: > This is the perfect use case for a Histogram. > > Rather than try and sample every N milliseconds, measure every user delay > as a histogram Observe(duration) and have Prometheus collect the histogram > at the usual 15 second interval. > > Also, rather than try and write a script, simply build the Prometheus > client library into your service such that the histogram observer is > integrated. This reduces the overhead to the minimum. > > On Tue, Jun 21, 2022 at 10:33 AM Nemanja Delic <[email protected]> wrote: > >> Main thing is to detect delays in user activity - since they are >> connecting remotely we are trying to catch response times in their regular >> activity. We are able to get this data - but don't see how this can be >> ingested in Prometheus. >> > >> On Tue, Jun 21, 2022 at 10:24 AM Ben Kochie <[email protected]> wrote: >> >>> Do you really need every sample every 10ms? Or maybe you need to put >>> your measurements into a histogram for more efficient aggregation? >>> >>> What are you actually measuring and what is your actual alert criteria? >>> >>> On Tue, Jun 21, 2022 at 10:18 AM dc3o <[email protected]> wrote: >>> >>>> We're having a script that measures a user activity on the server. >>>> Script outputs the data in text file(can make it be prometheus friendly). >>>> Problem is that intervals are really short ~ 10ms. Is prometheus proper >>>> tool for scraping these targets? Goal is to graph this data and make sure >>>> I >>>> can alert on some criteria. >>>> >>>> -- >>>> You received this message because you are subscribed to the Google >>>> Groups "Prometheus Users" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to [email protected]. >>>> To view this discussion on the web visit >>>> https://groups.google.com/d/msgid/prometheus-users/1af3ea74-625e-47f3-9ddd-9dcd9a87758dn%40googlegroups.com >>>> >>>> <https://groups.google.com/d/msgid/prometheus-users/1af3ea74-625e-47f3-9ddd-9dcd9a87758dn%40googlegroups.com?utm_medium=email&utm_source=footer> >>>> . >>>> >>> -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/d3539815-f7a8-4e2e-9b7e-b2c25f6b22a6n%40googlegroups.com.
