On Tue, 28 Apr 2020 at 10:25, Julien Pivotto <[email protected]> wrote:
> On 28 Apr 10:13, Brian Brazil wrote: > > On Tue, 28 Apr 2020 at 10:06, Julien Pivotto <[email protected]> > wrote: > > > > > On 28 Apr 09:51, Brian Brazil wrote: > > > > On Tue, 28 Apr 2020 at 09:33, Julien Pivotto <[email protected] > > > > > wrote: > > > > > > > > > Dear all, > > > > > > > > > > We had another request for a promql function, last_over_time. > > > > > > > > > > https://github.com/prometheus/prometheus/issues/7179 > > > > > https://github.com/prometheus/prometheus/pull/6530 > > > > > https://github.com/prometheus/prometheus/pull/1139 > > > > > > > > > > I would like to see more broadly comments about this - it seems > that > > > > > there are usescases for this (I needed this - I was lucky to be > able to > > > > > do it with recording rules - but you can not really do it with > > > > > subqueries). > > > > > > > > > > > > > Thus far we've had no valid concrete use cases. The requests thus far > > > > appear to have been a mix of misunderstanding of PromQL semantics, > and > > > > attempting to use Prometheus as an event logging system. > > > > > > > > What's your use case? > > > > > > > > Brian > > > > > > One of our partners have a flaky API ; we don't always get the data we > > > need, and have gaps of up to 1 hour every now and then. > > > > > > This is gauges (number of credits we still have) so only the last value > > > we could scrape matters. > > > > > > Since it is a third party there isn't much we can do about it. We did > > > not want to hack the exporter neither. We currently have a recording > rule > > > but that has drawbacks, e.g. upon prometheus restart. > > > > > > > For resource limit issues, I'd expect min/max over time to cover that > > sufficiently. I'd also argue that if an hour has passed with no new data > > then either it has already alerted, or it otherwise no longer matters. > > The real issue here is the lack of reliability, and I'm not sure if > trying > > to paper over it really helps. > > > > Brian > > min_over_time and the alerts keeps firing '1 hour' after we add new > resources. > > max_over_time and we are '1 hour' late. > > last_over_time would really be the best compromise we have there. > You could silence the alert, or look at a shorter range for the alert. Predict_linear may also help. I'm reluctant to add features to PromQL to workaround things that only occur when a target is down for an extended period, as the target being down is a bigger problem that should (in principle) be alerting you anyway. Brian > > > > > > > > > > > > > > > > > > > > > > > > > > I know that we are careful when adding new promql functions, but > this > > > > > seems to be a valid one. > > > > > > > > > > Maybe we can revisit the conclusion here and add this promql > function? > > > > > > > > > > > > > > > -- > > > > > (o- Julien Pivotto > > > > > //\ Open-Source Consultant > > > > > V_/_ Inuits - https://www.inuits.eu > > > > > > > > > > -- > > > > > You received this message because you are subscribed to the Google > > > Groups > > > > > "Prometheus Developers" group. > > > > > To unsubscribe from this group and stop receiving emails from it, > send > > > an > > > > > email to [email protected]. > > > > > To view this discussion on the web visit > > > > > > > > > https://groups.google.com/d/msgid/prometheus-developers/20200428083353.GA36342%40oxygen > > > > > . > > > > > > > > > > > > > > > > > -- > > > > Brian Brazil > > > > www.robustperception.io > > > > > > -- > > > (o- Julien Pivotto > > > //\ Open-Source Consultant > > > V_/_ Inuits - https://www.inuits.eu > > > > > > > > > -- > > Brian Brazil > > www.robustperception.io > > -- > (o- Julien Pivotto > //\ Open-Source Consultant > V_/_ Inuits - https://www.inuits.eu > -- Brian Brazil www.robustperception.io -- You received this message because you are subscribed to the Google Groups "Prometheus Developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-developers/CAHJKeLqVDr%2BdXshFwf110%2BAOE%3D0T4D0a56zMPm9yoJj_guE_sw%40mail.gmail.com.
