Re: [prometheus-users] one off in a counter

2024-02-26 Thread 'Fabian Stäber' via Prometheus Users
Thanks Ittay. Hmmm, I can only imagine two reasons: 1. Prometheus hasn't scraped yet, so you are seeing some older values than if you look directly at /metrics. 2. Prometheus scrapes another target than you are looking at. Maybe something with service discovery is going wrong. Could

Re: [prometheus-users] Re: Metrics from PUSH Consumer - Relabeled Metrics? Check "Up" state?

2024-02-26 Thread Alexander Wilke
Hello, absent_over_time() Looks good. It should allow me to alert If metrics is Not available for 10min. This will not alert If system reboots. If i use a query with Offset I think it would alert only AeS Long as the Offset still hast a value, eg Offset 10m i will get an Alarm For 10m but

Re: [prometheus-users] Re: Metrics from PUSH Consumer - Relabeled Metrics? Check "Up" state?

2024-02-26 Thread Chris Siebenmann
> Will I run into issues with "staleness" if there aren't any metrics anymore > for (more) than 5 minutes? > Or perhaps can I use this "staleness" indicator in some way? Perhaps this is a use for absent() or absent_over_time(), if you know specific metrics that should always be present from the

[prometheus-users] Re: Metrics from PUSH Consumer - Relabeled Metrics? Check "Up" state?

2024-02-26 Thread Alexander Wilke
Will I run into issues with "staleness" if there aren't any metrics anymore for (more) than 5 minutes? Or perhaps can I use this "staleness" indicator in some way? Brian Candler schrieb am Montag, 26. Februar 2024 um 16:15:57 UTC+1: > > I am still looking for a solution to identify if a device

[prometheus-users] Re: Integrating Prometheus with Splunk and ServiceNow for automated ticket creation.

2024-02-26 Thread 'Brian Candler' via Prometheus Users
> Invalid authorization Seems you're not authorizing to Splunk properly. Can you point to their documentation which says how you need to authenticate to their API? I note you're using http rather than https, so HTTP basic auth is probably not allowed (it's insecure, it sends the username and

[prometheus-users] Re: Metrics from PUSH Consumer - Relabeled Metrics? Check "Up" state?

2024-02-26 Thread 'Brian Candler' via Prometheus Users
> I am still looking for a solution to identify if a device which uses "PUSH" method is not sending data anmore for e.g. 10 minutes. Push an additional metric which is "last push time", and check when that value is more than 10 minutes earlier than the current time. If you already have a

Re: [prometheus-users] one off in a counter

2024-02-26 Thread Ittay Dror
It is literaly just 'my_total' On Monday, February 26, 2024 at 4:13:21 PM UTC+2 Fabian Stäber wrote: > Hi Ittay, > > Please post the PromQL query you are using. > > Fabian > > On Mon, Feb 26, 2024 at 3:03 PM Ittay Dror wrote: > >> I have a counter with a type label. The type can be one of 4

Re: [prometheus-users] one off in a counter

2024-02-26 Thread 'Fabian Stäber' via Prometheus Users
Hi Ittay, Please post the PromQL query you are using. Fabian On Mon, Feb 26, 2024 at 3:03 PM Ittay Dror wrote: > I have a counter with a type label. The type can be one of 4 values. Using > prom-client (15.0.0) in a node (express) app with Typescript. The code only > does 'inc' in the

[prometheus-users] Integrating Prometheus with Splunk and ServiceNow for automated ticket creation.

2024-02-26 Thread Er Ravikiran Paatil
while Integrating Prometheus with Splunk, - name: 'splunk-webhook' # Adding Splunk Webhook webhook_configs: - url: 'http://glchbs-st1001.eu.org.net:8088/services/collector' send_resolved: true http_config: basic_auth:

[prometheus-users] one off in a counter

2024-02-26 Thread Ittay Dror
I have a counter with a type label. The type can be one of 4 values. Using prom-client (15.0.0) in a node (express) app with Typescript. The code only does 'inc' in the counter. The counter is initialized only with name, help and labelNames configuration. If I fetch /metrics directly from the

[prometheus-users] Re: Metrics from PUSH Consumer - Relabeled Metrics? Check "Up" state?

2024-02-26 Thread Alexander Wilke
Hello, I am still looking for a solution to identify if a device which uses "PUSH" method is not sending data anmore for e.g. 10 minutes. The amount of devices is limited at around 20-30. any idea how to check if there are not any new metrics from it? All metrics from these devices sharing the