[prometheus-users] How to define metric type as a variable

[Agarwal ,Naveen]

Hi:

Our prometheus database contains around 5k+ unique type of metrics. With time, 
we have defined alerting rules to detect deviations.

However, given the number of growing metrics, it is becoming difficult to 
expand the alerting rules.

Generally we are interested in increase/decrease of values in the metrics when 
compared to a previous time-interval. Keeping this in mind, is it possible to 
write a query where metric name is not specified, instead it picks up all 
metric names available in database in sequence.


e.g. #metrics(5min) /#metrics(30 mins) > 50
all unique metric names are picked from database.

Thanks,
Naveen

Sent from Outlook for Android

-- 
You received this message because you are subscribed to the Google Groups 
"Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to prometheus-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/prometheus-users/PU4P216MB108850E092E6DD798A2FBEEBA6849%40PU4P216MB1088.KORP216.PROD.OUTLOOK.COM.

Re: [prometheus-users] file_sd with params

[Julien Pivotto]

It is not possible as target.json param only override the first entry and will 
not allow dots.

Prometheus 2.43 enables scrape_config_file so you can split your scrape
configs into multiple files. Maybe that can be helpful.


On 23 Mar 14:17, Christian Oelsner wrote:
> Hello all,
> 
> I am trying to use file_sd with params.
> I currently have a scrape job that looks similar to this.
> 
> - job_name: 'confluent-cloud'
>   scrape_interval: 1m
>   scrape_timeout: 20s
>   static_configs:
>   - targets:
> - api.telemetry.confluent.cloud
>   scheme: https
>   basic_auth:
> username: username
> password: password
>   metrics_path: /v2/metrics/cloud/export
>   params:
> "resource.kafka.id":
>   - lkc-a
>   - lkc-b
>   - lkc-c
>   - lkc-d
> 
> The params are my challenge. They are growing rapidly and i now have some 
> 40+ clusters.
> I have been searching around for guidance on how to do this with file_sd as 
> this would make my life easier, but alas. I havent been able to decode how 
> to achieve that.
> Is it even possible to have a targets.json/yml with params that would 
> result in the same as the job above?
> 
> Best regards
> Christian Oelsner
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Prometheus Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to prometheus-users+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/prometheus-users/b3e93bdb-6454-494a-a77a-15fb7d6c627en%40googlegroups.com.


-- 
Julien Pivotto
@roidelapluie

-- 
You received this message because you are subscribed to the Google Groups 
"Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to prometheus-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/prometheus-users/ZBzH3YxbG5wOkEuB%40nixos.

[prometheus-users] file_sd with params

[Christian Oelsner]

Hello all,

I am trying to use file_sd with params.
I currently have a scrape job that looks similar to this.

- job_name: 'confluent-cloud'
  scrape_interval: 1m
  scrape_timeout: 20s
  static_configs:
  - targets:
- api.telemetry.confluent.cloud
  scheme: https
  basic_auth:
username: username
password: password
  metrics_path: /v2/metrics/cloud/export
  params:
"resource.kafka.id":
  - lkc-a
  - lkc-b
  - lkc-c
  - lkc-d

The params are my challenge. They are growing rapidly and i now have some 
40+ clusters.
I have been searching around for guidance on how to do this with file_sd as 
this would make my life easier, but alas. I havent been able to decode how 
to achieve that.
Is it even possible to have a targets.json/yml with params that would 
result in the same as the job above?

Best regards
Christian Oelsner

-- 
You received this message because you are subscribed to the Google Groups 
"Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to prometheus-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/prometheus-users/b3e93bdb-6454-494a-a77a-15fb7d6c627en%40googlegroups.com.

[prometheus-users] Re: number of occurrences in a day

[Brian Candler]

(foo==1)[24h:1m]
creates a range vector, which you then need to process further. If you run 
a function over it to count the values in the range you will get an instant 
vector, which as I showed in my original post makes this:
*count_over_time((foo==1)[24h:1m])*
That is valid (I've tested it with 'up' instead of 'foo'), and that is the 
solution I propose.

foo[24h] == 1
is not currently valid prometheus syntax, as I already said. There are some 
use cases where it would be nice *if* prometheus implemented this - but it 
doesn't.

On Thursday, 23 March 2023 at 03:34:06 UTC BHARATH KUMAR wrote:

> (foo==1)[24h:1m]. I tried this query. But I am getting an error message " 
> *bad_data: 
> invalid expression type "range vector" for range query, must be Scalar or 
> instant Vector* ".
>
> foo[24h] == 1. Error message: "* bad_data: 1:1: parse error: binary 
> expression must contain only scalar and instant vector types* "
> I am scraping at 1-minute intervals. 
>
> *Brian: I also note that you're only sampling periodically whether 
> something is in state 0, 1, or 2: you don't really know what happened in 
> between those samples, so you're never going to get a truly accurate value 
> for how long it was in each state. For example, it could flip from 1 to 2 
> and back to 1 between scrapes.*
>
> *Me: It's fine. *Even if it flips from 1 to 2 and back to 1 between 
> scrapes. I only bother about the output samples. For example last 24 hours 
> I will have 1440 samples. I want to find a number of 1's. It's fine even 
> though it flips in between scrapes.
>
> On Wednesday, 22 March 2023 at 13:36:23 UTC+5:30 Brian Candler wrote:
>
>> It's not easy to do exactly.
>>
>> To get a rough answer, you can do a subquery 
>> :  
>> (foo == 1)[24h:1m] will resample the timeseries at 1 minute intervals, then 
>> you can wrap that with count_over_time, giving:
>> count_over_time((foo == 1)[24h:1m])
>>
>> But if you weren't scraping at exactly 1 minute intervals, the count may 
>> not be accurate. Also if there are missed samples, the value of foo at time 
>> T will look back for the previous value (up to 5 minutes by default), which 
>> means in that situation some samples may be double-counted (in effect, 
>> assuming the metric value remained constant over that time, when you don't 
>> actually know what value it had).
>>
>> The only way I know to get an exact answer is to send the range vector 
>> query "foo[24h]" to the *instant* query endpoint 
>> ,
>>  
>> then filter and count the samples client-side.  A range vector like that 
>> gives the raw values with their raw timestamps as stored in the TSDB.
>>
>> For this use case it would be nice if Prometheus were to allow certain 
>> operators to work directly on range vectors, so you could write
>> foo[24h] == 1
>> But that would add quite a lot of complexity into the semantics of the 
>> query language, which already has to consider argument combinations for 
>> (scalar, scalar), (scalar, instant vector) and (instant vector, instant 
>> vector).
>>
>> On Wednesday, 22 March 2023 at 05:19:01 UTC BHARATH KUMAR wrote:
>>
>>> Hello All,
>>>
>>> I have a Prometheus metric that will give output as 0 or 1 or 2. It can 
>>> be anything(0 or 1 or 2). Could you tell me the number of 1's that occurred 
>>> in the last 24 hours?
>>>
>>> I tried with count_over_time. but I am getting errors. I tried 
>>> sum_over_time, but it is not working for a few test cases.
>>>
>>> Any lead?
>>>
>>> I really appreciate any help you can provide.
>>>
>>> Thanks & regards,
>>> Bharath Kumar
>>>
>>

-- 
You received this message because you are subscribed to the Google Groups 
"Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to prometheus-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/prometheus-users/13c10987-3f91-4770-ba74-f839550c898an%40googlegroups.com.