Re: [prometheus-users] Wildcard in PromQL "5.+" vs "5\\d{2}"

2023-10-03 Thread Stuart Clark
quot; followed by exactly 2 numbers. So the first one would also match "50" or "5frogs" which aren't valid status codes, but in reality your application would have to be having serious problems to be setting those values anyway. -- Stuart Clark -- You received this messag

Re: [prometheus-users] Not able add remote_write in Prometheus config for k8s monitoring

2023-09-18 Thread Stuart Clark
"; Not really a Prometheus thing, but that isn't a valid ConfigMap. It is expecting top level fields called "metatdata" and "data", with a filename within the data section that then contains whatever data you are wanting (which for Prometheus is a YAML config file)

Re: [prometheus-users] Re: Promteheus HA different metrics

2023-09-05 Thread Stuart Clark
or example) the Prometheus server will either need to be inside the cluster or connected to the same network mechanism as the pods. -- Stuart Clark -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubscribe from this group and s

Re: [prometheus-users] Re: Prometheus file format

2023-08-08 Thread Stuart Clark
ld tell you that the average job runtime is 5 minutes over the past 3 hours, but you'd then use the events system to find the exact durations for each run (or the number of events processed, or the error message returned, etc.). -- Stuart Clark -- You received this message because you are

Re: [prometheus-users] Re: Prometheus file format

2023-08-07 Thread Stuart Clark
0 seconds to 2 minutes. If you need to know when that set of metrics was created a common pattern is to include a metric who's value is the timestamp. -- Stuart Clark -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubs

Re: [prometheus-users] Re: Unusual traffic in prometheus nodes.

2023-07-27 Thread Stuart Clark
you could have had a web scan producing millions of combinations) * lots of changes to the targets, such as new instances of software or high churn of applications restarting -- Stuart Clark -- You received this message because you are subscribed to the Google Groups "Prometheus Users&qu

Re: [prometheus-users] Tracking Metrics Per Session/Request

2023-07-03 Thread Stuart Clark
id" label that I can use? Perhaps pull the Activity.Current trace id that is used for exemplars? Not sure what the best practices are here. This sounds like you might actually want traces rather than just metrics, which can be added using Tempo or other tools. -- Stuart Clark -- Y

Re: [prometheus-users] Prometheus counter reset

2023-05-25 Thread Stuart Clark
On 25/05/2023 15:59, Yogita Bhardwaj wrote: How can i apply rate or increase while using http api. You just need to include it in the query you are sending the Prometheus API. -- Stuart Clark -- You received this message because you are subscribed to the Google Groups "Prometheus

Re: [prometheus-users] Prometheus counter reset

2023-05-25 Thread Stuart Clark
, so you don't need to do anything. Prometheus handles counter resets automatically when you are using functions like rate(). -- Stuart Clark -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubscribe from this group and

Re: [prometheus-users] Blackbox exporter http module check interval - how to change default 15s

2023-05-18 Thread Stuart Clark
__param_target * source_labels: [__param_target] target_label: instance * target_label: ADDRESS Samples still every 15s. 2023-05-15 09:45:00 0.0624 2023-05-15 09:45:15 How are you producing that data? Did you remember to reload/restart Prometheus after making the change to your config?

Re: [prometheus-users] Sum up request duration correctly

2023-04-30 Thread Stuart Clark
nt system, such as Loki, Elasticsearch or a SQL database. -- Stuart Clark -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-users+unsubscr.

Re: [prometheus-users] Re: Do we need 2 config.yml for fetching metrics from 2 separate regions ?

2023-04-30 Thread Stuart Clark
be running multiple instances of the exporter. -- Stuart Clark -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-users+unsubscr...@google

Re: [prometheus-users] NLOG target?

2023-04-30 Thread Stuart Clark
& details, etc.) then no, as Prometheus is a metric system. Tempo is the tracing system from Grafana that can work alongside Prometheus (and there are other OS and commercial offerings too). -- Stuart Clark -- You received this message because you are subscribed to the Google Groups "P

Re: [prometheus-users] Cloudwatch Exporter configuration for existing Grafana Dashboards

2023-04-30 Thread Stuart Clark
still need slight tweaks due to different job names or use of labels). -- Stuart Clark -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-

Re: [prometheus-users] Can prometheus detect the periodicity of a metric?

2023-04-18 Thread Stuart Clark
That's definitely something that can be done _using_ Prometheus, but not something done _within_ it. You'd have an application which uses the query API (or is sent live data via remote write) to fetch metrics and then does whatever calculations are needed (for example using machine learning met

Re: [prometheus-users] Custom metric timestamps

2023-04-18 Thread Stuart Clark
The timestamp feature is only really designed for limited situations, such as ingesting metrics from another metric system. Scraping is generally expected to be returning the metric value from "now" (or in the case of coming via another metric system within the last few minutes). As Prometheus i

Re: [prometheus-users] mTLS was enabled but failed to access Prometheus via web

2023-04-08 Thread Stuart Clark
I know what I missed in the config? How have you configured the Windows machine? Have you just imported the CA into Windows, or did you generate a client certificate and import / configure that too? -- Stuart Clark -- You received this message because you are subscribed to the Google Groups

Re: [prometheus-users] Immediately pull metrics from target

2023-03-30 Thread Stuart Clark
x27;d only have to wait a short amount of time before metrics start appearing - although often you need to wait for at least a few scrapes to be able to do things like looking at counter increase rates, etc. -- Stuart Clark -- You received this message because you are subscribed to the Goo

Re: [prometheus-users] Alerts Description and Summary

2023-03-27 Thread Stuart Clark
nvironment) which you can then include in email/ticket subjects. -- Stuart Clark -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-users+un

Re: [prometheus-users] Graph how long a job takes

2023-03-27 Thread Stuart Clark
could describe what you are doing, what is happening & what you are wanting instead someone might be able to suggest something... -- Stuart Clark -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubscribe from this group and

Re: [prometheus-users] Separate endpoint for aggregate metrics?

2023-03-27 Thread Stuart Clark
of the metrics. Can it be removed? What makes it problematic? -- Stuart Clark -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to

Re: [prometheus-users] Query data is empty when step is 1h in query_range api

2023-03-17 Thread Stuart Clark
e way staleness handling works, so anything longer than that will likely result in what you are seeing. -- Stuart Clark -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubscribe from this group and stop receiving emails from it

Re: [prometheus-users] scrape targets on Prometheus

2023-03-02 Thread Stuart Clark
here . I would suggest using the file SD option and then having a tool creating a JSON/YAML file containing the servers to scrape. Is there really no list of those servers? For example are you using something like Ansible or Chef to provision/maintain them? -- Stuart Clark -- You received this

Re: [prometheus-users] Create custom metrics and add it to prometheus

2023-02-28 Thread Stuart Clark
Could anyone help me? What is the contents of those files in /var/lib/node_exporter? -- Stuart Clark -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubscribe from this group and stop receiving emails from it, send an emai

Re: [prometheus-users] How to keep servers up while computer is off. (Blackbox_Exporter)

2023-02-21 Thread Stuart Clark
computer is away? Do you mean you are switching off the machine which is running Blackbox Exporter or Prometheus? -- Stuart Clark -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubscribe from this group and stop receiving e

Re: [prometheus-users] server uptime

2023-02-21 Thread Stuart Clark
like this:     avg_over_time(up{job="node"}[30d]) * 100 You would need to adjust the query to be avg_over_time(up{job="node"}[28d]) * 100 -- Stuart Clark -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubscribe f

Re: [prometheus-users] metric_relabel_configs not dropping metrics

2023-02-21 Thread Stuart Clark
e any relabelling rules applied for that particular job only. The only way you can set relabelling rules for a job is by editing the job config. For the Prometheus Operator this is done by adjusting the PodMonitor/ServiceMonitor objects. -- Stuart Clark -- You received this message because yo

Re: [prometheus-users] fading out sample resolution for samples from longer ago possible?

2023-02-20 Thread Stuart Clark
have such features. -- Stuart Clark -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-users+unsubscr...@googlegroups.com. To view this discu

Re: [prometheus-users] metric_relabel_configs not dropping metrics

2023-02-20 Thread Stuart Clark
config jobs were added as part of loading prometheus-operator. I'm not sure I can change those. The other jobs are probably from PodMonitor & ServiceMonitor objects, so you'd need to adjust those. -- Stuart Clark -- You received this message because you are subscribed to the Google

Re: [prometheus-users] metric_relabel_configs not dropping metrics

2023-02-20 Thread Stuart Clark
On 20/02/2023 22:33, Jihui Yang wrote: I think these metrics are being scraped from another job. What I want is to drop any scraped metrics with names match the regex I provided Then you need to add the relabel config to that other job. -- Stuart Clark -- You received this message because you

Re: [prometheus-users] metric_relabel_configs not dropping metrics

2023-02-20 Thread Stuart Clark
cs msg="Scrape failed" err="server returned HTTP status 400 Bad Request" ``` The metrics are still not dropped I'm not really following exactly what your config is. Those errors suggest that at least some of the scrapes are failing. When you say "the metrics

Re: [prometheus-users] metric_relabel_configs not dropping metrics

2023-02-20 Thread Stuart Clark
r file based) to tell Prometheus where to scrape from. -- Stuart Clark -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-users+unsubscr...

Re: [prometheus-users] targetmissing alert is fooled by two endpoints on one job

2023-02-17 Thread Stuart Clark
s the same the instance label should be different. -- Stuart Clark -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-users+unsubscr...@go

Re: [prometheus-users] 404 Error as prometheus adds a port 443 to my aws app runner domain

2023-02-13 Thread Stuart Clark
config file as well still getting the same error 404 I'm not quite sure what you are meaning by "without a port number"? All connections are to a specific port. If you don't specify a port when doing a web request it will default to 80 or 443 (depending on if it is HTTP or

Re: [prometheus-users] Workaround to get the status of a service if port changes each time? (Blackbox_Exporter)

2023-02-03 Thread Stuart Clark
:5678 each time the service is stop/start. The hostname stays the same. Is there another exporter that I can use? or might know of an workaround that would work in this case. Since the url change each time, blackbox exporter cannot be used in this instance. How would you know what port to use?

Re: [prometheus-users] can node_exporter expose aggregated node_cpu_seconds_total?

2023-02-01 Thread Stuart Clark
e="603k09311-9-bjsimu01",instance="10.253.108.171:9100",ip="10.253.108.171",job="node-exporter",mode="iowait",namespace="product-coc-monitor",pod="coc-monitor-prometheus-node-exporter-c2plp",service="coc-monitor-prometheus-nod

Re: [prometheus-users] Prometheus same host multiple endpoints

2023-01-23 Thread Stuart Clark
int from a YAML/JSON file or changing AWS/Kubernetes tags. -- Stuart Clark -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-users+unsubscr

Re: [prometheus-users] Time series with change interval much less than scrape interval

2023-01-18 Thread Stuart Clark
much better fit for you than a timeseries database and ecosystem that is designed for handling metrics. -- Stuart Clark -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubscribe from this group and stop receiving emails from it

Re: [prometheus-users] AlertManager rules examples

2023-01-18 Thread Stuart Clark
u can set rules to send you a message every >> 10 days that alert is firing, meaning you'd get a message at 60, 50, 40, >> etc days until expiry. >> >> More complex alerting routing decisions are generally out of scope for >> Alertmanager and would be expecte

Re: [prometheus-users] AlertManager rules examples

2023-01-13 Thread Stuart Clark
PagerDuty, OpsGenie, Grafana On-Call, etc.). This would cover you example of wanting to escalate an alert after a period of time, but would also cover things like having on-call rotas where different people would be contacted by looking at a rota calendar. -- Stuart Clark -- You received this mes

Re: [prometheus-users] Prometheus-Thanos-Grafana Integration || error || open /etc/prometheus-shared/prometheus.yaml: no such file or directory

2023-01-06 Thread Stuart Clark
eus.yaml in the directory /etc/prometheus-shared doesn't exist, which is the configuration file for Prometheus. You need to either ensure the file does exist at that location or adjust the command line to give the correct location if different. -- Stuart Clark -- You received this mess

Re: [prometheus-users] Issue getting data in time range

2022-12-17 Thread Stuart Clark
expecting & what you actually see? -- Stuart Clark -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-users+unsubscr...@googlegroups.com.

Re: [prometheus-users] Issue while exposing Prometheus metric from Spring Boot application

2022-12-14 Thread Stuart Clark
It is a requirement that counters end with _total which is likely why it is being added for you. On 14 December 2022 17:50:32 GMT, "arnav...@gmail.com" wrote: >Hi, > >I am exposing two counter metrics from my spring boot application. The >metrics are getting generated and the counters increas

Re: [prometheus-users] Null value in alerts

2022-12-09 Thread Stuart Clark
x27;t listing a metric here as you are using !~. You need to ensure you are only using = in any labels. -- Stuart Clark -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubscribe from this group and stop receiving emails f

Re: [prometheus-users] How to combine counter data which scrapes every 5 min using node_exporter

2022-12-03 Thread Stuart Clark
n the metric as labels. These need splitting out into their own separate metrics (e.g. NNI_RTT_Min and Packet_Loss). As it stands you are creating new time series for basically every scrape, which results in the lines you are seeing in your graph. Also as it is a cardinality issue it will cause

Re: [prometheus-users] Prometheus reaction time

2022-11-23 Thread Stuart Clark
dressed. In general it would actually be more useful to alert if that threshold is breached for over say 5 minutes, or even more usefully when an SLO is failed or is projected to fail within the next 30 minutes. -- Stuart Clark -- You received this message because you are subscribed to the Goo

Re: [prometheus-users] prometheus federate transfter alerts stopped to main prometheus

2022-11-21 Thread Stuart Clark
bels: true     metrics_path: '/federate'     scheme      : http     params:       'match[]':        - '{job!=""}' The alert rules are set in each Prometheus, so if you are not wanting them you'd need to look at the configuration and change/remove. --

Re: [prometheus-users] Prometheus Growth

2022-11-21 Thread Stuart Clark
https://promcon.io/2022-munich/talks/why-is-it-so-big-analysing-the-m/) which is available on YouTube, but basically the main memory usage is generally due to the number of time series, with only a very minor usage due to queries. -- Stuart Clark -- You received this message because you are subscribed t

Re: [prometheus-users] Modify the check interval of blackbox exporter HTTP probe

2022-11-21 Thread Stuart Clark
point with the Blackbox Exporter. -- Stuart Clark -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-users+unsubscr...@googlegroups.com. To

Re: [prometheus-users] Prometheus jobs are not showing prometheus UI Page

2022-11-09 Thread Stuart Clark
? Where are you looking in the UI and what is showing? -- Stuart Clark -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-user

Re: [prometheus-users] Is there any plan for prometheus to support string type metrics

2022-11-01 Thread Stuart Clark
, which are very different and not limited to number (e.g. logs). Maybe it would be best to describe what you are trying to achieve? -- Stuart Clark -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubscribe from this grou

Re: [prometheus-users] Guaranteed ingestion of metrics with historical timestamps

2022-10-30 Thread Stuart Clark
global storage) it makes sense to support both, there are plenty of use cases which only require one or the other. -- Stuart Clark -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubscribe from this group and stop receiving

Re: [prometheus-users] Guaranteed ingestion of metrics with historical timestamps

2022-10-25 Thread Stuart Clark
2 18:24:56 BST, Omar Khazamov wrote: >Thanks, I'm importing metrics from our internal metrics system. Do you have >any advice on how to push withthe explicit timestamps? > >Le ven. 7 oct. 2022 à 13:30, Stuart Clark a >écrit : > >> On 07/10/2022 10:16, Omar Khaz

Re: [prometheus-users] Use Case Assessment

2022-10-17 Thread Stuart Clark
o have that as a label to the pending orders metric (which then would allow you to alert if one method stops working), but order ID isn't something you should ever put in the metrics. Instead once you were alerted about a potential issue you might query your order database directly or look

Re: [prometheus-users] Mysql handshake error with blackbox tcp_connect

2022-10-13 Thread Stuart Clark
g anything. You would need to look at the server configuration to whitelist the IP of the Blackbox Exporter so it doesn't get blocked. Alternatively look at using the MySQL Exporter instead of using Blackbox Exporter, which will give you more insight above just availability too? -- Stuar

Re: [prometheus-users] Guaranteed ingestion of metrics with historical timestamps

2022-10-07 Thread Stuart Clark
always be publishing the "latest" value of any metric when Prometheus scrapes the endpoint (or the push gateway in this case). -- Stuart Clark -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubscribe from this

Re: [prometheus-users] Metrics vs log level

2022-10-07 Thread Stuart Clark
cal "error" and "debug" levels? Are you meaning some metrics would only be exposed on some environments? -- Stuart Clark -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubscribe from this group and sto

Re: [prometheus-users] Re: Hisorical data

2022-09-26 Thread Stuart Clark
/querying/api/#delete-series -- Stuart Clark -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-users+unsubscr...@googlegroups.com. To

Re: [prometheus-users] Re: Hisorical data

2022-09-26 Thread Stuart Clark
stem was configured. If you have something stored in the local Prometheus store (rather than something you are using remote read with) and don't want to wait until it expires (14 days or whatever you configured) you can use the deletion API. -- Stuart Clark -- You received this message

Re: [prometheus-users] is it already a known issue with alertmanager integration to pagerduty

2022-09-26 Thread Stuart Clark
gt;, and also your alert grouping config. In general if you are sending alerts through to an external incident management system (PagerDuty, OpsGenie, JIRA, etc.) you are probably best not doing any grouping at all: |group_by: ['...']| -- Stuart Clark -- You received this message bec

Re: [prometheus-users] /metrics lifecycle

2022-09-08 Thread Stuart Clark
why you think resetting metrics is important / needed? -- Stuart Clark -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-users+unsubscr

Re: [prometheus-users] Re: up query

2022-08-16 Thread Stuart Clark
d be completely in an unreachable state. So you are only wanting it if every single scrape failed over the past 2 days? Try sum() instead of max_over_time(). -- Stuart Clark -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To un

Re: [prometheus-users] How could I trucate data after pull in Java client.

2022-08-12 Thread Stuart Clark
ot;slice & dice" the data, so for example to be able to see which specific HTTP response code was returned from a web call, etc. What is the value of the metric app_version supposed to signify? -- Stuart Clark -- You received this message because you are subscribed to the Google Groups &

Re: [prometheus-users] Prometheus Api to json to use in react.js

2022-08-12 Thread Stuart Clark
: https://prometheus.io/docs/prometheus/latest/querying/api/ -- Stuart Clark -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-user

Re: [prometheus-users] Re: evaluation interval cannot be recognized in seperate job

2022-08-12 Thread Stuart Clark
On 12/08/2022 08:29, nina guo wrote: So if with push gateway or textfile collector, we need to also to customize our metrics, am I right? What do you mean? -- Stuart Clark -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To u

Re: [prometheus-users] Re: evaluation interval cannot be recognized in seperate job

2022-08-12 Thread Stuart Clark
f the last time the metric was updated. -- Stuart Clark -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-users+unsubscr...@googlegrou

Re: [prometheus-users] Re: evaluation interval cannot be recognized in seperate job

2022-08-11 Thread Stuart Clark
interval you will end up with stale time series and "gaps" in all your graphs. -- Stuart Clark -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubscribe from this group and stop receiving emails from it, send an email

Re: [prometheus-users] node exporter text file collector

2022-08-08 Thread Stuart Clark
7;='+service+','+'ldap_search_base'+'='+ldap_search_base+','+'} '+str(query_check), file=open("/var/log/node_exporter/filecollector/ldap_query.prom", "a+")) File mode "a" will open for appending (so preserve anything

Re: [prometheus-users] node exporter text file collector

2022-08-08 Thread Stuart Clark
ime you do an update. Depending on the tooling used it is often better to create a new file with a random non *.prom filename and then rename it into the correct name - renames are generally atomic whereas file updates often aren't, meaning if you modify the file directly you could end up r

Re: [prometheus-users] synthetic histograms in Prometheus

2022-08-07 Thread Stuart Clark
ou aren't able to produce a histogram in the application (or externally via processing an event feed, such as logs)? By design a simple histogram is pretty low impact, being a set of counters for each bucket. -- Stuart Clark -- You received this message because you are subscribed to the Googl

Re: [prometheus-users] synthetic histograms in Prometheus

2022-08-07 Thread Stuart Clark
n be possible - which is only possible in the application, or if you store events elsewhere (e.g. in log files) in a tool that connects to your event store system. -- Stuart Clark -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group.

Re: [prometheus-users] Re: blackbox metrics scraping

2022-07-25 Thread Stuart Clark
t; would be returned, and you'd have gaps in your graphs. The recommended maximum of about 2-2.5 minutes is to allow for a single scrape failure not to result in gaps as well as all the various processing times to actually do the scrape. -- Stuart Clark -- You received this message because

Re: [prometheus-users] Re: blackbox metrics scraping

2022-07-25 Thread Stuart Clark
On 25/07/2022 08:28, nina guo wrote: And one more question pls, I checked the log that the probe is sending every 2-3 seconds, can I adjust this frequency to about 1min? Yes that's the scrape frequency, so you can adjust the job configuration up to a maximum of about 2.5 minutes. -- S

Re: [prometheus-users] Prom QL

2022-07-24 Thread Stuart Clark
is failing) then no data exists about that server. For the drop down selector in Grafana you could use the "up" metric, which will still have values for servers that weren't able to be scraped, but selecting one of those would then result in an empty dashboard as there wouldn&#x

Re: [prometheus-users] Re: blackbox metrics scraping

2022-07-24 Thread Stuart Clark
On 25/07/2022 01:08, nina guo wrote: Thank you Brian. " up to T - 5 minutes  ", this 5 mins is the scraping interval? No. The scraping interval doesn't matter. Prometheus will by default look back at most 5 minutes for a value. -- Stuart Clark -- You received this message

Re: [prometheus-users] deactive alert after hook

2022-07-24 Thread Stuart Clark
;m not sure if that helps. -- Stuart Clark -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-users+unsubscr...@googlegroups.com. To view this

Re: [prometheus-users] Prometheus agent mode flag

2022-07-21 Thread Stuart Clark
container. -- Stuart Clark -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-users+unsubscr...@googlegroups.com. To view this discussion

Re: [prometheus-users] Prom QL

2022-07-20 Thread Stuart Clark
any queries won't have any data to query. However Prometheus itself creates certain metrics for all scrape targets, including one called "up" which is either 0 or 1 - where 0 means the scrape failed. You can therefore create dashboards and alerts that list the servers which aren

Re: [prometheus-users] Re: Https issue when using prometheus federation

2022-07-19 Thread Stuart Clark
e metrics? Could it be that it takes a while and your load balancer is configured with a shorter timeout? -- Stuart Clark -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubscribe from this group and stop recei

Re: [prometheus-users] Use remote-write instead of federation

2022-07-19 Thread Stuart Clark
n is hoping to achieve it would help us advise better? -- Stuart Clark -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-users+unsubscr...@g

Re: [prometheus-users] Best way to export status

2022-07-19 Thread Stuart Clark
wouldn't actually help BTW as the statusNumber is a label so you could only use regex matches rather than >/<. If you wanted that as well you'd need a separate metric (idrac_amperage_probe_status_number or something) that has no labels and just the 1-10 value. The value of that pur

Re: [prometheus-users] Use remote-write instead of federation

2022-07-18 Thread Stuart Clark
or using agent mode? If you are just going to copy everything to the "central" instance on the same cluster, why not just do without the extra three clusters and have just the one instance that monitors everything? -- Stuart Clark -- You received this message because you are subsc

Re: [prometheus-users] Use remote-write instead of federation

2022-07-18 Thread Stuart Clark
might be a good or bad thing depending on team responsibilities/timings. -- Stuart Clark -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to prometheu

Re: [prometheus-users] Change response of Alert

2022-07-11 Thread Stuart Clark
a one-way process - the webhook is triggered when an alert fires (and optionally clears). I'm not really sure what you are hoping for? -- Stuart Clark -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubscribe from this

Re: [prometheus-users] Re: Alerts are getting auto resolved automatically

2022-07-05 Thread Stuart Clark
Two alerts suggests that the two instances aren't talking to each other. How have you configured them? Does the UI show the "other" instance? On 5 July 2022 08:34:45 BST, Venkatraman Natarajan wrote: >Thanks Brian. I have used last_over_time query in our expression instead of >turning off auto-

Re: [prometheus-users] AM Executor on Kubernetes Cluster

2022-07-05 Thread Stuart Clark
What are you meaning by "AM executor"? Are you meaning the Alertmanager application? If so, there are both community Helm charts and a Docker image available, so running within Kubernetes should be no problem. On 5 July 2022 08:03:45 BST, Test Kumar wrote: >Hi Team, > >I installed the AM exec

Re: [prometheus-users] Guaranteed ingestion of metrics with historical timestamps

2022-06-18 Thread Stuart Clark
rd scenarios would you expect timestamps to be attached to the returned metrics. It would be good to hear a bit more about what you are trying to do, as it is highly likely that the use of timestamps in your use case is probably not the right option and they should just be dropped. -- Stuart Cl

Re: [prometheus-users] Does Prometheus recommend exposing 2M timeseries per scrape endpoint?

2022-06-14 Thread Stuart Clark
HTTP status code, section of website, etc.). What is it about this system that you think it needs very high cardinality labels? -- Stuart Clark -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubscribe from this group and

Re: [prometheus-users] Does Prometheus recommend exposing 2M timeseries per scrape endpoint?

2022-06-14 Thread Stuart Clark
metrics system. Would you be able to explain a bit more about what you are wanting to be able to achieve and we can suggest an alternative to Prometheus/metrics that would help? -- Stuart Clark -- You received this message because you are subscribed to the Google Groups "Prometheus Users&qu

Re: [prometheus-users] Does Prometheus support Netapp Trident NFS storage backend?

2022-05-31 Thread Stuart Clark
, and you are unlikely to recenive any free/paid support in that architecture. -- Stuart Clark -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to p

Re: [prometheus-users] Does Prometheus support Netapp Trident NFS storage backend?

2022-05-03 Thread Stuart Clark
rted doesn't mean it won't work. However there are issues that people find, and if you do find those the recommendation would always be to switch the filesystem to something local. So basically you'd be fairly on your own if you did have problems. -- Stuart Clark -- You re

Re: [prometheus-users] Running a prometheus on kubernetes in an offline env

2022-04-28 Thread Stuart Clark
ironment has no Internet access? There is nothing within Prometheus which should require such access. What isn't working? What does the target page show regarding the things you are scraping? -- Stuart Clark -- You received this message because you are subscribed to the Google Groups &quo

Re: [prometheus-users] Re: Run two node exporters on same server

2022-04-21 Thread Stuart Clark
ort hasn't been exposed to the outside host. -- Stuart Clark -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-users+unsubscr...@

Re: [prometheus-users] Facing 5m staleness issue even with 2.x

2022-04-19 Thread Stuart Clark
unless there is an error, which should be visible via something like the "up" metric). As the standard staleness interval is 5 minutes it is recommended that the maximum scrape period should be no more that 2 minutes (to allow for a failed scrape without the time series being marked as

Re: [prometheus-users] Backfilling data into Prometheus

2022-04-17 Thread Stuart Clark
ve you set your Prometheus retention period to? By default it is 2 weeks. -- Stuart Clark -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to prome

Re: [prometheus-users] Counter metric resets

2022-04-07 Thread Stuart Clark
an integer (due to the way interpolation happens). -- Stuart Clark -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-user

Re: [prometheus-users] Prometheus storage wrt Pushgateway metrics

2022-03-18 Thread Stuart Clark
etric that isn't changing is next to nothing, so I wouldn't worry about it. What you describe would be exactly how I'd expect Pushgateway to be behaving - some metrics are updated more frequently and others less, but they are always there and being scraped at the same frequenc

Re: [prometheus-users] Scrape_interval

2022-02-28 Thread Stuart Clark
, due to staleness - less frequent scrapes would likely cause series to be marked as stale and therefore disappear from alerts/dashboards. -- Stuart Clark -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubscribe from this grou

Re: [prometheus-users] Alertmanager Sendgrid WEB API Integration

2022-02-23 Thread Stuart Clark
erts to MS Teams), but I'm not sure if a Sendgrid one has already been written. -- Stuart Clark -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to

Re: [prometheus-users] Target Server - Which Prometheus Server Is Scraping

2022-02-16 Thread Stuart Clark
details of what is going on on the target server itself, not so concerned about what the Prometheus server has log wise. I was talking about logs from the application being scraped. -- Stuart Clark -- You received this message because you are subscribed to the Google Groups "Prometheus

  1   2   3   4   5   6   >