Re: [prometheus-developers] match a metric path and split it on a colon (:)

This question is better suited to the prometheus-users mailing list, I'm moving it there. >From the config I take it you are trying to achieve this in the graphite exporter? What you are trying will not be possible using the (faster) glob match type. You are on the right track with using

Re: [prometheus-developers] Re: Call for Consensus: node_exporter 1.0.0 release

I agree, if we plan on releasing 1.0, have an RC, a security review for a feature marked experimental doesn't need to hold things up. We should make it clear when we consider TLS "ready for serious use" but that's for another release. /MR On Thu, Apr 23, 2020 at 11:47 AM Richard Hartmann <

Re: [prometheus-developers] Chef Server SD

As Ben said, write out file SD files, usually using chef-client, but you can also script something else up if you want to decouple the SD lifecycle from chef-client runs. The file SD is just JSON, so that's very easy to generate in Ruby/Chef. Because Chef queries are so slow, and Prometheus SD

Re: [prometheus-developers] [VOTE] Allow listing non-SNMP exporters for devices that can already be monitored via the SNMP Exporter

YES Looking beyond SNMP, I regularly encounter cases where it is technically possible to monitor something using the graphite or statsd exporters. Where this is painful I do guide users into a different integration that is more special-purpose. My stance is that we should promote what works best,

Re: [prometheus-developers] [VOTE] Allow Kelvin as temperature unit in some cases

YES Kelvin is the standard unit. On Fri, May 29, 2020 at 9:12 AM Tom Wilkie wrote: > YES > > On Fri, May 29, 2020 at 8:13 AM Ben Kochie wrote: > >> YES >> >> On Thu, May 28, 2020 at 8:52 PM Bjoern Rabenstein >> wrote: >> >>> Dear Prometheans, >>> >>> So far, we have recommended Celsius as

Re: [prometheus-developers] Changing `master` to `main` across the org?

ity.com/mysql-terminology-updates/ >> >> >> Le mer. 1 juil. 2020 à 09:54, Matthias Rampke a >> écrit : >> >>> +1 for changing the default branch, +1 to seeing what GitHub does to >>> help us, but if there is no movement on that we can also work o

Re: [prometheus-developers] Changing `master` to `main` across the org?

+1 for changing the default branch, +1 to seeing what GitHub does to help us, but if there is no movement on that we can also work out a plan ourselves. For the MySQL case, *if* we are willing to break compatibility and consistency, we could change the metric names even without waiting for the

Re: [prometheus-developers] Lazy consensus: Merging options

… which we used to encourage inside the team. I prefer real merging too, I like git and I don't feel the need to make history linear. I don't use rebase merging from PRs but I also don't think we should remove that while allowing squash merging. I am okay with every maintainer having their own

Re: [prometheus-developers] Alertmanager HA

The listen address is the one that alertmanager binds to. Generally, this can be 0.0.0.0 (all interfaces), I believe that is the default and thus sometimes omitted. It will try to guess which address it can be reached at by other AMs (which address it should advertise to them). In some

Re: [prometheus-developers] How. to deal with outdated series in alerts.

Restart the application that produces metrics. Generally, the client libraries will remember metrics even if they are not being incremented anymore (it cannot know that they won't be again). Restarting clears the "seen" label set in each process. Side note: this can also bite you the other way –

Re: [prometheus-developers] Add collector for database/sql#DBStats

Ah, even before following the link I had the same questions as Björn: Could this be a separate thing? Does it benefit significantly from being part of the standard client_golang, or could it just as well be something people pull in on demand? I see at least two

Re: [prometheus-developers] Re: [VOTE] Allow environment variable expansion on external label values

YES On Thu, Mar 25, 2021 at 10:09 PM Julien Pivotto wrote: > On 25 Mar 23:08, Julien Pivotto wrote: > > On 25 Mar 23:06, Julien Pivotto wrote: > > > Hereby I am calling a vote to allow the expansion on environment > > > variables in the prometheus configuration file. > > > Because it can be

Re: [prometheus-developers] Mount Point missing alarm

Moving this to prometheus-users where it fits better. Try using the "unless" operator to compare against a metric that is present for all instances that should have this mountpoint. Assuming that is the case for all targets under this job: up{job=XX"} unless on(instance)

Re: [prometheus-developers] Python Multiprocess

Would you mind making a PR to improve the documentation? As the expert it is easy to write documentation that is *technically correct* but not helpful to an unsuspecting user; you are now in the best position to make this clear to the next one reading it. Best, MR On Wed, Apr 7, 2021 at 6:14 PM

Re: [prometheus-developers] Deprecating https://github.com/prometheus/nagios_plugins (?)

I think the no-magic route is better. You can also archive the repo[0] to make it clear that it's read only (with this GitHub feature, do we still need to graveyard anything ourselves? /MR [0]

Re: [prometheus-developers] consul_exporter - Expose health statuses as values?

What would some common queries be that this affects, and how would they look in the future? For example, "what fraction of nodes is down" "which nodes have multiple services down?" /MR On Mon, Aug 16, 2021, 22:31 Matt Russi wrote: > Currently, the consul_exporter exposes 4 series per

Re: [prometheus-developers] Change language on blackbox vs whitebox to closedbox vs openbox

*resurrecting thread* I agree that we should rename it. I wince every time I need to call it by name. I often just call it prober anyway and even unaffiliated people immediately know what I'm talking about. More broadly, I think "blackbox" / "whitebox" monitoring is not very descriptive, and

Re: [prometheus-developers] HA alertmanager clusters may merge into one if they run in the same flat network

What should happen if the DNS resolution does not result in the expected number of peers either? How would a deliberate shrinking or growing of a cluster work? Another solution I have seen (e.g. in Cassandra) is to have a cluster identity, such as a cluster name. Instances would refuse to talk to

Re: [prometheus-developers] amtool: passwords passed in the command line

I take a less hard line on that … I think it's good not to *accept secrets* on the command line, but I think we should not categorically exclude generic features (like headers on the command line) because someone *might* put secrets there. I don't have a final opinion whether we should add more

Re: [prometheus-developers] amtool: passwords passed in the command line

That's fair, I only ask that we consider use cases when they come up :) /MR On Wed, Dec 1, 2021 at 12:38 PM Julien Pivotto wrote: > What usecase for amtool would not involve authorization or authentication? > I don't think there are. > > Le mer. 1 déc. 2021, 09:21, Matthias Rampk

Re: [prometheus-developers] Requirements / Best Practices to use Prometheus Metrics for Serverless environments

What properties would an ideal OpenMetrics push receiver have? In particular, I am wondering: - What tradeoff would it make when metric ingestion is slower than metric production? Backpressure or drop data? - What are the semantics of pushing a counter? - Where would the data move from there, and

Re: [prometheus-developers] Proposal: Adopting rust-open-metrics-client as official Rust client

+1 for putting it in the prometheus org, we should have an official client for all the popular languages, amd official clients should be in the community org with the implied promise of support. /MR On Sat, 27 Nov 2021, 18:12 Julien Pivotto, wrote: > We could follow python and call it

Re: [prometheus-developers] Enabling auto-merge

Ah, I didn't understand at first that this is a per-pull-request thing. What is the right way to enable this? Do I change the repository configuration by hand? /MR On Thu, Nov 25, 2021 at 5:16 PM Julien Pivotto wrote: > Hello, > > I have enabled "auto merge" in prometheus/prometheus with the

Re: [prometheus-developers] Welcoming Matthias Loibl as a new Prometheus team member

Welcome to the other Matthias >:) /MR On Thu, Nov 25, 2021 at 2:11 PM Goutham Veeramachaneni wrote: > Welcome Matthias! Looong overdue :) > > Thanks > Goutham > > On Thu, Nov 25, 2021 at 2:31 PM Julien Pivotto > wrote: > >> Welcome!! >> >> On 25 Nov 12:32, Julius Volz wrote: >> > Hi

Re: [prometheus-developers] Option to disable security on Prometheus health endpoints, /-/healthy and /-/ready

It seems to me that these are two different directions – locking down the admin endpoints more vs. not locking down the health endpoints at all. In what scenario would one want to have /-/healthy and /-/ready protected? /MR On Thu, Sep 23, 2021 at 6:11 PM Julien Pivotto wrote: > On 23 Sep

Re: [prometheus-developers] Proposal: Adopting rust-open-metrics-client as official Rust client

For the public record – with "and" the sentence was even more confusing. What I really meant to write was "official clients should be in the official org". +1 to the plan! /MR On Thu, Jan 13, 2022, 21:12 Matthias Rampke wrote: > > What does "amd" stand for

Re: [prometheus-developers] Proposal: Adopting rust-open-metrics-client as official Rust client

s. I would proceed with the following: > > 1. Move https://github.com/mxinden/rust-open-metrics-client to > https://github.com/prometheus/client_rust > > 2. Move https://crates.io/crates/open-metrics-client to > https://crates.io/crates/prometheus-client > > Any objectio

[prometheus-developers] Please welcome Fabian "fstab" Stäber to the Prometheus Team

Dear all, I am happy to announce that Fabian Stäber is joining the Prometheus Team. Fabian has been the maintainer of the official Java client library for a while, and has now become a full team member. Welcome to the team, Fabian! Best, Matthias -- You received

Re: [prometheus-developers] Add label to mysqld-exporter to show the mysql instance

This is not supported in the exporter and we have no plans to add it. Most exporters use a different approach, which we recommend for exporters in general. Deploy the expory as a sidecar alongside the MySQL instance. In Kubernetes, this means an additional container in the MySQL pod. This solves

Re: [prometheus-developers] Update Prometheus Readme on docker hub

I propose that we cut most of it, only keep a short paragraph about what Prometheus is, and link to prometheus.io/docs and/or the README for details? That way we only need to update it when things fundamentally change. I don't think people come to Docker Hub to read extended documentation, and

Re: [prometheus-developers] Service Discovery for Oracle Cloud Infrastructure

Hi Mayur, Yes, this is the correct document. Will you be able to support this service discovery mechanism long-term? In the past we had service discovery mechanisms slowly break due to lack of attention, and would like to avoid that in the future. Best, Matthias On Thu, Jan 6, 2022 at 10:57 PM

Re: [prometheus-developers] Update Prometheus Readme on docker hub

Ah, I didn't realize. I would have kept even less information but if this is not too much to keep up to date, I'm happy! Let's see how things develop :) /MR On Fri, Feb 4, 2022 at 1:19 PM Julien Pivotto wrote: > On 04 Feb 13:12, Matthias Rampke wrote: > > I propose that we cut most of

Re: [prometheus-developers] [VOTE] Rename blackbox_exporter to prober

YES On Thu, Jan 20, 2022, 14:59 Ben Kochie wrote: > YES > > On Thu, Jan 20, 2022 at 3:41 PM Julien Pivotto > wrote: > >> Dear Prometheans, >> >> As per our governance, I'd like to cast a vote to rename the Blackbox >> Exporter to Prober. >> This vote is based on the following thread: >> >>

Re: [prometheus-developers] Java Prometheus Exporter for Traffic Metrics

Hi, I think this discussion is better suited for the -users mailing list, moving it there. Metric systems, like Prometheus, offer you a specific tradeoff: they allow you to count a *large number* of events by *limited dimensions*. Fundamentally, for each combination of dimensions, it tracks a

[prometheus-developers] No Dev Summit today and in August

Hello, because too many people are out for the summer, today's Dev Summit unfortunately won't be happening. Unless unexpected things happen, this is also true for the one in August. Dev Summit will be back in September – see you then! /MR -- You received this message because you are

Re: [prometheus-developers] Understanding the structure of node_exporter

Not all collectors are available on all platforms – in many cases, due to the platform dependent code, they won't even compile. This structure allows us to selectively compile the various collector files, and those that are getting compiled register themselves. Additionally, this structure allows

Re: [prometheus-developers] Quick Question: do tests fail on macos?

Hi, I'm trying this out – I can't reproduce the zookeeper failure (on main as of today) but I ran into the same problem on the TSDB test. I think what's happening is that the test actually takes too long. The test

[prometheus-developers] Re: Governance Working Group

There is now a mailing list for this effort: https://groups.google.com/a/prometheus.io/g/governance-wg Please request to join if you want to be part of it – and fill in the Doodle! I will nail down a date later this week. Cheers, MR On Tue, Sep 13, 2022 at 4:05 PM Matthias Rampke wrote: >

[prometheus-developers] Governance Working Group

Hi, At the in-person Dev Summit in May there was a lot of interest in updating the Prometheus project governance . I was volunteered to organize this discussion. To that end, I want to kick off a working group. If you are interested in the topic, please fill in

Re: [prometheus-developers] Would tooling for PromQL formatting/manipulation be useful and where should it live?

Re: drawing the line – I often feel like "specialized" tools that try to solve all advanced use cases end up with very complex and hard to use configuration (looking at you, relabeling). I often find it more pleasant to express what I want to do as *code*. What would an API look like that you or

Re: [prometheus-developers] Return specific value if label regex not match

Hi, this mailing list is for development of Prometheus and related projects. Since your question is about usage, I'm moving the thread to the prometheus-users mailing list. To answer your question, in general a regular expression can have an unbounded number of matches, so Prometheus cannot

[prometheus-developers] FYI: CircleCI setup_remote_docker architecture change

Hey all, The influxdb_exporter master build is currently broken , even though it uses the same configuration as other projects, and the build configuration did

Re: [prometheus-developers] Why do Info metrics have two Label Sets?

I can't speak to the original thinking but I can speculate. In text format info metrics, we conflate two things into labels: the identification of the thing we are informing about, and the information itself. For example, an info metric about hard disks would have a label that identifies the disk

Re: [prometheus-developers] [VOTE] Promote Windows Exporter as an official exporter

YES On Mon, Dec 5, 2022 at 10:44 AM Julien Pivotto wrote: > Dear Prometheans, > > As per our governance [1], "any matter that needs a decision [...] may > be called to a vote by any member if they deem it necessary." > > I am therefore calling a vote to promote Prometheus-community's Windows >

Re: [prometheus-developers] Gauge Exemplars

At scrape time, how would I know which method was used? "always the total value of the gauge" seems like the least surprising choice to me. If users need to track separate exemplars for increment and decrement, they could use two counters (basically, a non-native UpDownCounter), which conceptually

Re: [prometheus-developers] Changing consensus on HTTP headers

In general, what is a foot gun to me can be a rocket shoe to you, so I am in favor of providing them to those who require them, with clear labeling of the dangers. Specifically in this case, it has become more common ("beyond corp", "zero trust") to use HTTPS over the public internet, combined

Re: [prometheus-developers] Ingesting OTLP

Hmm, my assumption was that ingesting OTLP would be equivalent to running the OTel Collector + remote write, but without the collector in the middle. That implies using the same mappings as specified in the OTel spec / implemented in the collector. This is something that can happen relatively

Re: [prometheus-developers] [feature/proposal] Amazon EFA collector

To clarify, you are asking about adding this to the node exporter? I am torn between "this seems very specific" and "I guess it won't hurt anyone who doesn't need it". IMO adding support to the procfs package makes sense, whether it's then consumed by node exporter or a more specific one. /MR

Re: [prometheus-developers] How Prometheus uses go-restful ?

w whether > Prometheus is affected or not. > > Thanks > > BRs > /Gavin > > > > On 11 Feb 2023, at 02:10, Tristan Colgate wrote: > >  > This is probably due to GitHub dependabot currently flagging a security > issue with go-restful (I hit the same issue yest

Re: [prometheus-developers] Should Alertmanager be more tolerant of templating errors?

I agree that silently sending *no* alert is the worst possible outcome. I wonder what would be "nicer" in case a template fails - send the alert with the fields that did not fail to render (possibly render the error *into* the fields that failed to make it very obvious?), or (as proposed) fall

Re: [prometheus-developers] How Prometheus uses go-restful ?

You are on the right track with go mod graph: go-restful is a dependency of k8s.io/client-go and k8s.io/kube-openapi, so colloquially "the Kubernetes client library". Prometheus uses it for service discovery, fetching information about pods, endpoints, and services. From a cursory look through

[prometheus-developers] macOS DNS resolving change in Go 1.20

Will this affect how DNS SD behaves on macOS? https://danp.net/posts/macos-dns-change-in-go-1-20/ -- You received this message because you are subscribed to the Google Groups "Prometheus Developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to

Re: [prometheus-developers] Reconsider marshalling secrets in Prometheus libraries

I agree that this should be possible. My first intuition was to make this a separate API call, like UnsafeMarshalYAML but I am not sure how well that would play with the YAML infrastructure in Go? Maybe we could have a helper (.MarshalSecrets()) that returns the struct with wrapped/aliased types

Re: [prometheus-developers] [feature/proposal] Changing alert fingerprint calculation in prometheus/common

For a very long time, Prometheus did not store apeet state across restarts, so the alert startsAt would update even though the condition had not changed. I don't think we ever considered this time to be very meaningful or stable, partially due to the originally stateless implementation, but also

Re: [prometheus-developers] How to delay de-duplication of alerts in AlertManager

t; label1 could be "from" and label2 could be "to". > If it is possible to do it in Alertmanager without having to generate a > combined label in Prometheus, can someone show an example configuration ? > > Thanks, > Dhiman > > > > > > > > > On Wed, Aug 14, 201

Re: [prometheus-developers] Official Prometheus video playlist

Do we have a somewhat up-to-date curated list of Prometheus-related talks / recordings that are already around? /MR On Sun, Mar 1, 2020 at 8:45 AM Julius Volz wrote: > I always wanted to create some YouTube videos covering certain topics, but > never did it yet because I have certain quality

Re: [prometheus-developers] Checking if NFS is hanged or not using node_exporter.

The trouble is that the only sure way to know if MFS hangs is to try and use it. For one, the node exporter is not a health prober per se, bur more importantly, when NFS does hang, this leaves a thread permanently stuck in an uninterruptible syscall. Very soon the exporter would run out of threads

Re: [prometheus-developers] Prometheus Alert handle/resolve handling

I think it helps to think about Alertmanager webhooks differently. Alertmanager does not notify about individual alerts but about *groups* of alerts. These groups come into being, the number of alert instances in them potentially changes over time. Subsequent webhooks about the same groups are

Re: [prometheus-developers] prometheus/prometheus Changelog Management

In the exporters that I maintain I specifically ask contributors not to fill in the changelog. I want to keep a somewhat editorial voice there. I often rephrase changes to highlight what the change means for users, and usually provide extra remarks like upgrade instructions or deprecation notices.

Re: [prometheus-developers] prometheus/prometheus Changelog Management

The friction is real – DCO is not a submission quality issue but a roundtrip one. This would be even more difficult with wording. I agree that in *many* cases contributors can write the changelog entry; having the field in the PR template would encourage them to do so proactively. /MR On Fri,

Re: [prometheus-developers] prometheus/prometheus Changelog Management

How do I make it so there is no entry? On Fri, 14 Feb 2020, 18:07 Simon Pasquier, wrote: > Correct, the PR is in the promu repository (I've updated it just now > to address comments from Brian though it should have been done long > ago): > https://github.com/prometheus/promu/pull/170 > > Right

Re: [prometheus-developers] Moving "official" JIRA Alertmanager integration (github.com/free/jiralert) to prometheus-community Organization.

+1 given the prominence of "warning alert = ticket" in SRE lore, having a building block available for this in -community is a good thing. /MR On Sun, Feb 16, 2020 at 11:08 AM Bartłomiej Płotka wrote: > Hi, > > As per https://github.com/prometheus-community/community/issues/6 I would > like to

Re: [prometheus-developers] prometheus/prometheus Changelog Management

mit message for this! We can > definitely build some automation around this and it looks like such > workflow would be a huge improvement! > > Thanks Matthias. > > Kind Regards, > Bartek > > On Fri, 14 Feb 2020 at 13:02, 'Matthias Rampke' via Prometheus Develo

Re: [prometheus-developers] [VOTE] New governance document

Yes On Thu, 20 Feb 2020, 06:32 Julien Pivotto, wrote: > Yes > > - Original Message - > From: Richard Hartmann > To: Prometheus Developers > Sent: Wed, 19 Feb 2020 21:43:52 +0100 (CET) > Subject: [prometheus-developers] [VOTE] New governance document > > Dear all, > > I am hereby

Re: [prometheus-developers] Extending tsdb tests

I am a little concerned about "tests that are okay to fail" … especially since merging one PR with that means we have no signal for subsequent PRs. Meanwhile corthanos might adjust their usage, breaking older branches that don't even touch this (solvable with rebasing but adds additional round