> I cannot have the hostname as Node Exporter won't be scraping any metrics at that point.
Can you give some more specific examples? What metric are you joining with - perhaps node_uname_info? Note that the "up" metric will still exist (with a value of 0) when a scrape fails - this means: (a) you can join on it, and (b) you can alert on this condition, i.e. scrape failed / node_exporter is down. This is a different condition than "blackbox_exporter says host/service is down, but node_exporter is still being scraped". Hence the alerting rule for (up == 0) can be written to avoid the join. There is actually a benefit here: you'll only get one alert when the host goes down, instead of lots. Other solutions you can consider: 1. Add labels to your targets at scrape time, either by adding static labels (file_sd_config) or using relabelling 2. Generate an entirely separate metadata timeseries, which is not scraped from the node itself. This can be done by: (a) a static recording rule as you suggested, see https://www.robustperception.io/using-time-series-as-alert-thresholds <https://www.robustperception.io/using-time-series-as-alert-thresholds> T <https://www.robustperception.io/using-time-series-as-alert-thresholds>here it's being used for alert thresholds, but you can just as well do this for metadata as per https://www.robustperception.io/how-to-have-labels-for-machine-roles/ https://www.robustperception.io/exposing-the-software-version-to-prometheus <https://www.robustperception.io/using-time-series-as-alert-thresholds> (b) a static web page that you scrape containing all the metadata for all the targets - for an example see https://github.com/candlerb/netbox-prometheus#metadata -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/776b156a-3b88-42fd-bcad-cfb6aa0fe1eb%40googlegroups.com.
