We are looking for a way to drop scrpaed labels dynamically. We are using consul-service discovery for scraping various targets/exporters. We are trying to define re-usable scrapeconfigs, where most parameters are defined in the consul-service definition. That works pretty well, we are also able to define custom-labels that way.
But we are struggeling when trying to delete labels from the scraped metrics. We have cases where the exporter exposes labels we don't want. But instead of defining separate scrape_configs with hard-codes drop-label actions, we want to re-use our scrape-config. We tried the following We are using a technique like the one described here ( https://www.robustperception.io/extracting-full-labels-from-consul-tags) to extract all labels that should be removed fom a comma-separated-string. This string comes from our consul service-discovery. We apply those relabeling in metric_relabel_configs. To remove the labels we set the value to an empty string. It looks like that relabel_configs: - source_labels: [__meta_consul_service_metadata_labeldrop] target_label: 'labeldrop' metric_relabel_configs: - source_labels: [labeldrop] regex: '(?:[^,]+,){0}([^,]+).*' replacement: '' target_label: ${1} - source_labels: [labeldrop] regex: '(?:[^,]+,){1}([^,]+).*' replacement: '' target_label: ${1} and so on But unfortunately it does not work, the labels are still there. After investigating a little bit further it seems like using an empty replacement in combination with the regex-capturing-group causes problems. When e.g. using a static value like 'val' for the replacement all labels from our "list" get that value. I opened an issue for this (https://github.com/prometheus/prometheus/issues/8186), but was told it is an configuration issue. Maybe that is not the right way to approach out problem - but i am not able to find a solution (besides having multiple scrape_configs with different droplabel actions) Any help is appreciated - thank you -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/d5120a6f-5a8f-4247-9c0e-12571048c7c2n%40googlegroups.com.
