On Fri, 18 Dec 2020 at 10:50, Skaven <[email protected]> wrote: > Hello, folks. > > The story. > In my org we have several k8s clusters and quite unreliable security > department, who has control over firewall and have a habbit of corrupting > the rules on said firewall. > The confusion is immense. The issue is that at any point in time *one or > several nodes can lose access to one or several external resources*. > > So, we had a brilliant idea to mitigate the confusion by deploying a *blackbox > daemonset *and *configure Prometheus to query external resources*. But! > We have many-to-many relationship here. We want to query multiple resources > from all of the worker nodes in cluster. > > This means, that we need auto discovery (to dynamically get instances of > daemon set) and multiple targets. > In docs we found an example of multi target requests though static > configs. Unfortunately, as the name suggest, it is a static config and we > can't get the *metadata of node* that lost connection or *reliably* > determine the fact that connection was lost at all (no way to properly > configure alert threshold, so it wouldn't misfire). >
> On the other hand - there is Kubernetes service discovery. But the only > way we found is to have a separate job for each of the remote resources. > Yes, that's the way to do it. Use the blackbox exporter as though it was a normal exporter, and then have a scrape config for each of your external resources. Brian > > Is there, maybe, a better way to approach the problem? > > > -- Brian Brazil www.robustperception.io -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/CAHJKeLrr2dG4dZE5uax5AheNdDJikqgrRLAiiaxp82EgdcUfcw%40mail.gmail.com.
