Hi Brian,
Thanks for the response.
I have configured static and sd both as different job and when I am hitting
SD job, I am getting targets as all the services inside the cluster but
they are throwing 400 or 404 error but when I check the logs, the status is
200.
I am attaching job for your reference. Let me know if I am missing
something. I also tried to check connection by going into blackbox-exporter
pod but it is still throwing same error. I have put all under same
namespace to avoid any communication failure.
- job_name: "blackbox-kubernetes-services"
metrics_path: /probe
params:
module: [http_2xx]
kubernetes_sd_configs:
- role: service
relabel_configs:
# Example relabel to probe only some services that have
"example.io/should_be_probed = true" annotation
# - source_labels:
[__meta_kubernetes_service_annotation_example_io_should_be_probed]
# action: keep
# regex: true
- source_labels: [__address__]
target_label: __param_target
- target_label: __address__
replacement: prometheus-blackbox-exporter:9115
- source_labels: [__param_target]
target_label: instance
- action: labelmap
regex: __meta_kubernetes_service_label_(.+)
- source_labels: [__meta_kubernetes_namespace]
target_label: kubernetes_namespace
- source_labels: [__meta_kubernetes_service_name]
target_label: kubernetes_service_name
On Friday, 17 March 2023 at 21:04:29 UTC+5:30 Brian Candler wrote:
> > When I am using static config then it shows fine but when I am using
> service discovery for status of service or pod some are coming up some are
> down
>
> Then clearly you're not scraping/probing the same endpoints as you did
> with the static config.
>
> The easiest way to check is to go to the Prometheus web interface and look
> at the "Status > Targets" page and see what *actual* targets you're
> probing: first with the static config, and then secondly with the SD config
> (actually you can do both at the same time, but configure them as separate
> jobs so you can distinguish them by the "job" label). The first column
> shows the URL being scraped and the URL params being passed ("module" and
> "target" are the ones that need to be set for blackbox_exporter, of course).
>
> If you can show your static config, and the service discovery config, the
> problem might be clearer. My guess is the SD discovery isn't returning
> data in the form that you expect, or that your rewrite rules aren't working
> in the way that you expect.
>
> On Friday, 17 March 2023 at 15:23:28 UTC dikshant rai wrote:
>
>> Hi,
>> I tried to configure blackbox exporter for checking api status with
>> prometheus on kubernetes. When I am using static config then it shows fine
>> but when I am using service discovery for status of service or pod some are
>> coming up some are down. I used telnet to check connectivity as well that
>> is fine but still don't know where it is failing.
>> Please help.
>>
>
--
You received this message because you are subscribed to the Google Groups
"Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to prometheus-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/prometheus-users/e362d6b2-27c5-43a9-a2b2-b7193b12d564n%40googlegroups.com.
