Host OS:
Linux monitoring-blackbox-exporter-66b8d48bf9-wxzmf 4.19.86-coreos #1
<https://github.com/prometheus/blackbox_exporter/pull/1> SMP Mon Dec 2
20:13:38 -00 2019 x86_64 GNU/Linux
Blackbox_exporter version:
blackbox_exporter, version 0.16.0 (branch: HEAD, revision:
991f89846ae10db22a3933356a7d196642fcb9a9)
build user: root@64f600555645
build date: 20191111-16:27:24
go version: go1.13.4
blackbox.yml module config
modules:
http_2xx:
prober: http
http:
method: GET
preferred_ip_protocol: "ip4"
valid_status_codes: [200]
http_post_2xx:
prober: http
http:
method: POST
http_kubernetes_service:
prober: http
timeout: 5s
http:
headers:
Accept: "*/*"
Accept-Language: "en-US"
tls_config:
ca_file: /var/run/secrets/kubernetes.io/serviceaccount/ca.crt
bearer_token_file: /var/run/secrets/kubernetes.io/serviceaccount/token
preferred_ip_protocol: "ip4"
prometheus.yml scrape config
- job_name: blackbox-exporter-kubernetes-services
tls_config:
ca_file: /var/run/secrets/kubernetes.io/serviceaccount/ca.crt
bearer_token_file: /var/run/secrets/kubernetes.io/serviceaccount/token
metrics_path: /probe
params:
module: [http_2xx]
kubernetes_sd_configs:
- role: service
relabel_configs:
- source_labels: [__meta_kubernetes_service_annotation_prometheus_io_probed]
action: keep
regex: true
- source_labels: [__address__]
target_label: __param_target
- target_label: __address__
replacement: monitoring-blackbox-exporter.kyma-system.svc.cluster.local:9115
- source_labels: [__param_target]
target_label: instance
- action: labelmap
regex: __meta_kubernetes_service_label_(.+)
- source_labels: [__meta_kubernetes_namespace]
target_label: kubernetes_namespace
- source_labels: [__meta_kubernetes_service_name]
target_label: kubernetes_name
Test Probe
http://localhost:9115/probe?target=prometheus.io&module=http_2xx&debug=true
Logs for the probe:
ts=2020-03-10T07:06:30.506042853Z caller=main.go:304 module=http_2xx
target=prometheus.io level=info msg="Beginning probe" probe=http
timeout_seconds=119.5
ts=2020-03-10T07:06:30.506279096Z caller=http.go:318 module=http_2xx
target=prometheus.io level=info msg="Resolving target address" ip_protocol=ip4
ts=2020-03-10T07:06:30.763089729Z caller=http.go:318 module=http_2xx
target=prometheus.io level=info msg="Resolved target address" ip=104.24.121.60
ts=2020-03-10T07:06:30.763216413Z caller=client.go:250 module=http_2xx
target=prometheus.io level=info msg="Making HTTP request"
url=http://104.24.121.60 host=prometheus.io
ts=2020-03-10T07:06:30.861500108Z caller=main.go:119 module=http_2xx
target=prometheus.io level=info msg="Received HTTP response" status_code=502
ts=2020-03-10T07:06:30.861586357Z caller=main.go:119 module=http_2xx
target=prometheus.io level=info msg="Invalid HTTP response status code"
status_code=502 valid_status_codes=[200]
ts=2020-03-10T07:06:30.861626907Z caller=main.go:119 module=http_2xx
target=prometheus.io level=info msg="Response timings for roundtrip"
roundtrip=0 start=2020-03-10T07:06:30.763348055Z
dnsDone=2020-03-10T07:06:30.763348055Z
connectDone=2020-03-10T07:06:30.763660973Z
gotConn=2020-03-10T07:06:30.763698243Z
responseStart=2020-03-10T07:06:30.76434826Z end=2020-03-10T07:06:30.861608527Z
ts=2020-03-10T07:06:30.861681091Z caller=main.go:304 module=http_2xx
target=prometheus.io level=error msg="Probe failed" duration_seconds=0.355511167
http://localhost:9115/probe?target=https://www.baidu.com&module=http_2xx&debug=true
Logs for the probe:
ts=2020-03-10T07:09:46.051420326Z caller=main.go:304 module=http_2xx
target=https://www.baidu.com level=info msg="Beginning probe" probe=http
timeout_seconds=119.5
ts=2020-03-10T07:09:46.051622723Z caller=http.go:318 module=http_2xx
target=https://www.baidu.com level=info msg="Resolving target address"
ip_protocol=ip4
ts=2020-03-10T07:09:47.440699378Z caller=http.go:318 module=http_2xx
target=https://www.baidu.com level=info msg="Resolved target address"
ip=104.193.88.77
ts=2020-03-10T07:09:47.440812901Z caller=client.go:250 module=http_2xx
target=https://www.baidu.com level=info msg="Making HTTP request"
url=https://104.193.88.77 host=www.baidu.com
ts=2020-03-10T07:09:47.461982292Z caller=main.go:119 module=http_2xx
target=https://www.baidu.com level=error msg="Error for HTTP request" err="Get
https://104.193.88.77: read tcp 100.96.15.99:59676->104.193.88.77:443: read:
connection reset by peer"
ts=2020-03-10T07:09:47.46204674Z caller=main.go:119 module=http_2xx
target=https://www.baidu.com level=info msg="Response timings for roundtrip"
roundtrip=0 start=2020-03-10T07:09:47.440976902Z
dnsDone=2020-03-10T07:09:47.440976902Z
connectDone=2020-03-10T07:09:47.441181016Z gotConn=0001-01-01T00:00:00Z
responseStart=0001-01-01T00:00:00Z end=0001-01-01T00:00:00Z
ts=2020-03-10T07:09:47.462122893Z caller=main.go:304 module=http_2xx
target=https://www.baidu.com level=error msg="Probe failed"
duration_seconds=1.410634767
Actual Result:
Have added according annotation for kubernetes services, in Prometheus, the
targets are shown as down with error
Get
http://monitoring-blackbox-exporter.kyma-system.svc.cluster.local:9115/probe?module=http_2xx&target=kiali.kyma-system.svc%3A20001:
read tcp 100.96.1.6:40728->100.70.162.45:9115: read: connection reset by peer
Get
http://monitoring-blackbox-exporter.kyma-system.svc.cluster.local:9115/probe?module=http_2xx&target=kiali.kyma-system.svc%3A20001:
read tcp 100.96.1.6:40728->100.70.162.45:9115: read: connection reset by peer
--
You received this message because you are subscribed to the Google Groups
"Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/prometheus-users/34bc49e7-6e5a-4634-9675-c66f4c2ff2a5%40googlegroups.com.
