I'm deploying Prometheus using the Helm chart
<https://github.com/prometheus-community/helm-charts/tree/main/charts/prometheus>
and I have it configured to scrape Istio mTLS-secured pods using the TLS
settings specified
<https://istio.io/latest/docs/ops/integrations/prometheus/#tls-settings> by
the Istio team to do so. Basically what this amounts to is:
- Add the Istio sidecar to the Prometheus instance but disable all
traffic proxying - you just want to get the certificates from it.
- Mount the certificates into the Prometheus container.
- Set up your scrape configuration to use the certificates when scraping
Istio-enabled pods.
The YAML for the scrape configuration looks like this:
- job_name: "kubernetes-pods-istio-secure"
scheme: https
tls_config:
ca_file: /etc/istio-certs/root-cert.pem
cert_file: /etc/istio-certs/cert-chain.pem
key_file: /etc/istio-certs/key.pem
insecure_skip_verify: true
*This totally works using Prometheus v2.20.1* packaged as `prom/prometheus`
from Docker Hub.
*This fails on Prometheus v2.28.0* packaged as
`quay.io/prometheus/prometheus`. Instead of getting a successful scrape, I
get "connection reset by peer." I've validated the files are there and
properly mounted; they have the expected contents; and there are no
Prometheus log messages to indicate anything is amiss.
I've been rolling back slowly to see where it starts working again. I've
tried v2.26.0 and it still fails. I thought I'd drop a note in here to see
if anyone knows what's up.
--
You received this message because you are subscribed to the Google Groups
"Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/prometheus-users/527c1143-d796-4068-aa09-cb6d548905a8n%40googlegroups.com.
