I created the image using RHEL 7 and I could see that the DNS is delegated to Openshift node hosting this pod. I was also able to run curl command from within the pod which was successful. But as you point out, issue could very well be within the image I built, will try to gather more information when it happens again. I updated the prometheus & alertmanager with most recent version and restarted the pods, keeping my fingers crossed. Thank you for your help.
sh-4.2$ cat /etc/resolv.conf nameserver 10.244.60.18 search prometheus-custom.svc.cluster.local svc.cluster.local cluster.local localdomain xyz.com options ndots:5 sh-4.2$ On Monday, November 23, 2020 at 10:32:54 AM UTC-5 [email protected] wrote: > The OS that the host is running makes no difference; the question is what > OS the container is built from. You'll see this in the Dockerfile used to > build the container. > > If you are using the off-the-shelf docker container for blackbox_exporter > then it will be this Dockerfile > <https://github.com/prometheus/blackbox_exporter/blob/master/Dockerfile> > which > builds from quay.io/prometheus/busybox-linux-amd64:latest > This in turn appears to come from here > <https://github.com/prometheus/busybox>, which in turn is based on > debian:buster > <https://github.com/prometheus/busybox/blob/master/uclibc/Dockerfile> or > debian:buster-slim > <https://github.com/prometheus/busybox/blob/master/glibc/Dockerfile>. I > think those are systemd-based. > > I think you should docker exec into the running container, and see if > systemd-resolved is running, and/or if /etc/resolv.conf points to > 127.0.0.53. If so, the systemd bug I pointed to is relevant. > > If not, then you can try resolving host > arp-executor-sy-shra-arp-p.icl1p.xyz.com yourself to see if it resolves > or not. Ultimately, this problem isn't with blackbox-exporter, it's a case > of debugging why DNS isn't resolving. Intermittent DNS resolution can also > be caused by problems with your authoritative DNS. > -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/3dc6e074-23b8-45d7-ab97-fdb8dade3fa2n%40googlegroups.com.
