GitHub user mwaag created a discussion: Kubernetes-dashboard unreachable (sometimes)
Environment-Info: Cloudstack 4.20.3.0 Hypervisor: VMware 7.0.3 Kubernetes-Iso: v1.36.0 from https://download.cloudstack.org/cks/setup-v1.36.0-x86_64.iso Hi, we're experimenting with cloudstack kubernetes service right now and in some cases the kubernetes-dashboard started locally with kubectl --kubeconfig /path/to/kube.conf proxy gives us a ServiceUnavailability-Message when called in browser <img width="969" height="261" alt="09_04_51-000418" src="https://github.com/user-attachments/assets/11e46dfa-924e-408a-8a1d-5486a00d3112"; /> So we started investigating and found out that the cloud-controller-manager often is in state CrashLoopBackOff and collect many restarts after a while: ``` kubectl --kubeconfig /mnt/d/Users/mwa/Downloads/vpc3-kube_kube.conf get pods --all-namespaces NAMESPACE NAME READY STATUS RESTARTS AGE kube-system cloud-controller-manager-8648c8d5f-fmcwt 0/1 CrashLoopBackOff 899 (5m6s ago) 3d20h kube-system coredns-589f44dc88-pwgz8 1/1 Running 0 3d20h kube-system coredns-589f44dc88-x7kdm 1/1 Running 0 3d20h kube-system etcd-vpc3-kube-control-19edf733c33 1/1 Running 0 3d20h kube-system kube-apiserver-vpc3-kube-control-19edf733c33 1/1 Running 0 3d20h kube-system kube-controller-manager-vpc3-kube-control-19edf733c33 1/1 Running 2 (26h ago) 3d20h kube-system kube-proxy-6sv67 1/1 Running 0 3d20h kube-system kube-proxy-srm5w 1/1 Running 0 3d20h kube-system kube-scheduler-vpc3-kube-control-19edf733c33 1/1 Running 2 (26h ago) 3d20h kube-system weave-net-d8zc8 2/2 Running 0 3d20h kube-system weave-net-ttdnl 2/2 Running 1 (3d20h ago) 3d20h kubernetes-dashboard dashboard-metrics-scraper-6c78786c6f-vvbkx 1/1 Running 0 3d20h kubernetes-dashboard kubernetes-dashboard-57db5448b5-smgln 1/1 Running 0 3d20h ``` The log shows us a timeout trying to reach the management-server (192.168.0.87) ``` I0623 07:03:27.461933 1 serving.go:348] Generated self-signed cert in-memory W0623 07:03:40.762340 1 client_config.go:617] Neither --kubeconfig nor --master was specified. Using the inClusterConfig. This might not work. F0623 07:04:10.764367 1 main.go:74] Cloud provider could not be initialized: could not init cloud provider "external-cloudstack": Get "https://cloudportal0.test.c.fraunhofer.de/clien t/api/?apiKey=0s-l6KLik5QE7lTSd6NzlWEI5k0PSlOuFRvHREMILPg2Pm9R-_Nu7KPcUd2UriWb5lEqAA7SVQbJnJfrmjo6aA&command=listManagementServersMetrics&expires=2026-06-23T07%3A18%3A40Z&response=json&sig natureversion=3&signature=R7yOS7KhVxOowH2g8rW4U2J753Y%3D": dial tcp 192.168.0.87:443: i/o timeout ``` This brought us to among other things to https://github.com/apache/cloudstack/discussions/12037 and https://github.com/apache/cloudstack/discussions/10258 and https://github.com/apache/cloudstack/discussions/9267 Leading us to the questions: - Is the impossible callback to our management-server the root cause for this? (we had some dashboards running in some tests before although we used a completely wrong endpoint.url in our globals because we forgot about this setting in some earlier stadium). - Will deployed applications be reachable at all? (we will test this parallel to this posting) Since we're consider the VR-Networks as insecure in general and the cloudstack-ui is an internal-only resource would the reverse-proxy + firewall solution mentioned in the other discussions be advised for our scenario as well? GitHub link: https://github.com/apache/cloudstack/discussions/13473 ---- This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected]
