Hi, to clarify, I no longer have the issue with the LoadBalancer stuck in ‘Pending’. The redeployment of my Kubernetes cluster while experimenting with the various parameters for deploying a Kubernetes cluster finally got me to a point where my LoadBalancer properly got an IP address.
I now have the nginx Ingress controller operating and have deployed an Ingress that is accepting traffic. I am also now getting traffic at my pods. The simplest service / Deployment / pod that I have is a simple haproxy that is on a /icons/(.*) url that simply forwards to a S3 bucket. I have a README in that s3 bucket and I can do e.g. curl https://myapp.mydomain.com/icons/README and see the contents of the README. So everything is working now. I can put that command in a shell script loop and it returns my README every time so there’s not any performance issues now that the networking is finally networking. All my other services / Deployments / pods also have health check URLs that similarly are now working. As you would expect, since I’ve had this working in Azure for months now with the same Helm chart other than that it’s now pulling images from my local private registry rather than from my Azure private registry. The only issue I now have is that it took a significant amount of time – like, almost an hour – before Kubernetes started routing traffic to my pods. I will see whether that remains a problem when I get the CICD pipeline working. After a git checkin and image build it’s going to do a Helm update of the app to update the Deployment to pull in the updated image which in turn will trigger a rolling update, I will see whether that will disrupt the networking and whether I will have an issue with the app going offline after an update. So I think I’m okay for the moment. If the networking delay proves a problem going forward, I will update this thread with whatever information I can find that might be applicable. Kubernetes on Azure – 1.29.7 Kubernetes on Cloudstack – 1.28.4 Cloudstack version: 4.19.1.3 From: Kiran Chavala <kiran.chav...@shapeblue.com> Date: Tuesday, December 17, 2024 at 9:15 PM To: users@cloudstack.apache.org <users@cloudstack.apache.org> Subject: Re: Kubernetes nginx ingress svc stuck in 'Pending' Hi Eric Thanks for the information. Let me try to reproduce the issue in my lab. What is the cloudstack version and Kubernetes version you are using ? Also could you please share the nginx ingress controller yaml It’s documented in the example to use the cloud user https://docs.cloudstack.apache.org/en/4.20.0.0/plugins/cloudstack-kubernetes-service.html#kubernetes-clusters Regards Kiran From: Eric Green <eric.lee.gr...@gmail.com> Date: Wednesday, 18 December 2024 at 10:22 AM To: users@cloudstack.apache.org <users@cloudstack.apache.org> Subject: Re: Kubernetes nginx ingress svc stuck in 'Pending' Hi, I am using KVM with VLAN-based advanced networking. I have allocated VLANs 1000-2000 to VPC’s and looking at the virtual router, it’s using VLAN 1016. I successfully can ssh through the virtual router IP address to the nodes via the virtual router port redirects that were set up by Cloudstack when I instantiated the Kubernetes cluster (2222 and up, each node incrementing the port number). I did have to experiment a bit to find that the user name ‘cloud’ was what I needed to use to get into the nodes, that does not appear to be documented anywhere. From: Kiran Chavala <kiran.chav...@shapeblue.com> Date: Tuesday, December 17, 2024 at 8:26 PM To: users@cloudstack.apache.org <users@cloudstack.apache.org> Subject: Re: Kubernetes nginx ingress svc stuck in 'Pending' Hi Eric Could you please let me know the hypervisor you are using with Cloudstack. Is it KVM or Vmware ? Regards Kiran From: Eric Green <eric.lee.gr...@gmail.com> Date: Wednesday, 18 December 2024 at 4:40 AM To: users@cloudstack.apache.org <users@cloudstack.apache.org> Subject: Re: Kubernetes nginx ingress svc stuck in 'Pending' Ugh. Literally the simplest possible configuration and it's stuck in 'Pending'. So it has to be an issue on the Cloudstack side? Let me get the configuration of that Kubernetes cluster.... Okay. I destroyed the Kubernetes cluster and re-created it trying various options until your two commands created a working LoadBalancer with an assigned IP address. Then I created a Nagios ingress controller using the described YAML and successfully created the nginx ingress controller with an IP address for its LoadBalancer service. Now to get a working Ingress that routes traffic through the Services to my pods... Wish me luck. (To reiterate, I already have all of this working on AKS, Azure Kubernetes Service, it's just Cloudstack's Kubernetes that's being a pain to me). ________________________________ From: Jayanth Babu Reddy <jayanth.b...@nxtgen.com.INVALID> Sent: Wednesday, December 11, 2024 7:22 PM To: users@cloudstack.apache.org <users@cloudstack.apache.org> Subject: Re: Kubernetes nginx ingress svc stuck in 'Pending' Just to confirm if there's not any misconfiguration at ACS, try doing these and let us know if you get the IP assigned. $ kubectl run --image=nginx nginx $ kubectl expose pod/nginx --type=LoadBalancer --port=80 For your nginx ingress controller service of type Load Balancer, what does $ kubectl describe svc ingress-nginx-controller say? Thanks, Jayanth
