On Wed, May 09, 2018 at 07:35:23AM -0700, a...@unbotify.com wrote: > Hi all, > > I'm building a K8s cluster with KOps. With 3 IGs, one in each AZ. Nodes are > automatically tagged with failure-domain tags - > e.g.```failure-domain.beta.kubernetes.io/region=us-east-1,failure-domain.beta.kubernetes.io/zone=us-east-1a``` > and so for each AZ. > I'v added the cluster-autoscaler with auto discovery based on: > https://github.com/kubernetes/autoscaler/blob/master/cluster-autoscaler/cloudprovider/aws/examples/cluster-autoscaler-autodiscover.yaml > > then created a few hundreds of dummy containers just to see it scale out, but > once it started scaling up the underlying ASG, it was completely uneven (one > ASG went to maximum and one other was added a couple of nodes, while the > third did nothing). Pods were distributed accordingly (this is a new cluster > so nothing else runs there).
I haven't used the cluster-autoscaler, so sorry if this doesn't really help. But I think as a workarround (or might even be better for some scenarios), you can just have one ASG (and on IG in kops, that matches that very same ASG). But, the trick is to make that ASG/IG have several subnets, one in every AZ you want. Then, the ASG (is documented by aws, not just by chance) will spread the nodes it creates in the different AZs. And it will do it evenly. And if one AZ fails, then it will move the nodes to another. And then if it recovers, it will move the nodes again so it's even. We are using that ASG feature (with kops) and it works fine. And I guess that if you have only one ASG, cluster-autoscaler will only increase the nodes numbers and AWS will do the spreding things as capacity is available and that stuff. -- You received this message because you are subscribed to the Google Groups "Kubernetes user discussion and Q&A" group. To unsubscribe from this group and stop receiving emails from it, send an email to kubernetes-users+unsubscr...@googlegroups.com. To post to this group, send email to kubernetes-users@googlegroups.com. Visit this group at https://groups.google.com/group/kubernetes-users. For more options, visit https://groups.google.com/d/optout.
