Re: [kubernetes-users] Looking for Kubernetes on AWS users to connect (preferably Berlin/Europe)

2016-10-13 Thread Yusuke KUOKA 
Hi,

FYI, we have most of what you've mentioned in our roadmap 
 and some of them 
already have working PRs:

   - Discrete etcd cluster 
    (3+ dedicated 
   etcd nodes)
   - HA control plane  
(multiple 
   EC2 instances and an ELB load balancer, a separate ASG for the 
   master=controller)
   - Cluster upgrades 

If you'd like to apply the cluster-autoscaler from kubernetes/contrib 
 for 
node auto-scaling, you would be interested in the issue I've reported 
.
Regarding the single CF template thing, I suppose splitting templates would 
be an implication of that issue.

Also, if you'd like to apply AWS native node auto-scaling based on 
CloudWatch alarms, especially when automatic down-scaling, you would be 
interested in this PR  to 
avoid small downtime while terminating nodes.

Yusuke

2016年10月14日金曜日 6時58分34秒 UTC+9 Henning Jacobs:
>
> Brief answer why we are not using kube-aws directly right now:
>
>- It uses a single master EC2 instance --- we want to have an ASG for 
>the master nodes (probably running with size 1 usually, but having the 
>option for more, e.g. during updates/migrations etc)
>- It runs etcd on the master --- we want to run etcd separately 
>(currently we use our own 3 node etcd appliance with DNS discovery (SRV 
>records))
>- It does not configure an ELB for the API server --- we want to 
>terminate SSL at ELB in order to leverage existing SSL infrastructure 
>(including ACM)
>- It uses a single CF template --- we want to split into at least 3 CF 
>templates to facilitate future upgrades and extra node pools (one for etcd 
>cluster, one for master and one for worker nodes)
>
> We therefore adapted the generated Cloud Formation to YAML and are using 
> our own Senza Cloud Formation tool 
>  for deployment (it's not doing 
> any magic, but e.g. makes ELB+DNS config easy).
>
> I'll put our current (hacked) config into some public repo tomorrow...
>
> - Henning
>
>
> 2016-10-13 21:23 GMT+02:00 Brandon Philips  >:
>
>> On Thu, Oct 13, 2016 at 9:28 AM Henning Jacobs > > wrote:
>>
>>>
>>>- 
>>>
>>>We currently deploy test clusters with Cloud Formation (adapted from 
>>>kube-aws 
>>>
>>>)
>>>
>>>
>> Why couldn't you use kube-aws directly? What would your ideal tool do 
>> differently?
>>
>> Thanks,
>>
>> Brandon 
>>
>> -- 
>> You received this message because you are subscribed to a topic in the 
>> Google Groups "Kubernetes user discussion and Q" group.
>> To unsubscribe from this topic, visit 
>> https://groups.google.com/d/topic/kubernetes-users/WRsVjG7vc9Y/unsubscribe
>> .
>> To unsubscribe from this group and all its topics, send an email to 
>> kubernetes-use...@googlegroups.com .
>> To post to this group, send email to kubernet...@googlegroups.com 
>> .
>> Visit this group at https://groups.google.com/group/kubernetes-users.
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Kubernetes user discussion and Q" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to kubernetes-users+unsubscr...@googlegroups.com.
To post to this group, send email to kubernetes-users@googlegroups.com.
Visit this group at https://groups.google.com/group/kubernetes-users.
For more options, visit https://groups.google.com/d/optout.

Re: [kubernetes-users] Looking for Kubernetes on AWS users to connect (preferably Berlin/Europe)

2016-10-13 Thread Henning Jacobs 
Brief answer why we are not using kube-aws directly right now:

   - It uses a single master EC2 instance --- we want to have an ASG for
   the master nodes (probably running with size 1 usually, but having the
   option for more, e.g. during updates/migrations etc)
   - It runs etcd on the master --- we want to run etcd separately
   (currently we use our own 3 node etcd appliance with DNS discovery (SRV
   records))
   - It does not configure an ELB for the API server --- we want to
   terminate SSL at ELB in order to leverage existing SSL infrastructure
   (including ACM)
   - It uses a single CF template --- we want to split into at least 3 CF
   templates to facilitate future upgrades and extra node pools (one for etcd
   cluster, one for master and one for worker nodes)

We therefore adapted the generated Cloud Formation to YAML and are using
our own Senza Cloud Formation tool 
for deployment (it's not doing any magic, but e.g. makes ELB+DNS config
easy).

I'll put our current (hacked) config into some public repo tomorrow...

- Henning


2016-10-13 21:23 GMT+02:00 Brandon Philips :

> On Thu, Oct 13, 2016 at 9:28 AM Henning Jacobs 
> wrote:
>
>>
>>-
>>
>>We currently deploy test clusters with Cloud Formation (adapted from
>>kube-aws
>>
>>)
>>
>>
> Why couldn't you use kube-aws directly? What would your ideal tool do
> differently?
>
> Thanks,
>
> Brandon
>
> --
> You received this message because you are subscribed to a topic in the
> Google Groups "Kubernetes user discussion and Q" group.
> To unsubscribe from this topic, visit https://groups.google.com/d/
> topic/kubernetes-users/WRsVjG7vc9Y/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> kubernetes-users+unsubscr...@googlegroups.com.
> To post to this group, send email to kubernetes-users@googlegroups.com.
> Visit this group at https://groups.google.com/group/kubernetes-users.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Kubernetes user discussion and Q" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to kubernetes-users+unsubscr...@googlegroups.com.
To post to this group, send email to kubernetes-users@googlegroups.com.
Visit this group at https://groups.google.com/group/kubernetes-users.
For more options, visit https://groups.google.com/d/optout.

Re: [kubernetes-users] Looking for Kubernetes on AWS users to connect (preferably Berlin/Europe)

2016-10-13 Thread Brandon Philips 
On Thu, Oct 13, 2016 at 9:28 AM Henning Jacobs 
wrote:

>
>-
>
>We currently deploy test clusters with Cloud Formation (adapted from
>kube-aws
>
>)
>
>
Why couldn't you use kube-aws directly? What would your ideal tool do
differently?

Thanks,

Brandon

-- 
You received this message because you are subscribed to the Google Groups 
"Kubernetes user discussion and Q" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to kubernetes-users+unsubscr...@googlegroups.com.
To post to this group, send email to kubernetes-users@googlegroups.com.
Visit this group at https://groups.google.com/group/kubernetes-users.
For more options, visit https://groups.google.com/d/optout.

[kubernetes-users] Looking for Kubernetes on AWS users to connect (preferably Berlin/Europe)

2016-10-13 Thread Henning Jacobs 


Hi all,

we (Zalando Tech  in Berlin) are currently in 
the middle of setting up Kubernetes on AWS for production workloads. We are 
looking for existing experiences and users to connect to (preferably in 
Berlin or Europe, we can travel ;-)).

Some background:

   - 
   
   We currently have hundreds of AWS accounts with thousands of production 
   EC2 instances (all running Docker, but one container per EC2 instance)
   - 
   
   We are planning to migrate to a more consolidated setup with less AWS 
   accounts, running Kubernetes
   - 
   
   We currently deploy test clusters with Cloud Formation (adapted from 
   kube-aws 
   )
   

Some open issues we are facing right now:

   - 
   
   Cluster/network setup, multi-AZ or not, etcd
   - 
   
   Kubernetes federation
   - 
   
   Operating a production cluster incl. upgrades
   - 
   
   Autoscaling for nodes
   - 
   
   AWS integration (e.g. Application Load Balancer)
   

For many of these we have potential solutions, but we would like to hear 
what you are doing and exchange ideas.

See you around :-)

PS: we have cake

-- 
You received this message because you are subscribed to the Google Groups 
"Kubernetes user discussion and Q" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to kubernetes-users+unsubscr...@googlegroups.com.
To post to this group, send email to kubernetes-users@googlegroups.com.
Visit this group at https://groups.google.com/group/kubernetes-users.
For more options, visit https://groups.google.com/d/optout.