I think you need to handle it on a case-by-case basis. I suspect that a stack update would still be appropriate for most cases. If you still have concerns, you could always launch a second stack (pointing at the same ZK location) and migrate over by scaling down the first stack.
I've had success with both approaches, so I encourage you to try them and see how they work for your environment. On Wed, Jul 16, 2014 at 3:37 PM, Nayeem Syed <nay...@cronycle.com> wrote: > say if i want to change things in the cluster, maybe change a SG setting, > change a ELB size, ... anything in the formation. > > > On Wed, Jul 16, 2014 at 10:33 PM, Michael Babineau < > michael.babin...@gmail.com> wrote: > >> What scenario did you have in mind? (Which resource do you expect to be >> problematic?) >> >> On Wed, Jul 16, 2014 at 2:27 PM, Nayeem Syed <nay...@cronycle.com> wrote: >> >>> Hi Michael, >>> >>> I was considering also using cloud-formation for setting up Mesos >>> clusters. So the update is only possible for AutoScaling? Thats a bit >>> limiting isnt it? What if you need to update other resources? >>> >>> >>> On Wed, Jul 16, 2014 at 5:47 PM, Michael Babineau < >>> michael.babin...@gmail.com> wrote: >>> >>>> I manage my Mesos clusters via CloudFormation. Upgrades are pretty >>>> straightforward -- just apply an updated template (and/or parameters) to an >>>> existing stack. For now, I manually terminate instances (one at a time) and >>>> let the auto scaling groups bring up replacements with the new >>>> configuration. >>>> >>>> This should be automated with an UpdatePolicy >>>> <http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-attribute-updatepolicy.html>, >>>> I just haven't taken the time yet to implement and test it. >>>> >>>> By the way, the templates are open source, so feel free to use or steal >>>> from them: https://github.com/thefactory/cloudformation-mesos >>>> >>>> As Dick mentioned, however, this still doesn't cover ad-hoc >>>> maintenance. But it should make provisioning much less of a headache :) >>>> >>>> >>>> On Wed, Jul 16, 2014 at 4:35 AM, Nayeem Syed <nay...@cronycle.com> >>>> wrote: >>>> >>>>> you mean ad-hoc maintenance formations wont help? >>>>> >>>>> probably should have it deployed using formations initially and use >>>>> user-data to customize the server with specific installs etc? >>>>> >>>>> >>>>> On Wed, Jul 16, 2014 at 12:15 PM, Dick Davies <d...@hellooperator.net> >>>>> wrote: >>>>> >>>>>> For provisioning yes , for ad-hoc maintenance tasks won't help at all. >>>>>> >>>>>> On 16 July 2014 11:29, Nayeem Syed <nay...@cronycle.com> wrote: >>>>>> > Thanks for those! I will give it a try to get some deployment >>>>>> through >>>>>> > ansible. >>>>>> > >>>>>> > I was also wondering if Cloudformation might be good for this? As >>>>>> it clears >>>>>> > up the things very cleanly when you remove the formation? Though I >>>>>> find >>>>>> > their JSON file very difficult to navigate and their Update Feature >>>>>> doesnt >>>>>> > seem to work too well.. >>>>>> > >>>>>> > >>>>>> > On Wed, Jul 16, 2014 at 10:46 AM, Dick Davies < >>>>>> d...@hellooperator.net> >>>>>> > wrote: >>>>>> >> >>>>>> >> I'd like to show you my playbooks, but unfortunately they're for a >>>>>> client >>>>>> >> - I can vouch for it being very easy to add nodes to a cluster >>>>>> etc. if you >>>>>> >> just have to edit an 'inventory' file and add IPs into the correct >>>>>> groups. >>>>>> >> >>>>>> >> (NB: puppet and chef will automate your infrastructure too, it's >>>>>> just >>>>>> >> they're >>>>>> >> not as useful for things like rolling deployments in my experience >>>>>> because >>>>>> >> they're agent based, so it's harder to control when each server >>>>>> will >>>>>> >> update and >>>>>> >> restart services). >>>>>> >> >>>>>> >> A quick Google found: >>>>>> >> >>>>>> >> >>>>>> >> >>>>>> http://blog.michaelhamrah.com/2014/06/setting-up-a-multi-node-mesos-cluster-running-docker-haproxy-and-marathon-with-ansible/ >>>>>> >> >>>>>> >> which might be useful. >>>>>> >> >>>>>> >> The play books linked from that post are for bootstrapping a >>>>>> cluster, but >>>>>> >> it's >>>>>> >> pretty simple to add a second playbook to manage rolling deploys >>>>>> etc. >>>>>> >> There's some Ansible examples of rolling deploys (not Mesos >>>>>> specific) >>>>>> >> at : >>>>>> >> >>>>>> >> http://docs.ansible.com/guide_rolling_upgrade.html >>>>>> >> >>>>>> >> >>>>>> >> On 15 July 2014 14:41, Nayeem Syed <nay...@cronycle.com> wrote: >>>>>> >> > thanks! >>>>>> >> > >>>>>> >> > do you have some examples of how you are using it with ansible? >>>>>> i dont >>>>>> >> > have >>>>>> >> > specific preferences, whatever works really. >>>>>> >> > >>>>>> >> > >>>>>> >> > On Tue, Jul 15, 2014 at 2:35 PM, Dick Davies < >>>>>> d...@hellooperator.net> >>>>>> >> > wrote: >>>>>> >> >> >>>>>> >> >> You want a rolling restart i'd guess, unless you want downtime >>>>>> for some >>>>>> >> >> reason. >>>>>> >> >> >>>>>> >> >> We use Ansible, it's pretty nice. >>>>>> >> >> >>>>>> >> >> On 15 July 2014 10:47, Nayeem Syed <nay...@cronycle.com> wrote: >>>>>> >> >> > whats the best way to update mesos master instances. eg I >>>>>> want to >>>>>> >> >> > update >>>>>> >> >> > things in there, install new frameworks, but at the moment I >>>>>> am >>>>>> >> >> > ssh'ing >>>>>> >> >> > to >>>>>> >> >> > the instances and installing them one by one. that feels >>>>>> wrong, >>>>>> >> >> > shouldnt >>>>>> >> >> > it >>>>>> >> >> > be done in parallel to all the instances? >>>>>> >> >> > >>>>>> >> >> > what do people currently do to keep all the masters in sync? >>>>>> >> > >>>>>> >> > >>>>>> > >>>>>> > >>>>>> >>>>> >>>>> >>>> >>> >> >
