On Mon, Nov 02, 2015 at 11:05:04AM -0800, Spencer Krum wrote: > I agree with the strategy put forward by Clark and Jeremy. As for the > dynamic control Paul is talking about creating, I think it does have > value, but we can get away with this upgrade without it. After we're on > puppet apply, building automation so that different nodes can use > different modules makes more sense. > I am not sure how we can pass the gate with out it. Once we bump puppet-mysql to 3.6.1, everything will break. Depends-On won't help us, since we'll have a bunch of circular dependencies.
> It is my goal to work on puppet apply before the puppet-mysql upgrade. > We have elasticsearch and gerrit upgrades coming down the pipe as well. > So I'm not sure when we can do this work or who can do it. > Right, so waiting on puppet apply changes we are blocking the puppet-mysql upgrade path. What do you think your timelines are for fully depolying puppet apply into production? I know crinkle had another patch up[1] that sandboxes infra-cloud modules into a new directory, but not sure status of that. If infra-roots are fine with that, that will at least allow infra-cloud to move forward with the newer version of puppet-mysql. This method is the first step of the dynamic modulepath too. [1] https://review.openstack.org/#/c/209617/ > -- > Spencer Krum > [email protected] > > On Mon, Nov 2, 2015, at 10:49 AM, Paul Belanger wrote: > > On Mon, Nov 02, 2015 at 10:27:26AM -0800, Clark Boylan wrote: > > > On Mon, Nov 2, 2015, at 07:58 AM, Paul Belanger wrote: > > > > Greetings, > > > > > > > > I'd like to start a thread talking about the effort to upgrade our > > > > version of > > > > puppet-mysql to a newer / latest version. I know there has been some > > > > talk > > > > on this already, would somebody mind adding some information? > > > > > > > > I have heard 3 things: > > > > > > > > 1. Remove database support from current modules, and only use > > > > sql_connection > > > > strings. > > > > 2. Move everything to trove > > > > 3. Setup mysql cluster > > > I think this is mostly orthogonal to updating the puppet-mysql module > > > because 2 + 1ish (use sql_connection) is basically what we do everywhere > > > but Jenkins slave image builds, paste.openstack.org, and > > > wiki.openstack.org. > > > > > > > > Again, I don't know if there are true or not, just things I have seen > > > > people > > > > talk about. > > > > > > > > The obviously part that is missing, is _how_ we are going to do the > > > > upgrade. I > > > > know some people (clarkb?) already have some ideas on that. > > > > > > > > The reason for me asking, 2 weeks ago I offered to help the infra-cloud > > > > move > > > > forward and upgrading puppet-mysql was one of the items discussed. So, > > > > here I > > > > am, offering to do the grunt work, just need some understanding on what > > > > people > > > > want to do. > > > I was hoping that there was a forward and backward compatible > > > intermediate step we could make where old and new configs were supported > > > but I am told that isn't possible. As a result we will likely need to > > > update puppet-mysql, then all three of the above uses of puppet-mysql > > > semi atomically to keep everything working. > > > > > > Testing that image builds work before the update is simple as we can > > > just run > > > openstack-infra/project-config/nodepool/scripts/prepare_node_bare.sh to > > > see if that works. Paste.o.o is simple and has gone from local Drizzle > > > to Trove to local MySQL without much fuss so I doubt it will have much > > > trouble but a test deployment is easy if we are worried. > > > > > > The tricky one is going to be wiki.openstack.org and this is maybe where > > > the above list is relevant to the discussion. We could move it to an off > > > host database hosted by trove and not worry about updating puppet-mysql > > > in that module at all. Or we can test a deployment of wiki using the > > > newer mysql module before prod deployment. In either case we should > > > probably announce a wiki downtime prior to the upgrade, stop apache/php, > > > perform a database backup, switch to trove/run puppet with newer module, > > > restart apache/php. > > > > > So one of the big things I see, is once we bump puppet-mysql to the newer > > / > > latest version, all our current nodes will start consuming it (unless we > > stop > > puppet on each server). > > > > One thought I had was some like this: > > > > 1. Create /opt/puppet/modules/old, install current puppet-mysql module > > into > > it. > > 2. Remove /etc/puppet/modules/mysql and append --modulepath to now > > include > > /opt/puppet/modules/old > > 3. Add dynamic logic to build modulepath for puppet-mysql based on flag > > and > > force all nodes to it. > > 4. install puppet-mysql latest into /etc/puppet/modules. > > 5. Start migration process, paste.o.o for example. > > 6. Work through all nodes. > > 7. Disable dynamic modulepath logic (can be used for other module > > upgrades). > > 8. Delete /opt/puppet/modules/old/mysql > > > > Something like this, would allow us to some how control which nodes start > > consuming the newer version of puppet-mysql. Instead of a massive > > cutover for > > all our nodes. > > > > > Hope this helps, > > > Clark > > > > > > _______________________________________________ > > > OpenStack-Infra mailing list > > > [email protected] > > > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra > > > > _______________________________________________ > > OpenStack-Infra mailing list > > [email protected] > > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra > > _______________________________________________ > OpenStack-Infra mailing list > [email protected] > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra _______________________________________________ OpenStack-Infra mailing list [email protected] http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
