Hi Pradeep, Firstly, as discussed on IRC, I echo all of bnemec's concerns, this is not well aligned with our stable branch policy[1], or the stable-maint direction towards "critical bugfixes only"[2], so if possible I'd rather we figured out a general way to solve this problem that doesn't involve invasive/risky feature backports.
[1] http://lists.openstack.org/pipermail/openstack-dev/2016-March/090855.html [2] http://lists.openstack.org/pipermail/openstack-dev/2016-May/094440.html On Mon, May 16, 2016 at 03:33:29PM -0400, James Slagle wrote: > On Mon, May 16, 2016 at 10:34 AM, Pradeep Kilambi <[email protected]> wrote: > > Hi Everyone: > > > > I wanted to start a discussion around considering backporting Aodh to > > stable/liberty for upgrades. We have been discussing quite a bit on whats > > the best way for our users to upgrade ceilometer alarms to Aodh when moving > > from liberty to mitaka. A quick refresh on what changed, In Mitaka, > > ceilometer alarms were replaced by Aodh. So only way to get alarms > > functionality is use aodh. Now when the user kicks off upgrades from liberty > > to Mitaka, we want to make sure alarms continue to function as expected > > during the process which could take multiple days. To accomplish this I > > propose the following approach: > > > > * Backport Aodh functionality to stable/liberty. Note, Aodh functionality is > > backwards compatible, so with Aodh running, ceilometer api and client will > > redirect requests to Aodh api. So this should not impact existing users who > > are using ceilometer api or client. > > > > * As part of Aodh deployed via heat stack update, ceilometer alarms services > > will be replaced by openstack-aodh-*. This will be done by the puppet apply > > as part of stack convergence phase. > > > > * Add checks in the Mitaka pre upgrade steps when overcloud install kicks > > off to check and warn the user to update to liberty + aodh to ensure aodh is > > running. This will ensure heat stack update is run and, if alarming is used, > > Aodh is running as expected. > > > > The upgrade scenarios between various releases would work as follows: > > > > Liberty -> Mitaka > > > > * Upgrade starts with ceilometer alarms running > > * A pre-flight check will kick in to make sure Liberty is upgraded to > > liberty + aodh with stack update > > * Run heat stack update to upgrade to aodh > > * Now ceilometer alarms should be removed and Aodh should be running > > * Proceed with mitaka upgrade > > * End result, Aodh continue to run as expected > > > > Liberty + aodh -> Mitaka: > > > > * Upgrade starts with Aodh running > > * A pre-flight check will kick in to make sure Liberty is upgraded to Aodh > > with stack update > > * Confirming Aodh is indeed running, proceed with Mitaka upgrade with Aodh > > running > > * End result, Aodh continue to be run as expected > > > > > > This seems to be a good way to get the upgrades working for aodh. Other less > > effective options I can think of are: > > > > 1. Let the Mitaka upgrade kick off and do "yum update" which replace aodh > > during migration, alarm functionality will be down until puppet converge > > runs and configures Aodh. This means alarms will be down during upgrade > > which is not ideal. > > > > 2. During Mitaka upgrades, replace with Aodh and add a bash script that > > fully configures Aodh and ensures aodh is functioning. This will involve > > significant work and results in duplicating everything puppet does today. > > How much duplication would this really be? Why would it have to be in bash? > > Could it be: > > Liberty -> Mitaka > > * Upgrade starts with ceilometer alarms running > * Add a new hook for the first step of Mitaka upgrade that does: > ** sets up mitaka repos > ** migrates from ceilometer alarms to aodh, can use puppet > ** ensures aodh is running > * Proceed with rest of mitaka upgrade +1, I was thinking the same thing - I also don't get why it has to be bash, surely we could have a script that can apply a puppet manifest that uses our existing puppet profile/module implementation to bring up aodh? If we can figure out a clean way to do that, we could add a pre-upgrade interface to composable services that allows the same thing, e.g implement a supportable upgrade workflow that can be reused vs a one-off workaround. > At most, it seems we'd have to surround the puppet apply with some > pacemaker commands to possibly set maintenance mode and migrate > constraints. > > The puppet manifest itself would just be the includes and classes for aodh. +1 > One complication might be that the aodh packages from Mitaka might > pull in new deps that required updating other OpenStack services, > which we wouldn't yet want to do. That is probably worth confirming > though. It seems like we should at least investigate this approach before going ahead with the backport proposed - I'll -2 the backports pending further discussion and investigation into this alternate approach. Thanks, Steve __________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: [email protected]?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
