On 07/09/2015 07:06 PM, Andrew Woodward wrote:
> It has come to my attention that we spend an enormous amount of time in
> Fuel when switching between OpenStack releases. A good amount of this
> seems to be due to a chicken and the egg problem. We cant deploy updated
> packages with out updated manifests, and we we cant update (in that CI
> won't pass) the manifests without updated packages. In order to
> accommodate this we either have to set up a new separate set of CI
> pipelines for both or freeze various parts of CI, merging patch sets
> that can't pass until everything is in and working, or some monstrous
> combination of both. This has become quite the time sink for everyone
> involved and there must be a better way to accommodate switching between
> releases.
"We can't (...), and we can't (...)." :
This is a matter of orchestration and I think the module composition has
nothing to do/help with upgrade support.
Since I've been working on OpenStack, I contributed to two Open-Source
installers, and both were able to upgrade OpenStack using *upstream*
Puppet modules.
For example, Spinal Stack is running image-based deployment, that means
we don't upgrade packages on all nodes (it does not scale and it's not
safe, but this is out of topic here). We also use Puppetfile and r10k to
upgrade modules on our puppet client nodes (we run masterless). There is
no magic here, we use Ansible to do that things.
> Thinking about this I wonder how much easier this process could be if
> the manifests supported two releases at once.
Please define "support".
To me, you can deploy current master on Kilo, even though master targets
Liberty right now.
If you missed that, we had a whole dedicated session about that during
the last Summit, and here is the blueprint that our group is writing:
https://review.openstack.org/180141
Feel free to be part of the discussion!
Another thing is, we are currently working on testing upgrade between
OpenStack releases. You probably noticed, but we have a CI job that is
called "gate-puppet-*-puppet-beaker-rspec-dsvm-*".
It basically aims to checkout stable/kilo, run acceptance (beaker will
deploy OpenStack and create OpenStack resources), checkout the patchset
in sent in master, run acceptance again (beaker will upgrade OpenStack
and test if resources survived to the upgrade). This is a `grenade` like.
> I propose that we consider supporting both the dev release, and the
> prior release (as stable and default for option precedence). We would
> need to come up with some option versioning scheme in which we can
> identify options which are only relevant to the specific release (I'm
> not yet sure what sure what this would look line) This seems like it
> would be most of the need to support multiple versions at once.
Is your intention to open stable branches to multiple release support?
It sounds hard and expensive to manage in term of maintenance & release.
> But why should we go through this effort? Yes this would become
> complicated to set up, however I cant imagine that other folks are not
> having a similar issues.
>
> * Moving to a two version system should nearly reduce the impact of
> switching to nearly none (given you have packages)
Does OpenStack work like this? Can you install Nova from the same branch
and run Kilo/Liberty?
> * It should also help accelerate our releases of puppet-openstack, since
> devs can switch between versions more easily
It will be a mess to maintain and manage.
> * It should help our release support story since non-option changes will
> automatically hit dev and (current) stable manifests at the same time
I don't know how it will help, really. Maybe can you give some examples
here at the difficulties we have today and how you can fix it with
having 2 same releases in one branch.
> * This can also help the current stable release process, instead of
> branching, we can simply tag the first stable release and maintain it in
> the master branch, only pushing it out when we drop it to shift the
> dev/stable versions
> * While not a direct benefit to puppet-openstack, this would help
> simplify fuel running CI on master openstack and master puppet-openstack
> which will help with our efforts in puppet-openstack
I have the feeling you're trying to make something that will arrange
Fuel project while it breaks the whole Puppet OpenStack model we're
building, that tries to follow other OpenStack projects to be more
consistent.
I'm very happy if Fuel team is pushing some efforts in Puppet OpenStack
(and even more lately), but one thing is sure, we won't change our model
for Fuel.
> So next steps
> Lets discuss the proposal in general (good idea / bad idea). I'd like to
> hash out what this versioned interface may need to look like, that way I
> can take up attempting to make this work in a single module so we can
> vet/discuss any issues more thoroughly.
We have around 20 Puppet modules with stable branches. Are you sure to
have the bandwidth t
