On 07/08/15 00:12, Dan Prince wrote:
On Thu, 2015-07-23 at 07:40 +0100, Derek Higgins wrote:
See below
On 21/07/15 20:29, Derek Higgins wrote:
Hi All,
Something we discussed at the summit was to switch the focus of
tripleo's deployment method to deploy using instack using images
built
with tripleo-puppet-elements. Up to now all the instack work has
been
done downstream of tripleo as part of rdo. Having parts of our
deployment story outside of upstream gives us problems mainly
because it
becomes very difficult to CI what we expect deployers to use while
we're
developing the upstream parts.
Essentially what I'm talking about here is pulling instack
-undercloud
upstream along with a few of its dependency projects (instack,
tripleo-common, tuskar-ui-extras etc..) into tripleo and using them
in
our CI in place of devtest.
Getting our CI working with instack is close to working but has
taken
longer then I expected because of various complications and
distractions
but I hope to have something over the next few days that we can use
to
replace devtest in CI, in a lot of ways this will start out by
taking a
step backwards but we should finish up in a better place where we
will
be developing (and running CI on) what we expect deployers to use.
Once I have something that works I think it makes sense to drop the
jobs
undercloud-precise-nonha and overcloud-precise-nonha, while
switching
overcloud-f21-nonha to use instack, this has a few effects that
need to
be called out
1. We will no longer be running CI on (and as a result not
supporting)
most of the the bash based elements
2. We will no longer be running CI on (and as a result not
supporting)
ubuntu
One more side effect is that I think it also means we no longer have
the capability to test arbitrary Zuul refspecs for projects like Heat,
Neutron, Nova, or Ironic in our undercloud CI jobs. We've relied on the
source-repositories element to do this for us in the undercloud and
since most of the instack stuff uses packages I think we would loose
this capability.
I'm all for testing with packages mind you... would just like to see us
build packages for any projects that have Zuul refspecs inline, create
a per job repo, and then use that to build out the resulting instack
undercloud.
This to me is the biggest loss in our initial switch to instack
undercloud for CI. Perhaps there is a middle ground here where instack
(which used to support tripleo-image-elements itself) could still
support use of the source-repositories element in one CI job until we
get our package building processes up to speed?
/me really wants 'check experimental' to give us TripleO coverage for
select undercloud projects
If Derek is receptive, I would find it useful if Delorean became a
stackforge/openstack hosted project with better support for building
packages from local git trees rather than remote checkouts.
With a bit of hackery I was doing this for a while, developing features
locally on heat and other repos, then deploying an undercloud from a
locally hosted delorean repo.
This would help getting CI working with Zuul refspecs, but it may be
what Dan was meaning anyway when he said "get our package building
processes up to speed"
Should anybody come along in the future interested in either of
these
things (and prepared to put the time in) we can pick them back up
again.
In fact the move to puppet element based images should mean we can
more
easily add in extra distros in the future.
3. While we find our feet we should remove all tripleo-ci jobs from
non
tripleo projects, once we're confident with it we can explore
adding our
jobs back into other projects again
Nothing has changed yet, I order to check we're all on the same
page
this is high level details of how I see things should proceed so
shout
now if I got anything wrong or you disagree.
Ok, I have a POC that has worked end to end in our CI environment[1],
there are a *LOT* of workarounds in there so before we can merge it I
need to clean up and remove some of those workarounds and todo that a
few things need to move around, below is a list of what has to happen
(as best I can tell)
1) Pull in tripleo-heat-template spec changes to master delorean
We had two patches in the tripleo-heat-template midstream packaging
that
havn't made it into the master packaging, these are
https://review.gerrithub.io/241056 Package firstboot and extraconfig
templates
https://review.gerrithub.io/241057 Package environments and newtork
directories
2) Fixes for instack-undercloud (I didn't push these directly incase
it
effected people on old versions of puppet modules)
https://github.com/rdo-management/instack-undercloud/pull/5
3) Add packaging for various repositories into openstack-packaging
I've pulled the packaging for 5 repositories into
https://github.com/openstack-packages
https://github.com/openstack-packages/python-ironic-inspector-client
https://github.com/openstack-packages/python-rdomanager-oscplugin
https://github.com/openstack-packages/tuskar-ui-extras
https://github.com/openstack-packages/ironic-discoverd
https://github.com/openstack-packages/tripleo-common
I havn't imported these into gerrithub (incase following discussion
we
need to delete them again) but assuming we're in agreement we should
pull them into gerrithub)
4) update rdoinfo
https://github.com/redhat-openstack/rdoinfo/pull/69
If everybody is happy with all above we should merge this, all of the
packages needed will now be on the delorean master repository
5) Move DELOREAN_REPO_URL in tripleo-ci to a new delorean repo that
includes all of the new packages
There is a puppet-neutron change I needed to use the latest Neutron
packages BTW:
https://review.openstack.org/#/c/207861/
Just a heads up if we rebase the Delorean URL to something more recent
(last 2 weeks or so) that includes this neutron commit.
6) Take most of the workarounds out of this patch[1] and merge it
7) Reorg the tripleo ci tests (essentially remove all of the bash
element based tests).
8) Pull instack, instack-undercloud, python-rdomanager-oscplugin,
triple-common, tuskar-ui-extras and maybe more into the upstream
gerrit
I would really like to see us rename python-rdomanager-oscplugin. I
think any project having the name "RDO" in it probably doesn't belong
under TripleO proper. Looking at the project there are some distro
specific things... but those are fairly encapsulated (or could be made
so fairly easily).
From here on the way to run the tripleo will be to follow the
documentation in instack-undercloud, we should no longer be using
devtest, this means we've lost the automation devtest gave us, so we
will have to slowly build that up again. The main thing we have
gained
is that we will now be developing upstream all parts of how we expect
deployers to use tripleo.
- we will still have dependencies on tripleo-incubator we need to
remove
these (or move things into other repositories), essentially were
finished with this process once we're no longer installing the
"tripleo"
package.
- The new CI (as is) is running on Fedora jenkins nodes but building
(and deploying) centos images, we also discussed that some people
would
want to develop on Fedora we will have to create a Fedora job as well
(which I'm sure will involve the need for adding Fedora support into
instack)
Yes. I would like to see instack support Fedora as well.
Everything in the first 4 steps is ready to go now, once done we can
investigate moving DELOREAN_REPO_URL and remove the workarounds from
the
CI patch.
Thanks,
Derek
[1] - https://review.openstack.org/#/c/185151/
_____________________________________________________________________
_____
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: [email protected]?subject:unsubs
cribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: [email protected]?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: [email protected]?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
