On 8/28/2018 6:20 AM, Chris Dent wrote:
On Mon, 27 Aug 2018, melanie witt wrote:
I think we should use the openstack review system (gerrit) for moving
the code. We're moving a critical piece of nova to its own repo and I
think it's worth having the review and history contained in the
openstack review system.
This seems a reasonable enough strategy, in broad strokes. I want to
be sure that we're all actually in agreement on the details, as
we've had a few false starts and I think some of the details are
getting confused in the shuffle and the general busy-ness in progress.
Is anyone aware of anyone who hasn't commented yet that should? If
you are, please poke them so we don't surprise them.
Using smaller changes that make it easy to see import vs content
changes might make review faster than fewer, larger changes.
I _think_ we ought to be able to use the existing commits from the
runs-throughs-to-passing-tests already done, but if we use the
strategy described below it doesn't really matter: the TDD approach
(after fixing paths and test config) is pretty fast.
The most important bit of all of this is making sure we don't break
anything in the process for operators and users consuming nova and
placement, and ensure the upgrade path from rocky => stein is tested
in grenade.
This is one of the areas where pretty active support from all of
nova will be required: getting zuul, upgrade paths, and the like
clearly defined and executed.
The steps I think we should take are:
1. We copy the placement code into the openstack/placement repo and
have it passing all of its own unit and functional tests.
To break that down to more detail, how does this look?
(note the ALL CAPS where more than acknowledgement is requested)
1.1 Run the git filter-branch on a copy of nova
1.1.1 Add missing files to the file list:
1.1.1.1 .gitignore
1.1.1.2 # ANYTHING ELSE?
Unless I were to actually run the git filter-branch tooling to see what
is excluded from the new repo, I can't really say what is missing at
this time. I assume it would be clear during review - which is why I'm
asking that we do this stuff in gerrit where we do reviews.
1.2 Push -f that thing, acknowledge to be broken, to a seed repo on github
(ed's repo should be fine)
1.3 Do the repo creation bits described in
https://docs.openstack.org/infra/manual/creators.html
to seed openstack/placement
1.3.1 set zuul jobs. Either to noop-jobs, or non voting basic
func and unit # INPUT DESIRED HERE
Agree. As I said to gibi elsewhere in this thread, I would hold off on
adding a tempest-full job to the repo until we're at the end.
1.4 Once the repo exists with some content, incrementally bring it to
working
1.4.1 Update tox.ini to be placement oriented
1.4.2 Update setup.cfg to be placement oriented
1.4.3 Correct .stesr.conf
1.4.4 Move base of placement to "right" place
1.4.5 Move unit and functionals to right place
1.4.6 Do automated path fixings
1.4.7 Set up translation domain and i18n.py corectly
1.4.8 Trim placement/conf to just the conf settings required
(api, base, database, keystone, paths, placement)
1.4.9 Remove database files that are not relevant (the db api is
not used by placement)
1.4.10 Fix the Database Fixture to be just one database
1.4.11 Disable migrations that can't work (because of
dependencies on nova code, 014 and 030 are examples)
# INPUT DESIRED HERE AND ON SCHEMA MIGRATIONS IN GENERAL
1.4.12 Incrementally get tests working
1.4.13 Fix pep8
1.5 Make zuul pep, unit and functional voting
1.6 Create tools for db table sync/create
1.7 Concurrently go to step 2, where the harder magic happens.
1.8 Find and remove dead code (there will be some).
1.9 Tune up and confirm docs
1.10 Grep for remaining "nova" (as string and spirit) and fix
Item 1.4.12 may deserve some discussion. When I've done this the
several times before, the strategy I've used is to be test driven:
run either functional or unit tests, find and fix one of the errors
revealed, commit, move on.
This strategy has worked very well for me because of the "test
driven" part, but I'm hesitant to do it if reviewers are going to
get to a patch and say "why didn't you also change X?" The answer to
that question is "because this is incremental and test driven and
the tests didn't demand that change (yet)". Sometimes that will mean
that things of the same class of change are in different commits.
Are people okay with that and willing to commit to being okay with
that answer in reviews? To some extent we need to have some faith on
the end result: the tests work. If people are not okay with that, we
need the people who are not to determine and prove the alternate
strategy. I've had this one work and work well.
Seems reasonable to me. But to be clear, if there are 70 failed tests,
are you going to have 70 separate patches? Or this is just one of those
things where you start with 70, fix something, get down to 50 failed
tests, and iterate until you're down to all passing. If so, I'm OK with
that. It's hard to say without knowing how many patches get from 70
failures to 0 and what the size/complexity of those changes is, but
without knowing I'd default to the incremental approach for ease of review.
Please help to refine the above, thank you.
2. We have a stack of changes to zuul jobs that show nova working but
deploying placement in devstack from the new repo instead of nova's
repo. This includes the grenade job, ensuring that upgrade works.
If we can make a list for this (and the subsequent) major items that
is as detailed as I've made for step 1 above, I think that will help
us avoid some of the confusion and frustration that comes up. I'm
neither able nor willing to be responsible for creating those lists
for all these points, but very happy to help.
Grenade uses devstack so once we have devstack on master installing (and
configuring) placement from the new repo and disable installing and
configuring it from the nova repo, that's the majority of the change I'd
think.
Grenade will likely need a from-rocky script to move any config that is
necessary, but as you already noted below, if the new repo can live with
an existing nova.conf, then we might not need to do anything in grenade
since placement from the new repo (in stein) could then run with
nova.conf created for placement from the nova repo (in rocky).
3. When those pass, we merge them, effectively orphaning nova's copy
of placement. Switch those jobs to voting.
4. Finally, we delete the orphaned code from nova (without needing to
make any changes to non-placement-only test code -- code is truly
orphaned).
In case you missed it, one of the things I did earlier in the
discussion was make it so that the wsgi script for placement defined
in nova's setup.cfg [1] could:
* continue to exist
* with the same name
* using the nova.conf file
* running the extracted placement code
That was easy to do because of the work over the last year or so
that has been hardening the boundary between placement and nova, in
place. I've been assuming that maintaining the option to use
original conf file is a helpful trick for people. Is that the case?
Yes, as noted above, that might mean we don't need any changes in
grenade, but I can't foretell what will be needed until we start to
actually work on the devstack/grenade changes and see what fails and
then iterate on fixing those issues - exactly what you're doing with
fixing the in-tree unit/functional tests.
Thanks.
[1]
https://review.openstack.org/#/c/596291/3/nova/api/openstack/placement/wsgi.py
__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
--
Thanks,
Matt
__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
