Re: [openstack-dev] [nova] [placement] extraction (technical) update

2018-09-06 Thread Mohammed Naser
On Wed, Sep 5, 2018 at 12:41 PM Dan Smith wrote: > > > I think there was a period in time where the nova_api database was created > > where entires would try to get pulled out from the original nova database > > and > > then checking nova_api if it doesn't exist afterwards (or vice versa). One

Re: [openstack-dev] [nova] [placement] extraction (technical) update

2018-09-05 Thread Dan Smith
> I think there was a period in time where the nova_api database was created > where entires would try to get pulled out from the original nova database and > then checking nova_api if it doesn't exist afterwards (or vice versa). One > of the cases that this was done to deal with was for things

Re: [openstack-dev] [nova] [placement] extraction (technical) update

2018-09-05 Thread Matt Riedemann
On 9/5/2018 10:03 AM, Mohammed Naser wrote: On Wed, Sep 5, 2018 at 10:57 AM Matt Riedemann wrote: On 9/5/2018 8:47 AM, Mohammed Naser wrote: Could placement not do what happened for a while when the nova_api database was created? Can you be more specific? I'm having a brain fart here and not

Re: [openstack-dev] [nova] [placement] extraction (technical) update

2018-09-05 Thread Mohammed Naser
On Wed, Sep 5, 2018 at 10:57 AM Matt Riedemann wrote: > > On 9/5/2018 8:47 AM, Mohammed Naser wrote: > > Could placement not do what happened for a while when the nova_api > > database was created? > > Can you be more specific? I'm having a brain fart here and not > remembering what you are

Re: [openstack-dev] [nova] [placement] extraction (technical) update

2018-09-05 Thread Matt Riedemann
On 9/5/2018 8:47 AM, Mohammed Naser wrote: Could placement not do what happened for a while when the nova_api database was created? Can you be more specific? I'm having a brain fart here and not remembering what you are referring to with respect to the nova_api DB. I say this because I

Re: [openstack-dev] [nova] [placement] extraction (technical) update

2018-09-05 Thread Sylvain Bauza
On Wed, Sep 5, 2018 at 3:56 PM, Matt Riedemann wrote: > On 9/5/2018 8:39 AM, Dan Smith wrote: > >> Why not? >> > > Because of the versions table as noted earlier. Up until this point no one > had mentioned that but it would be an issue. > > >> I think the safest/cleanest thing to do here is

Re: [openstack-dev] [nova] [placement] extraction (technical) update

2018-09-05 Thread Matt Riedemann
On 9/5/2018 8:39 AM, Dan Smith wrote: Why not? Because of the versions table as noted earlier. Up until this point no one had mentioned that but it would be an issue. I think the safest/cleanest thing to do here is renumber placement-related migrations from 1, and provide a script or

Re: [openstack-dev] [nova] [placement] extraction (technical) update

2018-09-05 Thread Mohammed Naser
Could placement not do what happened for a while when the nova_api database was created? I say this because I know that moving the database is a huge task for us, considering how big it can be in certain cases for us, and it means control plane outage too On Wed, Sep 5, 2018 at 9:39 AM Dan Smith

Re: [openstack-dev] [nova] [placement] extraction (technical) update

2018-09-05 Thread Dan Smith
>> Yes, we should definitely trim the placement DB migrations to only >> things relevant to placement. And we can use this opportunity to get >> rid of cruft too and squash all of the placement migrations together >> to start at migration 1 for the placement repo. If anyone can think >> of a

Re: [openstack-dev] [nova] [placement] extraction (technical) update

2018-09-04 Thread Matt Riedemann
On 9/4/2018 4:27 PM, melanie witt wrote: Yes, we should definitely trim the placement DB migrations to only things relevant to placement. And we can use this opportunity to get rid of cruft too and squash all of the placement migrations together to start at migration 1 for the placement repo.

Re: [openstack-dev] [nova] [placement] extraction (technical) update

2018-09-04 Thread Chris Dent
On Tue, 4 Sep 2018, Eric Fried wrote: 030 is okay as long as nothing goes wrong. If something does it raises exceptions which would currently fail as the exceptions are not there. See below for more about exceptions. Maybe I'm misunderstanding what these migration thingies are supposed to be

Re: [openstack-dev] [nova] [placement] extraction (technical) update

2018-09-04 Thread melanie witt
On Tue, 4 Sep 2018 16:16:31 -0500, Eric Fried wrote: 030 is okay as long as nothing goes wrong. If something does it raises exceptions which would currently fail as the exceptions are not there. See below for more about exceptions. Maybe I'm misunderstanding what these migration thingies are

Re: [openstack-dev] [nova] [placement] extraction (technical) update

2018-09-04 Thread Eric Fried
> 030 is okay as long as nothing goes wrong. If something does it > raises exceptions which would currently fail as the exceptions are > not there. See below for more about exceptions. Maybe I'm misunderstanding what these migration thingies are supposed to be doing, but 030 [1] seems like it's

Re: [openstack-dev] [nova] [placement] extraction (technical) update

2018-09-03 Thread Chris Dent
There's been some progress on the technical side of extracting placement to it own repo. The summary is: * https://git.openstack.org/cgit/openstack/placement exists * https://review.openstack.org/#/c/599416/ is at the top of a series of patches. That patch is passing and voting on unit and

Re: [openstack-dev] [nova] [placement] extraction (technical) update

2018-08-28 Thread Ed Leafe
On Aug 28, 2018, at 10:47 AM, Matt Riedemann wrote: > >> 1.1 Run the git filter-branch on a copy of nova >> 1.1.1 Add missing files to the file list: >> 1.1.1.1 .gitignore >> 1.1.1.2 # ANYTHING ELSE? > > Unless I were to actually run the git filter-branch tooling to see

Re: [openstack-dev] [nova] [placement] extraction (technical) update

2018-08-28 Thread Dan Smith
> Grenade already has it's own "resources db" right? So we can shove > things in there before we upgrade and then verify they are still there > after the upgrade? Yep, I'm working on something right now. We create an instance that survives the upgrade and validate it on the other side. I'll just

Re: [openstack-dev] [nova] [placement] extraction (technical) update

2018-08-28 Thread Matt Riedemann
On 8/28/2018 10:57 AM, Dan Smith wrote: The from-rocky will also need to extract data from the nova-api database for the placement tables and put it into the new placement database (as real operators will have to do). It'll need to do this after the split code has been installed and the schema

Re: [openstack-dev] [nova] [placement] extraction (technical) update

2018-08-28 Thread Chris Dent
On Tue, 28 Aug 2018, Matt Riedemann wrote: Are people okay with that and willing to commit to being okay with that answer in reviews? To some extent we need to have some faith on the end result: the tests work. If people are not okay with that, we need the people who are not to determine and

Re: [openstack-dev] [nova] [placement] extraction (technical) update

2018-08-28 Thread Dan Smith
> Grenade uses devstack so once we have devstack on master installing > (and configuring) placement from the new repo and disable installing > and configuring it from the nova repo, that's the majority of the > change I'd think. > > Grenade will likely need a from-rocky script to move any config

Re: [openstack-dev] [nova] [placement] extraction (technical) update

2018-08-28 Thread Mohammed Naser
Forgive me for barging into this, but just with my deployer and PTL of a deployment project hat on.. As part of the split, wouldn't we *not* need to make a devstack change if this is done correctly because placement will become a nova dependency, which is pulled out of Git and when using Zuul,

Re: [openstack-dev] [nova] [placement] extraction (technical) update

2018-08-28 Thread Matt Riedemann
On 8/28/2018 6:20 AM, Chris Dent wrote: On Mon, 27 Aug 2018, melanie witt wrote: I think we should use the openstack review system (gerrit) for moving the code. We're moving a critical piece of nova to its own repo and I think it's worth having the review and history contained in the

Re: [openstack-dev] [nova] [placement] extraction (technical) update

2018-08-28 Thread Matt Riedemann
On 8/28/2018 8:11 AM, Balázs Gibizer wrote: I also think that we can add a non-voting tempest full job as well. Making it green depends on how hard to deploy placement from the new repo to tempest. I think as soon as placement repo has passing gabbits (e.g functional job) and we can deploy

Re: [openstack-dev] [nova] [placement] extraction (technical) update

2018-08-28 Thread Dan Smith
>> 2. We have a stack of changes to zuul jobs that show nova working but >> deploying placement in devstack from the new repo instead of nova's >> repo. This includes the grenade job, ensuring that upgrade works. > > I'm guessing there would need to be changes to Devstack itself, outside > of

Re: [openstack-dev] [nova] [placement] extraction (technical) update

2018-08-28 Thread Balázs Gibizer
On Tue, Aug 28, 2018 at 1:20 PM, Chris Dent wrote: On Mon, 27 Aug 2018, melanie witt wrote: I think we should use the openstack review system (gerrit) for moving the code. We're moving a critical piece of nova to its own repo and I think it's worth having the review and history

Re: [openstack-dev] [nova] [placement] extraction (technical) update

2018-08-28 Thread Balázs Gibizer
On Mon, Aug 27, 2018 at 5:31 PM, Matt Riedemann wrote: On 8/24/2018 7:36 AM, Chris Dent wrote: Over the past few days a few of us have been experimenting with extracting placement to its own repo, as has been discussed at length on this list, and in some etherpads:

Re: [openstack-dev] [nova] [placement] extraction (technical) update

2018-08-28 Thread Chris Dent
On Mon, 27 Aug 2018, melanie witt wrote: I think we should use the openstack review system (gerrit) for moving the code. We're moving a critical piece of nova to its own repo and I think it's worth having the review and history contained in the openstack review system. This seems a

Re: [openstack-dev] [nova] [placement] extraction (technical) update

2018-08-28 Thread Alex Xu
2018-08-27 23:31 GMT+08:00 Matt Riedemann : > On 8/24/2018 7:36 AM, Chris Dent wrote: > >> >> Over the past few days a few of us have been experimenting with >> extracting placement to its own repo, as has been discussed at >> length on this list, and in some etherpads: >> >>

Re: [openstack-dev] [nova] [placement] extraction (technical) update

2018-08-28 Thread Stephen Finucane
On Mon, 2018-08-27 at 14:23 -0700, melanie witt wrote: > On Mon, 27 Aug 2018 10:31:50 -0500, Matt Riedemann wrote: > > On 8/24/2018 7:36 AM, Chris Dent wrote: > > > > > > Over the past few days a few of us have been experimenting with > > > extracting placement to its own repo, as has been

Re: [openstack-dev] [nova] [placement] extraction (technical) update

2018-08-27 Thread melanie witt
On Mon, 27 Aug 2018 10:31:50 -0500, Matt Riedemann wrote: On 8/24/2018 7:36 AM, Chris Dent wrote: Over the past few days a few of us have been experimenting with extracting placement to its own repo, as has been discussed at length on this list, and in some etherpads:

Re: [openstack-dev] [nova] [placement] extraction (technical) update

2018-08-27 Thread Jay Pipes
On 08/27/2018 11:31 AM, Matt Riedemann wrote: On 8/24/2018 7:36 AM, Chris Dent wrote: Over the past few days a few of us have been experimenting with extracting placement to its own repo, as has been discussed at length on this list, and in some etherpads:

Re: [openstack-dev] [nova] [placement] extraction (technical) update

2018-08-27 Thread Eric Fried
Thanks Matt, you summed it up nicely. Just one thing to point out... > Option 1 would clearly be a drain on at least 2 nova cores to go through > the changes. I think Eric is on board for reviewing options 1 or 2 in > either case, but he prefers option 2. Since I'm throwing a wrench in the >

Re: [openstack-dev] [nova] [placement] extraction (technical) update

2018-08-27 Thread Matt Riedemann
On 8/24/2018 7:36 AM, Chris Dent wrote: Over the past few days a few of us have been experimenting with extracting placement to its own repo, as has been discussed at length on this list, and in some etherpads: https://etherpad.openstack.org/p/placement-extract-stein

Re: [openstack-dev] [nova] [placement] extraction (technical) update

2018-08-24 Thread Ed Leafe
On Aug 24, 2018, at 7:36 AM, Chris Dent wrote: > Over the past few days a few of us have been experimenting with > extracting placement to its own repo, as has been discussed at > length on this list, and in some etherpads: > >https://etherpad.openstack.org/p/placement-extract-stein >

Re: [openstack-dev] [nova] [placement] extraction (technical) update

2018-08-24 Thread Sean McGinnis
> > After some prompting from gibi, that code has now been adjusted so > that requirements.txt and tox.ini [1] make sure that the extract > placement branch is installed into the test virtualenvs. So in the > gate the unit and functional tests pass. Other jobs do not because > of [1]. > > In the

Re: [openstack-dev] [nova] [placement] extraction (technical) update

2018-08-24 Thread Chris Dent
On Fri, 24 Aug 2018, Chris Dent wrote: That work is in gerrit at https://review.openstack.org/#/c/596291/ with a hopefully clear commit message about what's going on. As with the rest of this work, this is not something to merge, rather an experiment to learn from. The hot spots in the