Re: [openstack-dev] [placement] devstack, grenade, database management

2018-10-16 Thread Matt Riedemann 

On 10/16/2018 5:48 AM, Chris Dent wrote:

* We need to address database creation scripts and database migrations.

   There's a general consensus that we should use alembic, and start
   things from a collapsed state. That is, we don't need to represent
   already existing migrations in the new repo, just the present-day
   structure of the tables.

   Right now the devstack code relies on a stubbed out command line
   tool at https://review.openstack.org/#/c/600161/ to create tables
   with a metadata.create_all(). This is a useful thing to have but
   doesn't follow the "db_sync" pattern set elsewhere, so I haven't
   followed through on making it pretty but can do so if people think
   it is useful. Whether we do that or not, we'll still need some
   kind of "db_sync" command. Do people want me to make a cleaned up
   "create" command?

   Ed has expressed some interest in exploring setting up alembic and
   the associated tools but that can easily be a more than one person
   job. Is anyone else interested?

It would be great to get all this stuff working sooner than later.
Without it we can't do two important tasks:

* Integration tests with the extracted placement [1].
* Hacking on extracted placement in/with devstack.


Another thing that came up today in IRC [1] which is maybe not as 
obvious from this email is what happens with the one online data 
migration we have for placement (create_incomplete_consumers). If we 
drop that online data migration from the placement repo, then ideally 
we'd have something to check it's done before people upgrade to stein 
and the extracted placement repo. There are some options there: 
placement-manage db sync could fail if there are missing consumers or we 
could simply have a placement-status upgrade check for it.




Another issue that needs some attention, but is not quite as urgent
is the desire to support other databases during the upgrade,
captured in this change

https://review.openstack.org/#/c/604028/


I have a grenade patch to test the postgresql-migrate-db.sh script now. [2]

[1] 
http://eavesdrop.openstack.org/irclogs/%23openstack-nova/%23openstack-nova.2018-10-16.log.html#t2018-10-16T19:37:25

[2] https://review.openstack.org/#/c/611020/

--

Thanks,

Matt

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [placement] devstack, grenade, database management

2018-10-16 Thread Chris Dent 


TL;DR: We need reviews on
https://review.openstack.org/#/q/topic:cd/placement-solo+status:open
and work on database management command line tools. More detail
within.

The stack of code, mostly put together by Matt, to get migrating
placement-in-nova to placement-in-placement working is passing its
tests. You can see the remaining pieces of not yet merged code at

https://review.openstack.org/#/q/topic:cd/placement-solo+status:open

Once that is fully merged, the first bullet point on the extraction
plan at


http://lists.openstack.org/pipermail/openstack-dev/2018-September/134541.html

will be complete and we'll have a model for how the next two bullet
points can be done.

At this time, there are two main sticking points to getting things
merged:

* The devstack, grenade, and devstack-gate changes need some review
  to make sure that some of the tricks Matt and I performed are
  acceptable to everyone. They are at:

  https://review.openstack.org/600162
  https://review.openstack.org/604454
  https://review.openstack.org/606853

* We need to address database creation scripts and database migrations.

  There's a general consensus that we should use alembic, and start
  things from a collapsed state. That is, we don't need to represent
  already existing migrations in the new repo, just the present-day
  structure of the tables.

  Right now the devstack code relies on a stubbed out command line
  tool at https://review.openstack.org/#/c/600161/ to create tables
  with a metadata.create_all(). This is a useful thing to have but
  doesn't follow the "db_sync" pattern set elsewhere, so I haven't
  followed through on making it pretty but can do so if people think
  it is useful. Whether we do that or not, we'll still need some
  kind of "db_sync" command. Do people want me to make a cleaned up
  "create" command?

  Ed has expressed some interest in exploring setting up alembic and
  the associated tools but that can easily be a more than one person
  job. Is anyone else interested?

It would be great to get all this stuff working sooner than later.
Without it we can't do two important tasks:

* Integration tests with the extracted placement [1].
* Hacking on extracted placement in/with devstack.

Another issue that needs some attention, but is not quite as urgent
is the desire to support other databases during the upgrade,
captured in this change

https://review.openstack.org/#/c/604028/

[1] There's a stack of code for enabling placement integration tests
starting at https://review.openstack.org/#/c/601614/ . It depends on
the devstack changes.

--
Chris Dent   ٩◔̯◔۶   https://anticdent.org/
freenode: cdent tw: @anticdent__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev