Looks like this was the regression: https://review.openstack.org/#/c/541435/
Because before that the placement_context_manager was configured in the sqlalchemy DB API code. Now it's only configured in a few select places, one of which is not the online_data_migrations code. This is also noticeable in devstack: http://logs.openstack.org/08/599208/2/check/tempest- full/608d60a/controller/logs/devstacklog.txt.gz#_2018-09-02_15_04_31_949 But because we try/except Exception in online_data_migrations we ignored the failure. ** Also affects: nova/rocky Importance: Undecided Status: New ** Changed in: nova Status: New => Triaged ** Changed in: nova Importance: Undecided => Critical ** Changed in: nova/rocky Status: New => Triaged ** Changed in: nova/rocky Importance: Undecided => Critical -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1790701 Title: online_data_migrations fail in rocky+ Status in OpenStack Compute (nova): Triaged Status in OpenStack Compute (nova) rocky series: Triaged Bug description: # nova-manage --debug db online_data_migrations Running batches of 50 until complete 2018-09-04 13:39:12.628 15100 ERROR oslo_db.sqlalchemy.exc_filters [req-73eac47f-96f5-4393-8104-80d64e4f281f - - - - -] DBAPIError exception wrapped from (psycopg2.ProgrammingError) relation "projects" does not exist LINE 2: FROM projects ^ [SQL: 'SELECT projects.id \nFROM projects \nWHERE projects.external_id = %(external_id_1)s'] [parameters: {'external_id_1': '00000000-0000-0000-0000-000000000000'}] (Background on this error at: http://sqlalche.me/e/f405): psycopg2.ProgrammingError: relation "projects" does not exist LINE 2: FROM projects ^ 2018-09-04 13:39:12.628 15100 ERROR oslo_db.sqlalchemy.exc_filters Traceback (most recent call last): 2018-09-04 13:39:12.628 15100 ERROR oslo_db.sqlalchemy.exc_filters File "/usr/lib64/python3.5/site-packages/sqlalchemy/engine/base.py", line 1193, in _execute_context 2018-09-04 13:39:12.628 15100 ERROR oslo_db.sqlalchemy.exc_filters context) 2018-09-04 13:39:12.628 15100 ERROR oslo_db.sqlalchemy.exc_filters File "/usr/lib64/python3.5/site-packages/sqlalchemy/engine/default.py", line 507, in do_execute 2018-09-04 13:39:12.628 15100 ERROR oslo_db.sqlalchemy.exc_filters cursor.execute(statement, parameters) 2018-09-04 13:39:12.628 15100 ERROR oslo_db.sqlalchemy.exc_filters psycopg2.ProgrammingError: relation "projects" does not exist 2018-09-04 13:39:12.628 15100 ERROR oslo_db.sqlalchemy.exc_filters LINE 2: FROM projects 2018-09-04 13:39:12.628 15100 ERROR oslo_db.sqlalchemy.exc_filters ^ 2018-09-04 13:39:12.628 15100 ERROR oslo_db.sqlalchemy.exc_filters 2018-09-04 13:39:12.628 15100 ERROR oslo_db.sqlalchemy.exc_filters Error attempting to run <function create_incomplete_consumers at 0x7fe62f7d19d8> +---------------------------------------------+--------------+-----------+ | Migration | Total Needed | Completed | +---------------------------------------------+--------------+-----------+ | create_incomplete_consumers | 0 | 0 | | delete_build_requests_with_no_instance_uuid | 0 | 0 | | migrate_instances_add_request_spec | 0 | 0 | | migrate_keypairs_to_api_db | 0 | 0 | | migrate_quota_classes_to_api_db | 0 | 0 | | migrate_quota_limits_to_api_db | 0 | 0 | | migration_migrate_to_uuid | 0 | 0 | | populate_missing_availability_zones | 0 | 0 | | populate_queued_for_delete | 0 | 0 | | populate_uuids | 0 | 0 | | service_uuids_online_data_migration | 0 | 0 | +---------------------------------------------+--------------+-----------+ To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1790701/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : [email protected] Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
