Reviewed: https://review.opendev.org/c/openstack/neutron/+/880867 Committed: https://opendev.org/openstack/neutron/commit/82029c2c514aaca5b9d45af46d2f7d945ff3099b Submitter: "Zuul (22348)" Branch: master
commit 82029c2c514aaca5b9d45af46d2f7d945ff3099b Author: Rodolfo Alonso Hernandez <[email protected]> Date: Thu Apr 20 02:13:05 2023 +0200 Use a writer context for the online alembic migrations ``DBConnection`` class does not retrieve the URL string correctly and cannot connect to the database. This patch replaces it with default writer context that any database operation transaction uses in Neutron. Closes-Bug: #2015847 Change-Id: I60523690bc170ec4ba81312ada0f2d83542cae67 ** Changed in: neutron Status: In Progress => Fix Released -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/2015847 Title: [sqlalchemy-20][FT] Functional job db missconfiguration with SQLAlchemy2.0 Status in neutron: Fix Released Bug description: There is (probably) some issue with configuration of the functional tests job neutron-functional-with-sqlalchemy-master which runs in periodic queue. It is failing every day with same error: 2023-04-11 03:57:17.753314 | controller | neutron.tests.functional.db.test_migrations.TestWalkMigrationsPostgreSQL.test_walk_versions 2023-04-11 03:57:17.753325 | controller | ------------------------------------------------------------------------------------------- 2023-04-11 03:57:17.753336 | controller | 2023-04-11 03:57:17.753346 | controller | Captured traceback: 2023-04-11 03:57:17.753357 | controller | ~~~~~~~~~~~~~~~~~~~ 2023-04-11 03:57:17.753368 | controller | Traceback (most recent call last): 2023-04-11 03:57:17.753379 | controller | 2023-04-11 03:57:17.753390 | controller | File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-functional-gate/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 145, in __init__ 2023-04-11 03:57:17.753401 | controller | self._dbapi_connection = engine.raw_connection() 2023-04-11 03:57:17.753412 | controller | 2023-04-11 03:57:17.753429 | controller | File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-functional-gate/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 3275, in raw_connection 2023-04-11 03:57:17.753441 | controller | return self.pool.connect() 2023-04-11 03:57:17.753452 | controller | 2023-04-11 03:57:17.753463 | controller | File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-functional-gate/lib/python3.10/site-packages/sqlalchemy/pool/base.py", line 455, in connect 2023-04-11 03:57:17.753474 | controller | return _ConnectionFairy._checkout(self) 2023-04-11 03:57:17.753492 | controller | 2023-04-11 03:57:17.753504 | controller | File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-functional-gate/lib/python3.10/site-packages/sqlalchemy/pool/base.py", line 1271, in _checkout 2023-04-11 03:57:17.753514 | controller | fairy = _ConnectionRecord.checkout(pool) 2023-04-11 03:57:17.753525 | controller | 2023-04-11 03:57:17.753536 | controller | File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-functional-gate/lib/python3.10/site-packages/sqlalchemy/pool/base.py", line 719, in checkout 2023-04-11 03:57:17.753548 | controller | rec = pool._do_get() 2023-04-11 03:57:17.753558 | controller | 2023-04-11 03:57:17.753570 | controller | File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-functional-gate/lib/python3.10/site-packages/sqlalchemy/pool/impl.py", line 168, in _do_get 2023-04-11 03:57:17.753581 | controller | with util.safe_reraise(): 2023-04-11 03:57:17.753591 | controller | 2023-04-11 03:57:17.753602 | controller | File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-functional-gate/lib/python3.10/site-packages/sqlalchemy/util/langhelpers.py", line 147, in __exit__ 2023-04-11 03:57:17.753613 | controller | raise exc_value.with_traceback(exc_tb) 2023-04-11 03:57:17.753624 | controller | 2023-04-11 03:57:17.753635 | controller | File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-functional-gate/lib/python3.10/site-packages/sqlalchemy/pool/impl.py", line 166, in _do_get 2023-04-11 03:57:17.753645 | controller | return self._create_connection() 2023-04-11 03:57:17.753656 | controller | 2023-04-11 03:57:17.753667 | controller | File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-functional-gate/lib/python3.10/site-packages/sqlalchemy/pool/base.py", line 396, in _create_connection 2023-04-11 03:57:17.753678 | controller | return _ConnectionRecord(self) 2023-04-11 03:57:17.753689 | controller | 2023-04-11 03:57:17.753700 | controller | File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-functional-gate/lib/python3.10/site-packages/sqlalchemy/pool/base.py", line 681, in __init__ 2023-04-11 03:57:17.753711 | controller | self.__connect() 2023-04-11 03:57:17.753722 | controller | 2023-04-11 03:57:17.753733 | controller | File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-functional-gate/lib/python3.10/site-packages/sqlalchemy/pool/base.py", line 905, in __connect 2023-04-11 03:57:17.753743 | controller | with util.safe_reraise(): 2023-04-11 03:57:17.753754 | controller | 2023-04-11 03:57:17.753769 | controller | File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-functional-gate/lib/python3.10/site-packages/sqlalchemy/util/langhelpers.py", line 147, in __exit__ 2023-04-11 03:57:17.753781 | controller | raise exc_value.with_traceback(exc_tb) 2023-04-11 03:57:17.753792 | controller | 2023-04-11 03:57:17.753803 | controller | File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-functional-gate/lib/python3.10/site-packages/sqlalchemy/pool/base.py", line 901, in __connect 2023-04-11 03:57:17.753814 | controller | self.dbapi_connection = connection = pool._invoke_creator(self) 2023-04-11 03:57:17.753825 | controller | 2023-04-11 03:57:17.753836 | controller | File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-functional-gate/lib/python3.10/site-packages/sqlalchemy/engine/create.py", line 636, in connect 2023-04-11 03:57:17.753847 | controller | return dialect.connect(*cargs, **cparams) 2023-04-11 03:57:17.753858 | controller | 2023-04-11 03:57:17.753868 | controller | File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-functional-gate/lib/python3.10/site-packages/sqlalchemy/engine/default.py", line 579, in connect 2023-04-11 03:57:17.753879 | controller | return self.loaded_dbapi.connect(*cargs, **cparams) 2023-04-11 03:57:17.753890 | controller | 2023-04-11 03:57:17.753901 | controller | File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-functional-gate/lib/python3.10/site-packages/psycopg2/__init__.py", line 122, in connect 2023-04-11 03:57:17.753912 | controller | conn = _connect(dsn, connection_factory=connection_factory, **kwasync) 2023-04-11 03:57:17.753923 | controller | 2023-04-11 03:57:17.753934 | controller | File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-functional-gate/lib/python3.10/site-packages/eventlet/support/psycopg2_patcher.py", line 46, in eventlet_wait_callback 2023-04-11 03:57:17.753945 | controller | state = conn.poll() 2023-04-11 03:57:17.753962 | controller | 2023-04-11 03:57:17.753974 | controller | psycopg2.OperationalError: FATAL: password authentication failed for user "openstack_citest" 2023-04-11 03:57:17.753985 | controller | connection to server at "localhost" (::1), port 5432 failed: FATAL: password authentication failed for user "openstack_citest" 2023-04-11 03:57:17.753997 | controller | 2023-04-11 03:57:17.754007 | controller | 2023-04-11 03:57:17.754018 | controller | 2023-04-11 03:57:17.754029 | controller | The above exception was the direct cause of the following exception: 2023-04-11 03:57:17.754040 | controller | 2023-04-11 03:57:17.754051 | controller | 2023-04-11 03:57:17.754062 | controller | Traceback (most recent call last): 2023-04-11 03:57:17.754073 | controller | 2023-04-11 03:57:17.754084 | controller | File "/home/zuul/src/opendev.org/openstack/neutron/neutron/tests/base.py", line 182, in func 2023-04-11 03:57:17.754095 | controller | return f(self, *args, **kwargs) 2023-04-11 03:57:17.754105 | controller | 2023-04-11 03:57:17.754116 | controller | File "/home/zuul/src/opendev.org/openstack/neutron/neutron/tests/functional/db/test_migrations.py", line 576, in test_walk_versions 2023-04-11 03:57:17.754128 | controller | migration.do_alembic_command(config, 'upgrade', dest) 2023-04-11 03:57:17.754138 | controller | 2023-04-11 03:57:17.754149 | controller | File "/home/zuul/src/opendev.org/openstack/neutron/neutron/db/migration/cli.py", line 90, in do_alembic_command 2023-04-11 03:57:17.754160 | controller | getattr(alembic_command, cmd)(config, *args, **kwargs) 2023-04-11 03:57:17.754171 | controller | 2023-04-11 03:57:17.754188 | controller | File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-functional-gate/lib/python3.10/site-packages/alembic/command.py", line 378, in upgrade 2023-04-11 03:57:17.754200 | controller | script.run_env() 2023-04-11 03:57:17.754211 | controller | 2023-04-11 03:57:17.754222 | controller | File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-functional-gate/lib/python3.10/site-packages/alembic/script/base.py", line 569, in run_env 2023-04-11 03:57:17.754233 | controller | util.load_python_file(self.dir, "env.py") 2023-04-11 03:57:17.754244 | controller | 2023-04-11 03:57:17.754255 | controller | File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-functional-gate/lib/python3.10/site-packages/alembic/util/pyfiles.py", line 94, in load_python_file 2023-04-11 03:57:17.754266 | controller | module = load_module_py(module_id, path) 2023-04-11 03:57:17.754277 | controller | 2023-04-11 03:57:17.754288 | controller | File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-functional-gate/lib/python3.10/site-packages/alembic/util/pyfiles.py", line 110, in load_module_py 2023-04-11 03:57:17.754299 | controller | spec.loader.exec_module(module) # type: ignore 2023-04-11 03:57:17.754310 | controller | 2023-04-11 03:57:17.754321 | controller | File "<frozen importlib._bootstrap_external>", line 883, in exec_module 2023-04-11 03:57:17.754332 | controller | 2023-04-11 03:57:17.754343 | controller | File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed 2023-04-11 03:57:17.754354 | controller | 2023-04-11 03:57:17.754365 | controller | File "/home/zuul/src/opendev.org/openstack/neutron/neutron/db/migration/alembic_migrations/env.py", line 120, in <module> 2023-04-11 03:57:17.754376 | controller | run_migrations_online() 2023-04-11 03:57:17.754387 | controller | 2023-04-11 03:57:17.754398 | controller | File "/home/zuul/src/opendev.org/openstack/neutron/neutron/db/migration/alembic_migrations/env.py", line 106, in run_migrations_online 2023-04-11 03:57:17.754409 | controller | with DBConnection(neutron_config.database.connection, connection) as conn: 2023-04-11 03:57:17.754420 | controller | 2023-04-11 03:57:17.754431 | controller | File "/home/zuul/src/opendev.org/openstack/neutron/neutron/db/migration/connection.py", line 32, in __enter__ 2023-04-11 03:57:17.754442 | controller | self.engine = session.create_engine(self.connection_url) 2023-04-11 03:57:17.754453 | controller | 2023-04-11 03:57:17.754464 | controller | File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-functional-gate/lib/python3.10/site-packages/debtcollector/removals.py", line 260, in wrapper 2023-04-11 03:57:17.754475 | controller | return f(*args, **kwargs) 2023-04-11 03:57:17.754485 | controller | 2023-04-11 03:57:17.754496 | controller | File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-functional-gate/lib/python3.10/site-packages/debtcollector/renames.py", line 41, in decorator 2023-04-11 03:57:17.754517 | controller | return wrapped(*args, **kwargs) 2023-04-11 03:57:17.754529 | controller | 2023-04-11 03:57:17.754540 | controller | File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-functional-gate/lib/python3.10/site-packages/oslo_db/sqlalchemy/engines.py", line 257, in create_engine 2023-04-11 03:57:17.754551 | controller | test_conn = _test_connection(engine, max_retries, retry_interval) 2023-04-11 03:57:17.754562 | controller | 2023-04-11 03:57:17.754573 | controller | File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-functional-gate/lib/python3.10/site-packages/oslo_db/sqlalchemy/engines.py", line 453, in _test_connection 2023-04-11 03:57:17.754584 | controller | return engine.connect() 2023-04-11 03:57:17.754595 | controller | 2023-04-11 03:57:17.754606 | controller | File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-functional-gate/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 3251, in connect 2023-04-11 03:57:17.754617 | controller | return self._connection_cls(self) 2023-04-11 03:57:17.754628 | controller | 2023-04-11 03:57:17.754639 | controller | File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-functional-gate/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 147, in __init__ 2023-04-11 03:57:17.754650 | controller | Connection._handle_dbapi_exception_noconnection( 2023-04-11 03:57:17.754660 | controller | 2023-04-11 03:57:17.754671 | controller | File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-functional-gate/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 2410, in _handle_dbapi_exception_noconnection 2023-04-11 03:57:17.754682 | controller | raise newraise.with_traceback(exc_info[2]) from e 2023-04-11 03:57:17.754693 | controller | 2023-04-11 03:57:17.754704 | controller | File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-functional-gate/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 145, in __init__ 2023-04-11 03:57:17.754715 | controller | self._dbapi_connection = engine.raw_connection() 2023-04-11 03:57:17.754726 | controller | 2023-04-11 03:57:17.754737 | controller | File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-functional-gate/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 3275, in raw_connection 2023-04-11 03:57:17.754748 | controller | return self.pool.connect() 2023-04-11 03:57:17.754764 | controller | 2023-04-11 03:57:17.754776 | controller | File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-functional-gate/lib/python3.10/site-packages/sqlalchemy/pool/base.py", line 455, in connect 2023-04-11 03:57:17.754787 | controller | return _ConnectionFairy._checkout(self) 2023-04-11 03:57:17.754798 | controller | 2023-04-11 03:57:17.754809 | controller | File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-functional-gate/lib/python3.10/site-packages/sqlalchemy/pool/base.py", line 1271, in _checkout 2023-04-11 03:57:17.754820 | controller | fairy = _ConnectionRecord.checkout(pool) 2023-04-11 03:57:17.754831 | controller | 2023-04-11 03:57:17.754842 | controller | File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-functional-gate/lib/python3.10/site-packages/sqlalchemy/pool/base.py", line 719, in checkout 2023-04-11 03:57:17.754853 | controller | rec = pool._do_get() 2023-04-11 03:57:17.754864 | controller | 2023-04-11 03:57:17.754881 | controller | File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-functional-gate/lib/python3.10/site-packages/sqlalchemy/pool/impl.py", line 168, in _do_get 2023-04-11 03:57:17.754893 | controller | with util.safe_reraise(): 2023-04-11 03:57:17.754905 | controller | 2023-04-11 03:57:17.754916 | controller | File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-functional-gate/lib/python3.10/site-packages/sqlalchemy/util/langhelpers.py", line 147, in __exit__ 2023-04-11 03:57:17.754926 | controller | raise exc_value.with_traceback(exc_tb) 2023-04-11 03:57:17.754938 | controller | 2023-04-11 03:57:17.754949 | controller | File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-functional-gate/lib/python3.10/site-packages/sqlalchemy/pool/impl.py", line 166, in _do_get 2023-04-11 03:57:17.754960 | controller | return self._create_connection() 2023-04-11 03:57:17.754971 | controller | 2023-04-11 03:57:17.754982 | controller | File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-functional-gate/lib/python3.10/site-packages/sqlalchemy/pool/base.py", line 396, in _create_connection 2023-04-11 03:57:17.754999 | controller | return _ConnectionRecord(self) 2023-04-11 03:57:17.755011 | controller | 2023-04-11 03:57:17.755022 | controller | File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-functional-gate/lib/python3.10/site-packages/sqlalchemy/pool/base.py", line 681, in __init__ 2023-04-11 03:57:17.755033 | controller | self.__connect() 2023-04-11 03:57:17.755044 | controller | 2023-04-11 03:57:17.755055 | controller | File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-functional-gate/lib/python3.10/site-packages/sqlalchemy/pool/base.py", line 905, in __connect 2023-04-11 03:57:17.755066 | controller | with util.safe_reraise(): 2023-04-11 03:57:17.755076 | controller | 2023-04-11 03:57:17.755091 | controller | File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-functional-gate/lib/python3.10/site-packages/sqlalchemy/util/langhelpers.py", line 147, in __exit__ 2023-04-11 03:57:17.755103 | controller | raise exc_value.with_traceback(exc_tb) 2023-04-11 03:57:17.755114 | controller | 2023-04-11 03:57:17.755125 | controller | File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-functional-gate/lib/python3.10/site-packages/sqlalchemy/pool/base.py", line 901, in __connect 2023-04-11 03:57:17.755136 | controller | self.dbapi_connection = connection = pool._invoke_creator(self) 2023-04-11 03:57:17.755147 | controller | 2023-04-11 03:57:17.755158 | controller | File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-functional-gate/lib/python3.10/site-packages/sqlalchemy/engine/create.py", line 636, in connect 2023-04-11 03:57:17.755169 | controller | return dialect.connect(*cargs, **cparams) 2023-04-11 03:57:17.755180 | controller | 2023-04-11 03:57:17.755191 | controller | File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-functional-gate/lib/python3.10/site-packages/sqlalchemy/engine/default.py", line 579, in connect 2023-04-11 03:57:17.755202 | controller | return self.loaded_dbapi.connect(*cargs, **cparams) 2023-04-11 03:57:17.755213 | controller | 2023-04-11 03:57:17.755224 | controller | File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-functional-gate/lib/python3.10/site-packages/psycopg2/__init__.py", line 122, in connect 2023-04-11 03:57:17.755235 | controller | conn = _connect(dsn, connection_factory=connection_factory, **kwasync) 2023-04-11 03:57:17.755246 | controller | 2023-04-11 03:57:17.755257 | controller | File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-functional-gate/lib/python3.10/site-packages/eventlet/support/psycopg2_patcher.py", line 46, in eventlet_wait_callback 2023-04-11 03:57:17.755268 | controller | state = conn.poll() 2023-04-11 03:57:17.755279 | controller | 2023-04-11 03:57:17.755290 | controller | sqlalchemy.exc.OperationalError: (psycopg2.OperationalError) FATAL: password authentication failed for user "openstack_citest" 2023-04-11 03:57:17.755301 | controller | connection to server at "localhost" (::1), port 5432 failed: FATAL: password authentication failed for user "openstack_citest" 2023-04-11 03:57:17.755311 | controller | 2023-04-11 03:57:17.755322 | controller | (Background on this error at: https://sqlalche.me/e/20/e3q8) To manage notifications about this bug go to: https://bugs.launchpad.net/neutron/+bug/2015847/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : [email protected] Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
