** Also affects: nova/rocky Importance: Undecided Status: New ** Changed in: nova/rocky Status: New => In Progress
** Changed in: nova/rocky Importance: Undecided => Medium ** Changed in: nova/rocky Assignee: (unassigned) => Lee Yarwood (lyarwood) -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1799246 Title: module level init of db transation contexts cause failure under mod_wsgi on module reload Status in OpenStack Compute (nova): Fix Released Status in OpenStack Compute (nova) rocky series: In Progress Bug description: Description =========== This is related to a downstream bug first reported here:https://bugzilla.redhat.com/show_bug.cgi?id=1630069 where in it was discovers that due to how triplo currently deploys the placemetn api under mod_wsgi when deployed under mod_wsgi if the wsgi application exits with an error it is reloaded back into the same python interperter instance. As a result of this behavior module level variable have a longer lifetime then the lifetime of the application. when run under uwsgi when the application is reloaded it is reloaded in a new python interperter meaning the lifetime of the module level variables is scoped to the life time of the application. As a result of the life time semantics of mod_wsgi the placment api and nova-api must assume that the wsgi applications init can be invoked multiple times on failure. The current use of the sqlalcamey enginefacade transaction_context is not reentrant resulting in a type error being raised on subsequet calls to configure when the wsgi application is reloaded. Expected result =============== it should be possible to reload the nova and placement api wsgi application under mod_wsgi on failure Actual result ============= 46087 [Wed Oct 10 15:10:49.433284 2018] [:error] [pid 14] [remote 172.25.0.10:208] mod_wsgi (pid=14): Target WSGI script '/var/www/cgi-bin/nova/nova-placement-api' cannot be loaded as Python module. 46088 [Wed Oct 10 15:10:49.433305 2018] [:error] [pid 14] [remote 172.25.0.10:208] mod_wsgi (pid=14): Exception occurred processing WSGI script '/var/www/cgi-bin/nova/nova-placement-api'. 46089 [Wed Oct 10 15:10:49.433320 2018] [:error] [pid 14] [remote 172.25.0.10:208] Traceback (most recent call last): 46090 [Wed Oct 10 15:10:49.433333 2018] [:error] [pid 14] [remote 172.25.0.10:208] File "/var/www/cgi-bin/nova/nova-placement-api", line 54, in <module> 46091 [Wed Oct 10 15:10:49.433354 2018] [:error] [pid 14] [remote 172.25.0.10:208] application = init_application() 46092 [Wed Oct 10 15:10:49.433361 2018] [:error] [pid 14] [remote 172.25.0.10:208] File "/usr/lib/python2.7/site-packages/nova/api/openstack/placement/wsgi.py", line 108, in init_application 46093 [Wed Oct 10 15:10:49.433386 2018] [:error] [pid 14] [remote 172.25.0.10:208] db_api.configure(conf.CONF) 46094 [Wed Oct 10 15:10:49.433392 2018] [:error] [pid 14] [remote 172.25.0.10:208] File "/usr/lib/python2.7/site-packages/nova/api/openstack/placement/db_api.py", line 35, in configure 46095 [Wed Oct 10 15:10:49.433403 2018] [:error] [pid 14] [remote 172.25.0.10:208] **_get_db_conf(conf.placement_database)) 46096 [Wed Oct 10 15:10:49.433408 2018] [:error] [pid 14] [remote 172.25.0.10:208] File "/usr/lib/python2.7/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 788, in configure 46097 [Wed Oct 10 15:10:49.433420 2018] [:error] [pid 14] [remote 172.25.0.10:208] self._factory.configure(**kw) 46098 [Wed Oct 10 15:10:49.433425 2018] [:error] [pid 14] [remote 172.25.0.10:208] File "/usr/lib/python2.7/site-packages/debtcollector/renames.py", line 43, in decorator 46099 [Wed Oct 10 15:10:49.433435 2018] [:error] [pid 14] [remote 172.25.0.10:208] return wrapped(*args, **kwargs) 46100 [Wed Oct 10 15:10:49.433440 2018] [:error] [pid 14] [remote 172.25.0.10:208] File "/usr/lib/python2.7/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 312, in configure 46101 [Wed Oct 10 15:10:49.433449 2018] [:error] [pid 14] [remote 172.25.0.10:208] self._configure(False, kw) 46102 [Wed Oct 10 15:10:49.433453 2018] [:error] [pid 14] [remote 172.25.0.10:208] File "/usr/lib/python2.7/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 317, in _configure 46103 [Wed Oct 10 15:10:49.433462 2018] [:error] [pid 14] [remote 172.25.0.10:208] raise TypeError("this TransactionFactory is already started") 46104 [Wed Oct 10 15:10:49.433473 2018] [:error] [pid 14] [remote 172.25.0.10:208] TypeError: this TransactionFactory is already started To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1799246/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp