Public bug reported: Running Rocky, in a DVR environment
We are seeing repeated errors like this in neutron-server logs. Not sure at this point what the effect is, or harmless. This is in a build test environment where a lot of automated VMs get spawned and deleted then associated/disassociated with existing floating IPs, as well as a lot of VMs and floating IPs that are created and deleted manually by user. The logs aren't indicating what the floating IP is, what host, or what port/VM they were bound to: http://paste.openstack.org/show/789444/ 2020-02-11 19:38:52,839.839 30190 ERROR oslo_db.api [req-3a00ad99-d6ed-46cc-9f18-6835854799c8 - - - - -] DB exceeded retry limit.: StaleDataError: UPDATE statement on table 'standardattributes' expected to update 1 row(s); 0 were matched. 2020-02-11 19:38:52,839.839 30190 ERROR oslo_db.api Traceback (most recent call last): 2020-02-11 19:38:52,839.839 30190 ERROR oslo_db.api File "/opt/pf9/neutron/lib/python2.7/site-packages/oslo_db/api.py", line 142, in wrapper 2020-02-11 19:38:52,839.839 30190 ERROR oslo_db.api return f(*args, **kwargs) 2020-02-11 19:38:52,839.839 30190 ERROR oslo_db.api File "/opt/pf9/neutron/lib/python2.7/site-packages/neutron_lib/db/api.py", line 183, in wrapped 2020-02-11 19:38:52,839.839 30190 ERROR oslo_db.api LOG.debug("Retry wrapper got retriable exception: %s", e) 2020-02-11 19:38:52,839.839 30190 ERROR oslo_db.api File "/opt/pf9/neutron/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ 2020-02-11 19:38:52,839.839 30190 ERROR oslo_db.api self.force_reraise() 2020-02-11 19:38:52,839.839 30190 ERROR oslo_db.api File "/opt/pf9/neutron/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise 2020-02-11 19:38:52,839.839 30190 ERROR oslo_db.api six.reraise(self.type_, self.value, self.tb) 2020-02-11 19:38:52,839.839 30190 ERROR oslo_db.api File "/opt/pf9/neutron/lib/python2.7/site-packages/neutron_lib/db/api.py", line 179, in wrapped 2020-02-11 19:38:52,839.839 30190 ERROR oslo_db.api return f(*dup_args, **dup_kwargs) 2020-02-11 19:38:52,839.839 30190 ERROR oslo_db.api File "/opt/pf9/neutron/lib/python2.7/site-packages/neutron/api/rpc/handlers/l3_rpc.py", line 273, in update_floatingip_statuses 2020-02-11 19:38:52,839.839 30190 ERROR oslo_db.api status) 2020-02-11 19:38:52,839.839 30190 ERROR oslo_db.api File "/opt/pf9/neutron/lib/python2.7/site-packages/neutron/db/api.py", line 123, in wrapped 2020-02-11 19:38:52,839.839 30190 ERROR oslo_db.api return method(*args, **kwargs) 2020-02-11 19:38:52,839.839 30190 ERROR oslo_db.api File "/opt/pf9/neutron/lib/python2.7/site-packages/neutron/db/l3_db.py", line 1524, in update_floatingip_status 2020-02-11 19:38:52,839.839 30190 ERROR oslo_db.api context, {'status': status}, id=floatingip_id) 2020-02-11 19:38:52,839.839 30190 ERROR oslo_db.api File "/opt/pf9/neutron/lib/python2.7/site-packages/neutron/objects/base.py", line 669, in update_object 2020-02-11 19:38:52,839.839 30190 ERROR oslo_db.api context, values, validate_filters=False, **kwargs) 2020-02-11 19:38:52,839.839 30190 ERROR oslo_db.api File "/opt/pf9/neutron/lib/python2.7/site-packages/neutron/objects/base.py", line 304, in update_object 2020-02-11 19:38:52,839.839 30190 ERROR oslo_db.api cls._update_objects(obj, values) 2020-02-11 19:38:52,839.839 30190 ERROR oslo_db.api File "/opt/pf9/neutron/lib/python2.7/site-packages/neutron/objects/base.py", line 296, in _update_objects 2020-02-11 19:38:52,839.839 30190 ERROR oslo_db.api obj.update() 2020-02-11 19:38:52,839.839 30190 ERROR oslo_db.api File "/opt/pf9/neutron/lib/python2.7/site-packages/neutron/objects/base.py", line 342, in decorator 2020-02-11 19:38:52,839.839 30190 ERROR oslo_db.api return func(self, *args, **kwargs) 2020-02-11 19:38:52,839.839 30190 ERROR oslo_db.api File "/opt/pf9/neutron/lib/python2.7/site-packages/neutron/objects/base.py", line 876, in update 2020-02-11 19:38:52,839.839 30190 ERROR oslo_db.api self._get_composite_keys())) 2020-02-11 19:38:52,839.839 30190 ERROR oslo_db.api File "/opt/pf9/neutron/lib/python2.7/site-packages/neutron/objects/db/api.py", line 82, in update_object 2020-02-11 19:38:52,839.839 30190 ERROR oslo_db.api db_obj.save(session=context.session) 2020-02-11 19:38:52,839.839 30190 ERROR oslo_db.api File "/opt/pf9/neutron/lib/python2.7/site-packages/oslo_db/sqlalchemy/models.py", line 50, in save 2020-02-11 19:38:52,839.839 30190 ERROR oslo_db.api session.flush() 2020-02-11 19:38:52,839.839 30190 ERROR oslo_db.api File "/opt/pf9/neutron/lib/python2.7/site-packages/sqlalchemy/orm/session.py", line 2254, in flush 2020-02-11 19:38:52,839.839 30190 ERROR oslo_db.api self._flush(objects) 2020-02-11 19:38:52,839.839 30190 ERROR oslo_db.api File "/opt/pf9/neutron/lib/python2.7/site-packages/sqlalchemy/orm/session.py", line 2380, in _flush 2020-02-11 19:38:52,839.839 30190 ERROR oslo_db.api transaction.rollback(_capture_exception=True) 2020-02-11 19:38:52,839.839 30190 ERROR oslo_db.api File "/opt/pf9/neutron/lib/python2.7/site-packages/sqlalchemy/util/langhelpers.py", line 66, in __exit__ 2020-02-11 19:38:52,839.839 30190 ERROR oslo_db.api compat.reraise(exc_type, exc_value, exc_tb) 2020-02-11 19:38:52,839.839 30190 ERROR oslo_db.api File "/opt/pf9/neutron/lib/python2.7/site-packages/sqlalchemy/orm/session.py", line 2344, in _flush 2020-02-11 19:38:52,839.839 30190 ERROR oslo_db.api flush_context.execute() 2020-02-11 19:38:52,839.839 30190 ERROR oslo_db.api File "/opt/pf9/neutron/lib/python2.7/site-packages/sqlalchemy/orm/unitofwork.py", line 391, in execute 2020-02-11 19:38:52,839.839 30190 ERROR oslo_db.api rec.execute(self) 2020-02-11 19:38:52,839.839 30190 ERROR oslo_db.api File "/opt/pf9/neutron/lib/python2.7/site-packages/sqlalchemy/orm/unitofwork.py", line 556, in execute 2020-02-11 19:38:52,839.839 30190 ERROR oslo_db.api uow 2020-02-11 19:38:52,839.839 30190 ERROR oslo_db.api File "/opt/pf9/neutron/lib/python2.7/site-packages/sqlalchemy/orm/persistence.py", line 177, in save_obj 2020-02-11 19:38:52,839.839 30190 ERROR oslo_db.api mapper, table, update) 2020-02-11 19:38:52,839.839 30190 ERROR oslo_db.api File "/opt/pf9/neutron/lib/python2.7/site-packages/sqlalchemy/orm/persistence.py", line 791, in _emit_update_statements 2020-02-11 19:38:52,839.839 30190 ERROR oslo_db.api (table.description, len(records), rows)) 2020-02-11 19:38:52,839.839 30190 ERROR oslo_db.api StaleDataError: UPDATE statement on table 'standardattributes' expected to update 1 row(s); 0 were matched. 2020-02-11 19:38:52,839.839 30190 ERROR oslo_db.api 2020-02-11 19:38:52,841.841 30190 ERROR oslo_messaging.rpc.server [req-3a00ad99-d6ed-46cc-9f18-6835854799c8 - - - - -] Exception during message handling: StaleDataError: UPDATE statement on table 'standardattributes' expected to update 1 row(s); 0 were matched. 2020-02-11 19:38:52,841.841 30190 ERROR oslo_messaging.rpc.server Traceback (most recent call last): 2020-02-11 19:38:52,841.841 30190 ERROR oslo_messaging.rpc.server File "/opt/pf9/neutron/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 166, in _process_incoming 2020-02-11 19:38:52,841.841 30190 ERROR oslo_messaging.rpc.server res = self.dispatcher.dispatch(message) 2020-02-11 19:38:52,841.841 30190 ERROR oslo_messaging.rpc.server File "/opt/pf9/neutron/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 265, in dispatch 2020-02-11 19:38:52,841.841 30190 ERROR oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args) 2020-02-11 19:38:52,841.841 30190 ERROR oslo_messaging.rpc.server File "/opt/pf9/neutron/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 194, in _do_dispatch 2020-02-11 19:38:52,841.841 30190 ERROR oslo_messaging.rpc.server result = func(ctxt, **new_args) 2020-02-11 19:38:52,841.841 30190 ERROR oslo_messaging.rpc.server File "/opt/pf9/neutron/lib/python2.7/site-packages/neutron_lib/db/api.py", line 140, in wrapped 2020-02-11 19:38:52,841.841 30190 ERROR oslo_messaging.rpc.server setattr(e, '_RETRY_EXCEEDED', True) 2020-02-11 19:38:52,841.841 30190 ERROR oslo_messaging.rpc.server File "/opt/pf9/neutron/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ 2020-02-11 19:38:52,841.841 30190 ERROR oslo_messaging.rpc.server self.force_reraise() 2020-02-11 19:38:52,841.841 30190 ERROR oslo_messaging.rpc.server File "/opt/pf9/neutron/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise 2020-02-11 19:38:52,841.841 30190 ERROR oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb) 2020-02-11 19:38:52,841.841 30190 ERROR oslo_messaging.rpc.server File "/opt/pf9/neutron/lib/python2.7/site-packages/neutron_lib/db/api.py", line 136, in wrapped 2020-02-11 19:38:52,841.841 30190 ERROR oslo_messaging.rpc.server return f(*args, **kwargs) 2020-02-11 19:38:52,841.841 30190 ERROR oslo_messaging.rpc.server File "/opt/pf9/neutron/lib/python2.7/site-packages/oslo_db/api.py", line 154, in wrapper 2020-02-11 19:38:52,841.841 30190 ERROR oslo_messaging.rpc.server ectxt.value = e.inner_exc 2020-02-11 19:38:52,841.841 30190 ERROR oslo_messaging.rpc.server File "/opt/pf9/neutron/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ 2020-02-11 19:38:52,841.841 30190 ERROR oslo_messaging.rpc.server self.force_reraise() 2020-02-11 19:38:52,841.841 30190 ERROR oslo_messaging.rpc.server File "/opt/pf9/neutron/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise 2020-02-11 19:38:52,841.841 30190 ERROR oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb) 2020-02-11 19:38:52,841.841 30190 ERROR oslo_messaging.rpc.server File "/opt/pf9/neutron/lib/python2.7/site-packages/oslo_db/api.py", line 142, in wrapper 2020-02-11 19:38:52,841.841 30190 ERROR oslo_messaging.rpc.server return f(*args, **kwargs) 2020-02-11 19:38:52,841.841 30190 ERROR oslo_messaging.rpc.server File "/opt/pf9/neutron/lib/python2.7/site-packages/neutron_lib/db/api.py", line 183, in wrapped 2020-02-11 19:38:52,841.841 30190 ERROR oslo_messaging.rpc.server LOG.debug("Retry wrapper got retriable exception: %s", e) 2020-02-11 19:38:52,841.841 30190 ERROR oslo_messaging.rpc.server File "/opt/pf9/neutron/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ 2020-02-11 19:38:52,841.841 30190 ERROR oslo_messaging.rpc.server self.force_reraise() 2020-02-11 19:38:52,841.841 30190 ERROR oslo_messaging.rpc.server File "/opt/pf9/neutron/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise 2020-02-11 19:38:52,841.841 30190 ERROR oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb) 2020-02-11 19:38:52,841.841 30190 ERROR oslo_messaging.rpc.server File "/opt/pf9/neutron/lib/python2.7/site-packages/neutron_lib/db/api.py", line 179, in wrapped 2020-02-11 19:38:52,841.841 30190 ERROR oslo_messaging.rpc.server return f(*dup_args, **dup_kwargs) 2020-02-11 19:38:52,841.841 30190 ERROR oslo_messaging.rpc.server File "/opt/pf9/neutron/lib/python2.7/site-packages/neutron/api/rpc/handlers/l3_rpc.py", line 273, in update_floatingip_statuses 2020-02-11 19:38:52,841.841 30190 ERROR oslo_messaging.rpc.server status) 2020-02-11 19:38:52,841.841 30190 ERROR oslo_messaging.rpc.server File "/opt/pf9/neutron/lib/python2.7/site-packages/neutron/db/api.py", line 123, in wrapped 2020-02-11 19:38:52,841.841 30190 ERROR oslo_messaging.rpc.server return method(*args, **kwargs) 2020-02-11 19:38:52,841.841 30190 ERROR oslo_messaging.rpc.server File "/opt/pf9/neutron/lib/python2.7/site-packages/neutron/db/l3_db.py", line 1524, in update_floatingip_status 2020-02-11 19:38:52,841.841 30190 ERROR oslo_messaging.rpc.server context, {'status': status}, id=floatingip_id) 2020-02-11 19:38:52,841.841 30190 ERROR oslo_messaging.rpc.server File "/opt/pf9/neutron/lib/python2.7/site-packages/neutron/objects/base.py", line 669, in update_object 2020-02-11 19:38:52,841.841 30190 ERROR oslo_messaging.rpc.server context, values, validate_filters=False, **kwargs) 2020-02-11 19:38:52,841.841 30190 ERROR oslo_messaging.rpc.server File "/opt/pf9/neutron/lib/python2.7/site-packages/neutron/objects/base.py", line 304, in update_object 2020-02-11 19:38:52,841.841 30190 ERROR oslo_messaging.rpc.server cls._update_objects(obj, values) 2020-02-11 19:38:52,841.841 30190 ERROR oslo_messaging.rpc.server File "/opt/pf9/neutron/lib/python2.7/site-packages/neutron/objects/base.py", line 296, in _update_objects 2020-02-11 19:38:52,841.841 30190 ERROR oslo_messaging.rpc.server obj.update() 2020-02-11 19:38:52,841.841 30190 ERROR oslo_messaging.rpc.server File "/opt/pf9/neutron/lib/python2.7/site-packages/neutron/objects/base.py", line 342, in decorator 2020-02-11 19:38:52,841.841 30190 ERROR oslo_messaging.rpc.server return func(self, *args, **kwargs) 2020-02-11 19:38:52,841.841 30190 ERROR oslo_messaging.rpc.server File "/opt/pf9/neutron/lib/python2.7/site-packages/neutron/objects/base.py", line 876, in update 2020-02-11 19:38:52,841.841 30190 ERROR oslo_messaging.rpc.server self._get_composite_keys())) 2020-02-11 19:38:52,841.841 30190 ERROR oslo_messaging.rpc.server File "/opt/pf9/neutron/lib/python2.7/site-packages/neutron/objects/db/api.py", line 82, in update_object 2020-02-11 19:38:52,841.841 30190 ERROR oslo_messaging.rpc.server db_obj.save(session=context.session) 2020-02-11 19:38:52,841.841 30190 ERROR oslo_messaging.rpc.server File "/opt/pf9/neutron/lib/python2.7/site-packages/oslo_db/sqlalchemy/models.py", line 50, in save 2020-02-11 19:38:52,841.841 30190 ERROR oslo_messaging.rpc.server session.flush() 2020-02-11 19:38:52,841.841 30190 ERROR oslo_messaging.rpc.server File "/opt/pf9/neutron/lib/python2.7/site-packages/sqlalchemy/orm/session.py", line 2254, in flush 2020-02-11 19:38:52,841.841 30190 ERROR oslo_messaging.rpc.server self._flush(objects) 2020-02-11 19:38:52,841.841 30190 ERROR oslo_messaging.rpc.server File "/opt/pf9/neutron/lib/python2.7/site-packages/sqlalchemy/orm/session.py", line 2380, in _flush 2020-02-11 19:38:52,841.841 30190 ERROR oslo_messaging.rpc.server transaction.rollback(_capture_exception=True) 2020-02-11 19:38:52,841.841 30190 ERROR oslo_messaging.rpc.server File "/opt/pf9/neutron/lib/python2.7/site-packages/sqlalchemy/util/langhelpers.py", line 66, in __exit__ 2020-02-11 19:38:52,841.841 30190 ERROR oslo_messaging.rpc.server compat.reraise(exc_type, exc_value, exc_tb) 2020-02-11 19:38:52,841.841 30190 ERROR oslo_messaging.rpc.server File "/opt/pf9/neutron/lib/python2.7/site-packages/sqlalchemy/orm/session.py", line 2344, in _flush 2020-02-11 19:38:52,841.841 30190 ERROR oslo_messaging.rpc.server flush_context.execute() 2020-02-11 19:38:52,841.841 30190 ERROR oslo_messaging.rpc.server File "/opt/pf9/neutron/lib/python2.7/site-packages/sqlalchemy/orm/unitofwork.py", line 391, in execute 2020-02-11 19:38:52,841.841 30190 ERROR oslo_messaging.rpc.server rec.execute(self) 2020-02-11 19:38:52,841.841 30190 ERROR oslo_messaging.rpc.server File "/opt/pf9/neutron/lib/python2.7/site-packages/sqlalchemy/orm/unitofwork.py", line 556, in execute 2020-02-11 19:38:52,841.841 30190 ERROR oslo_messaging.rpc.server uow 2020-02-11 19:38:52,841.841 30190 ERROR oslo_messaging.rpc.server File "/opt/pf9/neutron/lib/python2.7/site-packages/sqlalchemy/orm/persistence.py", line 177, in save_obj 2020-02-11 19:38:52,841.841 30190 ERROR oslo_messaging.rpc.server mapper, table, update) 2020-02-11 19:38:52,841.841 30190 ERROR oslo_messaging.rpc.server File "/opt/pf9/neutron/lib/python2.7/site-packages/sqlalchemy/orm/persistence.py", line 791, in _emit_update_statements 2020-02-11 19:38:52,841.841 30190 ERROR oslo_messaging.rpc.server (table.description, len(records), rows)) 2020-02-11 19:38:52,841.841 30190 ERROR oslo_messaging.rpc.server StaleDataError: UPDATE statement on table 'standardattributes' expected to update 1 row(s); 0 were matched. 2020-02-11 19:38:52,841.841 30190 ERROR oslo_messaging.rpc.server ** Affects: neutron Importance: Undecided Status: New -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/1862851 Title: update_floatingip_statuses: StaleDataError: UPDATE statement on table 'standardattributes' expected to update 1 row(s); 0 were matched. Status in neutron: New Bug description: Running Rocky, in a DVR environment We are seeing repeated errors like this in neutron-server logs. Not sure at this point what the effect is, or harmless. This is in a build test environment where a lot of automated VMs get spawned and deleted then associated/disassociated with existing floating IPs, as well as a lot of VMs and floating IPs that are created and deleted manually by user. The logs aren't indicating what the floating IP is, what host, or what port/VM they were bound to: http://paste.openstack.org/show/789444/ 2020-02-11 19:38:52,839.839 30190 ERROR oslo_db.api [req-3a00ad99-d6ed-46cc-9f18-6835854799c8 - - - - -] DB exceeded retry limit.: StaleDataError: UPDATE statement on table 'standardattributes' expected to update 1 row(s); 0 were matched. 2020-02-11 19:38:52,839.839 30190 ERROR oslo_db.api Traceback (most recent call last): 2020-02-11 19:38:52,839.839 30190 ERROR oslo_db.api File "/opt/pf9/neutron/lib/python2.7/site-packages/oslo_db/api.py", line 142, in wrapper 2020-02-11 19:38:52,839.839 30190 ERROR oslo_db.api return f(*args, **kwargs) 2020-02-11 19:38:52,839.839 30190 ERROR oslo_db.api File "/opt/pf9/neutron/lib/python2.7/site-packages/neutron_lib/db/api.py", line 183, in wrapped 2020-02-11 19:38:52,839.839 30190 ERROR oslo_db.api LOG.debug("Retry wrapper got retriable exception: %s", e) 2020-02-11 19:38:52,839.839 30190 ERROR oslo_db.api File "/opt/pf9/neutron/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ 2020-02-11 19:38:52,839.839 30190 ERROR oslo_db.api self.force_reraise() 2020-02-11 19:38:52,839.839 30190 ERROR oslo_db.api File "/opt/pf9/neutron/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise 2020-02-11 19:38:52,839.839 30190 ERROR oslo_db.api six.reraise(self.type_, self.value, self.tb) 2020-02-11 19:38:52,839.839 30190 ERROR oslo_db.api File "/opt/pf9/neutron/lib/python2.7/site-packages/neutron_lib/db/api.py", line 179, in wrapped 2020-02-11 19:38:52,839.839 30190 ERROR oslo_db.api return f(*dup_args, **dup_kwargs) 2020-02-11 19:38:52,839.839 30190 ERROR oslo_db.api File "/opt/pf9/neutron/lib/python2.7/site-packages/neutron/api/rpc/handlers/l3_rpc.py", line 273, in update_floatingip_statuses 2020-02-11 19:38:52,839.839 30190 ERROR oslo_db.api status) 2020-02-11 19:38:52,839.839 30190 ERROR oslo_db.api File "/opt/pf9/neutron/lib/python2.7/site-packages/neutron/db/api.py", line 123, in wrapped 2020-02-11 19:38:52,839.839 30190 ERROR oslo_db.api return method(*args, **kwargs) 2020-02-11 19:38:52,839.839 30190 ERROR oslo_db.api File "/opt/pf9/neutron/lib/python2.7/site-packages/neutron/db/l3_db.py", line 1524, in update_floatingip_status 2020-02-11 19:38:52,839.839 30190 ERROR oslo_db.api context, {'status': status}, id=floatingip_id) 2020-02-11 19:38:52,839.839 30190 ERROR oslo_db.api File "/opt/pf9/neutron/lib/python2.7/site-packages/neutron/objects/base.py", line 669, in update_object 2020-02-11 19:38:52,839.839 30190 ERROR oslo_db.api context, values, validate_filters=False, **kwargs) 2020-02-11 19:38:52,839.839 30190 ERROR oslo_db.api File "/opt/pf9/neutron/lib/python2.7/site-packages/neutron/objects/base.py", line 304, in update_object 2020-02-11 19:38:52,839.839 30190 ERROR oslo_db.api cls._update_objects(obj, values) 2020-02-11 19:38:52,839.839 30190 ERROR oslo_db.api File "/opt/pf9/neutron/lib/python2.7/site-packages/neutron/objects/base.py", line 296, in _update_objects 2020-02-11 19:38:52,839.839 30190 ERROR oslo_db.api obj.update() 2020-02-11 19:38:52,839.839 30190 ERROR oslo_db.api File "/opt/pf9/neutron/lib/python2.7/site-packages/neutron/objects/base.py", line 342, in decorator 2020-02-11 19:38:52,839.839 30190 ERROR oslo_db.api return func(self, *args, **kwargs) 2020-02-11 19:38:52,839.839 30190 ERROR oslo_db.api File "/opt/pf9/neutron/lib/python2.7/site-packages/neutron/objects/base.py", line 876, in update 2020-02-11 19:38:52,839.839 30190 ERROR oslo_db.api self._get_composite_keys())) 2020-02-11 19:38:52,839.839 30190 ERROR oslo_db.api File "/opt/pf9/neutron/lib/python2.7/site-packages/neutron/objects/db/api.py", line 82, in update_object 2020-02-11 19:38:52,839.839 30190 ERROR oslo_db.api db_obj.save(session=context.session) 2020-02-11 19:38:52,839.839 30190 ERROR oslo_db.api File "/opt/pf9/neutron/lib/python2.7/site-packages/oslo_db/sqlalchemy/models.py", line 50, in save 2020-02-11 19:38:52,839.839 30190 ERROR oslo_db.api session.flush() 2020-02-11 19:38:52,839.839 30190 ERROR oslo_db.api File "/opt/pf9/neutron/lib/python2.7/site-packages/sqlalchemy/orm/session.py", line 2254, in flush 2020-02-11 19:38:52,839.839 30190 ERROR oslo_db.api self._flush(objects) 2020-02-11 19:38:52,839.839 30190 ERROR oslo_db.api File "/opt/pf9/neutron/lib/python2.7/site-packages/sqlalchemy/orm/session.py", line 2380, in _flush 2020-02-11 19:38:52,839.839 30190 ERROR oslo_db.api transaction.rollback(_capture_exception=True) 2020-02-11 19:38:52,839.839 30190 ERROR oslo_db.api File "/opt/pf9/neutron/lib/python2.7/site-packages/sqlalchemy/util/langhelpers.py", line 66, in __exit__ 2020-02-11 19:38:52,839.839 30190 ERROR oslo_db.api compat.reraise(exc_type, exc_value, exc_tb) 2020-02-11 19:38:52,839.839 30190 ERROR oslo_db.api File "/opt/pf9/neutron/lib/python2.7/site-packages/sqlalchemy/orm/session.py", line 2344, in _flush 2020-02-11 19:38:52,839.839 30190 ERROR oslo_db.api flush_context.execute() 2020-02-11 19:38:52,839.839 30190 ERROR oslo_db.api File "/opt/pf9/neutron/lib/python2.7/site-packages/sqlalchemy/orm/unitofwork.py", line 391, in execute 2020-02-11 19:38:52,839.839 30190 ERROR oslo_db.api rec.execute(self) 2020-02-11 19:38:52,839.839 30190 ERROR oslo_db.api File "/opt/pf9/neutron/lib/python2.7/site-packages/sqlalchemy/orm/unitofwork.py", line 556, in execute 2020-02-11 19:38:52,839.839 30190 ERROR oslo_db.api uow 2020-02-11 19:38:52,839.839 30190 ERROR oslo_db.api File "/opt/pf9/neutron/lib/python2.7/site-packages/sqlalchemy/orm/persistence.py", line 177, in save_obj 2020-02-11 19:38:52,839.839 30190 ERROR oslo_db.api mapper, table, update) 2020-02-11 19:38:52,839.839 30190 ERROR oslo_db.api File "/opt/pf9/neutron/lib/python2.7/site-packages/sqlalchemy/orm/persistence.py", line 791, in _emit_update_statements 2020-02-11 19:38:52,839.839 30190 ERROR oslo_db.api (table.description, len(records), rows)) 2020-02-11 19:38:52,839.839 30190 ERROR oslo_db.api StaleDataError: UPDATE statement on table 'standardattributes' expected to update 1 row(s); 0 were matched. 2020-02-11 19:38:52,839.839 30190 ERROR oslo_db.api 2020-02-11 19:38:52,841.841 30190 ERROR oslo_messaging.rpc.server [req-3a00ad99-d6ed-46cc-9f18-6835854799c8 - - - - -] Exception during message handling: StaleDataError: UPDATE statement on table 'standardattributes' expected to update 1 row(s); 0 were matched. 2020-02-11 19:38:52,841.841 30190 ERROR oslo_messaging.rpc.server Traceback (most recent call last): 2020-02-11 19:38:52,841.841 30190 ERROR oslo_messaging.rpc.server File "/opt/pf9/neutron/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 166, in _process_incoming 2020-02-11 19:38:52,841.841 30190 ERROR oslo_messaging.rpc.server res = self.dispatcher.dispatch(message) 2020-02-11 19:38:52,841.841 30190 ERROR oslo_messaging.rpc.server File "/opt/pf9/neutron/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 265, in dispatch 2020-02-11 19:38:52,841.841 30190 ERROR oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args) 2020-02-11 19:38:52,841.841 30190 ERROR oslo_messaging.rpc.server File "/opt/pf9/neutron/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 194, in _do_dispatch 2020-02-11 19:38:52,841.841 30190 ERROR oslo_messaging.rpc.server result = func(ctxt, **new_args) 2020-02-11 19:38:52,841.841 30190 ERROR oslo_messaging.rpc.server File "/opt/pf9/neutron/lib/python2.7/site-packages/neutron_lib/db/api.py", line 140, in wrapped 2020-02-11 19:38:52,841.841 30190 ERROR oslo_messaging.rpc.server setattr(e, '_RETRY_EXCEEDED', True) 2020-02-11 19:38:52,841.841 30190 ERROR oslo_messaging.rpc.server File "/opt/pf9/neutron/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ 2020-02-11 19:38:52,841.841 30190 ERROR oslo_messaging.rpc.server self.force_reraise() 2020-02-11 19:38:52,841.841 30190 ERROR oslo_messaging.rpc.server File "/opt/pf9/neutron/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise 2020-02-11 19:38:52,841.841 30190 ERROR oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb) 2020-02-11 19:38:52,841.841 30190 ERROR oslo_messaging.rpc.server File "/opt/pf9/neutron/lib/python2.7/site-packages/neutron_lib/db/api.py", line 136, in wrapped 2020-02-11 19:38:52,841.841 30190 ERROR oslo_messaging.rpc.server return f(*args, **kwargs) 2020-02-11 19:38:52,841.841 30190 ERROR oslo_messaging.rpc.server File "/opt/pf9/neutron/lib/python2.7/site-packages/oslo_db/api.py", line 154, in wrapper 2020-02-11 19:38:52,841.841 30190 ERROR oslo_messaging.rpc.server ectxt.value = e.inner_exc 2020-02-11 19:38:52,841.841 30190 ERROR oslo_messaging.rpc.server File "/opt/pf9/neutron/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ 2020-02-11 19:38:52,841.841 30190 ERROR oslo_messaging.rpc.server self.force_reraise() 2020-02-11 19:38:52,841.841 30190 ERROR oslo_messaging.rpc.server File "/opt/pf9/neutron/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise 2020-02-11 19:38:52,841.841 30190 ERROR oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb) 2020-02-11 19:38:52,841.841 30190 ERROR oslo_messaging.rpc.server File "/opt/pf9/neutron/lib/python2.7/site-packages/oslo_db/api.py", line 142, in wrapper 2020-02-11 19:38:52,841.841 30190 ERROR oslo_messaging.rpc.server return f(*args, **kwargs) 2020-02-11 19:38:52,841.841 30190 ERROR oslo_messaging.rpc.server File "/opt/pf9/neutron/lib/python2.7/site-packages/neutron_lib/db/api.py", line 183, in wrapped 2020-02-11 19:38:52,841.841 30190 ERROR oslo_messaging.rpc.server LOG.debug("Retry wrapper got retriable exception: %s", e) 2020-02-11 19:38:52,841.841 30190 ERROR oslo_messaging.rpc.server File "/opt/pf9/neutron/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ 2020-02-11 19:38:52,841.841 30190 ERROR oslo_messaging.rpc.server self.force_reraise() 2020-02-11 19:38:52,841.841 30190 ERROR oslo_messaging.rpc.server File "/opt/pf9/neutron/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise 2020-02-11 19:38:52,841.841 30190 ERROR oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb) 2020-02-11 19:38:52,841.841 30190 ERROR oslo_messaging.rpc.server File "/opt/pf9/neutron/lib/python2.7/site-packages/neutron_lib/db/api.py", line 179, in wrapped 2020-02-11 19:38:52,841.841 30190 ERROR oslo_messaging.rpc.server return f(*dup_args, **dup_kwargs) 2020-02-11 19:38:52,841.841 30190 ERROR oslo_messaging.rpc.server File "/opt/pf9/neutron/lib/python2.7/site-packages/neutron/api/rpc/handlers/l3_rpc.py", line 273, in update_floatingip_statuses 2020-02-11 19:38:52,841.841 30190 ERROR oslo_messaging.rpc.server status) 2020-02-11 19:38:52,841.841 30190 ERROR oslo_messaging.rpc.server File "/opt/pf9/neutron/lib/python2.7/site-packages/neutron/db/api.py", line 123, in wrapped 2020-02-11 19:38:52,841.841 30190 ERROR oslo_messaging.rpc.server return method(*args, **kwargs) 2020-02-11 19:38:52,841.841 30190 ERROR oslo_messaging.rpc.server File "/opt/pf9/neutron/lib/python2.7/site-packages/neutron/db/l3_db.py", line 1524, in update_floatingip_status 2020-02-11 19:38:52,841.841 30190 ERROR oslo_messaging.rpc.server context, {'status': status}, id=floatingip_id) 2020-02-11 19:38:52,841.841 30190 ERROR oslo_messaging.rpc.server File "/opt/pf9/neutron/lib/python2.7/site-packages/neutron/objects/base.py", line 669, in update_object 2020-02-11 19:38:52,841.841 30190 ERROR oslo_messaging.rpc.server context, values, validate_filters=False, **kwargs) 2020-02-11 19:38:52,841.841 30190 ERROR oslo_messaging.rpc.server File "/opt/pf9/neutron/lib/python2.7/site-packages/neutron/objects/base.py", line 304, in update_object 2020-02-11 19:38:52,841.841 30190 ERROR oslo_messaging.rpc.server cls._update_objects(obj, values) 2020-02-11 19:38:52,841.841 30190 ERROR oslo_messaging.rpc.server File "/opt/pf9/neutron/lib/python2.7/site-packages/neutron/objects/base.py", line 296, in _update_objects 2020-02-11 19:38:52,841.841 30190 ERROR oslo_messaging.rpc.server obj.update() 2020-02-11 19:38:52,841.841 30190 ERROR oslo_messaging.rpc.server File "/opt/pf9/neutron/lib/python2.7/site-packages/neutron/objects/base.py", line 342, in decorator 2020-02-11 19:38:52,841.841 30190 ERROR oslo_messaging.rpc.server return func(self, *args, **kwargs) 2020-02-11 19:38:52,841.841 30190 ERROR oslo_messaging.rpc.server File "/opt/pf9/neutron/lib/python2.7/site-packages/neutron/objects/base.py", line 876, in update 2020-02-11 19:38:52,841.841 30190 ERROR oslo_messaging.rpc.server self._get_composite_keys())) 2020-02-11 19:38:52,841.841 30190 ERROR oslo_messaging.rpc.server File "/opt/pf9/neutron/lib/python2.7/site-packages/neutron/objects/db/api.py", line 82, in update_object 2020-02-11 19:38:52,841.841 30190 ERROR oslo_messaging.rpc.server db_obj.save(session=context.session) 2020-02-11 19:38:52,841.841 30190 ERROR oslo_messaging.rpc.server File "/opt/pf9/neutron/lib/python2.7/site-packages/oslo_db/sqlalchemy/models.py", line 50, in save 2020-02-11 19:38:52,841.841 30190 ERROR oslo_messaging.rpc.server session.flush() 2020-02-11 19:38:52,841.841 30190 ERROR oslo_messaging.rpc.server File "/opt/pf9/neutron/lib/python2.7/site-packages/sqlalchemy/orm/session.py", line 2254, in flush 2020-02-11 19:38:52,841.841 30190 ERROR oslo_messaging.rpc.server self._flush(objects) 2020-02-11 19:38:52,841.841 30190 ERROR oslo_messaging.rpc.server File "/opt/pf9/neutron/lib/python2.7/site-packages/sqlalchemy/orm/session.py", line 2380, in _flush 2020-02-11 19:38:52,841.841 30190 ERROR oslo_messaging.rpc.server transaction.rollback(_capture_exception=True) 2020-02-11 19:38:52,841.841 30190 ERROR oslo_messaging.rpc.server File "/opt/pf9/neutron/lib/python2.7/site-packages/sqlalchemy/util/langhelpers.py", line 66, in __exit__ 2020-02-11 19:38:52,841.841 30190 ERROR oslo_messaging.rpc.server compat.reraise(exc_type, exc_value, exc_tb) 2020-02-11 19:38:52,841.841 30190 ERROR oslo_messaging.rpc.server File "/opt/pf9/neutron/lib/python2.7/site-packages/sqlalchemy/orm/session.py", line 2344, in _flush 2020-02-11 19:38:52,841.841 30190 ERROR oslo_messaging.rpc.server flush_context.execute() 2020-02-11 19:38:52,841.841 30190 ERROR oslo_messaging.rpc.server File "/opt/pf9/neutron/lib/python2.7/site-packages/sqlalchemy/orm/unitofwork.py", line 391, in execute 2020-02-11 19:38:52,841.841 30190 ERROR oslo_messaging.rpc.server rec.execute(self) 2020-02-11 19:38:52,841.841 30190 ERROR oslo_messaging.rpc.server File "/opt/pf9/neutron/lib/python2.7/site-packages/sqlalchemy/orm/unitofwork.py", line 556, in execute 2020-02-11 19:38:52,841.841 30190 ERROR oslo_messaging.rpc.server uow 2020-02-11 19:38:52,841.841 30190 ERROR oslo_messaging.rpc.server File "/opt/pf9/neutron/lib/python2.7/site-packages/sqlalchemy/orm/persistence.py", line 177, in save_obj 2020-02-11 19:38:52,841.841 30190 ERROR oslo_messaging.rpc.server mapper, table, update) 2020-02-11 19:38:52,841.841 30190 ERROR oslo_messaging.rpc.server File "/opt/pf9/neutron/lib/python2.7/site-packages/sqlalchemy/orm/persistence.py", line 791, in _emit_update_statements 2020-02-11 19:38:52,841.841 30190 ERROR oslo_messaging.rpc.server (table.description, len(records), rows)) 2020-02-11 19:38:52,841.841 30190 ERROR oslo_messaging.rpc.server StaleDataError: UPDATE statement on table 'standardattributes' expected to update 1 row(s); 0 were matched. 2020-02-11 19:38:52,841.841 30190 ERROR oslo_messaging.rpc.server To manage notifications about this bug go to: https://bugs.launchpad.net/neutron/+bug/1862851/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : [email protected] Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
