Looking at logstash: http://logstash.openstack.org/#dashboard/file/logstash.json?query=message%3A%5C%22_send_notification%5C%22%20AND%20message%3A%5C%22ValueError%3A%20Circular%20reference%20detected%5C%22&from=7d
This seems to only show up on that aggregate negative test, and it's probably this error notification wrapper: https://github.com/openstack/nova/blob/2380659e358770a3f36253b93a112b9779a23958/nova/compute/api.py#L4602 The AggregateHostExists duplicate error comes out of the call to "aggregate.add_host(host_name)". I'm not sure what's breaking in the actual notification driver. It also looks like this is happening on non-py35 jobs too. ** Summary changed: - Duplicate entry for key 'uniq_aggregate_hosts0host0aggregate_id + "ValueError: Circular reference detected" in send_notification ** Also affects: oslo.messaging Importance: Undecided Status: New -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1673375 Title: "ValueError: Circular reference detected" in send_notification Status in OpenStack Compute (nova): New Status in oslo.messaging: New Bug description: I guess something is wrong with nova, causing a tempest failure. http://logs.openstack.org/08/445308/3/check/gate-tempest-dsvm-py35 -ubuntu- xenial/7bf0d72/logs/screen-n-api.txt.gz#_2017-03-16_05_31_09_399 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging Traceback (most recent call last): 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging File "/usr/local/lib/python3.5/dist-packages/sqlalchemy/engine/base.py", line 1139, in _execute_context 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging context) 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging File "/usr/local/lib/python3.5/dist-packages/sqlalchemy/engine/default.py", line 450, in do_execute 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging cursor.execute(statement, parameters) 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging File "/usr/local/lib/python3.5/dist-packages/pymysql/cursors.py", line 166, in execute 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging result = self._query(query) 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging File "/usr/local/lib/python3.5/dist-packages/pymysql/cursors.py", line 322, in _query 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging conn.query(q) 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging File "/usr/local/lib/python3.5/dist-packages/pymysql/connections.py", line 852, in query 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging self._affected_rows = self._read_query_result(unbuffered=unbuffered) 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging File "/usr/local/lib/python3.5/dist-packages/pymysql/connections.py", line 1053, in _read_query_result 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging result.read() 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging File "/usr/local/lib/python3.5/dist-packages/pymysql/connections.py", line 1336, in read 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging first_packet = self.connection._read_packet() 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging File "/usr/local/lib/python3.5/dist-packages/pymysql/connections.py", line 1010, in _read_packet 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging packet.check_error() 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging File "/usr/local/lib/python3.5/dist-packages/pymysql/connections.py", line 393, in check_error 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging err.raise_mysql_exception(self._data) 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging File "/usr/local/lib/python3.5/dist-packages/pymysql/err.py", line 107, in raise_mysql_exception 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging raise errorclass(errno, errval) 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging pymysql.err.IntegrityError: (1062, "Duplicate entry 'ubuntu-xenial-rax-iad-7906449-6' for key 'uniq_aggregate_hosts0host0aggregate_id'") To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1673375/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : [email protected] Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
