Reviewed: https://review.openstack.org/374975 Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=2ce193e16d7ca5b93671d7f0d2e3b35761f5d386 Submitter: Jenkins Branch: master
commit 2ce193e16d7ca5b93671d7f0d2e3b35761f5d386 Author: Matt Riedemann <[email protected]> Date: Thu Sep 22 12:41:46 2016 -0400 libvirt: ignore conflict when defining network filters We have a latent race in the libvirt firewall code when setting up static filters which is now an error with libvirt>=1.2.7, which is why we started seeing this in CI failures starting in newton which run on xenial nodes that have libvirt 1.3.1 (but didn't see it on trusty nodes with libvirt 1.2.2). Libvirt commit 46a811db0731cedaea0153fc223faa6096cee5b5 checks for an existing filter with the same name but a different uuid when defining network filters and raises an error if found. That was added in the 1.2.7 release. This change simply handles the error and ignores it so we don't fail to boot the instance. Unfortunately we don't have a specific error code from libvirt when this happens so the best we can do is compare the error message from the libvirt error which is only going to work for English locales because the error message from libvirt is translated. Change-Id: I161be26d605351f168e351d3ed3d308234346f6f Closes-Bug: #1612875 ** Changed in: nova Status: In Progress => Fix Released -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1612875 Title: FixedIPsTestJson fails server build with "was re-scheduled: operation failed: filter 'nova-no-nd-reflection' already exists with uuid" Status in OpenStack Compute (nova): Fix Released Status in OpenStack Compute (nova) newton series: Confirmed Bug description: Seen here: http://logs.openstack.org/98/355098/1/check/gate-tempest-dsvm-full- ubuntu- xenial/3c301f3/logs/screen-n-cond.txt.gz?level=TRACE#_2016-08-13_00_53_27_621 2016-08-13 00:53:27.621 15971 ERROR nova.scheduler.utils [req-e7b83619-01ae-43f2-b293-af02c5cb35a8 tempest-FixedIPsTestJson-2017152444 tempest-FixedIPsTestJson-2017152444] [instance: ac2c9a4a-1b07-43e1-8f4c-b75541331307] Error from last host: ubuntu-xenial-rax-ord-3453779 (node ubuntu-xenial-rax-ord-3453779): [u'Traceback (most recent call last):\n', u' File "/opt/stack/new/nova/nova/compute/manager.py", line 1778, in _do_build_and_run_instance\n filter_properties)\n', u' File "/opt/stack/new/nova/nova/compute/manager.py", line 1973, in _build_and_run_instance\n instance_uuid=instance.uuid, reason=six.text_type(e))\n', u"RescheduledException: Build of instance ac2c9a4a-1b07-43e1-8f4c-b75541331307 was re-scheduled: operation failed: filter 'nova-no-nd-reflection' already exists with uuid 1f47eeb2-d473-481e-998a-c4d64a44ac5e\n"] 2016-08-13 00:53:27.676 15971 WARNING nova.scheduler.utils [req-e7b83619-01ae-43f2-b293-af02c5cb35a8 tempest-FixedIPsTestJson-2017152444 tempest-FixedIPsTestJson-2017152444] Failed to compute_task_build_instances: No valid host was found. There are not enough hosts available. Traceback (most recent call last): File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/server.py", line 199, in inner return func(*args, **kwargs) File "/opt/stack/new/nova/nova/scheduler/manager.py", line 104, in select_destinations dests = self.driver.select_destinations(ctxt, spec_obj) File "/opt/stack/new/nova/nova/scheduler/filter_scheduler.py", line 74, in select_destinations raise exception.NoValidHost(reason=reason) NoValidHost: No valid host was found. There are not enough hosts available. 2016-08-13 00:53:27.676 15971 WARNING nova.scheduler.utils [req- e7b83619-01ae-43f2-b293-af02c5cb35a8 tempest- FixedIPsTestJson-2017152444 tempest-FixedIPsTestJson-2017152444] [instance: ac2c9a4a-1b07-43e1-8f4c-b75541331307] Setting instance to ERROR state. http://logstash.openstack.org/#dashboard/file/logstash.json?query=message%3A%5C%22was %20re-scheduled%3A%20operation%20failed%3A%20filter%20'nova-no-nd- reflection'%20already%20exists%20with%20uuid%5C%22%20AND%20tags%3A%5C%22screen-n-cond.txt%5C%22&from=7d 5 hits in 7 days, check queue only, but multiple changes. To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1612875/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : [email protected] Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
