Public bug reported:
This bug was encountered in IntegrationsQA (also known as SolutionsQA)
testing with Foundation Cloud Engine in the IntegrationsQA lab
environment.
The Tempest tests failed because OpenStack was unable to bind a port to
a virtual machine instance during the test_two_vms_fips Tempest test
from the Neutron Tempest Plugin
(neutron_tempest_plugin.scenario.admin.test_floatingip.FloatingIpTestCasesAdmin).
Here is the error Tempest reported:
<testcase
classname="neutron_tempest_plugin.scenario.admin.test_floatingip.FloatingIpTestCasesAdmin"
id="6bba729b-3fb6-494b-9e1e-82bbd89a1045" name="test_two_vms_fips"
time="15.480" timestamp="2022-12-13T05:15:19+0000">
<failure>Traceback (most recent call last): File
"/snap/fcbtest/35/lib/python3.10/site-packages/neutron_tempest_plugin/scenario/admin/test_floatingip.py",
line 110, in test_two_vms_fips servers, server_ssh_clients, fips =
self._create_vms(hyper, avail_zone) File
"/snap/fcbtest/35/lib/python3.10/site-packages/neutron_tempest_plugin/scenario/admin/test_floatingip.py",
line 75, in _create_vms servers.append(self.create_server( File
"/snap/fcbtest/35/lib/python3.10/site-packages/neutron_tempest_plugin/scenario/base.py",
line 150, in create_server self.wait_for_server_active(server['server'],
client=client) File
"/snap/fcbtest/35/lib/python3.10/site-packages/neutron_tempest_plugin/scenario/base.py",
line 527, in wait_for_server_active self.wait_for_server_status( File
"/snap/fcbtest/35/lib/python3.10/site-packages/neutron_tempest_plugin/scenario/base.py",
line 518, in wait_for_server_status waiters.wait_for_server_status(client,
server['id'], status, **kwargs) File "/home/ubuntu/snap/fcbt
est/35/.rally/verification/verifier-6c2956be-9f82-47a0-baae-388878cb5981/repo/tempest/common/waiters.py",
line 81, in wait_for_server_status raise
exceptions.BuildErrorException(details, server_id=server_id)
tempest.exceptions.BuildErrorException: Server
3867e215-f17e-4a29-a5bd-763ad2544a82 failed to build and is in ERROR status
Details: Fault: {'code': 500, 'created': '2022-12-13T05:15:33Z', 'message':
'Binding failed for port efbe7ab9-b493-4d0b-a10d-0854f011bc9e, please check
neutron logs for more information.', 'details': 'Traceback (most recent call
last):\n File "/usr/lib/python3/dist-packages/nova/compute/manager.py", line
2489, in _build_and_run_instance\n self.driver.spawn(context, instance,
image_meta,\n File
"/usr/lib/python3/dist-packages/nova/virt/libvirt/driver.py", line 4329, in
spawn\n xml = self._get_guest_xml(context, instance, network_info,\n File
"/usr/lib/python3/dist-packages/nova/virt/libvirt/driver.py", line 7288, in
_get_guest_xml\n network_info_str = str(net
work_info)\n File "/usr/lib/python3/dist-packages/nova/network/model.py", line
620, in __str__\n return self._sync_wrapper(fn, *args, **kwargs)\n File
"/usr/lib/python3/dist-packages/nova/network/model.py", line 603, in
_sync_wrapper\n self.wait()\n File
"/usr/lib/python3/dist-packages/nova/network/model.py", line 635, in wait\n
self[:] = self._gt.wait()\n File
"/usr/lib/python3/dist-packages/eventlet/greenthread.py", line 181, in wait\n
return self._exit_event.wait()\n File
"/usr/lib/python3/dist-packages/eventlet/event.py", line 125, in wait\n result
= hub.switch()\n File "/usr/lib/python3/dist-packages/eventlet/hubs/hub.py",
line 313, in switch\n return self.greenlet.switch()\n File
"/usr/lib/python3/dist-packages/eventlet/greenthread.py", line 221, in main\n
result = function(*args, **kwargs)\n File
"/usr/lib/python3/dist-packages/nova/utils.py", line 656, in context_wrapper\n
return func(*args, **kwargs)\n File
"/usr/lib/python3/dist-packages/nova/compute/manager.py", line 1876
, in _allocate_network_async\n raise e\n File
"/usr/lib/python3/dist-packages/nova/compute/manager.py", line 1854, in
_allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n
File "/usr/lib/python3/dist-packages/nova/network/neutron.py", line 1213, in
allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n
File "/usr/lib/python3/dist-packages/nova/network/neutron.py", line 1351, in
_update_ports_for_instance\n vif.destroy()\n File
"/usr/lib/python3/dist-packages/oslo_utils/excutils.py", line 227, in
__exit__\n self.force_reraise()\n File
"/usr/lib/python3/dist-packages/oslo_utils/excutils.py", line 200, in
force_reraise\n raise self.value\n File
"/usr/lib/python3/dist-packages/nova/network/neutron.py", line 1320, in
_update_ports_for_instance\n updated_port = self._update_port(\n File
"/usr/lib/python3/dist-packages/nova/network/neutron.py", line 584, in
_update_port\n _ensure_no_port_binding_failure(port)\n File
"/usr/lib/python3/dist-packa
ges/nova/network/neutron.py", line 293, in _ensure_no_port_binding_failure\n
raise
exception.PortBindingFailed(port_id=port[\'id\'])\nnova.exception.PortBindingFailed:
Binding failed for port efbe7ab9-b493-4d0b-a10d-0854f011bc9e, please check
neutron logs for more information.\n\nDuring handling of the above exception,
another exception occurred:\n\nTraceback (most recent call last):\n File
"/usr/lib/python3/dist-packages/nova/compute/manager.py", line 2312, in
_do_build_and_run_instance\n self._build_and_run_instance(context, instance,
image,\n File "/usr/lib/python3/dist-packages/nova/compute/manager.py", line
2585, in _build_and_run_instance\n raise
exception.RescheduledException(\nnova.exception.RescheduledException: Build of
instance 3867e215-f17e-4a29-a5bd-763ad2544a82 was re-scheduled: Binding failed
for port efbe7ab9-b493-4d0b-a10d-0854f011bc9e, please check neutron logs for
more information.\n'}. </failure>
</testcase>
The Tempest test results can be found at https://oil-
jenkins.canonical.com/artifacts/8cf589a3-8f0f-452b-b893-7f99d9d3a6a6/generated/generated/openstack/tempest_result.xml.
I could not find any errors in the Neutron logs for this during this
time. The full artifacts, including logs, for this test run can be
accessed at:
https://oil-
jenkins.canonical.com/artifacts/8cf589a3-8f0f-452b-b893-7f99d9d3a6a6/index.html
** Affects: neutron
Importance: Undecided
Status: New
** Tags: cdo-qa cdo-tempest foundations-engine
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1999540
Title:
Tempest test test_two_vms_fips failed due to port binding on instance
failing
Status in neutron:
New
Bug description:
This bug was encountered in IntegrationsQA (also known as SolutionsQA)
testing with Foundation Cloud Engine in the IntegrationsQA lab
environment.
The Tempest tests failed because OpenStack was unable to bind a port
to a virtual machine instance during the test_two_vms_fips Tempest
test from the Neutron Tempest Plugin
(neutron_tempest_plugin.scenario.admin.test_floatingip.FloatingIpTestCasesAdmin).
Here is the error Tempest reported:
<testcase
classname="neutron_tempest_plugin.scenario.admin.test_floatingip.FloatingIpTestCasesAdmin"
id="6bba729b-3fb6-494b-9e1e-82bbd89a1045" name="test_two_vms_fips"
time="15.480" timestamp="2022-12-13T05:15:19+0000">
<failure>Traceback (most recent call last): File
"/snap/fcbtest/35/lib/python3.10/site-packages/neutron_tempest_plugin/scenario/admin/test_floatingip.py",
line 110, in test_two_vms_fips servers, server_ssh_clients, fips =
self._create_vms(hyper, avail_zone) File
"/snap/fcbtest/35/lib/python3.10/site-packages/neutron_tempest_plugin/scenario/admin/test_floatingip.py",
line 75, in _create_vms servers.append(self.create_server( File
"/snap/fcbtest/35/lib/python3.10/site-packages/neutron_tempest_plugin/scenario/base.py",
line 150, in create_server self.wait_for_server_active(server['server'],
client=client) File
"/snap/fcbtest/35/lib/python3.10/site-packages/neutron_tempest_plugin/scenario/base.py",
line 527, in wait_for_server_active self.wait_for_server_status( File
"/snap/fcbtest/35/lib/python3.10/site-packages/neutron_tempest_plugin/scenario/base.py",
line 518, in wait_for_server_status waiters.wait_for_server_status(client,
server['id'], status, **kwargs) File "/home/ubuntu/snap/fc
btest/35/.rally/verification/verifier-6c2956be-9f82-47a0-baae-388878cb5981/repo/tempest/common/waiters.py",
line 81, in wait_for_server_status raise
exceptions.BuildErrorException(details, server_id=server_id)
tempest.exceptions.BuildErrorException: Server
3867e215-f17e-4a29-a5bd-763ad2544a82 failed to build and is in ERROR status
Details: Fault: {'code': 500, 'created': '2022-12-13T05:15:33Z', 'message':
'Binding failed for port efbe7ab9-b493-4d0b-a10d-0854f011bc9e, please check
neutron logs for more information.', 'details': 'Traceback (most recent call
last):\n File "/usr/lib/python3/dist-packages/nova/compute/manager.py", line
2489, in _build_and_run_instance\n self.driver.spawn(context, instance,
image_meta,\n File
"/usr/lib/python3/dist-packages/nova/virt/libvirt/driver.py", line 4329, in
spawn\n xml = self._get_guest_xml(context, instance, network_info,\n File
"/usr/lib/python3/dist-packages/nova/virt/libvirt/driver.py", line 7288, in
_get_guest_xml\n network_info_str = str(n
etwork_info)\n File "/usr/lib/python3/dist-packages/nova/network/model.py",
line 620, in __str__\n return self._sync_wrapper(fn, *args, **kwargs)\n File
"/usr/lib/python3/dist-packages/nova/network/model.py", line 603, in
_sync_wrapper\n self.wait()\n File
"/usr/lib/python3/dist-packages/nova/network/model.py", line 635, in wait\n
self[:] = self._gt.wait()\n File
"/usr/lib/python3/dist-packages/eventlet/greenthread.py", line 181, in wait\n
return self._exit_event.wait()\n File
"/usr/lib/python3/dist-packages/eventlet/event.py", line 125, in wait\n result
= hub.switch()\n File "/usr/lib/python3/dist-packages/eventlet/hubs/hub.py",
line 313, in switch\n return self.greenlet.switch()\n File
"/usr/lib/python3/dist-packages/eventlet/greenthread.py", line 221, in main\n
result = function(*args, **kwargs)\n File
"/usr/lib/python3/dist-packages/nova/utils.py", line 656, in context_wrapper\n
return func(*args, **kwargs)\n File
"/usr/lib/python3/dist-packages/nova/compute/manager.py", line 18
76, in _allocate_network_async\n raise e\n File
"/usr/lib/python3/dist-packages/nova/compute/manager.py", line 1854, in
_allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n
File "/usr/lib/python3/dist-packages/nova/network/neutron.py", line 1213, in
allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n
File "/usr/lib/python3/dist-packages/nova/network/neutron.py", line 1351, in
_update_ports_for_instance\n vif.destroy()\n File
"/usr/lib/python3/dist-packages/oslo_utils/excutils.py", line 227, in
__exit__\n self.force_reraise()\n File
"/usr/lib/python3/dist-packages/oslo_utils/excutils.py", line 200, in
force_reraise\n raise self.value\n File
"/usr/lib/python3/dist-packages/nova/network/neutron.py", line 1320, in
_update_ports_for_instance\n updated_port = self._update_port(\n File
"/usr/lib/python3/dist-packages/nova/network/neutron.py", line 584, in
_update_port\n _ensure_no_port_binding_failure(port)\n File
"/usr/lib/python3/dist-pac
kages/nova/network/neutron.py", line 293, in _ensure_no_port_binding_failure\n
raise
exception.PortBindingFailed(port_id=port[\'id\'])\nnova.exception.PortBindingFailed:
Binding failed for port efbe7ab9-b493-4d0b-a10d-0854f011bc9e, please check
neutron logs for more information.\n\nDuring handling of the above exception,
another exception occurred:\n\nTraceback (most recent call last):\n File
"/usr/lib/python3/dist-packages/nova/compute/manager.py", line 2312, in
_do_build_and_run_instance\n self._build_and_run_instance(context, instance,
image,\n File "/usr/lib/python3/dist-packages/nova/compute/manager.py", line
2585, in _build_and_run_instance\n raise
exception.RescheduledException(\nnova.exception.RescheduledException: Build of
instance 3867e215-f17e-4a29-a5bd-763ad2544a82 was re-scheduled: Binding failed
for port efbe7ab9-b493-4d0b-a10d-0854f011bc9e, please check neutron logs for
more information.\n'}. </failure>
</testcase>
The Tempest test results can be found at https://oil-
jenkins.canonical.com/artifacts/8cf589a3-8f0f-452b-b893-7f99d9d3a6a6/generated/generated/openstack/tempest_result.xml.
I could not find any errors in the Neutron logs for this during this
time. The full artifacts, including logs, for this test run can be
accessed at:
https://oil-
jenkins.canonical.com/artifacts/8cf589a3-8f0f-452b-b893-7f99d9d3a6a6/index.html
To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1999540/+subscriptions
--
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : [email protected]
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help : https://help.launchpad.net/ListHelp
