[Yahoo-eng-team] [Bug 1527061] Re: Nova should not throw exception when port binding fails for Ironic
[Expired for neutron because there has been no activity for 60 days.] ** Changed in: neutron Status: Incomplete => Expired -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/1527061 Title: Nova should not throw exception when port binding fails for Ironic Status in neutron: Expired Bug description: Neutron tries to bind port on compute where instance is launched. It doesn't make sense when hypervisor_type is ironic, since VM does not live on hypervisor in this case. Furthermore it leads to failed provisioning of baremetal node, when neutron is not configured on ironic compute node. When binding fails, nova starts cleanup and then throws exception. As part of cleanup ironic port is deleted. This should not be the case when virt-driver being used is Ironic. Nova versions: liberty on-wards. Setup: node-1: controller with neutron node-2: ironic-compute without neutron/neutron agents Register a BM node and perform nova boot. From Nova: = nova-compute.log:21147:2015-12-05 02:57:38.542 7649 ERROR nova.compute.manager [-] Instance failed network setup after 1 attempt(s) nova-compute.log:21148:2015-12-05 02:57:38.542 7649 ERROR nova.compute.manager Traceback (most recent call last): nova-compute.log:21149:2015-12-05 02:57:38.542 7649 ERROR nova.compute.manager File "/opt/stack/venv/nova-20151204T044743Z/lib/python2.7/site-packages/nova/compute/manager.py", line 1564, in _allocate_network_async nova-compute.log:21150:2015-12-05 02:57:38.542 7649 ERROR nova.compute.manager dhcp_options=dhcp_options) nova-compute.log:21151:2015-12-05 02:57:38.542 7649 ERROR nova.compute.manager File "/opt/stack/venv/nova-20151204T044743Z/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 727, in allocate_for_instance nova-compute.log:21152:2015-12-05 02:57:38.542 7649 ERROR nova.compute.manager self._delete_ports(neutron, instance, created_port_ids) nova-compute.log:21153:2015-12-05 02:57:38.542 7649 ERROR nova.compute.manager File "/opt/stack/venv/nova-20151204T044743Z/lib/python2.7/site-packages/oslo_utils/excutils.py", line 204, in __exit__ nova-compute.log:21154:2015-12-05 02:57:38.542 7649 ERROR nova.compute.manager six.reraise(self.type_, self.value, self.tb) nova-compute.log:21155:2015-12-05 02:57:38.542 7649 ERROR nova.compute.manager File "/opt/stack/venv/nova-20151204T044743Z/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 719, in allocate_for_instance nova-compute.log:21156:2015-12-05 02:57:38.542 7649 ERROR nova.compute.manager security_group_ids, available_macs, dhcp_opts) nova-compute.log:21157:2015-12-05 02:57:38.542 7649 ERROR nova.compute.manager File "/opt/stack/venv/nova-20151204T044743Z/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 342, in _create_port nova-compute.log:21158:2015-12-05 02:57:38.542 7649 ERROR nova.compute.manager raise exception.PortBindingFailed(port_id=port_id) nova-compute.log:21159:2015-12-05 02:57:38.542 7649 ERROR nova.compute.manager PortBindingFailed: Binding failed for port 0514b7e8-0408-4d5e-9b48-8292e686494f, please check neutron logs for more information. nova-compute.log:21160:2015-12-05 02:57:38.542 7649 ERROR nova.compute.manager 2015-12-05 02:57:43.375 7649 DEBUG nova.virt.ironic.driver [req-25ba4c81-4333-43e4-b8d9-e460827435e0 a989fe7fc89e4825b98d7e6584cc 1308f1a382ef43c79eeed0ebf8a9db3b] unplug: instance_uuid=d9295d4d-1104-47ab-8e75-5b30d0a3838b vif=[] _unp 2015-12-05 02:57:43.376 7649 ERROR nova.compute.manager [req-25ba4c81-4333-43e4-b8d9-e460827435e0 a989fe7fc89e4825b98d7e6584cc 1308f1a382ef43c79eeed0ebf8a9db3b] [instance: d9295d4d-1104-47ab-8e75-5b30d0a3838b] Instance failed to spawn 2015-12-05 02:57:43.376 7649 ERROR nova.compute.manager [instance: d9295d4d-1104-47ab-8e75-5b30d0a3838b] Traceback (most recent call last): 2015-12-05 02:57:43.376 7649 ERROR nova.compute.manager [instance: d9295d4d-1104-47ab-8e75-5b30d0a3838b] File "/opt/stack/venv/nova-20151204T044743Z/lib/python2.7/site-packages/nova/compute/manager.py", line 2157, in _build_resources 2015-12-05 02:57:43.376 7649 ERROR nova.compute.manager [instance: d9295d4d-1104-47ab-8e75-5b30d0a3838b] yield resources 2015-12-05 02:57:43.376 7649 ERROR nova.compute.manager [instance: d9295d4d-1104-47ab-8e75-5b30d0a3838b] File "/opt/stack/venv/nova-20151204T044743Z/lib/python2.7/site-packages/nova/compute/manager.py", line 2011, in _build_and_run_inst 2015-12-05 02:57:43.376 7649 ERROR nova.compute.manager [instance: d9295d4d-1104-47ab-8e75-5b30d0a3838b] block_device_info=block_device_info) 2015-12-05 02:57:43.376 7649 ERROR nova.compute.manager [instance: d9295d4d-1104-47ab-8e75-5b30d0a3838b] File
[Yahoo-eng-team] [Bug 1527061] Re: Nova should not throw exception when port binding fails for Ironic
hi Johnson,
actually, I don't know well about VMware VCDriver, and have no idea no
whey Neutron return a port with {'binding:vif_type': binding_failed} ,
in case this happened, nova should though it as a bad port and is not
usable. I personal suggest that we connect Neutron team to take care
about this.
Eli.
** Project changed: nova => neutron
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1527061
Title:
Nova should not throw exception when port binding fails for Ironic
Status in neutron:
Confirmed
Bug description:
Neutron tries to bind port on compute where instance is launched. It
doesn't make sense when hypervisor_type is ironic, since VM does not
live on hypervisor in this case. Furthermore it leads to failed
provisioning of baremetal node, when neutron is not configured on
ironic compute node.
When binding fails, nova starts cleanup and then throws exception. As
part of cleanup ironic port is deleted. This should not be the case
when virt-driver being used is Ironic.
Nova versions: liberty on-wards.
Setup:
node-1: controller with neutron
node-2: ironic-compute without neutron/neutron agents
Register a BM node and perform nova boot.
From Nova:
=
nova-compute.log:21147:2015-12-05 02:57:38.542 7649 ERROR
nova.compute.manager [-] Instance failed network setup after 1 attempt(s)
nova-compute.log:21148:2015-12-05 02:57:38.542 7649 ERROR
nova.compute.manager Traceback (most recent call last):
nova-compute.log:21149:2015-12-05 02:57:38.542 7649 ERROR
nova.compute.manager File
"/opt/stack/venv/nova-20151204T044743Z/lib/python2.7/site-packages/nova/compute/manager.py",
line 1564, in _allocate_network_async
nova-compute.log:21150:2015-12-05 02:57:38.542 7649 ERROR
nova.compute.manager dhcp_options=dhcp_options)
nova-compute.log:21151:2015-12-05 02:57:38.542 7649 ERROR
nova.compute.manager File
"/opt/stack/venv/nova-20151204T044743Z/lib/python2.7/site-packages/nova/network/neutronv2/api.py",
line 727, in allocate_for_instance
nova-compute.log:21152:2015-12-05 02:57:38.542 7649 ERROR
nova.compute.manager self._delete_ports(neutron, instance, created_port_ids)
nova-compute.log:21153:2015-12-05 02:57:38.542 7649 ERROR
nova.compute.manager File
"/opt/stack/venv/nova-20151204T044743Z/lib/python2.7/site-packages/oslo_utils/excutils.py",
line 204, in __exit__
nova-compute.log:21154:2015-12-05 02:57:38.542 7649 ERROR
nova.compute.manager six.reraise(self.type_, self.value, self.tb)
nova-compute.log:21155:2015-12-05 02:57:38.542 7649 ERROR
nova.compute.manager File
"/opt/stack/venv/nova-20151204T044743Z/lib/python2.7/site-packages/nova/network/neutronv2/api.py",
line 719, in allocate_for_instance
nova-compute.log:21156:2015-12-05 02:57:38.542 7649 ERROR
nova.compute.manager security_group_ids, available_macs, dhcp_opts)
nova-compute.log:21157:2015-12-05 02:57:38.542 7649 ERROR
nova.compute.manager File
"/opt/stack/venv/nova-20151204T044743Z/lib/python2.7/site-packages/nova/network/neutronv2/api.py",
line 342, in _create_port
nova-compute.log:21158:2015-12-05 02:57:38.542 7649 ERROR
nova.compute.manager raise exception.PortBindingFailed(port_id=port_id)
nova-compute.log:21159:2015-12-05 02:57:38.542 7649 ERROR
nova.compute.manager PortBindingFailed: Binding failed for port
0514b7e8-0408-4d5e-9b48-8292e686494f, please check neutron logs for more
information.
nova-compute.log:21160:2015-12-05 02:57:38.542 7649 ERROR nova.compute.manager
2015-12-05 02:57:43.375 7649 DEBUG nova.virt.ironic.driver
[req-25ba4c81-4333-43e4-b8d9-e460827435e0 a989fe7fc89e4825b98d7e6584cc
1308f1a382ef43c79eeed0ebf8a9db3b] unplug:
instance_uuid=d9295d4d-1104-47ab-8e75-5b30d0a3838b vif=[] _unp
2015-12-05 02:57:43.376 7649 ERROR nova.compute.manager
[req-25ba4c81-4333-43e4-b8d9-e460827435e0 a989fe7fc89e4825b98d7e6584cc
1308f1a382ef43c79eeed0ebf8a9db3b] [instance:
d9295d4d-1104-47ab-8e75-5b30d0a3838b] Instance failed to spawn
2015-12-05 02:57:43.376 7649 ERROR nova.compute.manager [instance:
d9295d4d-1104-47ab-8e75-5b30d0a3838b] Traceback (most recent call last):
2015-12-05 02:57:43.376 7649 ERROR nova.compute.manager [instance:
d9295d4d-1104-47ab-8e75-5b30d0a3838b] File
"/opt/stack/venv/nova-20151204T044743Z/lib/python2.7/site-packages/nova/compute/manager.py",
line 2157, in _build_resources
2015-12-05 02:57:43.376 7649 ERROR nova.compute.manager [instance:
d9295d4d-1104-47ab-8e75-5b30d0a3838b] yield resources
2015-12-05 02:57:43.376 7649 ERROR nova.compute.manager [instance:
d9295d4d-1104-47ab-8e75-5b30d0a3838b] File
"/opt/stack/venv/nova-20151204T044743Z/lib/python2.7/site-packages/nova/compute/manager.py",
line 2011, in _build_and_run_inst
2015-12-05 02:57:43.376 7649 ERROR nova.compute.manager [instance:
