Public bug reported: Environment: Stable/Liberty with neutron ML2 (Mechanism Driver is a custom asynchronous driver based off the Opendaylight V2 driver) No agents or OVS bridges in use. One controller and network node and two compute nodes.
When a VM fails to start on any compute node, nova removes the host
binding from the VM but doesn't send a port update to neutron notifying
it about the hostbinding change.
When the same error state VM is deleted, nova doesn't send any events to
neutron. As a result the driver thinks the port is still active, owned
by an existing VM and bound to the host properly.
NOTE: This behavior is only seen with VM's in the ERROR state, ports for
VMs in ACTIVE state are deleted properly.
$ nova show vm1
+--------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Property | Value
|
+--------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| OS-DCF:diskConfig | AUTO
|
| OS-EXT-AZ:availability_zone | nova
|
| OS-EXT-SRV-ATTR:host | -
|
| OS-EXT-SRV-ATTR:hostname | vm1
|
| OS-EXT-SRV-ATTR:hypervisor_hostname | -
|
| OS-EXT-SRV-ATTR:instance_name | instance-0000077a
|
| OS-EXT-SRV-ATTR:kernel_id | 72d053dc-aae8-4559-a7c4-107c980bd674
|
| OS-EXT-SRV-ATTR:launch_index | 0
|
| OS-EXT-SRV-ATTR:ramdisk_id | 3d25fd54-ffbc-4370-b971-2196a7963c24
|
| OS-EXT-SRV-ATTR:reservation_id | r-pujfofv9
|
| OS-EXT-SRV-ATTR:root_device_name | /dev/vda
|
| OS-EXT-SRV-ATTR:user_data | -
|
| OS-EXT-STS:power_state | 0
|
| OS-EXT-STS:task_state | -
|
| OS-EXT-STS:vm_state | error
|
| OS-SRV-USG:launched_at | -
|
| OS-SRV-USG:terminated_at | -
|
| accessIPv4 |
|
| accessIPv6 |
|
| config_drive |
|
| created | 2016-06-20T22:51:13Z
|
| fault | {"message": "No valid host was found.
There are not enough hosts available.", "code": 500, "details": " File
\"/opt/stack/nova/nova/conductor/manager.py\", line 739, in build_instances
|
| flavor | m1.small (2)
|
| hostId |
|
| id | 5d37f0b1-d3ac-4246-badc-b1a2aa6c3137
|
| image | cirros-0.3.4-x86_64-uec
(50976ce1-c00e-4a80-9d72-6bf5d1abe54a)
|
| key_name | -
|
| metadata | {}
|
| name | vm1
|
| net1 network | 30.0.0.231
|
| os-extended-volumes:volumes_attached | []
|
| security_groups | default
|
| status | ERROR
|
| tenant_id | 4c3ddd212d4f4af9b4d0d55e89b43125
|
| updated | 2016-06-20T22:51:23Z
|
| user_id | 1e3e7e3addc14861afe177f285e43b40
|
+--------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
$ neutron port-show 5efcfa1b-1300-4194-8229-11863f85d0a9
+-----------------------+-------------------------------------------------------------------------------------------------------------------------------+
| Field | Value
|
+-----------------------+-------------------------------------------------------------------------------------------------------------------------------+
| admin_state_up | True
|
| allowed_address_pairs |
|
| binding:host_id | bxb-ds-50
|
| binding:profile | {}
|
| binding:vif_details | {"port_filter": false, "vhostuser_ovs_plug": false,
"vhostuser_socket": "/tmp/sock-fa163eaba0d2", "vhostuser_mode": "server"} |
| binding:vif_type | vhostuser
|
| binding:vnic_type | normal
|
| device_id | 5d37f0b1-d3ac-4246-badc-b1a2aa6c3137
|
| device_owner | compute:nova
|
| dns_assignment | {"hostname": "host-30-0-0-231", "ip_address":
"30.0.0.231", "fqdn": "host-30-0-0-231.openstacklocal."}
|
| dns_name |
|
| extra_dhcp_opts |
|
| fixed_ips | {"subnet_id": "880aac1b-662e-4a4f-b640-7929d69a2b3d",
"ip_address": "30.0.0.231"} |
| id | 5efcfa1b-1300-4194-8229-11863f85d0a9
|
| mac_address | fa:16:3e:ab:a0:d2
|
| name |
|
| network_id | cf31ce68-4c5b-4e22-b725-dff0b6511a3d
|
| port_security_enabled | True
|
| security_groups | ae59a53e-38f2-471d-9de9-b8aea815dc27
|
| status | ACTIVE
|
| tenant_id | 4c3ddd212d4f4af9b4d0d55e89b43125
|
+-----------------------+-------------------------------------------------------------------------------------------------------------------------------+
$ nova delete vm1
Request to delete server vm1 has been accepted.
$ nova show vm1
ERROR (CommandError): No server with a name or ID of 'vm1' exists.
$ neutron port-show 5efcfa1b-1300-4194-8229-11863f85d0a9
+-----------------------+-------------------------------------------------------------------------------------------------------------------------------+
| Field | Value
|
+-----------------------+-------------------------------------------------------------------------------------------------------------------------------+
| admin_state_up | True
|
| allowed_address_pairs |
|
| binding:host_id | bxb-ds-50
|
| binding:profile | {}
|
| binding:vif_details | {"port_filter": false, "vhostuser_ovs_plug": false,
"vhostuser_socket": "/tmp/sock-fa163eaba0d2", "vhostuser_mode": "server"} |
| binding:vif_type | vhostuser
|
| binding:vnic_type | normal
|
| device_id | 5d37f0b1-d3ac-4246-badc-b1a2aa6c3137
|
| device_owner | compute:nova
|
| dns_assignment | {"hostname": "host-30-0-0-231", "ip_address":
"30.0.0.231", "fqdn": "host-30-0-0-231.openstacklocal."}
|
| dns_name |
|
| extra_dhcp_opts |
|
| fixed_ips | {"subnet_id": "880aac1b-662e-4a4f-b640-7929d69a2b3d",
"ip_address": "30.0.0.231"} |
| id | 5efcfa1b-1300-4194-8229-11863f85d0a9
|
| mac_address | fa:16:3e:ab:a0:d2
|
| name |
|
| network_id | cf31ce68-4c5b-4e22-b725-dff0b6511a3d
|
| port_security_enabled | True
|
| security_groups | ae59a53e-38f2-471d-9de9-b8aea815dc27
|
| status | ACTIVE
|
| tenant_id | 4c3ddd212d4f4af9b4d0d55e89b43125
|
+-----------------------+-------------------------------------------------------------------------------------------------------------------------------+
** Affects: nova
Importance: Undecided
Status: New
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1594604
Title:
Nova does not delete or update neutron port for failed VMs
Status in OpenStack Compute (nova):
New
Bug description:
Environment:
Stable/Liberty with neutron ML2 (Mechanism Driver is a custom asynchronous
driver based off the Opendaylight V2 driver)
No agents or OVS bridges in use.
One controller and network node and two compute nodes.
When a VM fails to start on any compute node, nova removes the host
binding from the VM but doesn't send a port update to neutron
notifying it about the hostbinding change.
When the same error state VM is deleted, nova doesn't send any events
to neutron. As a result the driver thinks the port is still active,
owned by an existing VM and bound to the host properly.
NOTE: This behavior is only seen with VM's in the ERROR state, ports
for VMs in ACTIVE state are deleted properly.
$ nova show vm1
+--------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Property | Value
|
+--------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| OS-DCF:diskConfig | AUTO
|
| OS-EXT-AZ:availability_zone | nova
|
| OS-EXT-SRV-ATTR:host | -
|
| OS-EXT-SRV-ATTR:hostname | vm1
|
| OS-EXT-SRV-ATTR:hypervisor_hostname | -
|
| OS-EXT-SRV-ATTR:instance_name | instance-0000077a
|
| OS-EXT-SRV-ATTR:kernel_id | 72d053dc-aae8-4559-a7c4-107c980bd674
|
| OS-EXT-SRV-ATTR:launch_index | 0
|
| OS-EXT-SRV-ATTR:ramdisk_id | 3d25fd54-ffbc-4370-b971-2196a7963c24
|
| OS-EXT-SRV-ATTR:reservation_id | r-pujfofv9
|
| OS-EXT-SRV-ATTR:root_device_name | /dev/vda
|
| OS-EXT-SRV-ATTR:user_data | -
|
| OS-EXT-STS:power_state | 0
|
| OS-EXT-STS:task_state | -
|
| OS-EXT-STS:vm_state | error
|
| OS-SRV-USG:launched_at | -
|
| OS-SRV-USG:terminated_at | -
|
| accessIPv4 |
|
| accessIPv6 |
|
| config_drive |
|
| created | 2016-06-20T22:51:13Z
|
| fault | {"message": "No valid host was
found. There are not enough hosts available.", "code": 500, "details": " File
\"/opt/stack/nova/nova/conductor/manager.py\", line 739, in build_instances
|
| flavor | m1.small (2)
|
| hostId |
|
| id | 5d37f0b1-d3ac-4246-badc-b1a2aa6c3137
|
| image | cirros-0.3.4-x86_64-uec
(50976ce1-c00e-4a80-9d72-6bf5d1abe54a)
|
| key_name | -
|
| metadata | {}
|
| name | vm1
|
| net1 network | 30.0.0.231
|
| os-extended-volumes:volumes_attached | []
|
| security_groups | default
|
| status | ERROR
|
| tenant_id | 4c3ddd212d4f4af9b4d0d55e89b43125
|
| updated | 2016-06-20T22:51:23Z
|
| user_id | 1e3e7e3addc14861afe177f285e43b40
|
+--------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
$ neutron port-show 5efcfa1b-1300-4194-8229-11863f85d0a9
+-----------------------+-------------------------------------------------------------------------------------------------------------------------------+
| Field | Value
|
+-----------------------+-------------------------------------------------------------------------------------------------------------------------------+
| admin_state_up | True
|
| allowed_address_pairs |
|
| binding:host_id | bxb-ds-50
|
| binding:profile | {}
|
| binding:vif_details | {"port_filter": false, "vhostuser_ovs_plug": false,
"vhostuser_socket": "/tmp/sock-fa163eaba0d2", "vhostuser_mode": "server"} |
| binding:vif_type | vhostuser
|
| binding:vnic_type | normal
|
| device_id | 5d37f0b1-d3ac-4246-badc-b1a2aa6c3137
|
| device_owner | compute:nova
|
| dns_assignment | {"hostname": "host-30-0-0-231", "ip_address":
"30.0.0.231", "fqdn": "host-30-0-0-231.openstacklocal."}
|
| dns_name |
|
| extra_dhcp_opts |
|
| fixed_ips | {"subnet_id":
"880aac1b-662e-4a4f-b640-7929d69a2b3d", "ip_address": "30.0.0.231"}
|
| id | 5efcfa1b-1300-4194-8229-11863f85d0a9
|
| mac_address | fa:16:3e:ab:a0:d2
|
| name |
|
| network_id | cf31ce68-4c5b-4e22-b725-dff0b6511a3d
|
| port_security_enabled | True
|
| security_groups | ae59a53e-38f2-471d-9de9-b8aea815dc27
|
| status | ACTIVE
|
| tenant_id | 4c3ddd212d4f4af9b4d0d55e89b43125
|
+-----------------------+-------------------------------------------------------------------------------------------------------------------------------+
$ nova delete vm1
Request to delete server vm1 has been accepted.
$ nova show vm1
ERROR (CommandError): No server with a name or ID of 'vm1' exists.
$ neutron port-show 5efcfa1b-1300-4194-8229-11863f85d0a9
+-----------------------+-------------------------------------------------------------------------------------------------------------------------------+
| Field | Value
|
+-----------------------+-------------------------------------------------------------------------------------------------------------------------------+
| admin_state_up | True
|
| allowed_address_pairs |
|
| binding:host_id | bxb-ds-50
|
| binding:profile | {}
|
| binding:vif_details | {"port_filter": false, "vhostuser_ovs_plug": false,
"vhostuser_socket": "/tmp/sock-fa163eaba0d2", "vhostuser_mode": "server"} |
| binding:vif_type | vhostuser
|
| binding:vnic_type | normal
|
| device_id | 5d37f0b1-d3ac-4246-badc-b1a2aa6c3137
|
| device_owner | compute:nova
|
| dns_assignment | {"hostname": "host-30-0-0-231", "ip_address":
"30.0.0.231", "fqdn": "host-30-0-0-231.openstacklocal."}
|
| dns_name |
|
| extra_dhcp_opts |
|
| fixed_ips | {"subnet_id":
"880aac1b-662e-4a4f-b640-7929d69a2b3d", "ip_address": "30.0.0.231"}
|
| id | 5efcfa1b-1300-4194-8229-11863f85d0a9
|
| mac_address | fa:16:3e:ab:a0:d2
|
| name |
|
| network_id | cf31ce68-4c5b-4e22-b725-dff0b6511a3d
|
| port_security_enabled | True
|
| security_groups | ae59a53e-38f2-471d-9de9-b8aea815dc27
|
| status | ACTIVE
|
| tenant_id | 4c3ddd212d4f4af9b4d0d55e89b43125
|
+-----------------------+-------------------------------------------------------------------------------------------------------------------------------+
To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1594604/+subscriptions
--
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : [email protected]
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help : https://help.launchpad.net/ListHelp
