Reviewed: https://review.openstack.org/406627 Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=f0d44c5b09f3f3c84038d40b621bb629a1f8110e Submitter: Jenkins Branch: master
commit f0d44c5b09f3f3c84038d40b621bb629a1f8110e Author: Matt Riedemann <[email protected]> Date: Sun Dec 4 15:08:04 2016 -0500 Handle ComputeHostNotFound when listing hypervisors Compute node resources must currently be deleted manually in the database, and as such they can reference service records which have been deleted via the services delete API. Because of this when listing hypervisors (compute nodes), we may get a ComputeHostNotFound error when trying to lookup a service record for a compute node where the service was deleted. This causes the API to fail with a 500 since it's not handled. This change handles the ComputeHostNotFound when looping over compute nodes in the hypervisors index and detail methods and simply ignores them. Change-Id: I2717274bb1bd370870acbf58c03dc59cee30cc5e Closes-Bug: #1646255 ** Changed in: nova Status: In Progress => Fix Released -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1646255 Title: removing compute node causes ComputeHostNotFound in nova-api Status in OpenStack Compute (nova): Fix Released Status in OpenStack Compute (nova) newton series: Confirmed Bug description: trying to remove compute node properly Steps to reproduce ================== 1) remove all instances from the hypervisor: (env) vance@zs95k5:~$ nova hypervisor-servers zs93k23 +----+------+---------------+---------------------+ | ID | Name | Hypervisor ID | Hypervisor Hostname | +----+------+---------------+---------------------+ +----+------+---------------+---------------------+ 2) disable the hypervisor: (env) vance@zs95k5:~$ nova service-list +----+----------------+---------------------+----------+----------+-------+----------------------------+-----------------+ | Id | Binary | Host | Zone | Status | State | Updated_at | Disabled Reason | +----+----------------+---------------------+----------+----------+-------+----------------------------+-----------------+ | 3 | nova-cert | juju-605709-2-lxd-3 | internal | enabled | up | 2016-11-30T21:13:34.000000 | - | | 4 | nova-scheduler | juju-605709-2-lxd-3 | internal | enabled | up | 2016-11-30T21:13:27.000000 | - | | 5 | nova-conductor | juju-605709-2-lxd-3 | internal | enabled | up | 2016-11-30T21:13:30.000000 | - | | 14 | nova-compute | u27-maas-machine-1 | nova | disabled | up | 2016-11-30T21:13:28.000000 | - | | 16 | nova-compute | zs95k181 | nova | enabled | up | 2016-11-30T21:13:33.000000 | - | | 17 | nova-compute | zs93k23 | nova | enabled | up | 2016-11-30T21:13:33.000000 | - | +----+----------------+---------------------+----------+----------+-------+----------------------------+-----------------+ (env) vance@zs95k5:~$ nova service-disable zs93k23 nova-compute +---------+--------------+----------+ | Host | Binary | Status | +---------+--------------+----------+ | zs93k23 | nova-compute | disabled | +---------+--------------+----------+ 3) delete the compute service (env) vance@zs95k5:~$ nova service-delete 17 (env) vance@zs95k5:~$ nova service-list +----+----------------+---------------------+----------+----------+-------+----------------------------+-----------------+ | Id | Binary | Host | Zone | Status | State | Updated_at | Disabled Reason | +----+----------------+---------------------+----------+----------+-------+----------------------------+-----------------+ | 3 | nova-cert | juju-605709-2-lxd-3 | internal | enabled | up | 2016-11-30T21:14:54.000000 | - | | 4 | nova-scheduler | juju-605709-2-lxd-3 | internal | enabled | up | 2016-11-30T21:14:47.000000 | - | | 5 | nova-conductor | juju-605709-2-lxd-3 | internal | enabled | up | 2016-11-30T21:14:56.000000 | - | | 14 | nova-compute | u27-maas-machine-1 | nova | disabled | up | 2016-11-30T21:14:48.000000 | - | | 16 | nova-compute | zs95k181 | nova | enabled | up | 2016-11-30T21:14:53.000000 | - | +----+----------------+---------------------+----------+----------+-------+----------------------------+-----------------+ 4) delete the neutron agent (env) vance@zs95k5:~$ openstack network agent list +--------------------------------------+--------------------+--------------------+-------------------+-------+-------+---------------------------+ | ID | Agent Type | Host | Availability Zone | Alive | State | Binary | +--------------------------------------+--------------------+--------------------+-------------------+-------+-------+---------------------------+ | 039e4b7a-3dbe-4e87-a9a5-d4b569d3113d | Open vSwitch agent | u27-maas-machine-2 | None | True | UP | neutron-openvswitch-agent | | 2aa13570-0e62-4198-96d9-dfe732d7874d | DHCP agent | u27-maas-machine-2 | nova | True | UP | neutron-dhcp-agent | | 2ab2320a-69cb-4a7f-8ae4-541d3b2bdd3b | L3 agent | u27-maas-machine-2 | nova | True | UP | neutron-l3-agent | | 48d6d83b-e459-46c8-945c-eea4197c01ec | Open vSwitch agent | zs95k181 | None | True | UP | neutron-openvswitch-agent | | a36eecd5-1fb4-436a-becd-fccc737518fd | Metering agent | u27-maas-machine-2 | None | True | UP | neutron-metering-agent | | aaee3bf0-f8bd-41b7-94ed-f9213f120016 | Open vSwitch agent | zs93k23 | None | True | UP | neutron-openvswitch-agent | | c6039c81-ad20-4258-a926-bc4a90dccc96 | Loadbalancer agent | u27-maas-machine-2 | None | True | UP | neutron-lbaas-agent | | cfecc66c-2888-4c3d-8241-1d3fcd018a1f | Metadata agent | u27-maas-machine-2 | None | True | UP | neutron-metadata-agent | | f60cbf28-f030-43ae-a598-0d0182529804 | Open vSwitch agent | u27-maas-machine-1 | None | True | UP | neutron-openvswitch-agent | +--------------------------------------+--------------------+--------------------+-------------------+-------+-------+---------------------------+ (env) vance@zs95k5:~$ openstack network agent delete aaee3bf0-f8bd-41b7-94ed-f9213f120016 (env) vance@zs95k5:~$ openstack network agent list +--------------------------------------+--------------------+--------------------+-------------------+-------+-------+---------------------------+ | ID | Agent Type | Host | Availability Zone | Alive | State | Binary | +--------------------------------------+--------------------+--------------------+-------------------+-------+-------+---------------------------+ | 039e4b7a-3dbe-4e87-a9a5-d4b569d3113d | Open vSwitch agent | u27-maas-machine-2 | None | True | UP | neutron-openvswitch-agent | | 2aa13570-0e62-4198-96d9-dfe732d7874d | DHCP agent | u27-maas-machine-2 | nova | True | UP | neutron-dhcp-agent | | 2ab2320a-69cb-4a7f-8ae4-541d3b2bdd3b | L3 agent | u27-maas-machine-2 | nova | True | UP | neutron-l3-agent | | 48d6d83b-e459-46c8-945c-eea4197c01ec | Open vSwitch agent | zs95k181 | None | True | UP | neutron-openvswitch-agent | | a36eecd5-1fb4-436a-becd-fccc737518fd | Metering agent | u27-maas-machine-2 | None | True | UP | neutron-metering-agent | | c6039c81-ad20-4258-a926-bc4a90dccc96 | Loadbalancer agent | u27-maas-machine-2 | None | True | UP | neutron-lbaas-agent | | cfecc66c-2888-4c3d-8241-1d3fcd018a1f | Metadata agent | u27-maas-machine-2 | None | True | UP | neutron-metadata-agent | | f60cbf28-f030-43ae-a598-0d0182529804 | Open vSwitch agent | u27-maas-machine-1 | None | True | UP | neutron-openvswitch-agent | +--------------------------------------+--------------------+--------------------+-------------------+-------+-------+---------------------------+ 5) check hypervisor list (env) vance@zs95k5:~$ nova hypervisor-list ERROR (ClientException): Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible. <class 'nova.exception.ComputeHostNotFound'> (HTTP 500) (Request-ID: req-2b773323-be4f-4c8c-be05-a41144ebed76) Okay, here is the error from the log: http://paste.ubuntu.com/23559892/ Expected result =============== Nova compute node is properly removed from the OS database. Environment =========== 1. OpenStack nova version 2:13.1.2-0ubuntu2 2. Hypervisor: Libvirt+KVM (KVM for IBM z Systems) 3. Networking type: OVS+GRE To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1646255/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : [email protected] Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
