Reviewed: https://review.openstack.org/603844 Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=542635034882e1b6897e1935f09d6feb6e77d1ce Submitter: Zuul Branch: master
commit 542635034882e1b6897e1935f09d6feb6e77d1ce Author: Jack Ding <[email protected]> Date: Wed Sep 19 11:54:44 2018 -0400 Correct instance port binding for rebuilds The following 2 scenarios could result in an instance with incorrect port binding and cause subsequent rebuilds to fail. If an evacuation of an instance fails part way through, after the point where we reassign the port binding to the new host but before we change the instance host, we end up with the ports assigned to the wrong host. This change adds a check to determine if there's any port binding host mismatches and if so trigger setup of instance network. During recovery of failed hosts, neutron could get overwhelmed and lose messages, for example when active controller was powered-off in the middle of instance evacuations. In this case the vif_type was set to 'binding_failed' or 'unbound'. We subsequently hit "Unsupported VIF type" exception during instance hard_reboot or rebuild, leaving the instance unrecoverable. This commit changes _heal_instance_info_cache periodic task to update port binding if evacuation fails due to above errors so that the instance can be recovered later. Closes-Bug: #1659062 Related-Bug: #1784579 Co-Authored-By: Gerry Kopec <[email protected]> Co-Authored-By: Jim Gauld <[email protected]> Change-Id: I75fd15ac2a29e420c09499f2c41d11259ca811ae Signed-off-by: Jack Ding <[email protected]> ** Changed in: nova Status: In Progress => Fix Released -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1659062 Title: Failed evacuations leave neutron ports on destination host Status in OpenStack Compute (nova): Fix Released Bug description: Description =========== This is related to https://bugs.launchpad.net/nova/+bug/1430042 and the associated fix https://review.openstack.org/#/c/169827/; if an evacuation fails there is no reverting of the neutron ports' host_id binding back to the source host. This may or may not be a bug, but if the evacuation fails and the source host comes back up and VMs are expected to be running, then the neutron ports should probably be rolled back. Steps to reproduce ================== * Raise an exception at some point in the evacuation flow after the setup_instance_network_on_host calls in _do_rebuild_instance in the manager * Issue an evacuation of a VM to the host that will fail Expected result =============== * If the evacuation fails the expectation would be to have the neutron ports have their host_id binding updated to be the source host. Actual result ============= * The ports host_id bindings remain as the destination host. Environment =========== 1. Exact version of OpenStack you are running. See the following Newton 2. Which hypervisor did you use? PowerVM 2. Which storage type did you use? N/A 3. Which networking type did you use? Neutron with SEA To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1659062/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : [email protected] Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
