Reviewed: https://review.openstack.org/393805 Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=9345ed40e85e291d2f9e9aac24d468f40140ae0e Submitter: Jenkins Branch: master
commit 9345ed40e85e291d2f9e9aac24d468f40140ae0e Author: Brent Tang <[email protected]> Date: Fri Nov 4 07:35:25 2016 -0700 Instance obj_clone leaves metadata as changed When performing an obj_clone on an Instance object, it relies on the base obj_clone's deep copy method which just goes through all of the fields and duplicates them on the clone. However, the Instance object has internal tracking attributes for metadata and system_metadata that keep track of which keys have changed. Since these don't get copied over on the deep copy, these will show up as having changed on the cloned Instance (if they had metadata set originally) and on a save on this cloned object will end up updating the metadata in the db which if these have stale information will wipe out other changes. Such a scenario to this is occurring during build_instance where the Claim constructor is saving off a clone of the Instance object so that it will have the current copy. In the case of a failure during the build_instance, the claim will use this current copy to then set the value of the host on the instance to None. Since the clone reflects that the system_metadata has changed, it will as part of the save try to update the system_metadata in the db, which since it is stale will wipe out changes made. In the case of the issue being fixed here, the value in the system_metdata being wiped out is the network_allocated=True attribute. By this being wiped out then reschedules of that build_instance fail in cases where the IP address has been assigned to the instance. Change-Id: Ie49a90993fb9643c04d75402dbaaa933c6b5222e Closes-Bug: #1639230 ** Changed in: nova Status: In Progress => Fix Released -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1639230 Title: reschedule fails with ip already allocated error Status in OpenStack Compute (nova): Fix Released Status in OpenStack Compute (nova) newton series: New Bug description: Tried to create a server in a multi-host environment. The create failed on the first host that was attempted due to a ClientException raised by nova.volume.cinder.API.initialize_connection while trying to attach a volume. When the build was rescheduled on a different host, it should have realized that the network was already allocated by the first attempt and reused that, but the network_allocated=True from instance.system_metadata somehow disappeared, leading to the following exception that causes the reschedule to fail: 2016-10-13 04:48:29.007 16273 WARNING nova.network.neutronv2.api [req-9b343ef7-e8d9-4a61-b86c-a61908afe4df 0688b01e6439ca32d698d20789d52169126fb41fb1a4ddafcebb97d854e836c9 94e1baed634145e0aade858973ae88e8 - - -] [instance: b85d6c6c-e385-4601-aa47-5c580f893c9b] Neutron error creating port on network 5038a36b-cb1e-4a61-b26c-a05a80b37ed6 2016-10-13 04:48:29.007 16273 ERROR nova.network.neutronv2.api [instance: b85d6c6c-e385-4601-aa47-5c580f893c9b] Traceback (most recent call last): 2016-10-13 04:48:29.007 16273 ERROR nova.network.neutronv2.api [instance: b85d6c6c-e385-4601-aa47-5c580f893c9b] File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 392, in _create_port_minimal 2016-10-13 04:48:29.007 16273 ERROR nova.network.neutronv2.api [instance: b85d6c6c-e385-4601-aa47-5c580f893c9b] port_response = port_client.create_port(port_req_body) 2016-10-13 04:48:29.007 16273 ERROR nova.network.neutronv2.api [instance: b85d6c6c-e385-4601-aa47-5c580f893c9b] File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 98, in wrapper 2016-10-13 04:48:29.007 16273 ERROR nova.network.neutronv2.api [instance: b85d6c6c-e385-4601-aa47-5c580f893c9b] ret = obj(*args, **kwargs) 2016-10-13 04:48:29.007 16273 ERROR nova.network.neutronv2.api [instance: b85d6c6c-e385-4601-aa47-5c580f893c9b] File "/usr/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 750, in create_port 2016-10-13 04:48:29.007 16273 ERROR nova.network.neutronv2.api [instance: b85d6c6c-e385-4601-aa47-5c580f893c9b] return self.post(self.ports_path, body=body) 2016-10-13 04:48:29.007 16273 ERROR nova.network.neutronv2.api [instance: b85d6c6c-e385-4601-aa47-5c580f893c9b] File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 98, in wrapper 2016-10-13 04:48:29.007 16273 ERROR nova.network.neutronv2.api [instance: b85d6c6c-e385-4601-aa47-5c580f893c9b] ret = obj(*args, **kwargs) 2016-10-13 04:48:29.007 16273 ERROR nova.network.neutronv2.api [instance: b85d6c6c-e385-4601-aa47-5c580f893c9b] File "/usr/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 365, in post 2016-10-13 04:48:29.007 16273 ERROR nova.network.neutronv2.api [instance: b85d6c6c-e385-4601-aa47-5c580f893c9b] headers=headers, params=params) 2016-10-13 04:48:29.007 16273 ERROR nova.network.neutronv2.api [instance: b85d6c6c-e385-4601-aa47-5c580f893c9b] File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 98, in wrapper 2016-10-13 04:48:29.007 16273 ERROR nova.network.neutronv2.api [instance: b85d6c6c-e385-4601-aa47-5c580f893c9b] ret = obj(*args, **kwargs) 2016-10-13 04:48:29.007 16273 ERROR nova.network.neutronv2.api [instance: b85d6c6c-e385-4601-aa47-5c580f893c9b] File "/usr/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 300, in do_request 2016-10-13 04:48:29.007 16273 ERROR nova.network.neutronv2.api [instance: b85d6c6c-e385-4601-aa47-5c580f893c9b] self._handle_fault_response(status_code, replybody, resp) 2016-10-13 04:48:29.007 16273 ERROR nova.network.neutronv2.api [instance: b85d6c6c-e385-4601-aa47-5c580f893c9b] File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 98, in wrapper 2016-10-13 04:48:29.007 16273 ERROR nova.network.neutronv2.api [instance: b85d6c6c-e385-4601-aa47-5c580f893c9b] ret = obj(*args, **kwargs) 2016-10-13 04:48:29.007 16273 ERROR nova.network.neutronv2.api [instance: b85d6c6c-e385-4601-aa47-5c580f893c9b] File "/usr/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 275, in _handle_fault_response 2016-10-13 04:48:29.007 16273 ERROR nova.network.neutronv2.api [instance: b85d6c6c-e385-4601-aa47-5c580f893c9b] exception_handler_v20(status_code, error_body) 2016-10-13 04:48:29.007 16273 ERROR nova.network.neutronv2.api [instance: b85d6c6c-e385-4601-aa47-5c580f893c9b] File "/usr/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 91, in exception_handler_v20 2016-10-13 04:48:29.007 16273 ERROR nova.network.neutronv2.api [instance: b85d6c6c-e385-4601-aa47-5c580f893c9b] request_ids=request_ids) 2016-10-13 04:48:29.007 16273 ERROR nova.network.neutronv2.api [instance: b85d6c6c-e385-4601-aa47-5c580f893c9b] Conflict: IP address 9.114.25.215 already allocated in subnet e2d1b84e-2308-48ab-9101-0d48721c9fd2 2016-10-13 04:48:29.007 16273 ERROR nova.network.neutronv2.api [instance: b85d6c6c-e385-4601-aa47-5c580f893c9b] Neutron server returns request_ids: ['req-682b2d6b-768f-413d-862c-32490cad5589'] 2016-10-13 04:48:29.007 16273 ERROR nova.network.neutronv2.api [instance: b85d6c6c-e385-4601-aa47-5c580f893c9b] found in newton To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1639230/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : [email protected] Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
