Reviewed: https://review.openstack.org/25521 Committed: http://github.com/openstack/nova/commit/1fa7879071310fda7d8393716441dd02be38fc30 Submitter: Jenkins Branch: milestone-proposed
commit 1fa7879071310fda7d8393716441dd02be38fc30 Author: Dan Smith <[email protected]> Date: Tue Mar 26 13:14:56 2013 -0700 Always store old instance_type during a migration Previously, we were trying to optimize for the migration (as opposed to resize) case by not storing the old_ instance_type. However, this is confusing to the vast amount of shared code that works on migrations and resizes because the "is old == new" logic must be replicated everywhere. To cut our losses and hopefully avoid other hidden dragons, this patch removes the optimization and makes both paths the same. Fixes bug #1160489 Change-Id: I2ad7b02239baada5fd2174f5b61cdc90ec91e6f5 (cherry picked from commit 985d703d2731fe9ec968ef6698bbf39743ede32a) ** Changed in: nova Status: Fix Committed => Fix Released -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1160489 Title: periodically seeing KeyError: 'old_instance_type_memory_mb' in compute.log Status in OpenStack Compute (Nova): Fix Released Bug description: When running multiple migrations concurrently, I see an error the repeats in the compute.log file. The trace can be seen here: http://pastebin.com/XRr70jpU Here is the steps I followed to recreate this issue: (1). Boot the instances nova boot --image be8b6475-26d8-410f-aaa5-8b278d98c8f9 --flavor 1 MIGRATE1 nova boot --image be8b6475-26d8-410f-aaa5-8b278d98c8f9 --flavor 1 MIGRATE2 nova boot --image be8b6475-26d8-410f-aaa5-8b278d98c8f9 --flavor 1 MIGRATE3 [root@vs347 ~]# nova list +--------------------------------------+----------+--------+-------------------+ | ID | Name | Status | Networks | +--------------------------------------+----------+--------+-------------------+ | ccf6212b-bb8d-4826-97db-e18b4d4cb2b0 | MIGRATE1 | ACTIVE | demonet=172.0.0.2 | | 739a6d71-cc63-4a34-b14f-7aef6f635a31 | MIGRATE2 | ACTIVE | demonet=172.0.0.4 | | 11dabeb8-2799-40e1-a355-4d58f935a9c6 | MIGRATE3 | ACTIVE | demonet=172.0.0.3 | +--------------------------------------+----------+--------+-------------------+ bvt-instance-00000029 - olyblade02 bvt-instance-0000002a - olyblade02 bvt-instance-0000002b - olyblade02 (2). migrate the instances simultaneously [root@vs347 ~]# nova migrate MIGRATE1 [root@vs347 ~]# nova migrate MIGRATE2 [root@vs347 ~]# nova migrate MIGRATE3 [root@vs347 ~]# * State as of 4:24 p.m. +--------------------------------------+----------+--------+-------------------+ | ID | Name | Status | Networks | +--------------------------------------+----------+--------+-------------------+ | ccf6212b-bb8d-4826-97db-e18b4d4cb2b0 | MIGRATE1 | RESIZE | demonet=172.0.0.2 | | 739a6d71-cc63-4a34-b14f-7aef6f635a31 | MIGRATE2 | RESIZE | demonet=172.0.0.4 | | 11dabeb8-2799-40e1-a355-4d58f935a9c6 | MIGRATE3 | RESIZE | demonet=172.0.0.3 | +--------------------------------------+----------+--------+-------------------+ * Now they are in resize-verify state so do confirm [root@vs347 ~]# nova resize-confirm MIGRATE1 [root@vs347 ~]# nova resize-confirm MIGRATE2 [root@vs347 ~]# nova resize-confirm MIGRATE3 [root@vs347 ~]# [root@vs347 ~]# nova list +--------------------------------------+----------+--------+-------------------+ | ID | Name | Status | Networks | +--------------------------------------+----------+--------+-------------------+ | ccf6212b-bb8d-4826-97db-e18b4d4cb2b0 | MIGRATE1 | ERROR | demonet=172.0.0.2 | | 739a6d71-cc63-4a34-b14f-7aef6f635a31 | MIGRATE2 | ACTIVE | demonet=172.0.0.4 | | 11dabeb8-2799-40e1-a355-4d58f935a9c6 | MIGRATE3 | ACTIVE | demonet=172.0.0.3 | +--------------------------------------+----------+--------+-------------------+ [root@vs347 ~]# nova show MIGRATE1 +-------------------------------------+------------------------------------------------------------------------------------------------------------------------+ | Property | Value | +-------------------------------------+------------------------------------------------------------------------------------------------------------------------+ | status | ERROR | | updated | 2013-03-22T21:52:21Z | | OS-EXT-STS:task_state | None | | OS-EXT-SRV-ATTR:host | vs347.rch.kstart.ibm.com | | key_name | None | | image | Rhel6MasterFile (be8b6475-26d8-410f-aaa5-8b278d98c8f9) | | hostId | a3883219ffb68903fc2ecc1e0a490f3f250fdf3773f446bcb398441c | | OS-EXT-STS:vm_state | error | | OS-EXT-SRV-ATTR:instance_name | bvt-instance-00000029 | | OS-EXT-SRV-ATTR:hypervisor_hostname | olyblade01.rch.stglabs.ibm.com | | flavor | m1.tiny (1) | | demonet network | 172.0.0.2 | | id | ccf6212b-bb8d-4826-97db-e18b4d4cb2b0 | | security_groups | [{u'name': u'default'}] | | user_id | 3ccf55fd609b45319f24fe681338886d | | name | MIGRATE1 | | created | 2013-03-22T21:19:00Z | | tenant_id | 67b1c37f4ca64283908c7077e9e59997 | | OS-DCF:diskConfig | MANUAL | | metadata | {} | | accessIPv4 | | | accessIPv6 | | | fault | {u'message': u'KeyError', u'code': 500, u'details': u'\'old_instance_type_memory_mb\' | | | File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 229, in decorated_function | | | return function(self, context, *args, **kwargs) | | | File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 1956, in confirm_resize | | | rt.confirm_resize(context, migration) | | | File "/usr/lib/python2.6/site-packages/nova/compute/resource_tracker.py", line 373, in confirm_resize | | | self.update_available_resource(elevated) | | | File "/usr/lib/python2.6/site-packages/nova/openstack/common/lockutils.py", line 242, in inner | | | retval = f(*args, **kwargs) | | | File "/usr/lib/python2.6/site-packages/nova/compute/resource_tracker.py", line 269, in update_available_resource | | | self._update_usage_from_migrations(resources, migrations) | | | File "/usr/lib/python2.6/site-packages/nova/compute/resource_tracker.py", line 468, in _update_usage_from_migrations | | | migration) | | | File "/usr/lib/python2.6/site-packages/nova/compute/resource_tracker.py", line 428, in _update_usage_from_migration | | | itype = instance_types.extract_instance_type(instance, \'old_\') | | | File "/usr/lib/python2.6/site-packages/nova/compute/instance_types.py", line 250, in extract_instance_type | | | instance_type[key] = type_fn(sys_meta[type_key]) | | | ', u'created': u'2013-03-22T21:52:21Z'} | | OS-EXT-STS:power_state | 1 | | OS-EXT-AZ:availability_zone | nova | | config_drive | | +-------------------------------------+------------------------------------------------------------------------------------------------------------------------+ To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1160489/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : [email protected] Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
