Public bug reported: Description =========== I have try to resize instaces but it show Unexpected error As I check from log nova import requests library from python version 3.7 but we use python 3.6
Environment =========== CentOS 8.2.2004 novepackage: openstack-nova-api 20.4.0-1.el8 openstack-repo: centos-openstack-train (from official guide) python-version: 3.6 Steps to reproduce ================== openstack server resize --flavor <flavor-id> <instance-id> Logs & Configs ============== nova-api.log (https://pastebin.com/raw/uPFApnHB) 2020-09-18 12:21:58.524 1034587 DEBUG nova.api.openstack.wsgi [req-f5a1119b-62d5-4b29-9c3a-bd9cbba20251 ed6cc4cad89a4eac96e582434c89d795 5c73d5ad91a84ffeb257a19560f09d17 - default default] Action: 'action', calling method: <bound method ServersController._action_resize of <nova.api.openstack.compute.servers.ServersController object at 0x7f029893bac8>>, body: {"resize": {"flavorRef": "b453b4d8-63ca-4399-b94f-12a7d755ec63"}} _process_stack /usr/local/lib/python3.6/site-packages/nova/api/openstack/wsgi.py:520 2020-09-18 12:21:58.526 1034587 DEBUG nova.compute.api [req-f5a1119b-62d5-4b29-9c3a-bd9cbba20251 ed6cc4cad89a4eac96e582434c89d795 5c73d5ad91a84ffeb257a19560f09d17 - default default] [instance: eadb92d2-cddd-4330-a57f-37087c390701] Fetching instance by UUID get /usr/local/lib/python3.6/site-packages/nova/compute/api.py:2647 2020-09-18 12:21:58.532 1034587 DEBUG oslo_concurrency.lockutils [req-f5a1119b-62d5-4b29-9c3a-bd9cbba20251 ed6cc4cad89a4eac96e582434c89d795 5c73d5ad91a84ffeb257a19560f09d17 - default default] Lock "2b40c006-7fc9-4310-a3ce-b3e09caef49e" acquired by "nova.context.set_target_cell.<locals>.get_or_set_cached_cell_and_set_connections" :: waited 0.000s inner /usr/lib/python3.6/site-packages/oslo_concurrency/lockutils.py:327 2020-09-18 12:21:58.532 1034587 DEBUG oslo_concurrency.lockutils [req-f5a1119b-62d5-4b29-9c3a-bd9cbba20251 ed6cc4cad89a4eac96e582434c89d795 5c73d5ad91a84ffeb257a19560f09d17 - default default] Lock "2b40c006-7fc9-4310-a3ce-b3e09caef49e" released by "nova.context.set_target_cell.<locals>.get_or_set_cached_cell_and_set_connections" :: held 0.000s inner /usr/lib/python3.6/site-packages/oslo_concurrency/lockutils.py:339 2020-09-18 12:21:58.533 1034587 DEBUG oslo_concurrency.lockutils [req-f5a1119b-62d5-4b29-9c3a-bd9cbba20251 ed6cc4cad89a4eac96e582434c89d795 5c73d5ad91a84ffeb257a19560f09d17 - default default] Lock "2b40c006-7fc9-4310-a3ce-b3e09caef49e" acquired by "nova.context.set_target_cell.<locals>.get_or_set_cached_cell_and_set_connections" :: waited 0.000s inner /usr/lib/python3.6/site-packages/oslo_concurrency/lockutils.py:327 2020-09-18 12:21:58.533 1034587 DEBUG oslo_concurrency.lockutils [req-f5a1119b-62d5-4b29-9c3a-bd9cbba20251 ed6cc4cad89a4eac96e582434c89d795 5c73d5ad91a84ffeb257a19560f09d17 - default default] Lock "2b40c006-7fc9-4310-a3ce-b3e09caef49e" released by "nova.context.set_target_cell.<locals>.get_or_set_cached_cell_and_set_connections" :: held 0.000s inner /usr/lib/python3.6/site-packages/oslo_concurrency/lockutils.py:339 2020-09-18 12:21:58.676 1034588 DEBUG nova.api.openstack.wsgi [req-b13db621-c393-4dff-a281-e5cb2b481765 34fc286937cb4ca08e8d60dace2e71c1 624396dc175f40f39058ae1788488783 - default default] Calling method '<bound method ServersController.detail of <nova.api.openstack.compute.servers.ServersController object at 0x7f02989c9a58>>' _process_stack /usr/local/lib/python3.6/site-packages/nova/api/openstack/wsgi.py:523 2020-09-18 12:21:58.683 1034588 DEBUG nova.compute.api [req-b13db621-c393-4dff-a281-e5cb2b481765 34fc286937cb4ca08e8d60dace2e71c1 624396dc175f40f39058ae1788488783 - default default] Searching by: {'changes-since': datetime.datetime(2020, 9, 18, 5, 16, 57, 836544, tzinfo=<FixedOffset '+00:00' datetime.timedelta(0)>)} get_all /usr/local/lib/python3.6/site-packages/nova/compute/api.py:2688 2020-09-18 12:21:58.697 1034588 DEBUG oslo_db.sqlalchemy.engines [req-b13db621-c393-4dff-a281-e5cb2b481765 34fc286937cb4ca08e8d60dace2e71c1 624396dc175f40f39058ae1788488783 - default default] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION _check_effective_sql_mode /usr/lib/python3.6/site-packages/oslo_db/sqlalchemy/engines.py:307 2020-09-18 12:21:58.709 1034588 DEBUG nova.context [req-b13db621-c393-4dff-a281-e5cb2b481765 34fc286937cb4ca08e8d60dace2e71c1 624396dc175f40f39058ae1788488783 - default default] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),2b40c006-7fc9-4310-a3ce-b3e09caef49e(cell1) load_cells /usr/local/lib/python3.6/site-packages/nova/context.py:466 2020-09-18 12:21:58.713 1034588 DEBUG oslo_concurrency.lockutils [req-b13db621-c393-4dff-a281-e5cb2b481765 34fc286937cb4ca08e8d60dace2e71c1 624396dc175f40f39058ae1788488783 - default default] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell.<locals>.get_or_set_cached_cell_and_set_connections" :: waited 0.000s inner /usr/lib/python3.6/site-packages/oslo_concurrency/lockutils.py:327 2020-09-18 12:21:58.714 1034588 DEBUG oslo_concurrency.lockutils [req-b13db621-c393-4dff-a281-e5cb2b481765 34fc286937cb4ca08e8d60dace2e71c1 624396dc175f40f39058ae1788488783 - default default] Lock "00000000-0000-0000-0000-000000000000" released by "nova.context.set_target_cell.<locals>.get_or_set_cached_cell_and_set_connections" :: held 0.001s inner /usr/lib/python3.6/site-packages/oslo_concurrency/lockutils.py:339 2020-09-18 12:21:58.715 1034588 DEBUG oslo_concurrency.lockutils [req-b13db621-c393-4dff-a281-e5cb2b481765 34fc286937cb4ca08e8d60dace2e71c1 624396dc175f40f39058ae1788488783 - default default] Lock "2b40c006-7fc9-4310-a3ce-b3e09caef49e" acquired by "nova.context.set_target_cell.<locals>.get_or_set_cached_cell_and_set_connections" :: waited 0.000s inner /usr/lib/python3.6/site-packages/oslo_concurrency/lockutils.py:327 2020-09-18 12:21:58.717 1034588 DEBUG oslo_concurrency.lockutils [req-b13db621-c393-4dff-a281-e5cb2b481765 34fc286937cb4ca08e8d60dace2e71c1 624396dc175f40f39058ae1788488783 - default default] Lock "2b40c006-7fc9-4310-a3ce-b3e09caef49e" released by "nova.context.set_target_cell.<locals>.get_or_set_cached_cell_and_set_connections" :: held 0.002s inner /usr/lib/python3.6/site-packages/oslo_concurrency/lockutils.py:339 2020-09-18 12:21:58.730 1034588 DEBUG oslo_db.sqlalchemy.engines [req-b13db621-c393-4dff-a281-e5cb2b481765 34fc286937cb4ca08e8d60dace2e71c1 624396dc175f40f39058ae1788488783 - default default] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION _check_effective_sql_mode /usr/lib/python3.6/site-packages/oslo_db/sqlalchemy/engines.py:307 2020-09-18 12:21:58.779 1034588 DEBUG oslo_db.sqlalchemy.engines [req-b13db621-c393-4dff-a281-e5cb2b481765 34fc286937cb4ca08e8d60dace2e71c1 624396dc175f40f39058ae1788488783 - default default] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION _check_effective_sql_mode /usr/lib/python3.6/site-packages/oslo_db/sqlalchemy/engines.py:307 2020-09-18 12:21:58.815 1034588 INFO nova.osapi_compute.wsgi.server [req-b13db621-c393-4dff-a281-e5cb2b481765 34fc286937cb4ca08e8d60dace2e71c1 624396dc175f40f39058ae1788488783 - default default] 10.10.61.21 "GET /v2.1/servers/detail?all_tenants=True&changes-since=2020-09-18T05%3A16%3A57.836544%2B00%3A00 HTTP/1.1" status: 200 len: 413 time: 0.9357138 2020-09-18 12:21:59.194 1034587 DEBUG nova.compute.api [req-f5a1119b-62d5-4b29-9c3a-bd9cbba20251 ed6cc4cad89a4eac96e582434c89d795 5c73d5ad91a84ffeb257a19560f09d17 - default default] [instance: eadb92d2-cddd-4330-a57f-37087c390701] Old instance type 2vCPU-4G-30G, new instance type 16vCPU-AVX-16G-40G resize /usr/local/lib/python3.6/site-packages/nova/compute/api.py:3686 2020-09-18 12:21:59.199 1034587 DEBUG nova.context [req-f5a1119b-62d5-4b29-9c3a-bd9cbba20251 ed6cc4cad89a4eac96e582434c89d795 5c73d5ad91a84ffeb257a19560f09d17 - default default] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),2b40c006-7fc9-4310-a3ce-b3e09caef49e(cell1) load_cells /usr/local/lib/python3.6/site-packages/nova/context.py:466 2020-09-18 12:21:59.200 1034587 DEBUG oslo_concurrency.lockutils [req-f5a1119b-62d5-4b29-9c3a-bd9cbba20251 ed6cc4cad89a4eac96e582434c89d795 5c73d5ad91a84ffeb257a19560f09d17 - default default] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell.<locals>.get_or_set_cached_cell_and_set_connections" :: waited 0.000s inner /usr/lib/python3.6/site-packages/oslo_concurrency/lockutils.py:327 2020-09-18 12:21:59.201 1034587 DEBUG oslo_concurrency.lockutils [req-f5a1119b-62d5-4b29-9c3a-bd9cbba20251 ed6cc4cad89a4eac96e582434c89d795 5c73d5ad91a84ffeb257a19560f09d17 - default default] Lock "00000000-0000-0000-0000-000000000000" released by "nova.context.set_target_cell.<locals>.get_or_set_cached_cell_and_set_connections" :: held 0.001s inner /usr/lib/python3.6/site-packages/oslo_concurrency/lockutils.py:339 2020-09-18 12:21:59.202 1034587 DEBUG oslo_concurrency.lockutils [req-f5a1119b-62d5-4b29-9c3a-bd9cbba20251 ed6cc4cad89a4eac96e582434c89d795 5c73d5ad91a84ffeb257a19560f09d17 - default default] Lock "2b40c006-7fc9-4310-a3ce-b3e09caef49e" acquired by "nova.context.set_target_cell.<locals>.get_or_set_cached_cell_and_set_connections" :: waited 0.000s inner /usr/lib/python3.6/site-packages/oslo_concurrency/lockutils.py:327 2020-09-18 12:21:59.202 1034587 DEBUG oslo_concurrency.lockutils [req-f5a1119b-62d5-4b29-9c3a-bd9cbba20251 ed6cc4cad89a4eac96e582434c89d795 5c73d5ad91a84ffeb257a19560f09d17 - default default] Lock "2b40c006-7fc9-4310-a3ce-b3e09caef49e" released by "nova.context.set_target_cell.<locals>.get_or_set_cached_cell_and_set_connections" :: held 0.000s inner /usr/lib/python3.6/site-packages/oslo_concurrency/lockutils.py:339 2020-09-18 12:21:59.219 1034587 DEBUG oslo_db.sqlalchemy.engines [req-f5a1119b-62d5-4b29-9c3a-bd9cbba20251 ed6cc4cad89a4eac96e582434c89d795 5c73d5ad91a84ffeb257a19560f09d17 - default default] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION _check_effective_sql_mode /usr/lib/python3.6/site-packages/oslo_db/sqlalchemy/engines.py:307 2020-09-18 12:21:59.238 1034587 DEBUG nova.quota [req-f5a1119b-62d5-4b29-9c3a-bd9cbba20251 ed6cc4cad89a4eac96e582434c89d795 5c73d5ad91a84ffeb257a19560f09d17 - default default] Getting quotas for project 5c73d5ad91a84ffeb257a19560f09d17. Resources: {'ram', 'cores'} _get_quotas /usr/local/lib/python3.6/site-packages/nova/quota.py:390 2020-09-18 12:21:59.242 1034587 DEBUG nova.quota [req-f5a1119b-62d5-4b29-9c3a-bd9cbba20251 ed6cc4cad89a4eac96e582434c89d795 5c73d5ad91a84ffeb257a19560f09d17 - default default] Getting quotas for user ed6cc4cad89a4eac96e582434c89d795 and project 5c73d5ad91a84ffeb257a19560f09d17. Resources: {'ram', 'cores'} _get_quotas /usr/local/lib/python3.6/site-packages/nova/quota.py:382 2020-09-18 12:21:59.258 1034587 DEBUG nova.virt.hardware [req-f5a1119b-62d5-4b29-9c3a-bd9cbba20251 ed6cc4cad89a4eac96e582434c89d795 5c73d5ad91a84ffeb257a19560f09d17 - default default] Flavor limits 0:0:0 get_cpu_topology_constraints /usr/local/lib/python3.6/site-packages/nova/virt/hardware.py:337 2020-09-18 12:21:59.259 1034587 DEBUG nova.virt.hardware [req-f5a1119b-62d5-4b29-9c3a-bd9cbba20251 ed6cc4cad89a4eac96e582434c89d795 5c73d5ad91a84ffeb257a19560f09d17 - default default] Image limits 0:0:0 get_cpu_topology_constraints /usr/local/lib/python3.6/site-packages/nova/virt/hardware.py:341 2020-09-18 12:21:59.259 1034587 DEBUG nova.virt.hardware [req-f5a1119b-62d5-4b29-9c3a-bd9cbba20251 ed6cc4cad89a4eac96e582434c89d795 5c73d5ad91a84ffeb257a19560f09d17 - default default] Flavor pref 0:0:0 get_cpu_topology_constraints /usr/local/lib/python3.6/site-packages/nova/virt/hardware.py:377 2020-09-18 12:21:59.259 1034587 DEBUG nova.virt.hardware [req-f5a1119b-62d5-4b29-9c3a-bd9cbba20251 ed6cc4cad89a4eac96e582434c89d795 5c73d5ad91a84ffeb257a19560f09d17 - default default] Image pref 0:0:0 get_cpu_topology_constraints /usr/local/lib/python3.6/site-packages/nova/virt/hardware.py:381 2020-09-18 12:21:59.260 1034587 DEBUG nova.virt.hardware [req-f5a1119b-62d5-4b29-9c3a-bd9cbba20251 ed6cc4cad89a4eac96e582434c89d795 5c73d5ad91a84ffeb257a19560f09d17 - default default] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 get_cpu_topology_constraints /usr/local/lib/python3.6/site-packages/nova/virt/hardware.py:421 2020-09-18 12:22:00.108 1034587 ERROR nova.api.openstack.wsgi [req-f5a1119b-62d5-4b29-9c3a-bd9cbba20251 ed6cc4cad89a4eac96e582434c89d795 5c73d5ad91a84ffeb257a19560f09d17 - default default] Unexpected exception in API method: ImportError: cannot import name 'universaldetector' from 'requests.packages.chardet' (/usr/local/lib/python3.7/dist-packages/requests/packages/chardet/__init__.py) 2020-09-18 12:22:00.108 1034587 ERROR nova.api.openstack.wsgi Traceback (most recent call last): 2020-09-18 12:22:00.108 1034587 ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.6/site-packages/nova/api/openstack/wsgi.py", line 671, in wrapped 2020-09-18 12:22:00.108 1034587 ERROR nova.api.openstack.wsgi return f(*args, **kwargs) 2020-09-18 12:22:00.108 1034587 ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.6/site-packages/nova/api/validation/__init__.py", line 110, in wrapper 2020-09-18 12:22:00.108 1034587 ERROR nova.api.openstack.wsgi return func(*args, **kwargs) 2020-09-18 12:22:00.108 1034587 ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.6/site-packages/nova/api/openstack/compute/servers.py", line 1044, in _action_resize 2020-09-18 12:22:00.108 1034587 ERROR nova.api.openstack.wsgi self._resize(req, id, flavor_ref, **kwargs) 2020-09-18 12:22:00.108 1034587 ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.6/site-packages/nova/api/openstack/compute/servers.py", line 963, in _resize 2020-09-18 12:22:00.108 1034587 ERROR nova.api.openstack.wsgi self.compute_api.resize(context, instance, flavor_id, **kwargs) 2020-09-18 12:22:00.108 1034587 ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.6/site-packages/nova/compute/api.py", line 225, in inner 2020-09-18 12:22:00.108 1034587 ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) 2020-09-18 12:22:00.108 1034587 ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.6/site-packages/nova/compute/api.py", line 152, in inner 2020-09-18 12:22:00.108 1034587 ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) 2020-09-18 12:22:00.108 1034587 ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.6/site-packages/nova/compute/api.py", line 215, in wrapped 2020-09-18 12:22:00.108 1034587 ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) 2020-09-18 12:22:00.108 1034587 ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.6/site-packages/nova/compute/api.py", line 3772, in resize 2020-09-18 12:22:00.108 1034587 ERROR nova.api.openstack.wsgi request_spec=request_spec) 2020-09-18 12:22:00.108 1034587 ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.6/site-packages/nova/conductor/api.py", line 96, in resize_instance 2020-09-18 12:22:00.108 1034587 ERROR nova.api.openstack.wsgi request_spec=request_spec, host_list=host_list) 2020-09-18 12:22:00.108 1034587 ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.6/site-packages/nova/conductor/rpcapi.py", line 340, in migrate_server 2020-09-18 12:22:00.108 1034587 ERROR nova.api.openstack.wsgi return cctxt.call(context, 'migrate_server', **kw) 2020-09-18 12:22:00.108 1034587 ERROR nova.api.openstack.wsgi File "/usr/lib/python3.6/site-packages/oslo_messaging/rpc/client.py", line 181, in call 2020-09-18 12:22:00.108 1034587 ERROR nova.api.openstack.wsgi transport_options=self.transport_options) 2020-09-18 12:22:00.108 1034587 ERROR nova.api.openstack.wsgi File "/usr/lib/python3.6/site-packages/oslo_messaging/transport.py", line 129, in _send 2020-09-18 12:22:00.108 1034587 ERROR nova.api.openstack.wsgi transport_options=transport_options) 2020-09-18 12:22:00.108 1034587 ERROR nova.api.openstack.wsgi File "/usr/lib/python3.6/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 646, in send 2020-09-18 12:22:00.108 1034587 ERROR nova.api.openstack.wsgi transport_options=transport_options) 2020-09-18 12:22:00.108 1034587 ERROR nova.api.openstack.wsgi File "/usr/lib/python3.6/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 636, in _send 2020-09-18 12:22:00.108 1034587 ERROR nova.api.openstack.wsgi raise result 2020-09-18 12:22:00.108 1034587 ERROR nova.api.openstack.wsgi ImportError: cannot import name 'universaldetector' from 'requests.packages.chardet' (/usr/local/lib/python3.7/dist-packages/requests/packages/chardet/__init__.py) 2020-09-18 12:22:00.108 1034587 ERROR nova.api.openstack.wsgi 2020-09-18 12:22:00.113 1034587 INFO nova.api.openstack.wsgi [req-f5a1119b-62d5-4b29-9c3a-bd9cbba20251 ed6cc4cad89a4eac96e582434c89d795 5c73d5ad91a84ffeb257a19560f09d17 - default default] HTTP exception thrown: Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible. <class 'ImportError'> 2020-09-18 12:22:00.115 1034587 DEBUG nova.api.openstack.wsgi [req-f5a1119b-62d5-4b29-9c3a-bd9cbba20251 ed6cc4cad89a4eac96e582434c89d795 5c73d5ad91a84ffeb257a19560f09d17 - default default] Returning 500 to user: Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible. <class 'ImportError'> __call__ /usr/local/lib/python3.6/site-packages/nova/api/openstack/wsgi.py:941 2020-09-18 12:22:00.116 1034587 INFO nova.osapi_compute.wsgi.server [req-f5a1119b-62d5-4b29-9c3a-bd9cbba20251 ed6cc4cad89a4eac96e582434c89d795 5c73d5ad91a84ffeb257a19560f09d17 - default default] 10.10.61.101 "POST /v2.1/servers/eadb92d2-cddd-4330-a57f-37087c390701/action HTTP/1.1" status: 500 len: 618 time: 1.5959389 ** Affects: nova Importance: Undecided Status: New ** Description changed: Description =========== I have try to resize instaces but it show Unexpected error As I check from log nova import requests library from python version 3.7 but we use python 3.6 Environment =========== CentOS 8.2.2004 novepackage: openstack-nova-api 20.4.0-1.el8 openstack-repo: centos-openstack-train (from official guide) python-version: 3.6 Steps to reproduce ================== openstack server resize --flavor <flavor-id> <instance-id> Logs & Configs ============== - nova-api.log + nova-api.log (https://pastebin.com/raw/uPFApnHB) 2020-09-18 12:21:58.524 1034587 DEBUG nova.api.openstack.wsgi [req-f5a1119b-62d5-4b29-9c3a-bd9cbba20251 ed6cc4cad89a4eac96e582434c89d795 5c73d5ad91a84ffeb257a19560f09d17 - default default] Action: 'action', calling method: <bound method ServersController._action_resize of <nova.api.openstack.compute.servers.ServersController object at 0x7f029893bac8>>, body: {"resize": {"flavorRef": "b453b4d8-63ca-4399-b94f-12a7d755ec63"}} _process_stack /usr/local/lib/python3.6/site-packages/nova/api/openstack/wsgi.py:520 2020-09-18 12:21:58.526 1034587 DEBUG nova.compute.api [req-f5a1119b-62d5-4b29-9c3a-bd9cbba20251 ed6cc4cad89a4eac96e582434c89d795 5c73d5ad91a84ffeb257a19560f09d17 - default default] [instance: eadb92d2-cddd-4330-a57f-37087c390701] Fetching instance by UUID get /usr/local/lib/python3.6/site-packages/nova/compute/api.py:2647 2020-09-18 12:21:58.532 1034587 DEBUG oslo_concurrency.lockutils [req-f5a1119b-62d5-4b29-9c3a-bd9cbba20251 ed6cc4cad89a4eac96e582434c89d795 5c73d5ad91a84ffeb257a19560f09d17 - default default] Lock "2b40c006-7fc9-4310-a3ce-b3e09caef49e" acquired by "nova.context.set_target_cell.<locals>.get_or_set_cached_cell_and_set_connections" :: waited 0.000s inner /usr/lib/python3.6/site-packages/oslo_concurrency/lockutils.py:327 2020-09-18 12:21:58.532 1034587 DEBUG oslo_concurrency.lockutils [req-f5a1119b-62d5-4b29-9c3a-bd9cbba20251 ed6cc4cad89a4eac96e582434c89d795 5c73d5ad91a84ffeb257a19560f09d17 - default default] Lock "2b40c006-7fc9-4310-a3ce-b3e09caef49e" released by "nova.context.set_target_cell.<locals>.get_or_set_cached_cell_and_set_connections" :: held 0.000s inner /usr/lib/python3.6/site-packages/oslo_concurrency/lockutils.py:339 2020-09-18 12:21:58.533 1034587 DEBUG oslo_concurrency.lockutils [req-f5a1119b-62d5-4b29-9c3a-bd9cbba20251 ed6cc4cad89a4eac96e582434c89d795 5c73d5ad91a84ffeb257a19560f09d17 - default default] Lock "2b40c006-7fc9-4310-a3ce-b3e09caef49e" acquired by "nova.context.set_target_cell.<locals>.get_or_set_cached_cell_and_set_connections" :: waited 0.000s inner /usr/lib/python3.6/site-packages/oslo_concurrency/lockutils.py:327 2020-09-18 12:21:58.533 1034587 DEBUG oslo_concurrency.lockutils [req-f5a1119b-62d5-4b29-9c3a-bd9cbba20251 ed6cc4cad89a4eac96e582434c89d795 5c73d5ad91a84ffeb257a19560f09d17 - default default] Lock "2b40c006-7fc9-4310-a3ce-b3e09caef49e" released by "nova.context.set_target_cell.<locals>.get_or_set_cached_cell_and_set_connections" :: held 0.000s inner /usr/lib/python3.6/site-packages/oslo_concurrency/lockutils.py:339 2020-09-18 12:21:58.676 1034588 DEBUG nova.api.openstack.wsgi [req-b13db621-c393-4dff-a281-e5cb2b481765 34fc286937cb4ca08e8d60dace2e71c1 624396dc175f40f39058ae1788488783 - default default] Calling method '<bound method ServersController.detail of <nova.api.openstack.compute.servers.ServersController object at 0x7f02989c9a58>>' _process_stack /usr/local/lib/python3.6/site-packages/nova/api/openstack/wsgi.py:523 2020-09-18 12:21:58.683 1034588 DEBUG nova.compute.api [req-b13db621-c393-4dff-a281-e5cb2b481765 34fc286937cb4ca08e8d60dace2e71c1 624396dc175f40f39058ae1788488783 - default default] Searching by: {'changes-since': datetime.datetime(2020, 9, 18, 5, 16, 57, 836544, tzinfo=<FixedOffset '+00:00' datetime.timedelta(0)>)} get_all /usr/local/lib/python3.6/site-packages/nova/compute/api.py:2688 2020-09-18 12:21:58.697 1034588 DEBUG oslo_db.sqlalchemy.engines [req-b13db621-c393-4dff-a281-e5cb2b481765 34fc286937cb4ca08e8d60dace2e71c1 624396dc175f40f39058ae1788488783 - default default] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION _check_effective_sql_mode /usr/lib/python3.6/site-packages/oslo_db/sqlalchemy/engines.py:307 2020-09-18 12:21:58.709 1034588 DEBUG nova.context [req-b13db621-c393-4dff-a281-e5cb2b481765 34fc286937cb4ca08e8d60dace2e71c1 624396dc175f40f39058ae1788488783 - default default] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),2b40c006-7fc9-4310-a3ce-b3e09caef49e(cell1) load_cells /usr/local/lib/python3.6/site-packages/nova/context.py:466 2020-09-18 12:21:58.713 1034588 DEBUG oslo_concurrency.lockutils [req-b13db621-c393-4dff-a281-e5cb2b481765 34fc286937cb4ca08e8d60dace2e71c1 624396dc175f40f39058ae1788488783 - default default] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell.<locals>.get_or_set_cached_cell_and_set_connections" :: waited 0.000s inner /usr/lib/python3.6/site-packages/oslo_concurrency/lockutils.py:327 2020-09-18 12:21:58.714 1034588 DEBUG oslo_concurrency.lockutils [req-b13db621-c393-4dff-a281-e5cb2b481765 34fc286937cb4ca08e8d60dace2e71c1 624396dc175f40f39058ae1788488783 - default default] Lock "00000000-0000-0000-0000-000000000000" released by "nova.context.set_target_cell.<locals>.get_or_set_cached_cell_and_set_connections" :: held 0.001s inner /usr/lib/python3.6/site-packages/oslo_concurrency/lockutils.py:339 2020-09-18 12:21:58.715 1034588 DEBUG oslo_concurrency.lockutils [req-b13db621-c393-4dff-a281-e5cb2b481765 34fc286937cb4ca08e8d60dace2e71c1 624396dc175f40f39058ae1788488783 - default default] Lock "2b40c006-7fc9-4310-a3ce-b3e09caef49e" acquired by "nova.context.set_target_cell.<locals>.get_or_set_cached_cell_and_set_connections" :: waited 0.000s inner /usr/lib/python3.6/site-packages/oslo_concurrency/lockutils.py:327 2020-09-18 12:21:58.717 1034588 DEBUG oslo_concurrency.lockutils [req-b13db621-c393-4dff-a281-e5cb2b481765 34fc286937cb4ca08e8d60dace2e71c1 624396dc175f40f39058ae1788488783 - default default] Lock "2b40c006-7fc9-4310-a3ce-b3e09caef49e" released by "nova.context.set_target_cell.<locals>.get_or_set_cached_cell_and_set_connections" :: held 0.002s inner /usr/lib/python3.6/site-packages/oslo_concurrency/lockutils.py:339 2020-09-18 12:21:58.730 1034588 DEBUG oslo_db.sqlalchemy.engines [req-b13db621-c393-4dff-a281-e5cb2b481765 34fc286937cb4ca08e8d60dace2e71c1 624396dc175f40f39058ae1788488783 - default default] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION _check_effective_sql_mode /usr/lib/python3.6/site-packages/oslo_db/sqlalchemy/engines.py:307 2020-09-18 12:21:58.779 1034588 DEBUG oslo_db.sqlalchemy.engines [req-b13db621-c393-4dff-a281-e5cb2b481765 34fc286937cb4ca08e8d60dace2e71c1 624396dc175f40f39058ae1788488783 - default default] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION _check_effective_sql_mode /usr/lib/python3.6/site-packages/oslo_db/sqlalchemy/engines.py:307 2020-09-18 12:21:58.815 1034588 INFO nova.osapi_compute.wsgi.server [req-b13db621-c393-4dff-a281-e5cb2b481765 34fc286937cb4ca08e8d60dace2e71c1 624396dc175f40f39058ae1788488783 - default default] 10.10.61.21 "GET /v2.1/servers/detail?all_tenants=True&changes-since=2020-09-18T05%3A16%3A57.836544%2B00%3A00 HTTP/1.1" status: 200 len: 413 time: 0.9357138 2020-09-18 12:21:59.194 1034587 DEBUG nova.compute.api [req-f5a1119b-62d5-4b29-9c3a-bd9cbba20251 ed6cc4cad89a4eac96e582434c89d795 5c73d5ad91a84ffeb257a19560f09d17 - default default] [instance: eadb92d2-cddd-4330-a57f-37087c390701] Old instance type 2vCPU-4G-30G, new instance type 16vCPU-AVX-16G-40G resize /usr/local/lib/python3.6/site-packages/nova/compute/api.py:3686 2020-09-18 12:21:59.199 1034587 DEBUG nova.context [req-f5a1119b-62d5-4b29-9c3a-bd9cbba20251 ed6cc4cad89a4eac96e582434c89d795 5c73d5ad91a84ffeb257a19560f09d17 - default default] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),2b40c006-7fc9-4310-a3ce-b3e09caef49e(cell1) load_cells /usr/local/lib/python3.6/site-packages/nova/context.py:466 2020-09-18 12:21:59.200 1034587 DEBUG oslo_concurrency.lockutils [req-f5a1119b-62d5-4b29-9c3a-bd9cbba20251 ed6cc4cad89a4eac96e582434c89d795 5c73d5ad91a84ffeb257a19560f09d17 - default default] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell.<locals>.get_or_set_cached_cell_and_set_connections" :: waited 0.000s inner /usr/lib/python3.6/site-packages/oslo_concurrency/lockutils.py:327 2020-09-18 12:21:59.201 1034587 DEBUG oslo_concurrency.lockutils [req-f5a1119b-62d5-4b29-9c3a-bd9cbba20251 ed6cc4cad89a4eac96e582434c89d795 5c73d5ad91a84ffeb257a19560f09d17 - default default] Lock "00000000-0000-0000-0000-000000000000" released by "nova.context.set_target_cell.<locals>.get_or_set_cached_cell_and_set_connections" :: held 0.001s inner /usr/lib/python3.6/site-packages/oslo_concurrency/lockutils.py:339 2020-09-18 12:21:59.202 1034587 DEBUG oslo_concurrency.lockutils [req-f5a1119b-62d5-4b29-9c3a-bd9cbba20251 ed6cc4cad89a4eac96e582434c89d795 5c73d5ad91a84ffeb257a19560f09d17 - default default] Lock "2b40c006-7fc9-4310-a3ce-b3e09caef49e" acquired by "nova.context.set_target_cell.<locals>.get_or_set_cached_cell_and_set_connections" :: waited 0.000s inner /usr/lib/python3.6/site-packages/oslo_concurrency/lockutils.py:327 2020-09-18 12:21:59.202 1034587 DEBUG oslo_concurrency.lockutils [req-f5a1119b-62d5-4b29-9c3a-bd9cbba20251 ed6cc4cad89a4eac96e582434c89d795 5c73d5ad91a84ffeb257a19560f09d17 - default default] Lock "2b40c006-7fc9-4310-a3ce-b3e09caef49e" released by "nova.context.set_target_cell.<locals>.get_or_set_cached_cell_and_set_connections" :: held 0.000s inner /usr/lib/python3.6/site-packages/oslo_concurrency/lockutils.py:339 2020-09-18 12:21:59.219 1034587 DEBUG oslo_db.sqlalchemy.engines [req-f5a1119b-62d5-4b29-9c3a-bd9cbba20251 ed6cc4cad89a4eac96e582434c89d795 5c73d5ad91a84ffeb257a19560f09d17 - default default] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION _check_effective_sql_mode /usr/lib/python3.6/site-packages/oslo_db/sqlalchemy/engines.py:307 2020-09-18 12:21:59.238 1034587 DEBUG nova.quota [req-f5a1119b-62d5-4b29-9c3a-bd9cbba20251 ed6cc4cad89a4eac96e582434c89d795 5c73d5ad91a84ffeb257a19560f09d17 - default default] Getting quotas for project 5c73d5ad91a84ffeb257a19560f09d17. Resources: {'ram', 'cores'} _get_quotas /usr/local/lib/python3.6/site-packages/nova/quota.py:390 2020-09-18 12:21:59.242 1034587 DEBUG nova.quota [req-f5a1119b-62d5-4b29-9c3a-bd9cbba20251 ed6cc4cad89a4eac96e582434c89d795 5c73d5ad91a84ffeb257a19560f09d17 - default default] Getting quotas for user ed6cc4cad89a4eac96e582434c89d795 and project 5c73d5ad91a84ffeb257a19560f09d17. Resources: {'ram', 'cores'} _get_quotas /usr/local/lib/python3.6/site-packages/nova/quota.py:382 2020-09-18 12:21:59.258 1034587 DEBUG nova.virt.hardware [req-f5a1119b-62d5-4b29-9c3a-bd9cbba20251 ed6cc4cad89a4eac96e582434c89d795 5c73d5ad91a84ffeb257a19560f09d17 - default default] Flavor limits 0:0:0 get_cpu_topology_constraints /usr/local/lib/python3.6/site-packages/nova/virt/hardware.py:337 2020-09-18 12:21:59.259 1034587 DEBUG nova.virt.hardware [req-f5a1119b-62d5-4b29-9c3a-bd9cbba20251 ed6cc4cad89a4eac96e582434c89d795 5c73d5ad91a84ffeb257a19560f09d17 - default default] Image limits 0:0:0 get_cpu_topology_constraints /usr/local/lib/python3.6/site-packages/nova/virt/hardware.py:341 2020-09-18 12:21:59.259 1034587 DEBUG nova.virt.hardware [req-f5a1119b-62d5-4b29-9c3a-bd9cbba20251 ed6cc4cad89a4eac96e582434c89d795 5c73d5ad91a84ffeb257a19560f09d17 - default default] Flavor pref 0:0:0 get_cpu_topology_constraints /usr/local/lib/python3.6/site-packages/nova/virt/hardware.py:377 2020-09-18 12:21:59.259 1034587 DEBUG nova.virt.hardware [req-f5a1119b-62d5-4b29-9c3a-bd9cbba20251 ed6cc4cad89a4eac96e582434c89d795 5c73d5ad91a84ffeb257a19560f09d17 - default default] Image pref 0:0:0 get_cpu_topology_constraints /usr/local/lib/python3.6/site-packages/nova/virt/hardware.py:381 2020-09-18 12:21:59.260 1034587 DEBUG nova.virt.hardware [req-f5a1119b-62d5-4b29-9c3a-bd9cbba20251 ed6cc4cad89a4eac96e582434c89d795 5c73d5ad91a84ffeb257a19560f09d17 - default default] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 get_cpu_topology_constraints /usr/local/lib/python3.6/site-packages/nova/virt/hardware.py:421 2020-09-18 12:22:00.108 1034587 ERROR nova.api.openstack.wsgi [req-f5a1119b-62d5-4b29-9c3a-bd9cbba20251 ed6cc4cad89a4eac96e582434c89d795 5c73d5ad91a84ffeb257a19560f09d17 - default default] Unexpected exception in API method: ImportError: cannot import name 'universaldetector' from 'requests.packages.chardet' (/usr/local/lib/python3.7/dist-packages/requests/packages/chardet/__init__.py) 2020-09-18 12:22:00.108 1034587 ERROR nova.api.openstack.wsgi Traceback (most recent call last): 2020-09-18 12:22:00.108 1034587 ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.6/site-packages/nova/api/openstack/wsgi.py", line 671, in wrapped 2020-09-18 12:22:00.108 1034587 ERROR nova.api.openstack.wsgi return f(*args, **kwargs) 2020-09-18 12:22:00.108 1034587 ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.6/site-packages/nova/api/validation/__init__.py", line 110, in wrapper 2020-09-18 12:22:00.108 1034587 ERROR nova.api.openstack.wsgi return func(*args, **kwargs) 2020-09-18 12:22:00.108 1034587 ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.6/site-packages/nova/api/openstack/compute/servers.py", line 1044, in _action_resize 2020-09-18 12:22:00.108 1034587 ERROR nova.api.openstack.wsgi self._resize(req, id, flavor_ref, **kwargs) 2020-09-18 12:22:00.108 1034587 ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.6/site-packages/nova/api/openstack/compute/servers.py", line 963, in _resize 2020-09-18 12:22:00.108 1034587 ERROR nova.api.openstack.wsgi self.compute_api.resize(context, instance, flavor_id, **kwargs) 2020-09-18 12:22:00.108 1034587 ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.6/site-packages/nova/compute/api.py", line 225, in inner 2020-09-18 12:22:00.108 1034587 ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) 2020-09-18 12:22:00.108 1034587 ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.6/site-packages/nova/compute/api.py", line 152, in inner 2020-09-18 12:22:00.108 1034587 ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) 2020-09-18 12:22:00.108 1034587 ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.6/site-packages/nova/compute/api.py", line 215, in wrapped 2020-09-18 12:22:00.108 1034587 ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) 2020-09-18 12:22:00.108 1034587 ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.6/site-packages/nova/compute/api.py", line 3772, in resize 2020-09-18 12:22:00.108 1034587 ERROR nova.api.openstack.wsgi request_spec=request_spec) 2020-09-18 12:22:00.108 1034587 ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.6/site-packages/nova/conductor/api.py", line 96, in resize_instance 2020-09-18 12:22:00.108 1034587 ERROR nova.api.openstack.wsgi request_spec=request_spec, host_list=host_list) 2020-09-18 12:22:00.108 1034587 ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.6/site-packages/nova/conductor/rpcapi.py", line 340, in migrate_server 2020-09-18 12:22:00.108 1034587 ERROR nova.api.openstack.wsgi return cctxt.call(context, 'migrate_server', **kw) 2020-09-18 12:22:00.108 1034587 ERROR nova.api.openstack.wsgi File "/usr/lib/python3.6/site-packages/oslo_messaging/rpc/client.py", line 181, in call 2020-09-18 12:22:00.108 1034587 ERROR nova.api.openstack.wsgi transport_options=self.transport_options) 2020-09-18 12:22:00.108 1034587 ERROR nova.api.openstack.wsgi File "/usr/lib/python3.6/site-packages/oslo_messaging/transport.py", line 129, in _send 2020-09-18 12:22:00.108 1034587 ERROR nova.api.openstack.wsgi transport_options=transport_options) 2020-09-18 12:22:00.108 1034587 ERROR nova.api.openstack.wsgi File "/usr/lib/python3.6/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 646, in send 2020-09-18 12:22:00.108 1034587 ERROR nova.api.openstack.wsgi transport_options=transport_options) 2020-09-18 12:22:00.108 1034587 ERROR nova.api.openstack.wsgi File "/usr/lib/python3.6/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 636, in _send 2020-09-18 12:22:00.108 1034587 ERROR nova.api.openstack.wsgi raise result 2020-09-18 12:22:00.108 1034587 ERROR nova.api.openstack.wsgi ImportError: cannot import name 'universaldetector' from 'requests.packages.chardet' (/usr/local/lib/python3.7/dist-packages/requests/packages/chardet/__init__.py) 2020-09-18 12:22:00.108 1034587 ERROR nova.api.openstack.wsgi 2020-09-18 12:22:00.113 1034587 INFO nova.api.openstack.wsgi [req-f5a1119b-62d5-4b29-9c3a-bd9cbba20251 ed6cc4cad89a4eac96e582434c89d795 5c73d5ad91a84ffeb257a19560f09d17 - default default] HTTP exception thrown: Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible. <class 'ImportError'> 2020-09-18 12:22:00.115 1034587 DEBUG nova.api.openstack.wsgi [req-f5a1119b-62d5-4b29-9c3a-bd9cbba20251 ed6cc4cad89a4eac96e582434c89d795 5c73d5ad91a84ffeb257a19560f09d17 - default default] Returning 500 to user: Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible. <class 'ImportError'> __call__ /usr/local/lib/python3.6/site-packages/nova/api/openstack/wsgi.py:941 2020-09-18 12:22:00.116 1034587 INFO nova.osapi_compute.wsgi.server [req-f5a1119b-62d5-4b29-9c3a-bd9cbba20251 ed6cc4cad89a4eac96e582434c89d795 5c73d5ad91a84ffeb257a19560f09d17 - default default] 10.10.61.101 "POST /v2.1/servers/eadb92d2-cddd-4330-a57f-37087c390701/action HTTP/1.1" status: 500 len: 618 time: 1.5959389 -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1896175 Title: nova crash on resize instance Status in OpenStack Compute (nova): New Bug description: Description =========== I have try to resize instaces but it show Unexpected error As I check from log nova import requests library from python version 3.7 but we use python 3.6 Environment =========== CentOS 8.2.2004 novepackage: openstack-nova-api 20.4.0-1.el8 openstack-repo: centos-openstack-train (from official guide) python-version: 3.6 Steps to reproduce ================== openstack server resize --flavor <flavor-id> <instance-id> Logs & Configs ============== nova-api.log (https://pastebin.com/raw/uPFApnHB) 2020-09-18 12:21:58.524 1034587 DEBUG nova.api.openstack.wsgi [req-f5a1119b-62d5-4b29-9c3a-bd9cbba20251 ed6cc4cad89a4eac96e582434c89d795 5c73d5ad91a84ffeb257a19560f09d17 - default default] Action: 'action', calling method: <bound method ServersController._action_resize of <nova.api.openstack.compute.servers.ServersController object at 0x7f029893bac8>>, body: {"resize": {"flavorRef": "b453b4d8-63ca-4399-b94f-12a7d755ec63"}} _process_stack /usr/local/lib/python3.6/site-packages/nova/api/openstack/wsgi.py:520 2020-09-18 12:21:58.526 1034587 DEBUG nova.compute.api [req-f5a1119b-62d5-4b29-9c3a-bd9cbba20251 ed6cc4cad89a4eac96e582434c89d795 5c73d5ad91a84ffeb257a19560f09d17 - default default] [instance: eadb92d2-cddd-4330-a57f-37087c390701] Fetching instance by UUID get /usr/local/lib/python3.6/site-packages/nova/compute/api.py:2647 2020-09-18 12:21:58.532 1034587 DEBUG oslo_concurrency.lockutils [req-f5a1119b-62d5-4b29-9c3a-bd9cbba20251 ed6cc4cad89a4eac96e582434c89d795 5c73d5ad91a84ffeb257a19560f09d17 - default default] Lock "2b40c006-7fc9-4310-a3ce-b3e09caef49e" acquired by "nova.context.set_target_cell.<locals>.get_or_set_cached_cell_and_set_connections" :: waited 0.000s inner /usr/lib/python3.6/site-packages/oslo_concurrency/lockutils.py:327 2020-09-18 12:21:58.532 1034587 DEBUG oslo_concurrency.lockutils [req-f5a1119b-62d5-4b29-9c3a-bd9cbba20251 ed6cc4cad89a4eac96e582434c89d795 5c73d5ad91a84ffeb257a19560f09d17 - default default] Lock "2b40c006-7fc9-4310-a3ce-b3e09caef49e" released by "nova.context.set_target_cell.<locals>.get_or_set_cached_cell_and_set_connections" :: held 0.000s inner /usr/lib/python3.6/site-packages/oslo_concurrency/lockutils.py:339 2020-09-18 12:21:58.533 1034587 DEBUG oslo_concurrency.lockutils [req-f5a1119b-62d5-4b29-9c3a-bd9cbba20251 ed6cc4cad89a4eac96e582434c89d795 5c73d5ad91a84ffeb257a19560f09d17 - default default] Lock "2b40c006-7fc9-4310-a3ce-b3e09caef49e" acquired by "nova.context.set_target_cell.<locals>.get_or_set_cached_cell_and_set_connections" :: waited 0.000s inner /usr/lib/python3.6/site-packages/oslo_concurrency/lockutils.py:327 2020-09-18 12:21:58.533 1034587 DEBUG oslo_concurrency.lockutils [req-f5a1119b-62d5-4b29-9c3a-bd9cbba20251 ed6cc4cad89a4eac96e582434c89d795 5c73d5ad91a84ffeb257a19560f09d17 - default default] Lock "2b40c006-7fc9-4310-a3ce-b3e09caef49e" released by "nova.context.set_target_cell.<locals>.get_or_set_cached_cell_and_set_connections" :: held 0.000s inner /usr/lib/python3.6/site-packages/oslo_concurrency/lockutils.py:339 2020-09-18 12:21:58.676 1034588 DEBUG nova.api.openstack.wsgi [req-b13db621-c393-4dff-a281-e5cb2b481765 34fc286937cb4ca08e8d60dace2e71c1 624396dc175f40f39058ae1788488783 - default default] Calling method '<bound method ServersController.detail of <nova.api.openstack.compute.servers.ServersController object at 0x7f02989c9a58>>' _process_stack /usr/local/lib/python3.6/site-packages/nova/api/openstack/wsgi.py:523 2020-09-18 12:21:58.683 1034588 DEBUG nova.compute.api [req-b13db621-c393-4dff-a281-e5cb2b481765 34fc286937cb4ca08e8d60dace2e71c1 624396dc175f40f39058ae1788488783 - default default] Searching by: {'changes-since': datetime.datetime(2020, 9, 18, 5, 16, 57, 836544, tzinfo=<FixedOffset '+00:00' datetime.timedelta(0)>)} get_all /usr/local/lib/python3.6/site-packages/nova/compute/api.py:2688 2020-09-18 12:21:58.697 1034588 DEBUG oslo_db.sqlalchemy.engines [req-b13db621-c393-4dff-a281-e5cb2b481765 34fc286937cb4ca08e8d60dace2e71c1 624396dc175f40f39058ae1788488783 - default default] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION _check_effective_sql_mode /usr/lib/python3.6/site-packages/oslo_db/sqlalchemy/engines.py:307 2020-09-18 12:21:58.709 1034588 DEBUG nova.context [req-b13db621-c393-4dff-a281-e5cb2b481765 34fc286937cb4ca08e8d60dace2e71c1 624396dc175f40f39058ae1788488783 - default default] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),2b40c006-7fc9-4310-a3ce-b3e09caef49e(cell1) load_cells /usr/local/lib/python3.6/site-packages/nova/context.py:466 2020-09-18 12:21:58.713 1034588 DEBUG oslo_concurrency.lockutils [req-b13db621-c393-4dff-a281-e5cb2b481765 34fc286937cb4ca08e8d60dace2e71c1 624396dc175f40f39058ae1788488783 - default default] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell.<locals>.get_or_set_cached_cell_and_set_connections" :: waited 0.000s inner /usr/lib/python3.6/site-packages/oslo_concurrency/lockutils.py:327 2020-09-18 12:21:58.714 1034588 DEBUG oslo_concurrency.lockutils [req-b13db621-c393-4dff-a281-e5cb2b481765 34fc286937cb4ca08e8d60dace2e71c1 624396dc175f40f39058ae1788488783 - default default] Lock "00000000-0000-0000-0000-000000000000" released by "nova.context.set_target_cell.<locals>.get_or_set_cached_cell_and_set_connections" :: held 0.001s inner /usr/lib/python3.6/site-packages/oslo_concurrency/lockutils.py:339 2020-09-18 12:21:58.715 1034588 DEBUG oslo_concurrency.lockutils [req-b13db621-c393-4dff-a281-e5cb2b481765 34fc286937cb4ca08e8d60dace2e71c1 624396dc175f40f39058ae1788488783 - default default] Lock "2b40c006-7fc9-4310-a3ce-b3e09caef49e" acquired by "nova.context.set_target_cell.<locals>.get_or_set_cached_cell_and_set_connections" :: waited 0.000s inner /usr/lib/python3.6/site-packages/oslo_concurrency/lockutils.py:327 2020-09-18 12:21:58.717 1034588 DEBUG oslo_concurrency.lockutils [req-b13db621-c393-4dff-a281-e5cb2b481765 34fc286937cb4ca08e8d60dace2e71c1 624396dc175f40f39058ae1788488783 - default default] Lock "2b40c006-7fc9-4310-a3ce-b3e09caef49e" released by "nova.context.set_target_cell.<locals>.get_or_set_cached_cell_and_set_connections" :: held 0.002s inner /usr/lib/python3.6/site-packages/oslo_concurrency/lockutils.py:339 2020-09-18 12:21:58.730 1034588 DEBUG oslo_db.sqlalchemy.engines [req-b13db621-c393-4dff-a281-e5cb2b481765 34fc286937cb4ca08e8d60dace2e71c1 624396dc175f40f39058ae1788488783 - default default] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION _check_effective_sql_mode /usr/lib/python3.6/site-packages/oslo_db/sqlalchemy/engines.py:307 2020-09-18 12:21:58.779 1034588 DEBUG oslo_db.sqlalchemy.engines [req-b13db621-c393-4dff-a281-e5cb2b481765 34fc286937cb4ca08e8d60dace2e71c1 624396dc175f40f39058ae1788488783 - default default] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION _check_effective_sql_mode /usr/lib/python3.6/site-packages/oslo_db/sqlalchemy/engines.py:307 2020-09-18 12:21:58.815 1034588 INFO nova.osapi_compute.wsgi.server [req-b13db621-c393-4dff-a281-e5cb2b481765 34fc286937cb4ca08e8d60dace2e71c1 624396dc175f40f39058ae1788488783 - default default] 10.10.61.21 "GET /v2.1/servers/detail?all_tenants=True&changes-since=2020-09-18T05%3A16%3A57.836544%2B00%3A00 HTTP/1.1" status: 200 len: 413 time: 0.9357138 2020-09-18 12:21:59.194 1034587 DEBUG nova.compute.api [req-f5a1119b-62d5-4b29-9c3a-bd9cbba20251 ed6cc4cad89a4eac96e582434c89d795 5c73d5ad91a84ffeb257a19560f09d17 - default default] [instance: eadb92d2-cddd-4330-a57f-37087c390701] Old instance type 2vCPU-4G-30G, new instance type 16vCPU-AVX-16G-40G resize /usr/local/lib/python3.6/site-packages/nova/compute/api.py:3686 2020-09-18 12:21:59.199 1034587 DEBUG nova.context [req-f5a1119b-62d5-4b29-9c3a-bd9cbba20251 ed6cc4cad89a4eac96e582434c89d795 5c73d5ad91a84ffeb257a19560f09d17 - default default] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),2b40c006-7fc9-4310-a3ce-b3e09caef49e(cell1) load_cells /usr/local/lib/python3.6/site-packages/nova/context.py:466 2020-09-18 12:21:59.200 1034587 DEBUG oslo_concurrency.lockutils [req-f5a1119b-62d5-4b29-9c3a-bd9cbba20251 ed6cc4cad89a4eac96e582434c89d795 5c73d5ad91a84ffeb257a19560f09d17 - default default] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell.<locals>.get_or_set_cached_cell_and_set_connections" :: waited 0.000s inner /usr/lib/python3.6/site-packages/oslo_concurrency/lockutils.py:327 2020-09-18 12:21:59.201 1034587 DEBUG oslo_concurrency.lockutils [req-f5a1119b-62d5-4b29-9c3a-bd9cbba20251 ed6cc4cad89a4eac96e582434c89d795 5c73d5ad91a84ffeb257a19560f09d17 - default default] Lock "00000000-0000-0000-0000-000000000000" released by "nova.context.set_target_cell.<locals>.get_or_set_cached_cell_and_set_connections" :: held 0.001s inner /usr/lib/python3.6/site-packages/oslo_concurrency/lockutils.py:339 2020-09-18 12:21:59.202 1034587 DEBUG oslo_concurrency.lockutils [req-f5a1119b-62d5-4b29-9c3a-bd9cbba20251 ed6cc4cad89a4eac96e582434c89d795 5c73d5ad91a84ffeb257a19560f09d17 - default default] Lock "2b40c006-7fc9-4310-a3ce-b3e09caef49e" acquired by "nova.context.set_target_cell.<locals>.get_or_set_cached_cell_and_set_connections" :: waited 0.000s inner /usr/lib/python3.6/site-packages/oslo_concurrency/lockutils.py:327 2020-09-18 12:21:59.202 1034587 DEBUG oslo_concurrency.lockutils [req-f5a1119b-62d5-4b29-9c3a-bd9cbba20251 ed6cc4cad89a4eac96e582434c89d795 5c73d5ad91a84ffeb257a19560f09d17 - default default] Lock "2b40c006-7fc9-4310-a3ce-b3e09caef49e" released by "nova.context.set_target_cell.<locals>.get_or_set_cached_cell_and_set_connections" :: held 0.000s inner /usr/lib/python3.6/site-packages/oslo_concurrency/lockutils.py:339 2020-09-18 12:21:59.219 1034587 DEBUG oslo_db.sqlalchemy.engines [req-f5a1119b-62d5-4b29-9c3a-bd9cbba20251 ed6cc4cad89a4eac96e582434c89d795 5c73d5ad91a84ffeb257a19560f09d17 - default default] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION _check_effective_sql_mode /usr/lib/python3.6/site-packages/oslo_db/sqlalchemy/engines.py:307 2020-09-18 12:21:59.238 1034587 DEBUG nova.quota [req-f5a1119b-62d5-4b29-9c3a-bd9cbba20251 ed6cc4cad89a4eac96e582434c89d795 5c73d5ad91a84ffeb257a19560f09d17 - default default] Getting quotas for project 5c73d5ad91a84ffeb257a19560f09d17. Resources: {'ram', 'cores'} _get_quotas /usr/local/lib/python3.6/site-packages/nova/quota.py:390 2020-09-18 12:21:59.242 1034587 DEBUG nova.quota [req-f5a1119b-62d5-4b29-9c3a-bd9cbba20251 ed6cc4cad89a4eac96e582434c89d795 5c73d5ad91a84ffeb257a19560f09d17 - default default] Getting quotas for user ed6cc4cad89a4eac96e582434c89d795 and project 5c73d5ad91a84ffeb257a19560f09d17. Resources: {'ram', 'cores'} _get_quotas /usr/local/lib/python3.6/site-packages/nova/quota.py:382 2020-09-18 12:21:59.258 1034587 DEBUG nova.virt.hardware [req-f5a1119b-62d5-4b29-9c3a-bd9cbba20251 ed6cc4cad89a4eac96e582434c89d795 5c73d5ad91a84ffeb257a19560f09d17 - default default] Flavor limits 0:0:0 get_cpu_topology_constraints /usr/local/lib/python3.6/site-packages/nova/virt/hardware.py:337 2020-09-18 12:21:59.259 1034587 DEBUG nova.virt.hardware [req-f5a1119b-62d5-4b29-9c3a-bd9cbba20251 ed6cc4cad89a4eac96e582434c89d795 5c73d5ad91a84ffeb257a19560f09d17 - default default] Image limits 0:0:0 get_cpu_topology_constraints /usr/local/lib/python3.6/site-packages/nova/virt/hardware.py:341 2020-09-18 12:21:59.259 1034587 DEBUG nova.virt.hardware [req-f5a1119b-62d5-4b29-9c3a-bd9cbba20251 ed6cc4cad89a4eac96e582434c89d795 5c73d5ad91a84ffeb257a19560f09d17 - default default] Flavor pref 0:0:0 get_cpu_topology_constraints /usr/local/lib/python3.6/site-packages/nova/virt/hardware.py:377 2020-09-18 12:21:59.259 1034587 DEBUG nova.virt.hardware [req-f5a1119b-62d5-4b29-9c3a-bd9cbba20251 ed6cc4cad89a4eac96e582434c89d795 5c73d5ad91a84ffeb257a19560f09d17 - default default] Image pref 0:0:0 get_cpu_topology_constraints /usr/local/lib/python3.6/site-packages/nova/virt/hardware.py:381 2020-09-18 12:21:59.260 1034587 DEBUG nova.virt.hardware [req-f5a1119b-62d5-4b29-9c3a-bd9cbba20251 ed6cc4cad89a4eac96e582434c89d795 5c73d5ad91a84ffeb257a19560f09d17 - default default] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 get_cpu_topology_constraints /usr/local/lib/python3.6/site-packages/nova/virt/hardware.py:421 2020-09-18 12:22:00.108 1034587 ERROR nova.api.openstack.wsgi [req-f5a1119b-62d5-4b29-9c3a-bd9cbba20251 ed6cc4cad89a4eac96e582434c89d795 5c73d5ad91a84ffeb257a19560f09d17 - default default] Unexpected exception in API method: ImportError: cannot import name 'universaldetector' from 'requests.packages.chardet' (/usr/local/lib/python3.7/dist-packages/requests/packages/chardet/__init__.py) 2020-09-18 12:22:00.108 1034587 ERROR nova.api.openstack.wsgi Traceback (most recent call last): 2020-09-18 12:22:00.108 1034587 ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.6/site-packages/nova/api/openstack/wsgi.py", line 671, in wrapped 2020-09-18 12:22:00.108 1034587 ERROR nova.api.openstack.wsgi return f(*args, **kwargs) 2020-09-18 12:22:00.108 1034587 ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.6/site-packages/nova/api/validation/__init__.py", line 110, in wrapper 2020-09-18 12:22:00.108 1034587 ERROR nova.api.openstack.wsgi return func(*args, **kwargs) 2020-09-18 12:22:00.108 1034587 ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.6/site-packages/nova/api/openstack/compute/servers.py", line 1044, in _action_resize 2020-09-18 12:22:00.108 1034587 ERROR nova.api.openstack.wsgi self._resize(req, id, flavor_ref, **kwargs) 2020-09-18 12:22:00.108 1034587 ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.6/site-packages/nova/api/openstack/compute/servers.py", line 963, in _resize 2020-09-18 12:22:00.108 1034587 ERROR nova.api.openstack.wsgi self.compute_api.resize(context, instance, flavor_id, **kwargs) 2020-09-18 12:22:00.108 1034587 ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.6/site-packages/nova/compute/api.py", line 225, in inner 2020-09-18 12:22:00.108 1034587 ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) 2020-09-18 12:22:00.108 1034587 ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.6/site-packages/nova/compute/api.py", line 152, in inner 2020-09-18 12:22:00.108 1034587 ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) 2020-09-18 12:22:00.108 1034587 ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.6/site-packages/nova/compute/api.py", line 215, in wrapped 2020-09-18 12:22:00.108 1034587 ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) 2020-09-18 12:22:00.108 1034587 ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.6/site-packages/nova/compute/api.py", line 3772, in resize 2020-09-18 12:22:00.108 1034587 ERROR nova.api.openstack.wsgi request_spec=request_spec) 2020-09-18 12:22:00.108 1034587 ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.6/site-packages/nova/conductor/api.py", line 96, in resize_instance 2020-09-18 12:22:00.108 1034587 ERROR nova.api.openstack.wsgi request_spec=request_spec, host_list=host_list) 2020-09-18 12:22:00.108 1034587 ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.6/site-packages/nova/conductor/rpcapi.py", line 340, in migrate_server 2020-09-18 12:22:00.108 1034587 ERROR nova.api.openstack.wsgi return cctxt.call(context, 'migrate_server', **kw) 2020-09-18 12:22:00.108 1034587 ERROR nova.api.openstack.wsgi File "/usr/lib/python3.6/site-packages/oslo_messaging/rpc/client.py", line 181, in call 2020-09-18 12:22:00.108 1034587 ERROR nova.api.openstack.wsgi transport_options=self.transport_options) 2020-09-18 12:22:00.108 1034587 ERROR nova.api.openstack.wsgi File "/usr/lib/python3.6/site-packages/oslo_messaging/transport.py", line 129, in _send 2020-09-18 12:22:00.108 1034587 ERROR nova.api.openstack.wsgi transport_options=transport_options) 2020-09-18 12:22:00.108 1034587 ERROR nova.api.openstack.wsgi File "/usr/lib/python3.6/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 646, in send 2020-09-18 12:22:00.108 1034587 ERROR nova.api.openstack.wsgi transport_options=transport_options) 2020-09-18 12:22:00.108 1034587 ERROR nova.api.openstack.wsgi File "/usr/lib/python3.6/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 636, in _send 2020-09-18 12:22:00.108 1034587 ERROR nova.api.openstack.wsgi raise result 2020-09-18 12:22:00.108 1034587 ERROR nova.api.openstack.wsgi ImportError: cannot import name 'universaldetector' from 'requests.packages.chardet' (/usr/local/lib/python3.7/dist-packages/requests/packages/chardet/__init__.py) 2020-09-18 12:22:00.108 1034587 ERROR nova.api.openstack.wsgi 2020-09-18 12:22:00.113 1034587 INFO nova.api.openstack.wsgi [req-f5a1119b-62d5-4b29-9c3a-bd9cbba20251 ed6cc4cad89a4eac96e582434c89d795 5c73d5ad91a84ffeb257a19560f09d17 - default default] HTTP exception thrown: Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible. <class 'ImportError'> 2020-09-18 12:22:00.115 1034587 DEBUG nova.api.openstack.wsgi [req-f5a1119b-62d5-4b29-9c3a-bd9cbba20251 ed6cc4cad89a4eac96e582434c89d795 5c73d5ad91a84ffeb257a19560f09d17 - default default] Returning 500 to user: Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible. <class 'ImportError'> __call__ /usr/local/lib/python3.6/site-packages/nova/api/openstack/wsgi.py:941 2020-09-18 12:22:00.116 1034587 INFO nova.osapi_compute.wsgi.server [req-f5a1119b-62d5-4b29-9c3a-bd9cbba20251 ed6cc4cad89a4eac96e582434c89d795 5c73d5ad91a84ffeb257a19560f09d17 - default default] 10.10.61.101 "POST /v2.1/servers/eadb92d2-cddd-4330-a57f-37087c390701/action HTTP/1.1" status: 500 len: 618 time: 1.5959389 To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1896175/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
