Public bug reported: After complete install openstack newton of nova-compute, when I start the nova-compute service, it dies immediately with error message:
2016-12-26 20:49:36.844 27086 ERROR oslo_service.service [req-44cf0677-f4c0-4fa1-9649-48e63c22486a - - - - -] Error starting thread. 2016-12-26 20:49:36.844 27086 ERROR oslo_service.service Traceback (most recent call last): 2016-12-26 20:49:36.844 27086 ERROR oslo_service.service File "/usr/lib/python2.7/dist-packages/oslo_service/service.py", line 708, in run_service 2016-12-26 20:49:36.844 27086 ERROR oslo_service.service service.start() 2016-12-26 20:49:36.844 27086 ERROR oslo_service.service File "/usr/lib/python2.7/dist-packages/nova/service.py", line 117, in start 2016-12-26 20:49:36.844 27086 ERROR oslo_service.service self.manager.init_host() 2016-12-26 20:49:36.844 27086 ERROR oslo_service.service File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 1139, in init_host 2016-12-26 20:49:36.844 27086 ERROR oslo_service.service context, self.host, expected_attrs=['info_cache', 'metadata']) 2016-12-26 20:49:36.844 27086 ERROR oslo_service.service File "/usr/lib/python2.7/dist-packages/oslo_versionedobjects/base.py", line 177, in wrapper 2016-12-26 20:49:36.844 27086 ERROR oslo_service.service args, kwargs) 2016-12-26 20:49:36.844 27086 ERROR oslo_service.service File "/usr/lib/python2.7/dist-packages/nova/conductor/rpcapi.py", line 236, in object_class_action_versions 2016-12-26 20:49:36.844 27086 ERROR oslo_service.service args=args, kwargs=kwargs) 2016-12-26 20:49:36.844 27086 ERROR oslo_service.service File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/client.py", line 169, in call 2016-12-26 20:49:36.844 27086 ERROR oslo_service.service retry=self.retry) 2016-12-26 20:49:36.844 27086 ERROR oslo_service.service File "/usr/lib/python2.7/dist-packages/oslo_messaging/transport.py", line 97, in _send 2016-12-26 20:49:36.844 27086 ERROR oslo_service.service timeout=timeout, retry=retry) 2016-12-26 20:49:36.844 27086 ERROR oslo_service.service File "/usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 464, in send 2016-12-26 20:49:36.844 27086 ERROR oslo_service.service retry=retry) 2016-12-26 20:49:36.844 27086 ERROR oslo_service.service File "/usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 455, in _send 2016-12-26 20:49:36.844 27086 ERROR oslo_service.service raise result 2016-12-26 20:49:36.844 27086 ERROR oslo_service.service RemoteError: Remote error: UnsupportedVersion Endpoint does not support RPC version 3.0 2016-12-26 20:49:36.844 27086 ERROR oslo_service.service [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line 133, in _dispatch_and_reply\n incoming.message))\n', u' File "/usr/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line 185, in _dispatch\n raise UnsupportedVersion(version)\n', u'UnsupportedVersion: Endpoint does not support RPC version 3.0\n']. 2016-12-26 20:49:36.844 27086 ERROR oslo_service.service I have the same nova version on controller node and compute node, and also when I try to run "openstack compute service list" on the controller node, it has error: Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible. <type 'exceptions.ValueError'> (HTTP 500) (Request-ID: req-499359d4-4362-48dc-8d7f-260d29c6900f) complete log: START with options: [u'--debug', u'compute', u'service', u'list'] options: Namespace(access_key='', access_secret='***', access_token='***', access_token_endpoint='', access_token_type='', auth_type='', auth_url='http://controller1:35357/v3', authorization_code='', cacert=None, cert='', client_id='', client_secret='***', cloud='', consumer_key='', consumer_secret='***', debug=True, default_domain='default', default_domain_id='', default_domain_name='', deferred_help=False, discovery_endpoint='', domain_id='', domain_name='', endpoint='', identity_provider='', identity_provider_url='', insecure=None, interface='', key='', log_file=None, old_profile=None, openid_scope='', os_beta_command=False, os_compute_api_version='', os_identity_api_version='3', os_image_api_version='2', os_network_api_version='', os_object_api_version='', os_project_id=None, os_project_name=None, os_volume_api_version='', passcode='', password='***', profile=None, project_domain_id='', project_domain_name='default', project_id='', project_name='admin', protocol='', redirect_uri= '', region_name='', timing=False, token='***', trust_id='', url='', user_domain_id='', user_domain_name='default', user_id='', username='admin', verbose_level=3, verify=None) Auth plugin password selected auth_config_hook(): {'auth_type': 'password', 'beta_command': False, u'compute_api_version': u'2', 'key': None, u'database_api_version': u'1.0', u'interface': None, 'auth_url': 'http://controller1:35357/v3', u'network_api_version': u'2', u'image_format': u'qcow2', 'networks': [], u'image_api_version': '2', 'verify': True, u'dns_api_version': u'2', u'object_store_api_version': u'1', 'username': 'admin', 'verbose_level': 3, 'region_name': '', 'api_timeout': None, u'baremetal_api_version': u'1', 'auth': {'user_domain_name': 'default', 'project_name': 'admin', 'project_domain_name': 'default'}, 'default_domain': 'default', 'debug': True, u'image_api_use_tasks': False, u'floating_ip_source': u'neutron', u'orchestration_api_version': u'1', 'timing': False, 'password': 'D34EC9030D', 'cacert': None, u'key_manager_api_version': u'v1', u'metering_api_version': u'2', 'deferred_help': False, u'identity_api_version': '3', u'volume_api_version': u'2', 'cert': None, u'secgroup_source': u'neutron', u'container_api_version': u'1', u'disable_vendor_agent': {}} defaults: {u'auth_type': 'password', u'compute_api_version': u'2', 'key': None, u'database_api_version': u'1.0', 'api_timeout': None, u'baremetal_api_version': u'1', u'image_api_version': u'2', 'cacert': None, u'image_api_use_tasks': False, u'floating_ip_source': u'neutron', u'orchestration_api_version': u'1', u'interface': None, u'network_api_version': u'2', u'image_format': u'qcow2', u'key_manager_api_version': u'v1', u'metering_api_version': u'2', 'verify': True, u'identity_api_version': u'2.0', u'volume_api_version': u'2', 'cert': None, u'secgroup_source': u'neutron', u'container_api_version': u'1', u'dns_api_version': u'2', u'object_store_api_version': u'1', u'disable_vendor_agent': {}} cloud cfg: {'auth_type': 'password', 'beta_command': False, u'compute_api_version': u'2', u'orchestration_api_version': u'1', u'database_api_version': u'1.0', 'timing': False, 'auth_url': 'http://controller1:35357/v3', u'network_api_version': u'2', u'image_format': u'qcow2', 'networks': [], u'image_api_version': '2', 'verify': True, u'dns_api_version': u'2', u'object_store_api_version': u'1', 'username': 'admin', 'verbose_level': 3, 'region_name': '', 'api_timeout': None, u'baremetal_api_version': u'1', 'auth': {'username': 'admin', 'project_name': 'admin', 'user_domain_name': 'default', 'auth_url': 'http://controller1:35357/v3', 'password': '***', 'project_domain_name': 'default'}, 'default_domain': 'default', u'container_api_version': u'1', u'image_api_use_tasks': False, u'floating_ip_source': u'neutron', 'key': None, u'interface': None, 'password': '***', 'cacert': None, u'key_manager_api_version': u'v1', u'metering_api_version': u'2', 'deferred_help': False, u'identity_api_versio n': '3', u'volume_api_version': u'2', 'cert': None, u'secgroup_source': u'neutron', 'debug': True, u'disable_vendor_agent': {}} compute API version 2, cmd group openstack.compute.v2 network API version 2, cmd group openstack.network.v2 image API version 2, cmd group openstack.image.v2 volume API version 2, cmd group openstack.volume.v2 identity API version 3, cmd group openstack.identity.v3 object_store API version 1, cmd group openstack.object_store.v1 neutronclient API version 2, cmd group openstack.neutronclient.v2 Auth plugin password selected auth_config_hook(): {'auth_type': 'password', 'beta_command': False, u'compute_api_version': u'2', 'key': None, u'database_api_version': u'1.0', u'interface': None, 'auth_url': 'http://controller1:35357/v3', u'network_api_version': u'2', u'image_format': u'qcow2', 'networks': [], u'image_api_version': '2', 'verify': True, u'dns_api_version': u'2', u'object_store_api_version': u'1', 'username': 'admin', 'verbose_level': 3, 'region_name': '', 'api_timeout': None, u'baremetal_api_version': u'1', 'auth': {'user_domain_name': 'default', 'project_name': 'admin', 'project_domain_name': 'default'}, 'default_domain': 'default', 'debug': True, u'image_api_use_tasks': False, u'floating_ip_source': u'neutron', u'orchestration_api_version': u'1', 'timing': False, 'password': 'D34EC9030D', 'cacert': None, u'key_manager_api_version': u'v1', u'metering_api_version': u'2', 'deferred_help': False, u'identity_api_version': '3', u'volume_api_version': u'2', 'cert': None, u'secgroup_source': u'neutron', u'container_api_version': u'1', u'disable_vendor_agent': {}} command: compute service list -> openstackclient.compute.v2.service.ListService Using auth plugin: password Using parameters {'username': 'admin', 'project_name': 'admin', 'user_domain_name': 'default', 'auth_url': 'http://controller1:35357/v3', 'password': '***', 'project_domain_name': 'default'} Get auth_ref REQ: curl -g -i -X GET http://controller1:35357/v3 -H "Accept: application/json" -H "User-Agent: osc-lib keystoneauth1/2.12.1 python-requests/2.10.0 CPython/2.7.12" Starting new HTTP connection (1): controller1 "GET /v3 HTTP/1.1" 200 251 RESP: [200] Date: Mon, 26 Dec 2016 14:19:32 GMT Server: Apache/2.4.18 (Ubuntu) Vary: X-Auth-Token X-Distribution: Ubuntu x-openstack-request-id: req-ce039729-7ced-4ee2-9a0c-037232cc5668 Content-Length: 251 Keep-Alive: timeout=5, max=100 Connection: Keep-Alive Content-Type: application/json RESP BODY: {"version": {"status": "stable", "updated": "2016-10-06T00:00:00Z", "media-types": [{"base": "application/json", "type": "application/vnd.openstack.identity-v3+json"}], "id": "v3.7", "links": [{"href": "http://controller1:35357/v3/";, "rel": "self"}]}} Making authentication request to http://controller1:35357/v3/auth/tokens "POST /v3/auth/tokens HTTP/1.1" 201 2276 {"token": {"is_domain": false, "methods": ["password"], "roles": [{"id": "8305e967804a4a39ad21584ae093e972", "name": "admin"}], "expires_at": "2016-12-26T15:19:32.000000Z", "project": {"domain": {"id": "default", "name": "Default"}, "id": "057783c4e1334f0db55b6187bfcefab2", "name": "admin"}, "catalog": [{"endpoints": [{"url": "http://controller1:8774/v2.1/057783c4e1334f0db55b6187bfcefab2";, "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", "id": "031c2466602d42a9a4f00d1efdd2395d"}, {"url": "http://controller1:8774/v2.1/057783c4e1334f0db55b6187bfcefab2";, "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", "id": "297a33a473a94e9fb776d628e1fe4012"}, {"url": "http://controller1:8774/v2.1/057783c4e1334f0db55b6187bfcefab2";, "interface": "public", "region": "RegionOne", "region_id": "RegionOne", "id": "2e2c2107bd654df4855b952d74399848"}], "type": "compute", "id": "bc67c41b3653459e97bc82cfc203e294", "name": "nova"}, {"endpoints": [{"url": "http://con troller1:35357/v3/", "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", "id": "51963649520e4926872871ab79931518"}, {"url": "http://controller1:35357/v3/";, "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", "id": "630980effb51495db6e4c1b6a990acf7"}, {"url": "http://controller1:5000/v3/";, "interface": "public", "region": "RegionOne", "region_id": "RegionOne", "id": "d1e4503b71554d89b6a1f0c67d1e7f6f"}], "type": "identity", "id": "f62a4310574d4037a4e08cf4189dd2a2", "name": "keystone"}, {"endpoints": [{"url": "http://controller1:9292";, "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", "id": "8544172e19c747488ede2da28edc3ebc"}, {"url": "http://controller1:9292";, "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", "id": "bad7c412ed404c5d86a86cd3ea82b22f"}, {"url": "http://controller1:9292";, "interface": "public", "region": "RegionOne", "region_id": "RegionOne", "id": "c4f63f643f634e26ac8ec508c57de11a"}], "type": "image", "id": "f64d65ba68244068bf64adefd14d9beb", "name": "glance"}], "user": {"domain": {"id": "default", "name": "Default"}, "id": "6c9e552cae4c42aa81e2c0eed68bdb0d", "name": "admin"}, "audit_ids": ["vs0KOX9qQg258Nc5pTf0vQ"], "issued_at": "2016-12-26T14:19:32.000000Z"}} run(Namespace(columns=[], formatter='table', host=None, long=False, max_width=0, noindent=False, quote_mode='nonnumeric', service=None)) Instantiating compute client for API Version Major: 2, Minor: 0 Making authentication request to http://controller1:35357/v3/auth/tokens "POST /v3/auth/tokens HTTP/1.1" 201 2276 {"token": {"is_domain": false, "methods": ["password"], "roles": [{"id": "8305e967804a4a39ad21584ae093e972", "name": "admin"}], "expires_at": "2016-12-26T15:19:32.000000Z", "project": {"domain": {"id": "default", "name": "Default"}, "id": "057783c4e1334f0db55b6187bfcefab2", "name": "admin"}, "catalog": [{"endpoints": [{"url": "http://controller1:8774/v2.1/057783c4e1334f0db55b6187bfcefab2";, "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", "id": "031c2466602d42a9a4f00d1efdd2395d"}, {"url": "http://controller1:8774/v2.1/057783c4e1334f0db55b6187bfcefab2";, "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", "id": "297a33a473a94e9fb776d628e1fe4012"}, {"url": "http://controller1:8774/v2.1/057783c4e1334f0db55b6187bfcefab2";, "interface": "public", "region": "RegionOne", "region_id": "RegionOne", "id": "2e2c2107bd654df4855b952d74399848"}], "type": "compute", "id": "bc67c41b3653459e97bc82cfc203e294", "name": "nova"}, {"endpoints": [{"url": "http://con troller1:35357/v3/", "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", "id": "51963649520e4926872871ab79931518"}, {"url": "http://controller1:35357/v3/";, "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", "id": "630980effb51495db6e4c1b6a990acf7"}, {"url": "http://controller1:5000/v3/";, "interface": "public", "region": "RegionOne", "region_id": "RegionOne", "id": "d1e4503b71554d89b6a1f0c67d1e7f6f"}], "type": "identity", "id": "f62a4310574d4037a4e08cf4189dd2a2", "name": "keystone"}, {"endpoints": [{"url": "http://controller1:9292";, "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", "id": "8544172e19c747488ede2da28edc3ebc"}, {"url": "http://controller1:9292";, "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", "id": "bad7c412ed404c5d86a86cd3ea82b22f"}, {"url": "http://controller1:9292";, "interface": "public", "region": "RegionOne", "region_id": "RegionOne", "id": "c4f63f643f634e26ac8ec508c57de11a"}], "type": "image", "id": "f64d65ba68244068bf64adefd14d9beb", "name": "glance"}], "user": {"domain": {"id": "default", "name": "Default"}, "id": "6c9e552cae4c42aa81e2c0eed68bdb0d", "name": "admin"}, "audit_ids": ["kcbQsErPSd-Op13ULtXL6g"], "issued_at": "2016-12-26T14:19:32.000000Z"}} REQ: curl -g -i -X GET http://controller1:8774/v2.1/057783c4e1334f0db55b6187bfcefab2/os-services -H "User-Agent: python-novaclient" -H "Accept: application/json" -H "X-Auth-Token: {SHA1}a3778de62c6816c3939997781e0be41e3b7b339d" Starting new HTTP connection (1): controller1 "GET /v2.1/057783c4e1334f0db55b6187bfcefab2/os-services HTTP/1.1" 500 194 RESP: [500] Openstack-Api-Version: compute 2.1 X-Openstack-Nova-Api-Version: 2.1 Vary: OpenStack-API-Version, X-OpenStack-Nova-API-Version Content-Type: application/json; charset=UTF-8 Content-Length: 194 X-Compute-Request-Id: req-e9d97dd6-a861-414c-a979-0fb40ed651b9 Date: Mon, 26 Dec 2016 14:19:32 GMT Connection: keep-alive RESP BODY: {"computeFault": {"message": "Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.\n<type 'exceptions.ValueError'>", "code": 500}} GET call to compute for http://controller1:8774/v2.1/057783c4e1334f0db55b6187bfcefab2/os-services used request id req-e9d97dd6-a861-414c-a979-0fb40ed651b9 Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible. <type 'exceptions.ValueError'> (HTTP 500) (Request-ID: req-e9d97dd6-a861-414c-a979-0fb40ed651b9) Traceback (most recent call last): File "/usr/lib/python2.7/dist-packages/cliff/app.py", line 387, in run_subcommand result = cmd.run(parsed_args) File "/usr/lib/python2.7/dist-packages/osc_lib/command/command.py", line 41, in run return super(Command, self).run(parsed_args) File "/usr/lib/python2.7/dist-packages/cliff/display.py", line 100, in run column_names, data = self.take_action(parsed_args) File "/usr/lib/python2.7/dist-packages/openstackclient/compute/v2/service.py", line 110, in take_action parsed_args.service) File "/usr/lib/python2.7/dist-packages/novaclient/v2/services.py", line 52, in list return self._list(url, "services") File "/usr/lib/python2.7/dist-packages/novaclient/base.py", line 249, in _list resp, body = self.api.client.get(url) File "/usr/lib/python2.7/dist-packages/keystoneauth1/adapter.py", line 187, in get return self.request(url, 'GET', **kwargs) File "/usr/lib/python2.7/dist-packages/novaclient/client.py", line 117, in request raise exceptions.from_response(resp, body, url, method) ClientException: Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible. <type 'exceptions.ValueError'> (HTTP 500) (Request-ID: req-e9d97dd6-a861-414c-a979-0fb40ed651b9) clean_up ListService: Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible. <type 'exceptions.ValueError'> (HTTP 500) (Request-ID: req-e9d97dd6-a861-414c-a979-0fb40ed651b9) Traceback (most recent call last): File "/usr/lib/python2.7/dist-packages/osc_lib/shell.py", line 135, in run ret_val = super(OpenStackShell, self).run(argv) File "/usr/lib/python2.7/dist-packages/cliff/app.py", line 267, in run result = self.run_subcommand(remainder) File "/usr/lib/python2.7/dist-packages/osc_lib/shell.py", line 180, in run_subcommand ret_value = super(OpenStackShell, self).run_subcommand(argv) File "/usr/lib/python2.7/dist-packages/cliff/app.py", line 387, in run_subcommand result = cmd.run(parsed_args) File "/usr/lib/python2.7/dist-packages/osc_lib/command/command.py", line 41, in run return super(Command, self).run(parsed_args) File "/usr/lib/python2.7/dist-packages/cliff/display.py", line 100, in run column_names, data = self.take_action(parsed_args) File "/usr/lib/python2.7/dist-packages/openstackclient/compute/v2/service.py", line 110, in take_action parsed_args.service) File "/usr/lib/python2.7/dist-packages/novaclient/v2/services.py", line 52, in list return self._list(url, "services") File "/usr/lib/python2.7/dist-packages/novaclient/base.py", line 249, in _list resp, body = self.api.client.get(url) File "/usr/lib/python2.7/dist-packages/keystoneauth1/adapter.py", line 187, in get return self.request(url, 'GET', **kwargs) File "/usr/lib/python2.7/dist-packages/novaclient/client.py", line 117, in request raise exceptions.from_response(resp, body, url, method) ClientException: Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible. <type 'exceptions.ValueError'> (HTTP 500) (Request-ID: req-e9d97dd6-a861-414c-a979-0fb40ed651b9) END return value: 1 nova-api, nova-consoleauth, nova-scheduler, nova-conductor and nova-novncproxy are started succesfully on the controller node. root@maas:~# dpkg -l | grep nova ii nova-api 2:14.0.1-0ubuntu1~cloud0 all OpenStack Compute - API frontend ii nova-common 2:14.0.1-0ubuntu1~cloud0 all OpenStack Compute - common files ii nova-conductor 2:14.0.1-0ubuntu1~cloud0 all OpenStack Compute - conductor service ii nova-consoleauth 2:14.0.1-0ubuntu1~cloud0 all OpenStack Compute - Console Authenticator ii nova-novncproxy 2:14.0.1-0ubuntu1~cloud0 all OpenStack Compute - NoVNC proxy ii nova-scheduler 2:14.0.1-0ubuntu1~cloud0 all OpenStack Compute - virtual machine scheduler ii python-nova 2:14.0.1-0ubuntu1~cloud0 all OpenStack Compute Python libraries ii python-novaclient 2:6.0.0-0ubuntu1~cloud0 all client library for OpenStack Compute API - Python 2.7 ** Affects: nova Importance: Undecided Status: New ** Attachment added: "log" https://bugs.launchpad.net/bugs/1652627/+attachment/4796571/+files/sosreport-compute1-20161226213141.tar.xz -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1652627 Title: nova-compute died immediatelly after start Status in OpenStack Compute (nova): New Bug description: After complete install openstack newton of nova-compute, when I start the nova-compute service, it dies immediately with error message: 2016-12-26 20:49:36.844 27086 ERROR oslo_service.service [req-44cf0677-f4c0-4fa1-9649-48e63c22486a - - - - -] Error starting thread. 2016-12-26 20:49:36.844 27086 ERROR oslo_service.service Traceback (most recent call last): 2016-12-26 20:49:36.844 27086 ERROR oslo_service.service File "/usr/lib/python2.7/dist-packages/oslo_service/service.py", line 708, in run_service 2016-12-26 20:49:36.844 27086 ERROR oslo_service.service service.start() 2016-12-26 20:49:36.844 27086 ERROR oslo_service.service File "/usr/lib/python2.7/dist-packages/nova/service.py", line 117, in start 2016-12-26 20:49:36.844 27086 ERROR oslo_service.service self.manager.init_host() 2016-12-26 20:49:36.844 27086 ERROR oslo_service.service File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 1139, in init_host 2016-12-26 20:49:36.844 27086 ERROR oslo_service.service context, self.host, expected_attrs=['info_cache', 'metadata']) 2016-12-26 20:49:36.844 27086 ERROR oslo_service.service File "/usr/lib/python2.7/dist-packages/oslo_versionedobjects/base.py", line 177, in wrapper 2016-12-26 20:49:36.844 27086 ERROR oslo_service.service args, kwargs) 2016-12-26 20:49:36.844 27086 ERROR oslo_service.service File "/usr/lib/python2.7/dist-packages/nova/conductor/rpcapi.py", line 236, in object_class_action_versions 2016-12-26 20:49:36.844 27086 ERROR oslo_service.service args=args, kwargs=kwargs) 2016-12-26 20:49:36.844 27086 ERROR oslo_service.service File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/client.py", line 169, in call 2016-12-26 20:49:36.844 27086 ERROR oslo_service.service retry=self.retry) 2016-12-26 20:49:36.844 27086 ERROR oslo_service.service File "/usr/lib/python2.7/dist-packages/oslo_messaging/transport.py", line 97, in _send 2016-12-26 20:49:36.844 27086 ERROR oslo_service.service timeout=timeout, retry=retry) 2016-12-26 20:49:36.844 27086 ERROR oslo_service.service File "/usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 464, in send 2016-12-26 20:49:36.844 27086 ERROR oslo_service.service retry=retry) 2016-12-26 20:49:36.844 27086 ERROR oslo_service.service File "/usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 455, in _send 2016-12-26 20:49:36.844 27086 ERROR oslo_service.service raise result 2016-12-26 20:49:36.844 27086 ERROR oslo_service.service RemoteError: Remote error: UnsupportedVersion Endpoint does not support RPC version 3.0 2016-12-26 20:49:36.844 27086 ERROR oslo_service.service [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line 133, in _dispatch_and_reply\n incoming.message))\n', u' File "/usr/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line 185, in _dispatch\n raise UnsupportedVersion(version)\n', u'UnsupportedVersion: Endpoint does not support RPC version 3.0\n']. 2016-12-26 20:49:36.844 27086 ERROR oslo_service.service I have the same nova version on controller node and compute node, and also when I try to run "openstack compute service list" on the controller node, it has error: Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible. <type 'exceptions.ValueError'> (HTTP 500) (Request-ID: req-499359d4-4362-48dc-8d7f-260d29c6900f) complete log: START with options: [u'--debug', u'compute', u'service', u'list'] options: Namespace(access_key='', access_secret='***', access_token='***', access_token_endpoint='', access_token_type='', auth_type='', auth_url='http://controller1:35357/v3', authorization_code='', cacert=None, cert='', client_id='', client_secret='***', cloud='', consumer_key='', consumer_secret='***', debug=True, default_domain='default', default_domain_id='', default_domain_name='', deferred_help=False, discovery_endpoint='', domain_id='', domain_name='', endpoint='', identity_provider='', identity_provider_url='', insecure=None, interface='', key='', log_file=None, old_profile=None, openid_scope='', os_beta_command=False, os_compute_api_version='', os_identity_api_version='3', os_image_api_version='2', os_network_api_version='', os_object_api_version='', os_project_id=None, os_project_name=None, os_volume_api_version='', passcode='', password='***', profile=None, project_domain_id='', project_domain_name='default', project_id='', project_name='admin', protocol='', redirect_ur i='', region_name='', timing=False, token='***', trust_id='', url='', user_domain_id='', user_domain_name='default', user_id='', username='admin', verbose_level=3, verify=None) Auth plugin password selected auth_config_hook(): {'auth_type': 'password', 'beta_command': False, u'compute_api_version': u'2', 'key': None, u'database_api_version': u'1.0', u'interface': None, 'auth_url': 'http://controller1:35357/v3', u'network_api_version': u'2', u'image_format': u'qcow2', 'networks': [], u'image_api_version': '2', 'verify': True, u'dns_api_version': u'2', u'object_store_api_version': u'1', 'username': 'admin', 'verbose_level': 3, 'region_name': '', 'api_timeout': None, u'baremetal_api_version': u'1', 'auth': {'user_domain_name': 'default', 'project_name': 'admin', 'project_domain_name': 'default'}, 'default_domain': 'default', 'debug': True, u'image_api_use_tasks': False, u'floating_ip_source': u'neutron', u'orchestration_api_version': u'1', 'timing': False, 'password': 'D34EC9030D', 'cacert': None, u'key_manager_api_version': u'v1', u'metering_api_version': u'2', 'deferred_help': False, u'identity_api_version': '3', u'volume_api_version': u'2', 'cert': None, u'secgroup_source': u'neutron' , u'container_api_version': u'1', u'disable_vendor_agent': {}} defaults: {u'auth_type': 'password', u'compute_api_version': u'2', 'key': None, u'database_api_version': u'1.0', 'api_timeout': None, u'baremetal_api_version': u'1', u'image_api_version': u'2', 'cacert': None, u'image_api_use_tasks': False, u'floating_ip_source': u'neutron', u'orchestration_api_version': u'1', u'interface': None, u'network_api_version': u'2', u'image_format': u'qcow2', u'key_manager_api_version': u'v1', u'metering_api_version': u'2', 'verify': True, u'identity_api_version': u'2.0', u'volume_api_version': u'2', 'cert': None, u'secgroup_source': u'neutron', u'container_api_version': u'1', u'dns_api_version': u'2', u'object_store_api_version': u'1', u'disable_vendor_agent': {}} cloud cfg: {'auth_type': 'password', 'beta_command': False, u'compute_api_version': u'2', u'orchestration_api_version': u'1', u'database_api_version': u'1.0', 'timing': False, 'auth_url': 'http://controller1:35357/v3', u'network_api_version': u'2', u'image_format': u'qcow2', 'networks': [], u'image_api_version': '2', 'verify': True, u'dns_api_version': u'2', u'object_store_api_version': u'1', 'username': 'admin', 'verbose_level': 3, 'region_name': '', 'api_timeout': None, u'baremetal_api_version': u'1', 'auth': {'username': 'admin', 'project_name': 'admin', 'user_domain_name': 'default', 'auth_url': 'http://controller1:35357/v3', 'password': '***', 'project_domain_name': 'default'}, 'default_domain': 'default', u'container_api_version': u'1', u'image_api_use_tasks': False, u'floating_ip_source': u'neutron', 'key': None, u'interface': None, 'password': '***', 'cacert': None, u'key_manager_api_version': u'v1', u'metering_api_version': u'2', 'deferred_help': False, u'identity_api_vers ion': '3', u'volume_api_version': u'2', 'cert': None, u'secgroup_source': u'neutron', 'debug': True, u'disable_vendor_agent': {}} compute API version 2, cmd group openstack.compute.v2 network API version 2, cmd group openstack.network.v2 image API version 2, cmd group openstack.image.v2 volume API version 2, cmd group openstack.volume.v2 identity API version 3, cmd group openstack.identity.v3 object_store API version 1, cmd group openstack.object_store.v1 neutronclient API version 2, cmd group openstack.neutronclient.v2 Auth plugin password selected auth_config_hook(): {'auth_type': 'password', 'beta_command': False, u'compute_api_version': u'2', 'key': None, u'database_api_version': u'1.0', u'interface': None, 'auth_url': 'http://controller1:35357/v3', u'network_api_version': u'2', u'image_format': u'qcow2', 'networks': [], u'image_api_version': '2', 'verify': True, u'dns_api_version': u'2', u'object_store_api_version': u'1', 'username': 'admin', 'verbose_level': 3, 'region_name': '', 'api_timeout': None, u'baremetal_api_version': u'1', 'auth': {'user_domain_name': 'default', 'project_name': 'admin', 'project_domain_name': 'default'}, 'default_domain': 'default', 'debug': True, u'image_api_use_tasks': False, u'floating_ip_source': u'neutron', u'orchestration_api_version': u'1', 'timing': False, 'password': 'D34EC9030D', 'cacert': None, u'key_manager_api_version': u'v1', u'metering_api_version': u'2', 'deferred_help': False, u'identity_api_version': '3', u'volume_api_version': u'2', 'cert': None, u'secgroup_source': u'neutron' , u'container_api_version': u'1', u'disable_vendor_agent': {}} command: compute service list -> openstackclient.compute.v2.service.ListService Using auth plugin: password Using parameters {'username': 'admin', 'project_name': 'admin', 'user_domain_name': 'default', 'auth_url': 'http://controller1:35357/v3', 'password': '***', 'project_domain_name': 'default'} Get auth_ref REQ: curl -g -i -X GET http://controller1:35357/v3 -H "Accept: application/json" -H "User-Agent: osc-lib keystoneauth1/2.12.1 python-requests/2.10.0 CPython/2.7.12" Starting new HTTP connection (1): controller1 "GET /v3 HTTP/1.1" 200 251 RESP: [200] Date: Mon, 26 Dec 2016 14:19:32 GMT Server: Apache/2.4.18 (Ubuntu) Vary: X-Auth-Token X-Distribution: Ubuntu x-openstack-request-id: req-ce039729-7ced-4ee2-9a0c-037232cc5668 Content-Length: 251 Keep-Alive: timeout=5, max=100 Connection: Keep-Alive Content-Type: application/json RESP BODY: {"version": {"status": "stable", "updated": "2016-10-06T00:00:00Z", "media-types": [{"base": "application/json", "type": "application/vnd.openstack.identity-v3+json"}], "id": "v3.7", "links": [{"href": "http://controller1:35357/v3/";, "rel": "self"}]}} Making authentication request to http://controller1:35357/v3/auth/tokens "POST /v3/auth/tokens HTTP/1.1" 201 2276 {"token": {"is_domain": false, "methods": ["password"], "roles": [{"id": "8305e967804a4a39ad21584ae093e972", "name": "admin"}], "expires_at": "2016-12-26T15:19:32.000000Z", "project": {"domain": {"id": "default", "name": "Default"}, "id": "057783c4e1334f0db55b6187bfcefab2", "name": "admin"}, "catalog": [{"endpoints": [{"url": "http://controller1:8774/v2.1/057783c4e1334f0db55b6187bfcefab2";, "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", "id": "031c2466602d42a9a4f00d1efdd2395d"}, {"url": "http://controller1:8774/v2.1/057783c4e1334f0db55b6187bfcefab2";, "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", "id": "297a33a473a94e9fb776d628e1fe4012"}, {"url": "http://controller1:8774/v2.1/057783c4e1334f0db55b6187bfcefab2";, "interface": "public", "region": "RegionOne", "region_id": "RegionOne", "id": "2e2c2107bd654df4855b952d74399848"}], "type": "compute", "id": "bc67c41b3653459e97bc82cfc203e294", "name": "nova"}, {"endpoints": [{"url": "http://c ontroller1:35357/v3/", "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", "id": "51963649520e4926872871ab79931518"}, {"url": "http://controller1:35357/v3/";, "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", "id": "630980effb51495db6e4c1b6a990acf7"}, {"url": "http://controller1:5000/v3/";, "interface": "public", "region": "RegionOne", "region_id": "RegionOne", "id": "d1e4503b71554d89b6a1f0c67d1e7f6f"}], "type": "identity", "id": "f62a4310574d4037a4e08cf4189dd2a2", "name": "keystone"}, {"endpoints": [{"url": "http://controller1:9292";, "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", "id": "8544172e19c747488ede2da28edc3ebc"}, {"url": "http://controller1:9292";, "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", "id": "bad7c412ed404c5d86a86cd3ea82b22f"}, {"url": "http://controller1:9292";, "interface": "public", "region": "RegionOne", "region_id": "RegionOne", "id": "c4f63f643f634e26ac8ec508c57de11a"}] , "type": "image", "id": "f64d65ba68244068bf64adefd14d9beb", "name": "glance"}], "user": {"domain": {"id": "default", "name": "Default"}, "id": "6c9e552cae4c42aa81e2c0eed68bdb0d", "name": "admin"}, "audit_ids": ["vs0KOX9qQg258Nc5pTf0vQ"], "issued_at": "2016-12-26T14:19:32.000000Z"}} run(Namespace(columns=[], formatter='table', host=None, long=False, max_width=0, noindent=False, quote_mode='nonnumeric', service=None)) Instantiating compute client for API Version Major: 2, Minor: 0 Making authentication request to http://controller1:35357/v3/auth/tokens "POST /v3/auth/tokens HTTP/1.1" 201 2276 {"token": {"is_domain": false, "methods": ["password"], "roles": [{"id": "8305e967804a4a39ad21584ae093e972", "name": "admin"}], "expires_at": "2016-12-26T15:19:32.000000Z", "project": {"domain": {"id": "default", "name": "Default"}, "id": "057783c4e1334f0db55b6187bfcefab2", "name": "admin"}, "catalog": [{"endpoints": [{"url": "http://controller1:8774/v2.1/057783c4e1334f0db55b6187bfcefab2";, "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", "id": "031c2466602d42a9a4f00d1efdd2395d"}, {"url": "http://controller1:8774/v2.1/057783c4e1334f0db55b6187bfcefab2";, "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", "id": "297a33a473a94e9fb776d628e1fe4012"}, {"url": "http://controller1:8774/v2.1/057783c4e1334f0db55b6187bfcefab2";, "interface": "public", "region": "RegionOne", "region_id": "RegionOne", "id": "2e2c2107bd654df4855b952d74399848"}], "type": "compute", "id": "bc67c41b3653459e97bc82cfc203e294", "name": "nova"}, {"endpoints": [{"url": "http://c ontroller1:35357/v3/", "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", "id": "51963649520e4926872871ab79931518"}, {"url": "http://controller1:35357/v3/";, "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", "id": "630980effb51495db6e4c1b6a990acf7"}, {"url": "http://controller1:5000/v3/";, "interface": "public", "region": "RegionOne", "region_id": "RegionOne", "id": "d1e4503b71554d89b6a1f0c67d1e7f6f"}], "type": "identity", "id": "f62a4310574d4037a4e08cf4189dd2a2", "name": "keystone"}, {"endpoints": [{"url": "http://controller1:9292";, "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", "id": "8544172e19c747488ede2da28edc3ebc"}, {"url": "http://controller1:9292";, "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", "id": "bad7c412ed404c5d86a86cd3ea82b22f"}, {"url": "http://controller1:9292";, "interface": "public", "region": "RegionOne", "region_id": "RegionOne", "id": "c4f63f643f634e26ac8ec508c57de11a"}] , "type": "image", "id": "f64d65ba68244068bf64adefd14d9beb", "name": "glance"}], "user": {"domain": {"id": "default", "name": "Default"}, "id": "6c9e552cae4c42aa81e2c0eed68bdb0d", "name": "admin"}, "audit_ids": ["kcbQsErPSd-Op13ULtXL6g"], "issued_at": "2016-12-26T14:19:32.000000Z"}} REQ: curl -g -i -X GET http://controller1:8774/v2.1/057783c4e1334f0db55b6187bfcefab2/os-services -H "User-Agent: python-novaclient" -H "Accept: application/json" -H "X-Auth-Token: {SHA1}a3778de62c6816c3939997781e0be41e3b7b339d" Starting new HTTP connection (1): controller1 "GET /v2.1/057783c4e1334f0db55b6187bfcefab2/os-services HTTP/1.1" 500 194 RESP: [500] Openstack-Api-Version: compute 2.1 X-Openstack-Nova-Api-Version: 2.1 Vary: OpenStack-API-Version, X-OpenStack-Nova-API-Version Content-Type: application/json; charset=UTF-8 Content-Length: 194 X-Compute-Request-Id: req-e9d97dd6-a861-414c-a979-0fb40ed651b9 Date: Mon, 26 Dec 2016 14:19:32 GMT Connection: keep-alive RESP BODY: {"computeFault": {"message": "Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.\n<type 'exceptions.ValueError'>", "code": 500}} GET call to compute for http://controller1:8774/v2.1/057783c4e1334f0db55b6187bfcefab2/os-services used request id req-e9d97dd6-a861-414c-a979-0fb40ed651b9 Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible. <type 'exceptions.ValueError'> (HTTP 500) (Request-ID: req-e9d97dd6-a861-414c-a979-0fb40ed651b9) Traceback (most recent call last): File "/usr/lib/python2.7/dist-packages/cliff/app.py", line 387, in run_subcommand result = cmd.run(parsed_args) File "/usr/lib/python2.7/dist-packages/osc_lib/command/command.py", line 41, in run return super(Command, self).run(parsed_args) File "/usr/lib/python2.7/dist-packages/cliff/display.py", line 100, in run column_names, data = self.take_action(parsed_args) File "/usr/lib/python2.7/dist-packages/openstackclient/compute/v2/service.py", line 110, in take_action parsed_args.service) File "/usr/lib/python2.7/dist-packages/novaclient/v2/services.py", line 52, in list return self._list(url, "services") File "/usr/lib/python2.7/dist-packages/novaclient/base.py", line 249, in _list resp, body = self.api.client.get(url) File "/usr/lib/python2.7/dist-packages/keystoneauth1/adapter.py", line 187, in get return self.request(url, 'GET', **kwargs) File "/usr/lib/python2.7/dist-packages/novaclient/client.py", line 117, in request raise exceptions.from_response(resp, body, url, method) ClientException: Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible. <type 'exceptions.ValueError'> (HTTP 500) (Request-ID: req-e9d97dd6-a861-414c-a979-0fb40ed651b9) clean_up ListService: Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible. <type 'exceptions.ValueError'> (HTTP 500) (Request-ID: req-e9d97dd6-a861-414c-a979-0fb40ed651b9) Traceback (most recent call last): File "/usr/lib/python2.7/dist-packages/osc_lib/shell.py", line 135, in run ret_val = super(OpenStackShell, self).run(argv) File "/usr/lib/python2.7/dist-packages/cliff/app.py", line 267, in run result = self.run_subcommand(remainder) File "/usr/lib/python2.7/dist-packages/osc_lib/shell.py", line 180, in run_subcommand ret_value = super(OpenStackShell, self).run_subcommand(argv) File "/usr/lib/python2.7/dist-packages/cliff/app.py", line 387, in run_subcommand result = cmd.run(parsed_args) File "/usr/lib/python2.7/dist-packages/osc_lib/command/command.py", line 41, in run return super(Command, self).run(parsed_args) File "/usr/lib/python2.7/dist-packages/cliff/display.py", line 100, in run column_names, data = self.take_action(parsed_args) File "/usr/lib/python2.7/dist-packages/openstackclient/compute/v2/service.py", line 110, in take_action parsed_args.service) File "/usr/lib/python2.7/dist-packages/novaclient/v2/services.py", line 52, in list return self._list(url, "services") File "/usr/lib/python2.7/dist-packages/novaclient/base.py", line 249, in _list resp, body = self.api.client.get(url) File "/usr/lib/python2.7/dist-packages/keystoneauth1/adapter.py", line 187, in get return self.request(url, 'GET', **kwargs) File "/usr/lib/python2.7/dist-packages/novaclient/client.py", line 117, in request raise exceptions.from_response(resp, body, url, method) ClientException: Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible. <type 'exceptions.ValueError'> (HTTP 500) (Request-ID: req-e9d97dd6-a861-414c-a979-0fb40ed651b9) END return value: 1 nova-api, nova-consoleauth, nova-scheduler, nova-conductor and nova-novncproxy are started succesfully on the controller node. root@maas:~# dpkg -l | grep nova ii nova-api 2:14.0.1-0ubuntu1~cloud0 all OpenStack Compute - API frontend ii nova-common 2:14.0.1-0ubuntu1~cloud0 all OpenStack Compute - common files ii nova-conductor 2:14.0.1-0ubuntu1~cloud0 all OpenStack Compute - conductor service ii nova-consoleauth 2:14.0.1-0ubuntu1~cloud0 all OpenStack Compute - Console Authenticator ii nova-novncproxy 2:14.0.1-0ubuntu1~cloud0 all OpenStack Compute - NoVNC proxy ii nova-scheduler 2:14.0.1-0ubuntu1~cloud0 all OpenStack Compute - virtual machine scheduler ii python-nova 2:14.0.1-0ubuntu1~cloud0 all OpenStack Compute Python libraries ii python-novaclient 2:6.0.0-0ubuntu1~cloud0 all client library for OpenStack Compute API - Python 2.7 To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1652627/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : [email protected] Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
