This is an issue with the messaging service, make sure to configure it
correctly.
** Changed in: nova
Assignee: (unassigned) => Isa Fishman (notisa)
** Changed in: nova
Status: New => Invalid
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/2012749
Title:
Instance fails to initiate with <class
'amqp.exceptions.AccessRefused'> (HTTP 500)
Status in OpenStack Compute (nova):
Invalid
Bug description:
Description:
Here's the log I was told to include.
"
2023-03-24 15:46:03.378 1889 INFO nova.osapi_compute.wsgi.server
[req-e659e737-0124-4718-8578-b5c1637eb581 a829c22d15af4e919b25a6b8f49f6d35
629b0682bf9f4e878930b936f19b6ade - default default] 10.116.0.3 "GET /v2.1
HTTP/1.1" status: 200 len: 783 time: 0.0010614
2023-03-24 15:46:03.422 1891 INFO nova.osapi_compute.wsgi.server [-]
10.116.0.3 "GET /v2.1 HTTP/1.1" status: 200 len: 783 time: 0.0015924
2023-03-24 15:46:04.275 1891 INFO nova.osapi_compute.wsgi.server
[req-b4845a8b-526b-4516-a0ab-72db01cb6a17 a829c22d15af4e919b25a6b8f49f6d35
629b0682bf9f4e878930b936f19b6ade - default default] 10.116.0.3 "GET
/v2.1/flavors/detail HTTP/1.1" status: 200 len: 860 time: 0.5756984
2023-03-24 15:46:04.283 1889 INFO nova.osapi_compute.wsgi.server
[req-53f7f032-7d3e-41c3-b3c5-5359ed8d9de3 a829c22d15af4e919b25a6b8f49f6d35
629b0682bf9f4e878930b936f19b6ade - default default] 10.116.0.3 "GET
/v2.1/os-availability-zone HTTP/1.1" status: 200 len: 495 time: 0.7366662
2023-03-24 15:46:04.303 1891 INFO nova.osapi_compute.wsgi.server
[req-d5a4c3ea-bcdc-4c0c-bf33-bd1aed4deff6 a829c22d15af4e919b25a6b8f49f6d35
629b0682bf9f4e878930b936f19b6ade - default default] 10.116.0.3 "GET
/v2.1/flavors/9f4a158d-c81f-4e28-b9e0-aced44b0c7b7/os-extra_specs HTTP/1.1"
status: 200 len: 417 time: 0.0216706
2023-03-24 15:46:04.359 1892 INFO nova.osapi_compute.wsgi.server
[req-b04cc473-774a-4f81-8ef8-c9156b68f2f6 a829c22d15af4e919b25a6b8f49f6d35
629b0682bf9f4e878930b936f19b6ade - default default] 10.116.0.3 "GET
/v2.1/limits?reserved=1 HTTP/1.1" status: 200 len: 910 time: 0.8074837
2023-03-24 15:46:04.587 1889 INFO nova.osapi_compute.wsgi.server [-]
10.116.0.3 "GET /v2.1 HTTP/1.1" status: 200 len: 783 time: 0.0016103
2023-03-24 15:46:04.675 1889 WARNING oslo_db.sqlalchemy.utils
[req-f308f20c-7deb-43e3-b545-f31932ffcac6 a829c22d15af4e919b25a6b8f49f6d35
629b0682bf9f4e878930b936f19b6ade - default default] Unique keys not in
sort_keys. The sorting order may be unstable.
2023-03-24 15:46:04.680 1889 INFO nova.osapi_compute.wsgi.server
[req-f308f20c-7deb-43e3-b545-f31932ffcac6 a829c22d15af4e919b25a6b8f49f6d35
629b0682bf9f4e878930b936f19b6ade - default default] 10.116.0.3 "GET
/v2.1/os-keypairs HTTP/1.1" status: 200 len: 414 time: 0.5834584
2023-03-24 15:46:05.056 1892 INFO nova.osapi_compute.wsgi.server
[req-27886a59-07e2-4ba5-80f0-cf8b32e5d3c0 a829c22d15af4e919b25a6b8f49f6d35
629b0682bf9f4e878930b936f19b6ade - default default] 10.116.0.3 "GET
/v2.1/servers/640e5601-460d-4ee9-81e7-d2c033ed4041 HTTP/1.1" status: 200 len:
2021 time: 1.0985825
2023-03-24 15:46:05.359 1889 INFO nova.osapi_compute.wsgi.server
[req-e659e737-0124-4718-8578-b5c1637eb581 a829c22d15af4e919b25a6b8f49f6d35
629b0682bf9f4e878930b936f19b6ade - default default] 10.116.0.3 "GET /v2.1
HTTP/1.1" status: 200 len: 783 time: 0.0013490
2023-03-24 15:46:05.374 1891 INFO nova.osapi_compute.wsgi.server
[req-bbe0eb79-33a2-45c5-b4b1-34574217f2c6 a829c22d15af4e919b25a6b8f49f6d35
629b0682bf9f4e878930b936f19b6ade - default default] 10.116.0.3 "GET
/v2.1/os-server-groups HTTP/1.1" status: 200 len: 419 time: 0.3312027
2023-03-24 15:46:11.048 1889 INFO nova.osapi_compute.wsgi.server
[req-3739f405-f817-4ff1-8883-42710fb49dcb a829c22d15af4e919b25a6b8f49f6d35
629b0682bf9f4e878930b936f19b6ade - default default] 10.116.0.3 "GET
/v2.1/limits?reserved=1 HTTP/1.1" status: 200 len: 910 time: 0.2919271
2023-03-24 15:46:11.051 1889 INFO nova.osapi_compute.wsgi.server
[req-3739f405-f817-4ff1-8883-42710fb49dcb a829c22d15af4e919b25a6b8f49f6d35
629b0682bf9f4e878930b936f19b6ade - default default] 10.116.0.3 "GET /v2.1
HTTP/1.1" status: 200 len: 783 time: 0.0014436
2023-03-24 15:46:11.511 1890 INFO nova.osapi_compute.wsgi.server
[req-f4996eb9-133a-488f-8461-2c23385deca3 a829c22d15af4e919b25a6b8f49f6d35
629b0682bf9f4e878930b936f19b6ade - default default] 10.116.0.3 "GET
/v2.1/servers/640e5601-460d-4ee9-81e7-d2c033ed4041 HTTP/1.1" status: 200 len:
2021 time: 0.2696385
2023-03-24 15:46:11.741 1889 INFO nova.osapi_compute.wsgi.server
[req-3739f405-f817-4ff1-8883-42710fb49dcb a829c22d15af4e919b25a6b8f49f6d35
629b0682bf9f4e878930b936f19b6ade - default default] 10.116.0.3 "GET /v2.1
HTTP/1.1" status: 200 len: 783 time: 0.0009189
2023-03-24 15:46:18.815 1890 INFO nova.osapi_compute.wsgi.server [-]
10.116.0.3 "GET /v2.1 HTTP/1.1" status: 200 len: 783 time: 0.0011003
2023-03-24 15:46:19.177 1890 WARNING oslo_config.cfg
[req-b6e5cc67-e09b-4aaf-804a-2beade3a2174 a829c22d15af4e919b25a6b8f49f6d35
629b0682bf9f4e878930b936f19b6ade - default default] Deprecated: Option
"api_servers" from group "glance" is deprecated for removal (
Support for image service configuration via standard keystoneauth1 Adapter
options was added in the 17.0.0 Queens release. The api_servers option was
retained temporarily to allow consumers time to cut over to a real load
balancing solution.
). Its value may be silently ignored in the future.
2023-03-24 15:46:19.840 1891 INFO nova.osapi_compute.wsgi.server
[req-4ce507fd-ff77-4611-8b00-3d2b1cb18a48 a829c22d15af4e919b25a6b8f49f6d35
629b0682bf9f4e878930b936f19b6ade - default default] 10.116.0.3 "GET
/v2.1/limits?reserved=1 HTTP/1.1" status: 200 len: 910 time: 0.2121718
2023-03-24 15:46:19.849 1891 INFO nova.osapi_compute.wsgi.server
[req-4ce507fd-ff77-4611-8b00-3d2b1cb18a48 a829c22d15af4e919b25a6b8f49f6d35
629b0682bf9f4e878930b936f19b6ade - default default] 10.116.0.3 "GET /v2.1
HTTP/1.1" status: 200 len: 783 time: 0.0012639
2023-03-24 15:46:20.230 1890 ERROR nova.api.openstack.wsgi
[req-b6e5cc67-e09b-4aaf-804a-2beade3a2174 a829c22d15af4e919b25a6b8f49f6d35
629b0682bf9f4e878930b936f19b6ade - default default] Unexpected exception in API
method: amqp.exceptions.AccessRefused: (0, 0): (403) ACCESS_REFUSED - Login was
refused using authentication mechanism AMQPLAIN. For details see the broker
logfile.
2023-03-24 15:46:20.230 1890 ERROR nova.api.openstack.wsgi Traceback (most
recent call last):
2023-03-24 15:46:20.230 1890 ERROR nova.api.openstack.wsgi File
"/usr/lib/python3/dist-packages/nova/api/openstack/wsgi.py", line 658, in
wrapped
2023-03-24 15:46:20.230 1890 ERROR nova.api.openstack.wsgi return
f(*args, **kwargs)
2023-03-24 15:46:20.230 1890 ERROR nova.api.openstack.wsgi File
"/usr/lib/python3/dist-packages/nova/api/validation/__init__.py", line 110, in
wrapper
2023-03-24 15:46:20.230 1890 ERROR nova.api.openstack.wsgi return
func(*args, **kwargs)
2023-03-24 15:46:20.230 1890 ERROR nova.api.openstack.wsgi File
"/usr/lib/python3/dist-packages/nova/api/validation/__init__.py", line 110, in
wrapper
2023-03-24 15:46:20.230 1890 ERROR nova.api.openstack.wsgi return
func(*args, **kwargs)
2023-03-24 15:46:20.230 1890 ERROR nova.api.openstack.wsgi File
"/usr/lib/python3/dist-packages/nova/api/validation/__init__.py", line 110, in
wrapper
2023-03-24 15:46:20.230 1890 ERROR nova.api.openstack.wsgi return
func(*args, **kwargs)
2023-03-24 15:46:20.230 1890 ERROR nova.api.openstack.wsgi [Previous line
repeated 10 more times]
2023-03-24 15:46:20.230 1890 ERROR nova.api.openstack.wsgi File
"/usr/lib/python3/dist-packages/nova/api/openstack/compute/servers.py", line
784, in create
2023-03-24 15:46:20.230 1890 ERROR nova.api.openstack.wsgi instances,
resv_id = self.compute_api.create(
2023-03-24 15:46:20.230 1890 ERROR nova.api.openstack.wsgi File
"/usr/lib/python3/dist-packages/nova/compute/api.py", line 2149, in create
2023-03-24 15:46:20.230 1890 ERROR nova.api.openstack.wsgi return
self._create_instance(
2023-03-24 15:46:20.230 1890 ERROR nova.api.openstack.wsgi File
"/usr/lib/python3/dist-packages/nova/compute/api.py", line 1697, in
_create_instance
2023-03-24 15:46:20.230 1890 ERROR nova.api.openstack.wsgi
self.compute_task_api.schedule_and_build_instances(
2023-03-24 15:46:20.230 1890 ERROR nova.api.openstack.wsgi File
"/usr/lib/python3/dist-packages/nova/conductor/api.py", line 134, in
schedule_and_build_instances
2023-03-24 15:46:20.230 1890 ERROR nova.api.openstack.wsgi
self.conductor_compute_rpcapi.schedule_and_build_instances(
2023-03-24 15:46:20.230 1890 ERROR nova.api.openstack.wsgi File
"/usr/lib/python3/dist-packages/nova/conductor/rpcapi.py", line 413, in
schedule_and_build_instances
2023-03-24 15:46:20.230 1890 ERROR nova.api.openstack.wsgi
cctxt.cast(context, 'schedule_and_build_instances', **kw)
2023-03-24 15:46:20.230 1890 ERROR nova.api.openstack.wsgi File
"/usr/lib/python3/dist-packages/oslo_messaging/rpc/client.py", line 155, in cast
2023-03-24 15:46:20.230 1890 ERROR nova.api.openstack.wsgi
self.transport._send(self.target, msg_ctxt, msg,
2023-03-24 15:46:20.230 1890 ERROR nova.api.openstack.wsgi File
"/usr/lib/python3/dist-packages/oslo_messaging/transport.py", line 123, in _send
2023-03-24 15:46:20.230 1890 ERROR nova.api.openstack.wsgi return
self._driver.send(target, ctxt, message,
2023-03-24 15:46:20.230 1890 ERROR nova.api.openstack.wsgi File
"/usr/lib/python3/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line
689, in send
2023-03-24 15:46:20.230 1890 ERROR nova.api.openstack.wsgi return
self._send(target, ctxt, message, wait_for_reply, timeout,
2023-03-24 15:46:20.230 1890 ERROR nova.api.openstack.wsgi File
"/usr/lib/python3/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line
653, in _send
2023-03-24 15:46:20.230 1890 ERROR nova.api.openstack.wsgi with
self._get_connection(rpc_common.PURPOSE_SEND, retry) as conn:
2023-03-24 15:46:20.230 1890 ERROR nova.api.openstack.wsgi File
"/usr/lib/python3/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line
605, in _get_connection
2023-03-24 15:46:20.230 1890 ERROR nova.api.openstack.wsgi return
rpc_common.ConnectionContext(self._connection_pool,
2023-03-24 15:46:20.230 1890 ERROR nova.api.openstack.wsgi File
"/usr/lib/python3/dist-packages/oslo_messaging/_drivers/common.py", line 423,
in __init__
2023-03-24 15:46:20.230 1890 ERROR nova.api.openstack.wsgi
self.connection = connection_pool.get(retry=retry)
2023-03-24 15:46:20.230 1890 ERROR nova.api.openstack.wsgi File
"/usr/lib/python3/dist-packages/oslo_messaging/_drivers/pool.py", line 98, in
get
2023-03-24 15:46:20.230 1890 ERROR nova.api.openstack.wsgi return
self.create(retry=retry)
2023-03-24 15:46:20.230 1890 ERROR nova.api.openstack.wsgi File
"/usr/lib/python3/dist-packages/oslo_messaging/_drivers/pool.py", line 135, in
create
2023-03-24 15:46:20.230 1890 ERROR nova.api.openstack.wsgi return
self.connection_cls(self.conf, self.url, purpose, retry=retry)
2023-03-24 15:46:20.230 1890 ERROR nova.api.openstack.wsgi File
"/usr/lib/python3/dist-packages/oslo_messaging/_drivers/impl_rabbit.py", line
648, in __init__
2023-03-24 15:46:20.230 1890 ERROR nova.api.openstack.wsgi
self.ensure_connection()
2023-03-24 15:46:20.230 1890 ERROR nova.api.openstack.wsgi File
"/usr/lib/python3/dist-packages/oslo_messaging/_drivers/impl_rabbit.py", line
771, in ensure_connection
2023-03-24 15:46:20.230 1890 ERROR nova.api.openstack.wsgi
self.connection.ensure_connection(
2023-03-24 15:46:20.230 1890 ERROR nova.api.openstack.wsgi File
"/usr/lib/python3/dist-packages/kombu/connection.py", line 382, in
ensure_connection
2023-03-24 15:46:20.230 1890 ERROR nova.api.openstack.wsgi
self._ensure_connection(*args, **kwargs)
2023-03-24 15:46:20.230 1890 ERROR nova.api.openstack.wsgi File
"/usr/lib/python3/dist-packages/kombu/connection.py", line 434, in
_ensure_connection
2023-03-24 15:46:20.230 1890 ERROR nova.api.openstack.wsgi return
retry_over_time(
2023-03-24 15:46:20.230 1890 ERROR nova.api.openstack.wsgi File
"/usr/lib/python3/dist-packages/kombu/utils/functional.py", line 312, in
retry_over_time
2023-03-24 15:46:20.230 1890 ERROR nova.api.openstack.wsgi return
fun(*args, **kwargs)
2023-03-24 15:46:20.230 1890 ERROR nova.api.openstack.wsgi File
"/usr/lib/python3/dist-packages/kombu/connection.py", line 878, in
_connection_factory
2023-03-24 15:46:20.230 1890 ERROR nova.api.openstack.wsgi
self._connection = self._establish_connection()
2023-03-24 15:46:20.230 1890 ERROR nova.api.openstack.wsgi File
"/usr/lib/python3/dist-packages/kombu/connection.py", line 813, in
_establish_connection
2023-03-24 15:46:20.230 1890 ERROR nova.api.openstack.wsgi conn =
self.transport.establish_connection()
2023-03-24 15:46:20.230 1890 ERROR nova.api.openstack.wsgi File
"/usr/lib/python3/dist-packages/kombu/transport/pyamqp.py", line 201, in
establish_connection
2023-03-24 15:46:20.230 1890 ERROR nova.api.openstack.wsgi conn.connect()
2023-03-24 15:46:20.230 1890 ERROR nova.api.openstack.wsgi File
"/usr/lib/python3/dist-packages/amqp/connection.py", line 329, in connect
2023-03-24 15:46:20.230 1890 ERROR nova.api.openstack.wsgi
self.drain_events(timeout=self.connect_timeout)
2023-03-24 15:46:20.230 1890 ERROR nova.api.openstack.wsgi File
"/usr/lib/python3/dist-packages/amqp/connection.py", line 522, in drain_events
2023-03-24 15:46:20.230 1890 ERROR nova.api.openstack.wsgi while not
self.blocking_read(timeout):
2023-03-24 15:46:20.230 1890 ERROR nova.api.openstack.wsgi File
"/usr/lib/python3/dist-packages/amqp/connection.py", line 528, in blocking_read
2023-03-24 15:46:20.230 1890 ERROR nova.api.openstack.wsgi return
self.on_inbound_frame(frame)
2023-03-24 15:46:20.230 1890 ERROR nova.api.openstack.wsgi File
"/usr/lib/python3/dist-packages/amqp/method_framing.py", line 53, in on_frame
2023-03-24 15:46:20.230 1890 ERROR nova.api.openstack.wsgi
callback(channel, method_sig, buf, None)
2023-03-24 15:46:20.230 1890 ERROR nova.api.openstack.wsgi File
"/usr/lib/python3/dist-packages/amqp/connection.py", line 534, in
on_inbound_method
2023-03-24 15:46:20.230 1890 ERROR nova.api.openstack.wsgi return
self.channels[channel_id].dispatch_method(
2023-03-24 15:46:20.230 1890 ERROR nova.api.openstack.wsgi File
"/usr/lib/python3/dist-packages/amqp/abstract_channel.py", line 143, in
dispatch_method
2023-03-24 15:46:20.230 1890 ERROR nova.api.openstack.wsgi listener(*args)
2023-03-24 15:46:20.230 1890 ERROR nova.api.openstack.wsgi File
"/usr/lib/python3/dist-packages/amqp/connection.py", line 664, in _on_close
2023-03-24 15:46:20.230 1890 ERROR nova.api.openstack.wsgi raise
error_for_code(reply_code, reply_text,
2023-03-24 15:46:20.230 1890 ERROR nova.api.openstack.wsgi
amqp.exceptions.AccessRefused: (0, 0): (403) ACCESS_REFUSED - Login was refused
using authentication mechanism AMQPLAIN. For details see the broker logfile.
2023-03-24 15:46:20.230 1890 ERROR nova.api.openstack.wsgi
2023-03-24 15:46:20.236 1890 INFO nova.api.openstack.wsgi
[req-b6e5cc67-e09b-4aaf-804a-2beade3a2174 a829c22d15af4e919b25a6b8f49f6d35
629b0682bf9f4e878930b936f19b6ade - default default] HTTP exception thrown:
Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and
attach the Nova API log if possible.
<class 'amqp.exceptions.AccessRefused'>
"
I used the recommend Ubuntu install guide, setup everything according to the
guide, and then tried to initialize an instance, that's when it errored out.
I'm running 2 servers, One is only running compute and the other one is
running identity and such. Here is what the identity server is running:
ii nova-api 3:25.1.0-0ubuntu1~cloud0
all OpenStack Compute - API frontend
ii nova-common 3:25.1.0-0ubuntu1~cloud0
all OpenStack Compute - common files
ii nova-conductor 3:25.1.0-0ubuntu1~cloud0
all OpenStack Compute - conductor service
ii nova-novncproxy 3:25.1.0-0ubuntu1~cloud0
all OpenStack Compute - NoVNC proxy
ii nova-scheduler 3:25.1.0-0ubuntu1~cloud0
all OpenStack Compute - virtual machine scheduler
ii python3-nova 3:25.1.0-0ubuntu1~cloud0
all OpenStack Compute Python 3 libraries
ii python3-novaclient 2:17.6.0-0ubuntu1~cloud0
all client library for OpenStack Compute API - 3.x
I'm just trying to configure this in the cloud before i do it at my
home lab.
To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/2012749/+subscriptions
--
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : [email protected]
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help : https://help.launchpad.net/ListHelp
