[Expired for OpenStack Compute (nova) because there has been no activity
for 60 days.]
** Changed in: nova
Status: Incomplete => Expired
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1891973
Title:
Getting HTTP 500 exception frequently in Openstack Ussuri
Status in OpenStack Compute (nova):
Expired
Bug description:
I am using openstack ussuri with Centos 8, installed using RDO packstack.
I frequently getting http 500 EXCEPTION ERROR when ever am doing some
operation, i need to restart httpd service inorder to overcome this issue.
Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/
and attach the Nova API log if possible.
<class 'sqlalchemy.exc.OperationalError'> (HTTP 500) (Request-ID:
req-8868aed7-265d-4e8e-aa37-d427b1b561cb)
: nova.exception.ResourceProviderAllocationRetrievalFailed: Failed to
retrieve allocations for resource provider
3a80ea71-91a6-4da7-a05b-682b501fe6c9: <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML
2.0//EN">
2020-08-17 22:53:46.574 63567 ERROR nova.scheduler.client.report
[req-fb01576e-e540-4930-a4f0-5109a0df28c4 - - - - -] [None] Failed to retrieve
resource provider tree from placement API for UUID
3a80ea71-91a6-4da7-a05b-682b501fe6c9. Got 500: <!DOCTYPE HTML PUBLIC
"-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>500 Internal Server Error</title>
</head><body>
<h1>Internal Server Error</h1>
<p>The server encountered an internal error or
misconfiguration and was unable to complete
your request.</p>
<p>Please contact the server administrator at
[no address given] to inform them of the time this error occurred,
and the actions you performed just before this error.</p>
<p>More information about this error may be available
in the server error log.</p>
</body></html>
.
2020-08-17 22:53:46.575 63567 ERROR nova.compute.manager
[req-fb01576e-e540-4930-a4f0-5109a0df28c4 - - - - -] Error updating resources
for node openstack-Karty.: nova.exception.ResourceProviderRetrievalFailed:
Failed to get resource provider with UUID 3a80ea71-91a6-4da7-a05b-682b501fe6c9
2020-08-17 22:53:46.575 63567 ERROR nova.compute.manager Traceback (most
recent call last):
2020-08-17 22:53:46.575 63567 ERROR nova.compute.manager File
"/usr/lib/python3.6/site-packages/nova/compute/manager.py", line 9685, in
_update_available_resource_for_node
2020-08-17 22:53:46.575 63567 ERROR nova.compute.manager startup=startup)
2020-08-17 22:53:46.575 63567 ERROR nova.compute.manager File
"/usr/lib/python3.6/site-packages/nova/compute/resource_tracker.py", line 842,
in update_available_resource
2020-08-17 22:53:46.575 63567 ERROR nova.compute.manager
self._update_available_resource(context, resources, startup=startup)
2020-08-17 22:53:46.575 63567 ERROR nova.compute.manager File
"/usr/lib/python3.6/site-packages/oslo_concurrency/lockutils.py", line 359, in
inner
2020-08-17 22:53:46.575 63567 ERROR nova.compute.manager return f(*args,
**kwargs)
2020-08-17 22:53:46.575 63567 ERROR nova.compute.manager File
"/usr/lib/python3.6/site-packages/nova/compute/resource_tracker.py", line 927,
in _update_available_resource
2020-08-17 22:53:46.575 63567 ERROR nova.compute.manager
self._update(context, cn, startup=startup)
2020-08-17 22:53:46.575 63567 ERROR nova.compute.manager File
"/usr/lib/python3.6/site-packages/nova/compute/resource_tracker.py", line 1176,
in _update
2020-08-17 22:53:46.575 63567 ERROR nova.compute.manager
self._update_to_placement(context, compute_node, startup)
2020-08-17 22:53:46.575 63567 ERROR nova.compute.manager File
"/usr/lib/python3.6/site-packages/retrying.py", line 68, in wrapped_f
2020-08-17 22:53:46.575 63567 ERROR nova.compute.manager return
Retrying(*dargs, **dkw).call(f, *args, **kw)
2020-08-17 22:53:46.575 63567 ERROR nova.compute.manager File
"/usr/lib/python3.6/site-packages/retrying.py", line 223, in call
2020-08-17 22:53:46.575 63567 ERROR nova.compute.manager return
attempt.get(self._wrap_exception)
2020-08-17 22:53:46.575 63567 ERROR nova.compute.manager File
"/usr/lib/python3.6/site-packages/retrying.py", line 261, in get
2020-08-17 22:53:46.575 63567 ERROR nova.compute.manager
six.reraise(self.value[0], self.value[1], self.value[2])
2020-08-17 22:53:46.575 63567 ERROR nova.compute.manager File
"/usr/lib/python3.6/site-packages/six.py", line 703, in reraise
2020-08-17 22:53:46.575 63567 ERROR nova.compute.manager raise value
2020-08-17 22:53:46.575 63567 ERROR nova.compute.manager File
"/usr/lib/python3.6/site-packages/retrying.py", line 217, in call
2020-08-17 22:53:46.575 63567 ERROR nova.compute.manager attempt =
Attempt(fn(*args, **kwargs), attempt_number, False)
2020-08-17 22:53:46.575 63567 ERROR nova.compute.manager File
"/usr/lib/python3.6/site-packages/nova/compute/resource_tracker.py", line 1110,
in _update_to_placement
2020-08-17 22:53:46.575 63567 ERROR nova.compute.manager context,
compute_node.uuid, name=compute_node.hypervisor_hostname)
2020-08-17 22:53:46.575 63567 ERROR nova.compute.manager File
"/usr/lib/python3.6/site-packages/nova/scheduler/client/report.py", line 857,
in get_provider_tree_and_ensure_root
2020-08-17 22:53:46.575 63567 ERROR nova.compute.manager
parent_provider_uuid=parent_provider_uuid)
2020-08-17 22:53:46.575 63567 ERROR nova.compute.manager File
"/usr/lib/python3.6/site-packages/nova/scheduler/client/report.py", line 640,
in _ensure_resource_provider
2020-08-17 22:53:46.575 63567 ERROR nova.compute.manager rps_to_refresh =
self.get_providers_in_tree(context, uuid)
2020-08-17 22:53:46.575 63567 ERROR nova.compute.manager File
"/usr/lib/python3.6/site-packages/nova/scheduler/client/report.py", line 503,
in get_providers_in_tree
2020-08-17 22:53:46.575 63567 ERROR nova.compute.manager raise
exception.ResourceProviderRetrievalFailed(uuid=uuid)
2020-08-17 22:53:46.575 63567 ERROR nova.compute.manager
nova.exception.ResourceProviderRetrievalFailed: Failed to get resource provider
with UUID 3a80ea71-91a6-4da7-a05b-682b501fe6c9
2020-08-17 22:53:46.575 63567 ERROR nova.compute.manager
2020-08-17 22:54:46.284 63567 ERROR nova.scheduler.client.report
[req-fb01576e-e540-4930-a4f0-5109a0df28c4 - - - - -] [None] Failed to retrieve
resource provider tree from placement API for UUID
3a80ea71-91a6-4da7-a05b-682b501fe6c9. Got 500: <!DOCTYPE HTML PUBLIC
"-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>500 Internal Server Error</title>
</head><body>
<h1>Internal Server Error</h1>
<p>The server encountered an internal error or
misconfiguration and was unable to complete
your request.</p>
<p>Please contact the server administrator at
[no address given] to inform them of the time this error occurred,
and the actions you performed just before this error.</p>
<p>More information about this error may be available
in the server error log.</p>
</body></html>
.
2020-08-17 22:54:46.285 63567 ERROR nova.compute.manager
[req-fb01576e-e540-4930-a4f0-5109a0df28c4 - - - - -] Error updating resources
for node openstack-Karty.: nova.exception.ResourceProviderRetrievalFailed:
Failed to get resource provider with UUID 3a80ea71-91a6-4da7-a05b-682b501fe6c9
2020-08-17 22:54:46.285 63567 ERROR nova.compute.manager Traceback (most
recent call last):
2020-08-17 22:54:46.285 63567 ERROR nova.compute.manager File
"/usr/lib/python3.6/site-packages/nova/compute/manager.py", line 9685, in
_update_available_resource_for_node
2020-08-17 22:54:46.285 63567 ERROR nova.compute.manager startup=startup)
2020-08-17 22:54:46.285 63567 ERROR nova.compute.manager File
"/usr/lib/python3.6/site-packages/nova/compute/resource_tracker.py", line 842,
in update_available_resource
2020-08-17 22:54:46.285 63567 ERROR nova.compute.manager
self._update_available_resource(context, resources, startup=startup)
2020-08-17 22:54:46.285 63567 ERROR nova.compute.manager File
"/usr/lib/python3.6/site-packages/oslo_concurrency/lockutils.py", line 359, in
inner
2020-08-17 22:54:46.285 63567 ERROR nova.compute.manager return f(*args,
**kwargs)
2020-08-17 22:54:46.285 63567 ERROR nova.compute.manager File
"/usr/lib/python3.6/site-packages/nova/compute/resource_tracker.py", line 927,
in _update_available_resource
2020-08-17 22:54:46.285 63567 ERROR nova.compute.manager
self._update(context, cn, startup=startup)
2020-08-17 22:54:46.285 63567 ERROR nova.compute.manager File
"/usr/lib/python3.6/site-packages/nova/compute/resource_tracker.py", line 1176,
in _update
2020-08-17 22:54:46.285 63567 ERROR nova.compute.manager
self._update_to_placement(context, compute_node, startup)
2020-08-17 22:54:46.285 63567 ERROR nova.compute.manager File
"/usr/lib/python3.6/site-packages/retrying.py", line 68, in wrapped_f
2020-08-17 22:54:46.285 63567 ERROR nova.compute.manager return
Retrying(*dargs, **dkw).call(f, *args, **kw)
2020-08-17 22:54:46.285 63567 ERROR nova.compute.manager File
"/usr/lib/python3.6/site-packages/retrying.py", line 223, in call
2020-08-17 22:54:46.285 63567 ERROR nova.compute.manager return
attempt.get(self._wrap_exception)
2020-08-17 22:54:46.285 63567 ERROR nova.compute.manager File
"/usr/lib/python3.6/site-packages/retrying.py", line 261, in get
2020-08-17 22:54:46.285 63567 ERROR nova.compute.manager
six.reraise(self.value[0], self.value[1], self.value[2])
2020-08-17 22:54:46.285 63567 ERROR nova.compute.manager File
"/usr/lib/python3.6/site-packages/six.py", line 703, in reraise
2020-08-17 22:54:46.285 63567 ERROR nova.compute.manager raise value
2020-08-17 22:54:46.285 63567 ERROR nova.compute.manager File
"/usr/lib/python3.6/site-packages/retrying.py", line 217, in call
2020-08-17 22:54:46.285 63567 ERROR nova.compute.manager attempt =
Attempt(fn(*args, **kwargs), attempt_number, False)
2020-08-17 22:54:46.285 63567 ERROR nova.compute.manager File
"/usr/lib/python3.6/site-packages/nova/compute/resource_tracker.py", line 1110,
in _update_to_placement
2020-08-17 22:54:46.285 63567 ERROR nova.compute.manager context,
compute_node.uuid, name=compute_node.hypervisor_hostname)
2020-08-17 22:54:46.285 63567 ERROR nova.compute.manager File
"/usr/lib/python3.6/site-packages/nova/scheduler/client/report.py", line 857,
in get_provider_tree_and_ensure_root
2020-08-17 22:54:46.285 63567 ERROR nova.compute.manager
parent_provider_uuid=parent_provider_uuid)
2020-08-17 22:54:46.285 63567 ERROR nova.compute.manager File
"/usr/lib/python3.6/site-packages/nova/scheduler/client/report.py", line 640,
in _ensure_resource_provider
2020-08-17 22:54:46.285 63567 ERROR nova.compute.manager rps_to_refresh =
self.get_providers_in_tree(context, uuid)
2020-08-17 22:54:46.285 63567 ERROR nova.compute.manager File
"/usr/lib/python3.6/site-packages/nova/scheduler/client/report.py", line 503,
in get_providers_in_tree
2020-08-17 22:54:46.285 63567 ERROR nova.compute.manager raise
exception.ResourceProviderRetrievalFailed(uuid=uuid)
2020-08-17 22:54:46.285 63567 ERROR nova.compute.manager
nova.exception.ResourceProviderRetrievalFailed: Failed to get resource provider
with UUID 3a80ea71-91a6-4da7-a05b-682b501fe6c9
2020-08-17 22:54:46.285 63567 ERROR nova.compute.manager
root@openstack-Karty nova(keystone_admin)]# openstack server list
Unknown Error (HTTP 500)
[root@openstack-Karty nova(keystone_admin)]# openstack server list
Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/
and attach the Nova API log if possible.
<class 'TypeError'> (HTTP 500) (Request-ID:
req-7d3a09b6-2e6a-43b5-8b4b-03c1ac3d37dd)
[root@openstack-Karty nova(keystone_admin)]# tail -f -n 100 nova-compute.log
2020-08-17 22:56:47.807 63567 ERROR nova.compute.manager File
"/usr/lib/python3.6/site-packages/urllib3/connection.py", line 169, in _new_conn
2020-08-17 22:56:47.807 63567 ERROR nova.compute.manager self, "Failed to
establish a new connection: %s" % e
2020-08-17 22:56:47.807 63567 ERROR nova.compute.manager
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection
object at 0x7ff128340518>: Failed to establish a new connection: [Errno 111]
ECONNREFUSED
2020-08-17 22:56:47.807 63567 ERROR nova.compute.manager
2020-08-17 22:56:47.807 63567 ERROR nova.compute.manager During handling of
the above exception, another exception occurred:
2020-08-17 22:56:47.807 63567 ERROR nova.compute.manager
2020-08-17 22:56:47.807 63567 ERROR nova.compute.manager Traceback (most
recent call last):
2020-08-17 22:56:47.807 63567 ERROR nova.compute.manager File
"/usr/lib/python3.6/site-packages/requests/adapters.py", line 449, in send
2020-08-17 22:56:47.807 63567 ERROR nova.compute.manager timeout=timeout
2020-08-17 22:56:47.807 63567 ERROR nova.compute.manager File
"/usr/lib/python3.6/site-packages/urllib3/connectionpool.py", line 720, in
urlopen
2020-08-17 22:56:47.807 63567 ERROR nova.compute.manager method, url,
error=e, _pool=self, _stacktrace=sys.exc_info()[2]
2020-08-17 22:56:47.807 63567 ERROR nova.compute.manager File
"/usr/lib/python3.6/site-packages/urllib3/util/retry.py", line 436, in increment
2020-08-17 22:56:47.807 63567 ERROR nova.compute.manager raise
MaxRetryError(_pool, url, error or ResponseError(cause))
2020-08-17 22:56:47.807 63567 ERROR nova.compute.manager
urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='10.24.0.117',
port=8778): Max retries exceeded with url:
/placement/resource_providers?in_tree=3a80ea71-91a6-4da7-a05b-682b501fe6c9
(Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at
0x7ff128340518>: Failed to establish a new connection: [Errno 111]
ECONNREFUSED',))
2020-08-17 22:56:47.807 63567 ERROR nova.compute.manager
2020-08-17 22:56:47.807 63567 ERROR nova.compute.manager During handling of
the above exception, another exception occurred:
2020-08-17 22:56:47.807 63567 ERROR nova.compute.manager
2020-08-17 22:56:47.807 63567 ERROR nova.compute.manager Traceback (most
recent call last):
2020-08-17 22:56:47.807 63567 ERROR nova.compute.manager File
"/usr/lib/python3.6/site-packages/keystoneauth1/session.py", line 1004, in
_send_request
2020-08-17 22:56:47.807 63567 ERROR nova.compute.manager resp =
self.session.request(method, url, **kwargs)
2020-08-17 22:56:47.807 63567 ERROR nova.compute.manager File
"/usr/lib/python3.6/site-packages/requests/sessions.py", line 533, in request
2020-08-17 22:56:47.807 63567 ERROR nova.compute.manager resp =
self.send(prep, **send_kwargs)
2020-08-17 22:56:47.807 63567 ERROR nova.compute.manager File
"/usr/lib/python3.6/site-packages/requests/sessions.py", line 646, in send
2020-08-17 22:56:47.807 63567 ERROR nova.compute.manager r =
adapter.send(request, **kwargs)
2020-08-17 22:56:47.807 63567 ERROR nova.compute.manager File
"/usr/lib/python3.6/site-packages/requests/adapters.py", line 516, in send
2020-08-17 22:56:47.807 63567 ERROR nova.compute.manager raise
ConnectionError(e, request=request)
2020-08-17 22:56:47.807 63567 ERROR nova.compute.manager
requests.exceptions.ConnectionError: HTTPConnectionPool(host='10.24.0.117',
port=8778): Max retries exceeded with url:
/placement/resource_providers?in_tree=3a80ea71-91a6-4da7-a05b-682b501fe6c9
(Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at
0x7ff128340518>: Failed to establish a new connection: [Errno 111]
ECONNREFUSED',))
2020-08-17 22:56:47.807 63567 ERROR nova.compute.manager
2020-08-17 22:56:47.807 63567 ERROR nova.compute.manager During handling of
the above exception, another exception occurred:
2020-08-17 22:56:47.807 63567 ERROR nova.compute.manager
2020-08-17 22:56:47.807 63567 ERROR nova.compute.manager Traceback (most
recent call last):
2020-08-17 22:56:47.807 63567 ERROR nova.compute.manager File
"/usr/lib/python3.6/site-packages/nova/compute/manager.py", line 9685, in
_update_available_resource_for_node
2020-08-17 22:56:47.807 63567 ERROR nova.compute.manager startup=startup)
2020-08-17 22:56:47.807 63567 ERROR nova.compute.manager File
"/usr/lib/python3.6/site-packages/nova/compute/resource_tracker.py", line 842,
in update_available_resource
2020-08-17 22:56:47.807 63567 ERROR nova.compute.manager
self._update_available_resource(context, resources, startup=startup)
2020-08-17 22:56:47.807 63567 ERROR nova.compute.manager File
"/usr/lib/python3.6/site-packages/oslo_concurrency/lockutils.py", line 359, in
inner
2020-08-17 22:56:47.807 63567 ERROR nova.compute.manager return f(*args,
**kwargs)
2020-08-17 22:56:47.807 63567 ERROR nova.compute.manager File
"/usr/lib/python3.6/site-packages/nova/compute/resource_tracker.py", line 927,
in _update_available_resource
2020-08-17 22:56:47.807 63567 ERROR nova.compute.manager
self._update(context, cn, startup=startup)
2020-08-17 22:56:47.807 63567 ERROR nova.compute.manager File
"/usr/lib/python3.6/site-packages/nova/compute/resource_tracker.py", line 1176,
in _update
2020-08-17 22:56:47.807 63567 ERROR nova.compute.manager
self._update_to_placement(context, compute_node, startup)
2020-08-17 22:56:47.807 63567 ERROR nova.compute.manager File
"/usr/lib/python3.6/site-packages/retrying.py", line 68, in wrapped_f
2020-08-17 22:56:47.807 63567 ERROR nova.compute.manager return
Retrying(*dargs, **dkw).call(f, *args, **kw)
2020-08-17 22:56:47.807 63567 ERROR nova.compute.manager File
"/usr/lib/python3.6/site-packages/retrying.py", line 223, in call
2020-08-17 22:56:47.807 63567 ERROR nova.compute.manager return
attempt.get(self._wrap_exception)
2020-08-17 22:56:47.807 63567 ERROR nova.compute.manager File
"/usr/lib/python3.6/site-packages/retrying.py", line 261, in get
2020-08-17 22:56:47.807 63567 ERROR nova.compute.manager
six.reraise(self.value[0], self.value[1], self.value[2])
2020-08-17 22:56:47.807 63567 ERROR nova.compute.manager File
"/usr/lib/python3.6/site-packages/six.py", line 703, in reraise
2020-08-17 22:56:47.807 63567 ERROR nova.compute.manager raise value
2020-08-17 22:56:47.807 63567 ERROR nova.compute.manager File
"/usr/lib/python3.6/site-packages/retrying.py", line 217, in call
2020-08-17 22:56:47.807 63567 ERROR nova.compute.manager attempt =
Attempt(fn(*args, **kwargs), attempt_number, False)
2020-08-17 22:56:47.807 63567 ERROR nova.compute.manager File
"/usr/lib/python3.6/site-packages/nova/compute/resource_tracker.py", line 1110,
in _update_to_placement
2020-08-17 22:56:47.807 63567 ERROR nova.compute.manager context,
compute_node.uuid, name=compute_node.hypervisor_hostname)
2020-08-17 22:56:47.807 63567 ERROR nova.compute.manager File
"/usr/lib/python3.6/site-packages/nova/scheduler/client/report.py", line 857,
in get_provider_tree_and_ensure_root
2020-08-17 22:56:47.807 63567 ERROR nova.compute.manager
parent_provider_uuid=parent_provider_uuid)
2020-08-17 22:56:47.807 63567 ERROR nova.compute.manager File
"/usr/lib/python3.6/site-packages/nova/scheduler/client/report.py", line 640,
in _ensure_resource_provider
2020-08-17 22:56:47.807 63567 ERROR nova.compute.manager rps_to_refresh =
self.get_providers_in_tree(context, uuid)
2020-08-17 22:56:47.807 63567 ERROR nova.compute.manager File
"/usr/lib/python3.6/site-packages/nova/scheduler/client/report.py", line 486,
in get_providers_in_tree
2020-08-17 22:56:47.807 63567 ERROR nova.compute.manager
global_request_id=context.global_id)
2020-08-17 22:56:47.807 63567 ERROR nova.compute.manager File
"/usr/lib/python3.6/site-packages/nova/scheduler/client/report.py", line 237,
in get
2020-08-17 22:56:47.807 63567 ERROR nova.compute.manager
global_request_id=global_request_id)
2020-08-17 22:56:47.807 63567 ERROR nova.compute.manager File
"/usr/lib/python3.6/site-packages/keystoneauth1/adapter.py", line 386, in get
2020-08-17 22:56:47.807 63567 ERROR nova.compute.manager return
self.request(url, 'GET', **kwargs)
2020-08-17 22:56:47.807 63567 ERROR nova.compute.manager File
"/usr/lib/python3.6/site-packages/openstack/proxy.py", line 98, in request
2020-08-17 22:56:47.807 63567 ERROR nova.compute.manager **kwargs)
2020-08-17 22:56:47.807 63567 ERROR nova.compute.manager File
"/usr/lib/python3.6/site-packages/keystoneauth1/adapter.py", line 248, in
request
2020-08-17 22:56:47.807 63567 ERROR nova.compute.manager return
self.session.request(url, method, **kwargs)
2020-08-17 22:56:47.807 63567 ERROR nova.compute.manager File
"/usr/lib/python3.6/site-packages/keystoneauth1/session.py", line 913, in
request
2020-08-17 22:56:47.807 63567 ERROR nova.compute.manager resp =
send(**kwargs)
2020-08-17 22:56:47.807 63567 ERROR nova.compute.manager File
"/usr/lib/python3.6/site-packages/keystoneauth1/session.py", line 1043, in
_send_request
2020-08-17 22:56:47.807 63567 ERROR nova.compute.manager **kwargs)
2020-08-17 22:56:47.807 63567 ERROR nova.compute.manager File
"/usr/lib/python3.6/site-packages/keystoneauth1/session.py", line 1020, in
_send_request
2020-08-17 22:56:47.807 63567 ERROR nova.compute.manager raise
exceptions.ConnectFailure(msg)
2020-08-17 22:56:47.807 63567 ERROR nova.compute.manager
keystoneauth1.exceptions.connection.ConnectFailure: Unable to establish
connection to
http://10.24.0.117:8778/placement/resource_providers?in_tree=3a80ea71-91a6-4da7-a05b-682b501fe6c9:
HTTPConnectionPool(host='10.24.0.117', port=8778): Max retries exceeded with
url: /placement/resource_providers?in_tree=3a80ea71-91a6-4da7-a05b-682b501fe6c9
(Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at
0x7ff128340518>: Failed to establish a new connection: [Errno 111]
ECONNREFUSED',))
2020-08-17 22:56:47.807 63567 ERROR nova.compute.manager
2020-08-17 22:59:56.498 63567 WARNING nova.virt.libvirt.imagecache
[req-fb01576e-e540-4930-a4f0-5109a0df28c4 - - - - -] Unknown base file:
/var/lib/nova/instances/_base/5a8a3c88277cdf03ea71a4448af35b8e684f7943
2020-08-17 22:59:56.498 63567 INFO nova.virt.libvirt.imagecache
[req-fb01576e-e540-4930-a4f0-5109a0df28c4 - - - - -] Removable base files:
/var/lib/nova/instances/_base/5a8a3c88277cdf03ea71a4448af35b8e684f7943
2020-08-17 22:59:56.499 63567 INFO nova.virt.libvirt.imagecache
[req-fb01576e-e540-4930-a4f0-5109a0df28c4 - - - - -] Base or swap file too
young to remove:
/var/lib/nova/instances/_base/5a8a3c88277cdf03ea71a4448af35b8e684f7943
2020-08-17 23:00:05.503 63567 INFO nova.compute.manager
[req-fb01576e-e540-4930-a4f0-5109a0df28c4 - - - - -] Running instance usage
audit for host openstack-Karty from 2020-08-18 05:00:00 to 2020-08-18 06:00:00.
0 instances.
2020-08-17 23:02:10.221 63567 INFO nova.compute.claims
[req-1de4c903-89cf-4494-99d5-3e6237cd67bf 108551f604a24c23989a699b058e584b
9d3e3eadbc9c4d5fa6a3b27bfee40734 - default default] [instance:
1e296073-65af-4c9c-8574-8c150f539cec] Claim successful on node openstack-Karty
2020-08-17 23:02:10.621 63567 INFO nova.virt.libvirt.driver
[req-1de4c903-89cf-4494-99d5-3e6237cd67bf 108551f604a24c23989a699b058e584b
9d3e3eadbc9c4d5fa6a3b27bfee40734 - default default] [instance:
1e296073-65af-4c9c-8574-8c150f539cec] Creating image
2020-08-17 23:02:14.710 63567 INFO os_vif
[req-1de4c903-89cf-4494-99d5-3e6237cd67bf 108551f604a24c23989a699b058e584b
9d3e3eadbc9c4d5fa6a3b27bfee40734 - default default] Successfully plugged vif
VIFOpenVSwitch(active=False,address=fa:16:3e:15:cb:de,bridge_name='br-int',has_traffic_filtering=True,id=bb31ebbf-b1f4-48bd-9412-5b1a8e60fecd,network=Network(ca332b6f-a3c3-4c96-bc27-5a2243929fb9),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapbb31ebbf-b1')
2020-08-17 23:02:15.093 63567 INFO nova.compute.manager
[req-b9522c87-a4bb-42ab-a9a9-a60d987e60a8 - - - - -] [instance:
1e296073-65af-4c9c-8574-8c150f539cec] VM Started (Lifecycle Event)
2020-08-17 23:02:15.131 63567 INFO nova.compute.manager
[req-b9522c87-a4bb-42ab-a9a9-a60d987e60a8 - - - - -] [instance:
1e296073-65af-4c9c-8574-8c150f539cec] VM Paused (Lifecycle Event)
2020-08-17 23:02:15.199 63567 INFO nova.compute.manager
[req-b9522c87-a4bb-42ab-a9a9-a60d987e60a8 - - - - -] [instance:
1e296073-65af-4c9c-8574-8c150f539cec] During sync_power_state the instance has
a pending task (spawning). Skip.
2020-08-17 23:02:51.316 63567 ERROR nova.compute.resource_tracker
[req-fb01576e-e540-4930-a4f0-5109a0df28c4 - - - - -] Skipping removal of
allocations for deleted instances: Failed to retrieve allocations for resource
provider 3a80ea71-91a6-4da7-a05b-682b501fe6c9: <!DOCTYPE HTML PUBLIC
"-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>500 Internal Server Error</title>
</head><body>
<h1>Internal Server Error</h1>
<p>The server encountered an internal error or
misconfiguration and was unable to complete
your request.</p>
<p>Please contact the server administrator at
[no address given] to inform them of the time this error occurred,
and the actions you performed just before this error.</p>
<p>More information about this error may be available
in the server error log.</p>
</body></html>
: nova.exception.ResourceProviderAllocationRetrievalFailed: Failed to
retrieve allocations for resource provider
3a80ea71-91a6-4da7-a05b-682b501fe6c9: <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML
2.0//EN">
2020-08-17 23:03:53.500 63567 ERROR nova.scheduler.client.report
[req-fb01576e-e540-4930-a4f0-5109a0df28c4 - - - - -] [None] Failed to retrieve
resource provider tree from placement API for UUID
3a80ea71-91a6-4da7-a05b-682b501fe6c9. Got 500: <!DOCTYPE HTML PUBLIC
"-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>500 Internal Server Error</title>
</head><body>
<h1>Internal Server Error</h1>
<p>The server encountered an internal error or
misconfiguration and was unable to complete
your request.</p>
<p>Please contact the server administrator at
[no address given] to inform them of the time this error occurred,
and the actions you performed just before this error.</p>
<p>More information about this error may be available
in the server error log.</p>
</body></html>
.
2020-08-17 23:03:53.501 63567 ERROR nova.compute.manager
[req-fb01576e-e540-4930-a4f0-5109a0df28c4 - - - - -] Error updating resources
for node openstack-Karty.: nova.exception.ResourceProviderRetrievalFailed:
Failed to get resource provider with UUID 3a80ea71-91a6-4da7-a05b-682b501fe6c9
2020-08-17 23:03:53.501 63567 ERROR nova.compute.manager Traceback (most
recent call last):
2020-08-17 23:03:53.501 63567 ERROR nova.compute.manager File
"/usr/lib/python3.6/site-packages/nova/compute/manager.py", line 9685, in
_update_available_resource_for_node
2020-08-17 23:03:53.501 63567 ERROR nova.compute.manager startup=startup)
2020-08-17 23:03:53.501 63567 ERROR nova.compute.manager File
"/usr/lib/python3.6/site-packages/nova/compute/resource_tracker.py", line 842,
in update_available_resource
2020-08-17 23:03:53.501 63567 ERROR nova.compute.manager
self._update_available_resource(context, resources, startup=startup)
2020-08-17 23:03:53.501 63567 ERROR nova.compute.manager File
"/usr/lib/python3.6/site-packages/oslo_concurrency/lockutils.py", line 359, in
inner
2020-08-17 23:03:53.501 63567 ERROR nova.compute.manager return f(*args,
**kwargs)
2020-08-17 23:03:53.501 63567 ERROR nova.compute.manager File
"/usr/lib/python3.6/site-packages/nova/compute/resource_tracker.py", line 927,
in _update_available_resource
2020-08-17 23:03:53.501 63567 ERROR nova.compute.manager
self._update(context, cn, startup=startup)
2020-08-17 23:03:53.501 63567 ERROR nova.compute.manager File
"/usr/lib/python3.6/site-packages/nova/compute/resource_tracker.py", line 1176,
in _update
2020-08-17 23:03:53.501 63567 ERROR nova.compute.manager
self._update_to_placement(context, compute_node, startup)
2020-08-17 23:03:53.501 63567 ERROR nova.compute.manager File
"/usr/lib/python3.6/site-packages/retrying.py", line 68, in wrapped_f
2020-08-17 23:03:53.501 63567 ERROR nova.compute.manager return
Retrying(*dargs, **dkw).call(f, *args, **kw)
2020-08-17 23:03:53.501 63567 ERROR nova.compute.manager File
"/usr/lib/python3.6/site-packages/retrying.py", line 223, in call
2020-08-17 23:03:53.501 63567 ERROR nova.compute.manager return
attempt.get(self._wrap_exception)
2020-08-17 23:03:53.501 63567 ERROR nova.compute.manager File
"/usr/lib/python3.6/site-packages/retrying.py", line 261, in get
2020-08-17 23:03:53.501 63567 ERROR nova.compute.manager
six.reraise(self.value[0], self.value[1], self.value[2])
2020-08-17 23:03:53.501 63567 ERROR nova.compute.manager File
"/usr/lib/python3.6/site-packages/six.py", line 703, in reraise
2020-08-17 23:03:53.501 63567 ERROR nova.compute.manager raise value
2020-08-17 23:03:53.501 63567 ERROR nova.compute.manager File
"/usr/lib/python3.6/site-packages/retrying.py", line 217, in call
2020-08-17 23:03:53.501 63567 ERROR nova.compute.manager attempt =
Attempt(fn(*args, **kwargs), attempt_number, False)
2020-08-17 23:03:53.501 63567 ERROR nova.compute.manager File
"/usr/lib/python3.6/site-packages/nova/compute/resource_tracker.py", line 1110,
in _update_to_placement
2020-08-17 23:03:53.501 63567 ERROR nova.compute.manager context,
compute_node.uuid, name=compute_node.hypervisor_hostname)
2020-08-17 23:03:53.501 63567 ERROR nova.compute.manager File
"/usr/lib/python3.6/site-packages/nova/scheduler/client/report.py", line 857,
in get_provider_tree_and_ensure_root
2020-08-17 23:03:53.501 63567 ERROR nova.compute.manager
parent_provider_uuid=parent_provider_uuid)
2020-08-17 23:03:53.501 63567 ERROR nova.compute.manager File
"/usr/lib/python3.6/site-packages/nova/scheduler/client/report.py", line 640,
in _ensure_resource_provider
2020-08-17 23:03:53.501 63567 ERROR nova.compute.manager rps_to_refresh =
self.get_providers_in_tree(context, uuid)
2020-08-17 23:03:53.501 63567 ERROR nova.compute.manager File
"/usr/lib/python3.6/site-packages/nova/scheduler/client/report.py", line 503,
in get_providers_in_tree
2020-08-17 23:03:53.501 63567 ERROR nova.compute.manager raise
exception.ResourceProviderRetrievalFailed(uuid=uuid)
2020-08-17 23:03:53.501 63567 ERROR nova.compute.manager
nova.exception.ResourceProviderRetrievalFailed: Failed to get resource provider
with UUID 3a80ea71-91a6-4da7-a05b-682b501fe6c9
2020-08-17 23:03:53.501 63567 ERROR nova.compute.manager
Gateway Timeout
The gateway did not receive a timely response from the upstream server or
application.
I need to keep restarting service httpd restart inorder to overcome
this issue. Please let us know a permanent solution. Thanks.
To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1891973/+subscriptions
--
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : [email protected]
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help : https://help.launchpad.net/ListHelp
