Reviewed: https://review.opendev.org/724188 Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=d6f664524dfa4eecbe3e16fda0fec5359faf84c6 Submitter: Zuul Branch: master
commit d6f664524dfa4eecbe3e16fda0fec5359faf84c6 Author: Sean Mooney <[email protected]> Date: Tue Apr 28 23:20:16 2020 +0100 Silence amqp heartbeat warning When the nova api is executing under uWSGI or MOD_WSGI the lifetime of the amqp heartbeat thread is controlled by the wsgi server. As a result when the nova api is run in this configuration we expect that the heartbeat thread will be suspended and heartbeats will be missed when the wsgi server suspends execution of the wsgi application. This change adds a python logging filter to suppress the reporting of heartbeat warnings as this behavior is expected. Since the operator cannot do anything to address the issue the warning is just noise and many operators and customers find it to be off-putting. Change-Id: I642b1e3ed6de2be4dcc19fe214f84095d2e1d31a Closes-Bug: #1825584 ** Changed in: nova Status: In Progress => Fix Released -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1825584 Title: eventlet monkey-patching breaks AMQP heartbeat on uWSGI Status in OpenStack Compute (nova): Fix Released Bug description: Stein nova-api running under uWSGI presents an AMQP issue. The first API call that requires RPC creates an AMQP connection and successfully completes. Normally regular heartbeats would be sent from this point on, to maintain the connection. This is not happening. After a few minutes, the AMQP server (rabbitmq, in my case) notices that there have been no heartbeats, and drops the connection. A later nova API call that requires RPC tries to use the old connection, and throws a "connection reset by peer" exception and the API call fails. A mailing-list response suggests that this is affecting mod_wsgi also: http://lists.openstack.org/pipermail/openstack- discuss/2019-April/005310.html I've discovered that this problem seems to be caused by eventlet monkey-patching, which was introduced in: https://github.com/openstack/nova/commit/23ba1c690652832c655d57476630f02c268c87ae It was later rearranged in: https://github.com/openstack/nova/commit/3c5e2b0e9fac985294a949852bb8c83d4ed77e04 but this problem remains. If I comment out the import of nova.monkey_patch in nova/api/openstack/__init__.py the problem goes away. Seems that eventlet monkey-patching and uWSGI are not getting along for some reason... To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1825584/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : [email protected] Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
