Patched versions of eventlet for bionic and cosmic uploaded for SRU Team
review.
** Description changed:
+ [Impact]
+ Swift proxy server will sometimes fail to start due to racy-ness in monkey
patching by eventlet and daemon startup. Swift dies, no object storage.
+
+ [Test Case]
+ Deploy swift using the reference bundles for deploying OpenStack with Juju
and the OpenStack charms.
+ Depending on the number of cores being used the stacktrace in the original
bug report will be seen and swift-proxy will fail to startup.
+
+ [Regression Potential]
+ The workaround fix to ensure that the eventlet hub is correctly initialised
has been in the upstream codebase since 2017 so low potential for regression.
+
+ [Original Bug Report]
Charm options:
- swift-proxy:
- charm: cs:swift-proxy
- num_units: 3
- bindings:
- "": *oam-space
- public: *public-space
- admin: *admin-space
- internal: *internal-space
- swift-storage: *swift-cluster-space
- options:
- openstack-origin: *openstack-origin
- vip: *swift-proxy-vip
- region: *openstack-region
- zone-assignment: manual
- partition-power: *partition-power
- workers: *workers
- to:
- - lxd:0
- - lxd:1
- - lxd:2
+ swift-proxy:
+ charm: cs:swift-proxy
+ num_units: 3
+ bindings:
+ "": *oam-space
+ public: *public-space
+ admin: *admin-space
+ internal: *internal-space
+ swift-storage: *swift-cluster-space
+ options:
+ openstack-origin: *openstack-origin
+ vip: *swift-proxy-vip
+ region: *openstack-region
+ zone-assignment: manual
+ partition-power: *partition-power
+ workers: *workers
+ to:
+ - lxd:0
+ - lxd:1
+ - lxd:2
-
- Setting up swift-proxy on Bionic causes swift-proxy.service to fail at start
with:
+ Setting up swift-proxy on Bionic causes swift-proxy.service to fail at
+ start with:
Feb 21 11:16:07 juju-29125a-0-lxd-15 systemd[1]: Stopped LSB: Swift proxy
server.
Feb 21 11:16:07 juju-29125a-0-lxd-15 systemd[1]: swift-proxy.service: Failed
to reset devices.list: Operation not permitted
Feb 21 11:16:07 juju-29125a-0-lxd-15 systemd[1]: Starting LSB: Swift proxy
server...
Feb 21 11:16:07 juju-29125a-0-lxd-15 swift-proxy[41939]: * Starting Swift
Swift proxy server swift-init proxy-server
Feb 21 11:16:08 juju-29125a-0-lxd-15 swift-proxy[41939]: Starting
proxy-server...(/etc/swift/proxy-server.conf)
Feb 21 11:16:08 juju-29125a-0-lxd-15 swift-proxy[41939]: Traceback (most
recent call last):
Feb 21 11:16:08 juju-29125a-0-lxd-15 swift-proxy[41939]: File
"/usr/bin/swift-proxy-server", line 23, in <module>
Feb 21 11:16:08 juju-29125a-0-lxd-15 swift-proxy[41939]:
sys.exit(run_wsgi(conf_file, 'proxy-server', **options))
Feb 21 11:16:08 juju-29125a-0-lxd-15 swift-proxy[41939]: File
"/usr/lib/python2.7/dist-packages/swift/common/wsgi.py", line 912, in run_wsgi
Feb 21 11:16:08 juju-29125a-0-lxd-15 swift-proxy[41939]:
loadapp(conf_path, global_conf=global_conf)
Feb 21 11:16:08 juju-29125a-0-lxd-15 swift-proxy[41939]: File
"/usr/lib/python2.7/dist-packages/swift/common/wsgi.py", line 390, in loadapp
Feb 21 11:16:08 juju-29125a-0-lxd-15 swift-proxy[41939]: ctx =
loadcontext(loadwsgi.APP, conf_file, global_conf=global_conf)
Feb 21 11:16:08 juju-29125a-0-lxd-15 swift-proxy[41939]: File
"/usr/lib/python2.7/dist-packages/swift/common/wsgi.py", line 374, in
loadcontext
Feb 21 11:16:08 juju-29125a-0-lxd-15 swift-proxy[41939]:
global_conf=global_conf)
Feb 21 11:16:08 juju-29125a-0-lxd-15 swift-proxy[41939]: File
"/usr/lib/python2.7/dist-packages/paste/deploy/loadwsgi.py", line 296, in
loadcontext
Feb 21 11:16:08 juju-29125a-0-lxd-15 swift-proxy[41939]:
global_conf=global_conf)
Feb 21 11:16:08 juju-29125a-0-lxd-15 swift-proxy[41939]: File
"/usr/lib/python2.7/dist-packages/paste/deploy/loadwsgi.py", line 320, in
_loadconfig
Feb 21 11:16:08 juju-29125a-0-lxd-15 swift-proxy[41939]: return
loader.get_context(object_type, name, global_conf)
Feb 21 11:16:08 juju-29125a-0-lxd-15 swift-proxy[41939]: File
"/usr/lib/python2.7/dist-packages/swift/common/wsgi.py", line 66, in get_context
Feb 21 11:16:08 juju-29125a-0-lxd-15 swift-proxy[41939]: object_type,
name=name, global_conf=global_conf)
Feb 21 11:16:08 juju-29125a-0-lxd-15 swift-proxy[41939]: File
"/usr/lib/python2.7/dist-packages/paste/deploy/loadwsgi.py", line 450, in
get_context
Feb 21 11:16:08 juju-29125a-0-lxd-15 swift-proxy[41939]:
global_additions=global_additions)
Feb 21 11:16:08 juju-29125a-0-lxd-15 swift-proxy[41939]: File
"/usr/lib/python2.7/dist-packages/paste/deploy/loadwsgi.py", line 562, in
_pipeline_app_context
Feb 21 11:16:08 juju-29125a-0-lxd-15 swift-proxy[41939]: for name in
pipeline[:-1]]
Feb 21 11:16:08 juju-29125a-0-lxd-15 swift-proxy[41939]: File
"/usr/lib/python2.7/dist-packages/swift/common/wsgi.py", line 66, in get_context
Feb 21 11:16:08 juju-29125a-0-lxd-15 swift-proxy[41939]: object_type,
name=name, global_conf=global_conf)
Feb 21 11:16:08 juju-29125a-0-lxd-15 swift-proxy[41939]: File
"/usr/lib/python2.7/dist-packages/paste/deploy/loadwsgi.py", line 458, in
get_context
Feb 21 11:16:08 juju-29125a-0-lxd-15 swift-proxy[41939]: section)
Feb 21 11:16:08 juju-29125a-0-lxd-15 swift-proxy[41939]: File
"/usr/lib/python2.7/dist-packages/paste/deploy/loadwsgi.py", line 517, in
_context_from_explicit
Feb 21 11:16:08 juju-29125a-0-lxd-15 swift-proxy[41939]: value =
import_string(found_expr)
Feb 21 11:16:08 juju-29125a-0-lxd-15 swift-proxy[41939]: File
"/usr/lib/python2.7/dist-packages/paste/deploy/loadwsgi.py", line 22, in
import_string
Feb 21 11:16:08 juju-29125a-0-lxd-15 swift-proxy[41939]: return
pkg_resources.EntryPoint.parse("x=" + s).load(False)
Feb 21 11:16:08 juju-29125a-0-lxd-15 swift-proxy[41939]: File
"/usr/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 2324, in load
Feb 21 11:16:08 juju-29125a-0-lxd-15 swift-proxy[41939]: return
self.resolve()
Feb 21 11:16:08 juju-29125a-0-lxd-15 swift-proxy[41939]: File
"/usr/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 2330, in
resolve
Feb 21 11:16:08 juju-29125a-0-lxd-15 swift-proxy[41939]: module =
__import__(self.module_name, fromlist=['__name__'], level=0)
Feb 21 11:16:08 juju-29125a-0-lxd-15 swift-proxy[41939]: File
"/usr/lib/python2.7/dist-packages/keystonemiddleware/auth_token/__init__.py",
line 221, in <module>
Feb 21 11:16:08 juju-29125a-0-lxd-15 swift-proxy[41939]: from
keystoneauth1 import adapter
Feb 21 11:16:08 juju-29125a-0-lxd-15 swift-proxy[41939]: File
"/usr/lib/python2.7/dist-packages/keystoneauth1/adapter.py", line 16, in
<module>
Feb 21 11:16:08 juju-29125a-0-lxd-15 swift-proxy[41939]: from
keystoneauth1 import session
Feb 21 11:16:08 juju-29125a-0-lxd-15 swift-proxy[41939]: File
"/usr/lib/python2.7/dist-packages/keystoneauth1/session.py", line 40, in
<module>
Feb 21 11:16:08 juju-29125a-0-lxd-15 swift-proxy[41939]: import
osprofiler.web as osprofiler_web
Feb 21 11:16:08 juju-29125a-0-lxd-15 swift-proxy[41939]: File
"/usr/lib/python2.7/dist-packages/osprofiler/web.py", line 20, in <module>
Feb 21 11:16:08 juju-29125a-0-lxd-15 swift-proxy[41939]: from osprofiler
import profiler
Feb 21 11:16:08 juju-29125a-0-lxd-15 swift-proxy[41939]: File
"/usr/lib/python2.7/dist-packages/osprofiler/profiler.py", line 26, in <module>
Feb 21 11:16:08 juju-29125a-0-lxd-15 swift-proxy[41939]: from osprofiler
import notifier
Feb 21 11:16:08 juju-29125a-0-lxd-15 swift-proxy[41939]: File
"/usr/lib/python2.7/dist-packages/osprofiler/notifier.py", line 16, in <module>
Feb 21 11:16:08 juju-29125a-0-lxd-15 swift-proxy[41939]: from
osprofiler.drivers import base
Feb 21 11:16:08 juju-29125a-0-lxd-15 swift-proxy[41939]: File
"/usr/lib/python2.7/dist-packages/osprofiler/drivers/__init__.py", line 4, in
<module>
Feb 21 11:16:08 juju-29125a-0-lxd-15 swift-proxy[41939]: from
osprofiler.drivers import loginsight # noqa
Feb 21 11:16:08 juju-29125a-0-lxd-15 swift-proxy[41939]: File
"/usr/lib/python2.7/dist-packages/osprofiler/drivers/loginsight.py", line 24,
in <module>
Feb 21 11:16:08 juju-29125a-0-lxd-15 swift-proxy[41939]: from
oslo_concurrency.lockutils import synchronized
Feb 21 11:16:08 juju-29125a-0-lxd-15 swift-proxy[41939]: File
"/usr/lib/python2.7/dist-packages/oslo_concurrency/lockutils.py", line 27, in
<module>
Feb 21 11:16:08 juju-29125a-0-lxd-15 swift-proxy[41939]: import fasteners
Feb 21 11:16:08 juju-29125a-0-lxd-15 swift-proxy[41939]: File
"/usr/lib/python2.7/dist-packages/fasteners/__init__.py", line 23, in <module>
Feb 21 11:16:08 juju-29125a-0-lxd-15 swift-proxy[41939]: from
fasteners.lock import locked # noqa
Feb 21 11:16:08 juju-29125a-0-lxd-15 swift-proxy[41939]: File
"/usr/lib/python2.7/dist-packages/fasteners/lock.py", line 24, in <module>
Feb 21 11:16:08 juju-29125a-0-lxd-15 swift-proxy[41939]: from fasteners
import _utils
Feb 21 11:16:08 juju-29125a-0-lxd-15 swift-proxy[41939]: File
"/usr/lib/python2.7/dist-packages/fasteners/_utils.py", line 22, in <module>
Feb 21 11:16:08 juju-29125a-0-lxd-15 swift-proxy[41939]: from monotonic
import monotonic as now # noqa
Feb 21 11:16:08 juju-29125a-0-lxd-15 swift-proxy[41939]: File
"/usr/lib/python2.7/dist-packages/monotonic.py", line 164, in <module>
Feb 21 11:16:08 juju-29125a-0-lxd-15 swift-proxy[41939]: raise
RuntimeError('no suitable implementation for this system')
Feb 21 11:16:08 juju-29125a-0-lxd-15 swift-proxy[41939]: RuntimeError: no
suitable implementation for this system
Feb 21 11:16:08 juju-29125a-0-lxd-15 swift-proxy[41939]: ...fail!
Feb 21 11:16:08 juju-29125a-0-lxd-15 systemd[1]: Started LSB: Swift proxy
server.
-
Looking at errors for monotonic lib, I can find other references around
github:
https://github.com/nameko/nameko/issues/533
https://github.com/eventlet/eventlet/issues/401
Swift-proxy indeed depends on eventlet:
root@juju-29125a-0-lxd-15:~# dpkg -l | grep eventlet
ii python-eventlet 0.20.0-4
all concurrent networking library - Python 2.x
And then the issue described at:
https://github.com/eventlet/eventlet/issues/401 applies.
Moving back to Xenial seems to stabilize swift proxy.
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1817055
Title:
Broken dependency when setting swift-proxy on Bionic
To manage notifications about this bug go to:
https://bugs.launchpad.net/charm-swift-proxy/+bug/1817055/+subscriptions
--
ubuntu-bugs mailing list
[email protected]
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
