*** This bug is a duplicate of bug 1523379 ***
https://bugs.launchpad.net/bugs/1523379
Matt, this looks like a dupe of bug 1523379. Which has a fix merged to
oslo.cache master. Let me know if you need it backported to Mitaka or
Liberty.
** This bug has been marked a duplicate of bug 1523379
memcache pool driver - need sync with last changes in keystone
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Identity (keystone).
https://bugs.launchpad.net/bugs/1570158
Title:
memcache pool reap issue
Status in OpenStack Identity (keystone):
Triaged
Status in OpenStack Identity (keystone) liberty series:
In Progress
Bug description:
There seems to be a code error in the memcache pool cleanup. I'm
seeing this on stable/liberty (built as of 2 weeks ago). I don't have
a specific reproducer for this, it just seems to happen. Looking at
the code I don't really understand how this can happen.
2016-04-13 20:50:29.124 46 ERROR keystone.common.wsgi
[req-30e23a7d-754d-4a89-904f-1804a20029f2 - - - - -] deque index out of range
2016-04-13 20:50:29.124 46 ERROR keystone.common.wsgi Traceback (most recent
call last):
2016-04-13 20:50:29.124 46 ERROR keystone.common.wsgi File
"/venv/local/lib/python2.7/site-packages/keystone/common/wsgi.py", line 248, in
__call__
2016-04-13 20:50:29.124 46 ERROR keystone.common.wsgi result =
method(context, **params)
2016-04-13 20:50:29.124 46 ERROR keystone.common.wsgi File
"/venv/local/lib/python2.7/site-packages/keystone/common/controller.py", line
138, in inner
2016-04-13 20:50:29.124 46 ERROR keystone.common.wsgi
context['subject_token_id']))
2016-04-13 20:50:29.124 46 ERROR keystone.common.wsgi File
"/venv/local/lib/python2.7/site-packages/keystone/token/provider.py", line 188,
in validate_token
2016-04-13 20:50:29.124 46 ERROR keystone.common.wsgi token =
self._validate_token(unique_id)
2016-04-13 20:50:29.124 46 ERROR keystone.common.wsgi File
"/venv/local/lib/python2.7/site-packages/dogpile/cache/region.py", line 1053,
in decorate
2016-04-13 20:50:29.124 46 ERROR keystone.common.wsgi should_cache_fn)
2016-04-13 20:50:29.124 46 ERROR keystone.common.wsgi File
"/venv/local/lib/python2.7/site-packages/dogpile/cache/region.py", line 657, in
get_or_create
2016-04-13 20:50:29.124 46 ERROR keystone.common.wsgi async_creator) as
value:
2016-04-13 20:50:29.124 46 ERROR keystone.common.wsgi File
"/venv/local/lib/python2.7/site-packages/dogpile/core/dogpile.py", line 158, in
__enter__
2016-04-13 20:50:29.124 46 ERROR keystone.common.wsgi return self._enter()
2016-04-13 20:50:29.124 46 ERROR keystone.common.wsgi File
"/venv/local/lib/python2.7/site-packages/dogpile/core/dogpile.py", line 91, in
_enter
2016-04-13 20:50:29.124 46 ERROR keystone.common.wsgi value = value_fn()
2016-04-13 20:50:29.124 46 ERROR keystone.common.wsgi File
"/venv/local/lib/python2.7/site-packages/dogpile/cache/region.py", line 610, in
get_value
2016-04-13 20:50:29.124 46 ERROR keystone.common.wsgi value =
self.backend.get(key)
2016-04-13 20:50:29.124 46 ERROR keystone.common.wsgi File
"/venv/local/lib/python2.7/site-packages/dogpile/cache/backends/memcached.py",
line 161, in get
2016-04-13 20:50:29.124 46 ERROR keystone.common.wsgi value =
self.client.get(key)
2016-04-13 20:50:29.124 46 ERROR keystone.common.wsgi File
"/venv/local/lib/python2.7/site-packages/keystone/common/cache/backends/memcache_pool.py",
line 36, in _run_method
2016-04-13 20:50:29.124 46 ERROR keystone.common.wsgi return
getattr(client, __name)(*args, **kwargs)
2016-04-13 20:50:29.124 46 ERROR keystone.common.wsgi File
"/usr/lib/python2.7/contextlib.py", line 24, in __exit__
2016-04-13 20:50:29.124 46 ERROR keystone.common.wsgi self.gen.next()
2016-04-13 20:50:29.124 46 ERROR keystone.common.wsgi File
"/venv/local/lib/python2.7/site-packages/keystone/common/cache/_memcache_pool.py",
line 132, in acquire
2016-04-13 20:50:29.124 46 ERROR keystone.common.wsgi
self._drop_expired_connections()
2016-04-13 20:50:29.124 46 ERROR keystone.common.wsgi File
"/venv/local/lib/python2.7/site-packages/keystone/common/cache/_memcache_pool.py",
line 166, in _drop_expired_connections
2016-04-13 20:50:29.124 46 ERROR keystone.common.wsgi while self.queue
and self.queue[0].ttl < now:
2016-04-13 20:50:29.124 46 ERROR keystone.common.wsgi IndexError: deque index
out of range
To manage notifications about this bug go to:
https://bugs.launchpad.net/keystone/+bug/1570158/+subscriptions
--
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help : https://help.launchpad.net/ListHelp