#35757: memcached.PyMemcacheCache reentrancy problem (OSError: [Errno 9] Bad
file
descriptor)
-------------------------------+-----------------------------------------
Reporter: Harm Verhagen | Type: Uncategorized
Status: new | Component: Uncategorized
Version: 5.1 | Severity: Normal
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------+-----------------------------------------
I found that using `memcached.PyMemcacheCache` with runserver has
reentrancy problems.
I implemented a json api in django, when issuing multiple request
simultaneously from javascript, the requests sometimes crash.
from django.core.cache import cache
I have urls that do something like this.
{{{
def example_url(request):
cache.incr("some key", 1) # count requests
return JsonResponse(data=data, status=201)
}}}
When issuing 5-10 requests simultaneously on a runserver, it often (not
always) get this crash
{{{
File "..../views/api.py", line 2512, in example_url
cache.incr("some key", 1)
File "./venv/lib/python3.12/site-
packages/django/core/cache/backends/memcached.py", line 110, in incr
val = self._cache.incr(key, delta)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "./venv/lib/python3.12/site-packages/pymemcache/client/hash.py",
line 350, in incr
return self._run_cmd("incr", key, False, *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "./venv/lib/python3.12/site-packages/pymemcache/client/hash.py",
line 322, in _run_cmd
return self._safely_run_func(client, func, default_val, *args,
**kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "./venv/lib/python3.12/site-packages/pymemcache/client/hash.py",
line 211, in _safely_run_func
result = func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "./venv/lib/python3.12/site-packages/pymemcache/client/base.py",
line 827, in incr
results = self._misc_cmd([cmd], b"incr", noreply)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "./venv/lib/python3.12/site-packages/pymemcache/client/base.py",
line 1291, in _misc_cmd
buf, line = _reader(self.sock, buf)
^^^^^^^^^^^^^^^^^^^^^^^
File "./venv/lib/python3.12/site-packages/pymemcache/client/base.py",
line 1658, in _readline
buf = _recv(sock, RECV_SIZE)
^^^^^^^^^^^^^^^^^^^^^^
File "./venv/lib/python3.12/site-packages/pymemcache/client/base.py",
line 1750, in _recv
return sock.recv(size)
OSError: [Errno 9] Bad file descriptor
}}}
The following code looked a bit suspicious.
`cache.backends.memcached.py:99 BaseMemcachedCache`
{{{
def close(self, **kwargs):
# Many clients don't clean up connections properly.
self._cache.disconnect_all()
}}}
Doesn't this race? closing a all connections, might close a connection
of a parallel request that was just opened but not yet finished.
== Notes
I'm not putting a heavy load on runserver, its a simple javascript app,
that happens to do 10 requests on a single page. Django should be able
to handle that, also in a dev environment
== Version info
{{{
Django==5.1.1
pymemcache==4.0.0
python 3.12.6
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/35757>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
--
You received this message because you are subscribed to the Google Groups
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-updates/01070191e68f7b7f-3ab494c8-6fe6-439f-99f5-139b0f98826f-000000%40eu-central-1.amazonses.com.
