#20493: HttpResponse objects cached with cache_page don't unpickle across Django
version leading to strange errors
-------------------------------------+--------------------
Reporter: cataliniacob | Owner: nobody
Type: Bug | Status: new
Component: Core (Cache system) | Version: 1.5
Severity: Normal | Keywords:
Triage Stage: Unreviewed | Has patch: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+--------------------
I just upgraded from 1.4 to 1.5 and this took a while to track.
If one has a cache that persists across a Django version change
(filesystem cache, memcached that is not restarted), when executing any
view decorated with @cache_page unpickling the HttpResponse from the cache
leads to strange errors. For example, when upgrading from 1.4 to 1.5 I
get:
{{{
[24/May/2013 15:53:42] "GET / HTTP/1.1" 200 5
Traceback (most recent call last):
File "/usr/lib64/python2.7/wsgiref/handlers.py", line 86, in run
self.finish_response()
File "/usr/lib64/python2.7/wsgiref/handlers.py", line 129, in
finish_response
self.close()
File "/usr/lib64/python2.7/wsgiref/simple_server.py", line 36, in close
SimpleHandler.close(self)
File "/usr/lib64/python2.7/wsgiref/handlers.py", line 257, in close
self.result.close()
File "/home/catalin/hacking/envs/cached-httpresponse-across-
versions/lib/python2.7/site-packages/django/http/response.py", line 231,
in close
for closable in self._closable_objects:
AttributeError: 'HttpResponse' object has no attribute '_closable_objects'
[24/May/2013 15:53:42] "GET / HTTP/1.1" 500 59
----------------------------------------
Exception happened during processing of request from ('127.0.0.1', 49852)
Traceback (most recent call last):
File "/usr/lib64/python2.7/SocketServer.py", line 582, in
process_request_thread
self.finish_request(request, client_address)
File "/usr/lib64/python2.7/SocketServer.py", line 323, in finish_request
self.RequestHandlerClass(request, client_address, self)
File "/home/catalin/hacking/envs/cached-httpresponse-across-
versions/lib/python2.7/site-packages/django/core/servers/basehttp.py",
line 150, in __init__
super(WSGIRequestHandler, self).__init__(*args, **kwargs)
File "/usr/lib64/python2.7/SocketServer.py", line 638, in __init__
self.handle()
File "/usr/lib64/python2.7/wsgiref/simple_server.py", line 124, in
handle
handler.run(self.server.get_app())
File "/usr/lib64/python2.7/wsgiref/handlers.py", line 89, in run
self.handle_error()
File "/usr/lib64/python2.7/wsgiref/handlers.py", line 304, in
handle_error
self.finish_response()
File "/usr/lib64/python2.7/wsgiref/handlers.py", line 127, in
finish_response
self.write(data)
File "/usr/lib64/python2.7/wsgiref/handlers.py", line 210, in write
self.send_headers()
File "/usr/lib64/python2.7/wsgiref/handlers.py", line 267, in
send_headers
if not self.origin_server or self.client_is_modern():
File "/usr/lib64/python2.7/wsgiref/handlers.py", line 280, in
client_is_modern
return self.environ['SERVER_PROTOCOL'].upper() != 'HTTP/0.9'
TypeError: 'NoneType' object has no attribute '__getitem__'
----------------------------------------
}}}
This was at first quite perplexing because I checked response.py and
{{{self._closable_objects}}} gets assigned in
{{{HttpResponseBase.__init__}}} which gets called by
{{{HttpResponse.__init__}}} so it seemed impossible for an HttpResponse
instance not to have the attribute. Unpickling doesn't normally call
{{{__init__}}} but this is not at all obvious or easy to track.
There are similar issues but with a bit more clear stacktrace if
downgrading from 1.5 to 1.4:
{{{
Traceback:
File "/home/catalin/hacking/envs/cached-httpresponse-across-
versions/lib/python2.7/site-packages/django/core/handlers/base.py" in
get_response
89. response = middleware_method(request)
File "/home/catalin/hacking/envs/cached-httpresponse-across-
versions/lib/python2.7/site-packages/django/middleware/cache.py" in
process_request
147. response = self.cache.get(cache_key, None)
File "/home/catalin/hacking/envs/cached-httpresponse-across-
versions/lib/python2.7/site-
packages/django/core/cache/backends/filebased.py" in get
41. return pickle.load(f)
Exception Type: ImportError at /
Exception Value: No module named response
}}}
I'm surprised nobody seems to have run into this so far, googling for
AttributeError on _closable_objects and searching the other bugs didn't
show anything. I would imagine it's reasonably common to upgrade without
restarting memcached or to have a filesystem based cache on a development
machine where you don't bother to install memcached. And this happens on
'''every''' view with @cache_page.
I don't know what a good solution would be. I can imagine Django doesn't
want to guarantee pickle/unpickle round trips across versions as those can
be hard to maintain but I think there should at least be a warning in the
upgrade notes (of every version): when upgrading Django you need to delete
your cache.
--
Ticket URL: <https://code.djangoproject.com/ticket/20493>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
--
You received this message because you are subscribed to the Google Groups
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-updates/055.dbacffdd3e70e1677cf206377eee0a87%40djangoproject.com?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.
