#6527: A bug in HttpResponse with iterators
--------------------------------------------------+-------------------------
Reporter: daonb <[email protected]> | Owner: ccahoon
Status: new | Milestone:
Component: HTTP handling | Version: SVN
Resolution: | Keywords: http
iterators
Stage: Design decision needed | Has_patch: 1
Needs_docs: 0 | Needs_tests: 0
Needs_better_patch: 0 |
--------------------------------------------------+-------------------------
Changes (by tomevans222):
* stage: Fixed on a branch => Design decision needed
Comment:
The suggested fix, and the changes committed to soc2009/http-wsgi-
improvements cannot possibly be committed to trunk. This severely breaks
previously documented behaviour because, as mrmachine points out, you will
no longer be able to iteratively generate large responses, and in
particular, will not be able to iteratively deliver chunks of content to
the client.
This will break any page that takes a long time to fully generate, or
about 10% of one of my sites.
I think that any change will need to consider the different types of
middleware that operate on a response's contents.
Some types of middleware will only examine the generated response, eg
CacheMiddleware will not modify the response, only store a version in the
cache when complete.
Others will want to replace the content and modify headers, eg
GzipMiddleware.
The current 'fix' only satisfies the requirements of the modifying
middleware.
I fixed the limitation of the cache middleware to cope with iterator based
responses with code like so:
{{{
def buffer_and_cache_response(response, cache_key, timeout):
from copy import copy
def worker(rsp):
from cStringIO import StringIO
buf = StringIO()
# we need to copy the response before we iterate through it
# if we copy it after, then the response will have a generator object
# on self._iterator that will not be picklable
buffered_response = copy(rsp)
for chunk in rsp:
buf.write(chunk)
yield chunk
buffered_response.content = buf.getvalue()
buf.close()
rsp.close()
cache.set(cache_key, buffered_response, timeout)
new_resp = copy(response)
new_resp._container = worker(response)
new_resp._is_string = False
return new_resp
}}}
Our cache middleware (virtually copy/paste of UpdateCacheMiddleware)
determines if this is an iterator derived response by checking
response._is_string (hacky, I know), and returns the rv of this function.
This is just an example of a more intelligent, correct way to fix this
issue.
--
Ticket URL: <http://code.djangoproject.com/ticket/6527#comment:19>
Django <http://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
--
You received this message because you are subscribed to the Google Groups
"Django updates" group.
To post to this group, send email to [email protected].
To unsubscribe from this group, send email to
[email protected].
For more options, visit this group at
http://groups.google.com/group/django-updates?hl=en.
