#19895: Second iteration over an invalid queryset returns an empty list instead
of
an exception
-------------------------------------+-------------------------------------
Reporter: gnosek | Owner: gnosek
Type: Bug | Status: new
Component: Database layer | Version: 1.4
(models, ORM) | Resolution:
Severity: Release blocker | Triage Stage: Accepted
Keywords: | Needs documentation: 0
Has patch: 1 | Patch needs improvement: 1
Needs tests: 0 | UI/UX: 0
Easy pickings: 0 |
-------------------------------------+-------------------------------------
Comment (by akaariai):
Attached is a minimalistic test case that will show the memory leak. The
case is simple - have enough objects that one ITERATOR_CHUNK_SIZE will not
convert all the objects (that is, more than 100 objects in the queryset).
Do bool(qs). This will result in memory leak when this ticket's patch is
applied, but will not leak if this ticket's patch isn't applied.
The reason for the leak is a bug in Python itself. The
[http://docs.python.org/2/library/gc.html#gc.garbage gc.garbage] docs say
that:
"""
A list of objects which the collector found to be unreachable but could
not be freed (uncollectable objects). By default, this list contains only
objects with `__del__()` methods. Objects that have `__del__()` methods
and are part of a reference cycle cause the entire reference cycle to be
uncollectable, including objects not necessarily in the cycle but
reachable only from it. ...
"""
However, no `__del__` method is defined anywhere, so there should not be
any uncollectable objects. Also, pypy collects the garbage, so this is
another thing pointing to a bug in Python.
I have tested this with Python 2.7.3 and Python 3.2.3, and both of those
will leak. Pypy 1.8.0 collects the garbage correctly.
Steps to reproduce: unpack the attachment, run tester.py, see if
gc.garbage has reference to _safe_iterator.
Even if this is a bug in Python this has to be fixed in Django itself. The
memory leak can be bad. It seems just reverting the commit is the right
fix.
Interestingly enough doing this change in Query.iterator() is enough to
cause leak:
{{{
try:
iterator() code here...
except Exception:
raise
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/19895#comment:7>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
--
You received this message because you are subscribed to the Google Groups
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
For more options, visit https://groups.google.com/groups/opt_out.
