#17251: select_for_update tests threads should close connections manually
-------------------------------------+-------------------------------------
Reporter: akaariai | Owner: nobody
Type: | Status: new
Cleanup/optimization | Version:
Component: Uncategorized | Resolution:
Severity: Normal | Triage Stage:
Keywords: | Unreviewed
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by akaariai):
I did some more digging into when separate thread connections are closed.
It happens to be so that the current behaviour relies on `DatabaseWrapper`
being thread-local. The problem is that the chain from
django.db.connection to the real underlying connection (for example
psycopg2 connection) contains cycles. Because of the cycles, the
connections will not be GCed (and thus closed) immediately when last
reference to them is closed. It seems that an object which is
threading.local subclass has a bit different rules to when it will be GCed
(or more precisely, when its `__dict__` will be collected). This makes
select_for_update tests leave open connections behind if you move the
threading.local from `DatabaseWrapper` to the
`ConnectionHandler`._connections dictionary. This also has the possibility
to make for some really hard to debug problems if Python (or `PyPy` or
Jython or...) decides to change the rules how GCing is implemented.
Anyways, it would be a good idea to always close connections in separate
threads. The abovementioned patch does this for select_for_update tests.
Some method decorators would be useful so that you could easily ensure
your connections are closed when running command line tasks, or when
creating separate threads. I think it would also be a good idea to break
the cyclic references in `DatabaseWrapper` (for example `DatabaseWrapper`
-> `DatabaseFeatures` -> `DatabaseWrapper`). This will however break every
external backend... If this is not done, at least document the gotchas in
this area of the code.
Additional note: do not use `__del__` to check when the objects are GCed,
this will make any cyclic reference uncollectable by the
[http://docs.python.org/library/gc.html#module-gc GC]. I happen to know
this isn't nice to debug... :)
But most importantly: it seems it is essential connections for separate
threads are closed manually in the current Django implementation. So, lets
do that for select_for_update tests.
--
Ticket URL: <https://code.djangoproject.com/ticket/17251#comment:2>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
--
You received this message because you are subscribed to the Google Groups
"Django updates" group.
To post to this group, send email to django-updates@googlegroups.com.
To unsubscribe from this group, send email to
django-updates+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/django-updates?hl=en.
