#35618: response.close() in a TestCase prematurely closes PostgreSQL connection
and
leads to psycopg2.InterfaceError
-------------------------------------+-------------------------------------
Reporter: Anders Kaseorg | Type:
| Uncategorized
Status: new | Component: Database
| layer (models, ORM)
Version: 5.0 | Severity: Normal
Keywords: | Triage Stage:
| Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
If I call `response.close()` on an HTTP response from the test client
within a `django.test.TestCase`, Django incorrectly closes the active
PostgreSQL connection, causing future test requests to fail with
`psycopg2.InterfaceError: connection already closed`.
This happens because `HttpResponseBase.close`
[https://github.com/django/django/blob/5.0.7/django/http/response.py#L335
sends] `signals.request_finished`, which is
[https://github.com/django/django/blob/5.0.7/django/db/__init__.py#L60
handled] by `django.db.close_old_connections`, which
[https://github.com/django/django/blob/5.0.7/django/db/__init__.py#L57
calls] `BaseDatabaseWrapper.close_if_unusable_or_obsolete`, which
[https://github.com/django/django/blob/5.0.7/django/db/backends/base/base.py#L596
observes] `self.get_autocommit() != self.settings_dict["AUTOCOMMIT"]`
(`False != True`) and closes the connection.
Any connection that’s within a transaction (such as the one opened by
`TestCase`) will have `get_autocommit() == False`. It seems very wrong
that the finishing of any request automatically triggers all connections
in transactions to be immediately closed.
Minimal complete test project:
https://gist.github.com/andersk/b4da5a106995b4c525595ea204675ee0
{{{
$ ./manage.py test
Found 1 test(s).
Creating test database for alias 'default'...
System check identified no issues (0 silenced).
E
======================================================================
ERROR: test (tests.MyTest.test)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/tmp/close-test/.direnv/python-3.12/lib/python3.12/site-
packages/django/db/backends/base/base.py", line 294, in _cursor
return self._prepare_cursor(self.create_cursor(name))
^^^^^^^^^^^^^^^^^^^^^^^^
File "/tmp/close-test/.direnv/python-3.12/lib/python3.12/site-
packages/django/utils/asyncio.py", line 26, in inner
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/tmp/close-test/.direnv/python-3.12/lib/python3.12/site-
packages/django/db/backends/postgresql/base.py", line 332, in
create_cursor
cursor = self.connection.cursor()
^^^^^^^^^^^^^^^^^^^^^^^^
psycopg2.InterfaceError: connection already closed
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/tmp/close-test/tests.py", line 8, in test
response = self.client.get("/two")
^^^^^^^^^^^^^^^^^^^^^^^
File "/tmp/close-test/.direnv/python-3.12/lib/python3.12/site-
packages/django/test/client.py", line 1049, in get
response = super().get(path, data=data, secure=secure,
headers=headers, **extra)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/tmp/close-test/.direnv/python-3.12/lib/python3.12/site-
packages/django/test/client.py", line 465, in get
return self.generic(
^^^^^^^^^^^^^
File "/tmp/close-test/.direnv/python-3.12/lib/python3.12/site-
packages/django/test/client.py", line 617, in generic
return self.request(**r)
^^^^^^^^^^^^^^^^^
File "/tmp/close-test/.direnv/python-3.12/lib/python3.12/site-
packages/django/test/client.py", line 1013, in request
self.check_exception(response)
File "/tmp/close-test/.direnv/python-3.12/lib/python3.12/site-
packages/django/test/client.py", line 743, in check_exception
raise exc_value
File "/tmp/close-test/.direnv/python-3.12/lib/python3.12/site-
packages/django/core/handlers/exception.py", line 55, in inner
response = get_response(request)
^^^^^^^^^^^^^^^^^^^^^
File "/tmp/close-test/.direnv/python-3.12/lib/python3.12/site-
packages/django/core/handlers/base.py", line 197, in _get_response
response = wrapped_callback(request, *callback_args,
**callback_kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/tmp/close-test/my_urls.py", line 7, in view
with connection.cursor():
^^^^^^^^^^^^^^^^^^^
File "/tmp/close-test/.direnv/python-3.12/lib/python3.12/site-
packages/django/utils/asyncio.py", line 26, in inner
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/tmp/close-test/.direnv/python-3.12/lib/python3.12/site-
packages/django/db/backends/base/base.py", line 316, in cursor
return self._cursor()
^^^^^^^^^^^^^^
File "/tmp/close-test/.direnv/python-3.12/lib/python3.12/site-
packages/django/db/backends/base/base.py", line 293, in _cursor
with self.wrap_database_errors:
File "/tmp/close-test/.direnv/python-3.12/lib/python3.12/site-
packages/django/db/utils.py", line 91, in __exit__
raise dj_exc_value.with_traceback(traceback) from exc_value
File "/tmp/close-test/.direnv/python-3.12/lib/python3.12/site-
packages/django/db/backends/base/base.py", line 294, in _cursor
return self._prepare_cursor(self.create_cursor(name))
^^^^^^^^^^^^^^^^^^^^^^^^
File "/tmp/close-test/.direnv/python-3.12/lib/python3.12/site-
packages/django/utils/asyncio.py", line 26, in inner
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/tmp/close-test/.direnv/python-3.12/lib/python3.12/site-
packages/django/db/backends/postgresql/base.py", line 332, in
create_cursor
cursor = self.connection.cursor()
^^^^^^^^^^^^^^^^^^^^^^^^
django.db.utils.InterfaceError: connection already closed
----------------------------------------------------------------------
Ran 1 test in 0.018s
FAILED (errors=1)
Destroying test database for alias 'default'...
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/35618>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
--
You received this message because you are subscribed to the Google Groups
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-updates/01070190c7a8b61c-04ec0c5e-38fa-4457-ba7b-708c3abdd907-000000%40eu-central-1.amazonses.com.
