#36957: Django psycopg connection pool + fork()
-------------------------------+--------------------------------------
Reporter: doc | Owner: (none)
Type: Bug | Status: new
Component: Uncategorized | Version: 6.0
Severity: Normal | Resolution:
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------+--------------------------------------
Description changed by doc:
Old description:
> Django's PostgreSQL backend stores psycopg_pool.ConnectionPool objects in
> a class-level dict (DatabaseWrapper._connection_pools). When gunicorn (or
> any pre-forking server) forks worker processes, all children inherit
> references to the same pool objects — and crucially, the same underlying
> TCP sockets to PostgreSQL. Multiple workers then read/write the same
> socket concurrently, corrupting the PostgreSQL wire protocol.
>
> **Root cause**
>
> {{{
> # django/db/backends/postgresql/base.py
> class DatabaseWrapper(BaseDatabaseWrapper):
> _connection_pools = {} # class-level dict — survives fork()
>
> @property
> def pool(self):
> if self.alias not in self._connection_pools:
> pool = ConnectionPool(...)
> self._connection_pools.setdefault(self.alias, pool)
> return self._connection_pools[self.alias]
> }}}
>
> **Workaround**
>
> {{{
> # gunicorn.conf.py
> def post_fork(server, worker):
> from django.db.backends.postgresql.base import DatabaseWrapper
> DatabaseWrapper._connection_pools.clear()
> }}}
>
> **Suggested fix**
> Use `os.register_at_fork(after_in_child=...)` to clear
> `_connection_pools` in child processes, or check `os.getpid()` in the
> `pool` property and recreate when it differs from the creating process.
>
> **Tested with**
> Django 6.0.2
> psycopg 3.2.x – 3.3.2
> psycopg-pool 3.2.x – 3.3.0
> gunicorn 25.x (--worker-class asgi)
> Python 3.12 – 3.14
New description:
Django's PostgreSQL backend stores `psycopg_pool.ConnectionPool` objects
in a class-level dict (`DatabaseWrapper._connection_pools`). When gunicorn
(or any pre-forking server) forks worker processes, all children inherit
references to the same pool objects — and crucially, the same underlying
TCP sockets to PostgreSQL. Multiple workers then read/write the same
socket concurrently, corrupting the PostgreSQL wire protocol.
**Root cause**
{{{
# django/db/backends/postgresql/base.py
class DatabaseWrapper(BaseDatabaseWrapper):
_connection_pools = {} # class-level dict — survives fork()
@property
def pool(self):
if self.alias not in self._connection_pools:
pool = ConnectionPool(...)
self._connection_pools.setdefault(self.alias, pool)
return self._connection_pools[self.alias]
}}}
**Workaround**
{{{
# gunicorn.conf.py
def post_fork(server, worker):
from django.db.backends.postgresql.base import DatabaseWrapper
DatabaseWrapper._connection_pools.clear()
}}}
**Suggested fix**
Use `os.register_at_fork(after_in_child=...)` to clear `_connection_pools`
in child processes, or check `os.getpid()` in the `pool` property and
recreate when it differs from the creating process.
**Tested with**
Django 6.0.2
psycopg 3.2.x – 3.3.2
psycopg-pool 3.2.x – 3.3.0
gunicorn 25.x (--worker-class asgi)
Python 3.12 – 3.14
The minimal reproducible example project is in the attachments
--
--
Ticket URL: <https://code.djangoproject.com/ticket/36957#comment:1>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
--
You received this message because you are subscribed to the Google Groups
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion visit
https://groups.google.com/d/msgid/django-updates/0107019c9e8b022e-9e684d86-d7ab-438d-b61f-ae0acddd8b10-000000%40eu-central-1.amazonses.com.
