#36056: Fix ignored exceptions in OutputWrapper.flush()
-------------------------------------+-------------------------------------
Reporter: Adam Johnson | Type: Bug
Status: new | Component: Core
| (Management commands)
Version: dev | Severity: Normal
Keywords: | Triage Stage:
| Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Running pytest on two different projects with Python 3.13, I’ve seen this
error logged after tests pass:
{{{
$ pytest
...
==== 1 passed in 0.01s ====
Exception ignored in: <django.core.management.base.OutputWrapper object at
0x173326230>
Traceback (most recent call last):
File "/.../.venv/lib/python3.13/site-
packages/django/core/management/base.py", line 171, in flush
self._out.flush()
ValueError: I/O operation on closed file.
...
}}}
It can appear multiple times, depending on how many management commands
are tested.
This message is from Python’s
[https://docs.python.org/3.12/library/sys.html#sys.unraisablehook
unraisable exception hook], logging the minimal available information for
an exception raised during a destructor or garbage collection.
I minimized the test case to find that it requires a test that captures
the exception from a failing management command, minimized to:
{{{#!python
import pytest
from django.core.management import call_command
from django.core.management.base import CommandError
from django.test import SimpleTestCase
class ExampleTests(SimpleTestCase):
def test_it(self):
with pytest.raises(CommandError) as excinfo:
call_command("check", "--non")
}}}
This test runs the `check` management command with a non-existent option,
triggering an error.
It can be run with `pytest` and `pytest-django` installed like:
{{{
$ pytest --ds=test_example test_example.py
========================= test session starts =========================
platform darwin -- Python 3.13.1, pytest-8.3.4, pluggy-1.5.0
django: version: 5.1.4, settings: test_example (from option)
rootdir: /.../example
configfile: pyproject.toml
plugins: cov-6.0.0, django-4.9.0
collected 1 item
test_example.py . [100%]
========================== 1 passed in 0.01s ==========================
Exception ignored in: <django.core.management.base.OutputWrapper object at
0x10533b760>
Traceback (most recent call last):
File "/.../.venv/lib/python3.13/site-
packages/django/core/management/base.py", line 171, in flush
self._out.flush()
ValueError: I/O operation on closed file.
Exception ignored in: <django.core.management.base.OutputWrapper object at
0x10549b760>
Traceback (most recent call last):
File "/.../.venv/lib/python3.13/site-
packages/django/core/management/base.py", line 171, in flush
self._out.flush()
ValueError: I/O operation on closed file.
}}}
(`-ds=test_example` specifies the Django settings module as the test
module, so you don’t need a whole project set up.)
After some drilling down, I determined the cause is `OutputWrapper`
instances that are created during `BaseCommand.__init__` with references
to `sys.stdout` and `sys.stderr`. It seems that when they’re deleted, they
interact poorly if the underlying file objects have already been closed.
pytest’s output caputring installs a mock `sys.stdout` and closes it after
tests are done, but the captured traceback retains a reference to the
`OutputWrapper` until tests finish. When the `OutputWrapper` is finally
garbage collected, the error is logged. Disabling output capturing with
`pytest -s` makes the error go away.
I tried to replicate within Django’s test framework with its output
capturing option, `--buffer`, but it didn’t work. I think it has some
differences that mean the error doesn’t appear.
But I did manage to minimize down to this test script, free of pytest and
management commands:
{{{#!python
import io
from django.core.management.base import OutputWrapper
out = io.TextIOWrapper(io.BytesIO())
wrapper = OutputWrapper(out)
out.close()
}}}
Running it shows:
{{{
$ python example_simplest.py
Exception ignored in: <django.core.management.base.OutputWrapper object at
0x1049cbb20>
Traceback (most recent call last):
File "/.../.venv/lib/python3.13/site-
packages/django/core/management/base.py", line 172, in flush
ValueError: I/O operation on closed file.
}}}
I also found that the error is reproducible on Python < 3.13 when enabling
development mode with `python -X dev`. It turns out the logging was
special-cased to development mode within the io module, but this special-
casing was removed in Python 3.13 in
[https://github.com/python/cpython/commit/58a2e0981642dcddf49daa776ff68a43d3498cee
commit 58a2e0981642dcddf49daa776ff68a43d3498cee]. So the error has been
there all along, but now it’s more visible.
I think the fix is to stop `OutputWrapper` from inheriting from
`TextIOBase`, as was added in dc8834cad41aa407f402dc54788df3cd37ab3e22.
That commit was focused on removing `force_str()` calls and it’s not clear
why the inheritance was added. But it seems that the path
[https://github.com/python/cpython/blob/e1baa778f602ede66831eb34b9ef17f21e4d4347/Lib/_pyio.py#L397
from `IOBase.__del__()`] to the custom `flush()` is what’s causing the
error.
--
Ticket URL: <https://code.djangoproject.com/ticket/36056>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
--
You received this message because you are subscribed to the Google Groups
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion visit
https://groups.google.com/d/msgid/django-updates/0107019426a98c88-1b96bbba-d996-4c17-81e9-a3a2dacabd4e-000000%40eu-central-1.amazonses.com.
