Control: unmerge 1058317 1063345 Control: reassign 1063345 python3.12 Control: affects 1063345 celery Control: forwarded 1063345 https://github.com/python/cpython/issues/115874 Control: block 1058317 by 1063345 Control: tag 1063345 patch
On Tue, Feb 06, 2024 at 02:26:55PM +0100, Jérémy Lal wrote: > python3-celery test suite crashes with python 3.12 during gc before exit (see > attached stack trace). > > It can be reproduced quickly with > > apt build-dep celery > dget -xu https://deb.debian.org/debian/pool/main/c/celery/celery_5.3.4-2.dsc > cd celery-5.3.4 > python3.12 -m pytest -vvvvv > t/unit/tasks/test_canvas.py::test_group::test_apply_contains_chords_containing_empty_chain > t/unit/tasks/test_canvas.py::test_group::test_apply > > > Results (0.56s): > 1 passed > 1 xfailed > Erreur de segmentation (core dumped) There are two separate issues in the current celery FTBFS, so I'm unmerging these two bugs again because they ought to be tracked separately - and it turns out that this one really is a bug in Python 3.12. I happened to notice that something very similar to #1063345 had recently been reported to Python upstream as https://github.com/python/cpython/issues/115874, with a much more manageably-sized reproducer, and there's a patch for it in https://github.com/python/cpython/issues/115874#issuecomment-1965775536. I've built the Debian python3.12 package with this patch, and I've confirmed that Celery 5.3.6 passes its tests with this patch when it previously failed. Celery is unlucky to run into this, but it doesn't seem to be anything particularly intrinsic to Celery - it's an artifact of the particular set of pytest plugins it happens to use, which manage to tickle this particular bug. I think it would be worth applying this patch to Debian's python3.12 package, as it could easily bite in other places and this was a real time-sink for me before I managed to find the upstream bug that somebody else had fortunately filed. (It would also be nice to get Celery back into Debian testing and Ubuntu noble if at all possible, the latter of which has a fairly close deadline.) Separately, #1058317 still has the test failure in test_AsyncResult.test_del. That was fixed in Celery 5.3.5, and 5.3.6 is currently on salsa, but there's only limited point in uploading it until the Python bug is fixed. Suggested Python patch follows, though you might want to check whether it's progressed any further upstream in case there've been any additional tweaks: diff --git a/debian/changelog b/debian/changelog index 0e16636..b57f7dc 100644 --- a/debian/changelog +++ b/debian/changelog @@ -1,3 +1,10 @@ +python3.12 (3.12.2-1.1) UNRELEASED; urgency=medium + + * Don't rely on module state in teedataobject_clear (from Brandt Bucher in + https://github.com/python/cpython/issues/115874; closes: #1063345). + + -- Colin Watson <cjwat...@debian.org> Tue, 27 Feb 2024 10:15:37 +0000 + python3.12 (3.12.2-1) unstable; urgency=medium * Python 3.12.2 release. diff --git a/debian/patches/itertools-clear-crash.diff b/debian/patches/itertools-clear-crash.diff new file mode 100644 index 0000000..cdaeebb --- /dev/null +++ b/debian/patches/itertools-clear-crash.diff @@ -0,0 +1,19 @@ +Description: Don't rely on module state in teedataobject_clear +Origin: other, https://github.com/python/cpython/issues/115874#issuecomment-1965775536 +Bug: https://github.com/python/cpython/issues/115874 +Bug-Debian: https://bugs.debian.org/1063345 + +Index: b/Modules/itertoolsmodule.c +=================================================================== +--- a/Modules/itertoolsmodule.c ++++ b/Modules/itertoolsmodule.c +@@ -832,8 +832,7 @@ + Py_CLEAR(tdo->values[i]); + tmp = tdo->nextlink; + tdo->nextlink = NULL; +- itertools_state *state = get_module_state_by_cls(Py_TYPE(tdo)); +- teedataobject_safe_decref(tmp, state->teedataobject_type); ++ teedataobject_safe_decref(tmp, Py_TYPE(tdo)); + return 0; + } + diff --git a/debian/patches/series b/debian/patches/series index 63c72c2..0a85028 100644 --- a/debian/patches/series +++ b/debian/patches/series @@ -31,3 +31,4 @@ fix-py_compile.diff ntpath-import.diff python3.12-updates.diff issue108447.diff +itertools-clear-crash.diff Thanks, -- Colin Watson (he/him) [cjwat...@debian.org]
