Public bug reported: [ Impact ]
* mod_python crashes under certain circumstances, making the served request to be cut in half. One reliable reproducer is the new deployment of Merge O Matic, available here: https://merges.ubuntu.com/new_mom_py3/main.html Half the time the page is loaded, it's cropped, and half the time, it's full. That can be quickly spotted by looking if the stats are present at the bottom. Just reload a few times to observe the difference. Here is the backtrace I collected when debugging that: ``` (gdb) bt #0 __new_sem_post (sem=0x21) at ./nptl/sem_post.c:35 #1 0x00007f51f15428ad in PyThread_release_lock () from /lib/x86_64-linux-gnu/libpython3.10.so.1.0 #2 0x00007f51f15af551 in ?? () from /lib/x86_64-linux-gnu/libpython3.10.so.1.0 #3 0x00007f51f191d0aa in release_interpreter (idata=0x7f51e0071840) at /build/libapache2-mod-python-jVXfq1/libapache2-mod-python-3.5.0+git20211031/src/mod_python.c:306 #4 python_handler (req=0x7f51f12290a0, phase=<optimized out>) at /build/libapache2-mod-python-jVXfq1/libapache2-mod-python-3.5.0+git20211031/src/mod_python.c:1573 #5 0x00005638904ead18 in ap_run_handler () #6 0x00005638904ecc06 in ap_invoke_handler () #7 0x00005638905138f8 in ap_process_async_request () #8 0x0000563890513f9b in ?? () #9 0x00005638904ffe88 in ap_run_process_connection () #10 0x00007f51f194f755 in ?? () from /usr/lib/apache2/modules/mod_mpm_event.so #11 0x00007f51f19501ea in ?? () from /usr/lib/apache2/modules/mod_mpm_event.so #12 0x00007f51f1b66ac3 in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:442 #13 0x00007f51f1bf8850 in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81 ``` Here is also the OOPS collected from the same machine: https://errors.ubuntu.com/oops/b7bdd23a-f5c3-11ef-98c6-fa163e171f02 * This bug is already fixed upstream, and is described in the following issue and related links: https://github.com/grisha/mod_python/issues/100 The backtrace is more or less the same, but points to the same line (mod_python.c:306), giving me confidence it's the same issue, thus the same fix. [ Test Plan ] * Install the new version on the new server of Merge O Matic (or deploy everything locally, as you prefer). * Load https://merges.ubuntu.com/new_mom_py3/main.html a few times, the page should always come in full. [ Where problems could occur ] The patch touches a part of mod_python related to the release of the interpreter, and just skips calling PyThreadState_Clear(). I expect this could lead to some resource loss if the patch is wrong, but I don't expect other big outage, as there is not much logic change. Besides, since the thread is usually short lived, there shouldn't be too much long-lasting effects, but we could probably observe some particular pages failing to be processed by mod_python. [ Other Info ] * Since the patch is coming from upstream, it's actually already applied on Noble and onward. Only Jammy fell into the issue by combining the right Python version and an unpatched mod_python. ** Affects: libapache2-mod-python (Ubuntu) Importance: Undecided Status: Invalid ** Affects: libapache2-mod-python (Ubuntu Jammy) Importance: High Assignee: Skia (hyask) Status: In Progress ** Description changed: [ Impact ] - * mod_python crashes under certain circumstances, making the served request to be cut in half. - One reliable reproducer is the new deployment of Merge O Matic, available here: https://merges.ubuntu.com/new_mom_py3/main.html - Half the time the page is loaded, it's cropped, and half the time, it's full. That can be quickly spotted by looking if the stats are present at the bottom. Just reload a few times to observe the difference. + * mod_python crashes under certain circumstances, making the served request to be cut in half. + One reliable reproducer is the new deployment of Merge O Matic, available here: https://merges.ubuntu.com/new_mom_py3/main.html + Half the time the page is loaded, it's cropped, and half the time, it's full. That can be quickly spotted by looking if the stats are present at the bottom. Just reload a few times to observe the difference. Here is the backtrace I collected when debugging that: ``` (gdb) bt #0 __new_sem_post (sem=0x21) at ./nptl/sem_post.c:35 #1 0x00007f51f15428ad in PyThread_release_lock () from /lib/x86_64-linux-gnu/libpython3.10.so.1.0 #2 0x00007f51f15af551 in ?? () from /lib/x86_64-linux-gnu/libpython3.10.so.1.0 #3 0x00007f51f191d0aa in release_interpreter (idata=0x7f51e0071840) - at /build/libapache2-mod-python-jVXfq1/libapache2-mod-python-3.5.0+git20211031/src/mod_python.c:306 + at /build/libapache2-mod-python-jVXfq1/libapache2-mod-python-3.5.0+git20211031/src/mod_python.c:306 #4 python_handler (req=0x7f51f12290a0, phase=<optimized out>) - at /build/libapache2-mod-python-jVXfq1/libapache2-mod-python-3.5.0+git20211031/src/mod_python.c:1573 + at /build/libapache2-mod-python-jVXfq1/libapache2-mod-python-3.5.0+git20211031/src/mod_python.c:1573 #5 0x00005638904ead18 in ap_run_handler () #6 0x00005638904ecc06 in ap_invoke_handler () #7 0x00005638905138f8 in ap_process_async_request () #8 0x0000563890513f9b in ?? () #9 0x00005638904ffe88 in ap_run_process_connection () #10 0x00007f51f194f755 in ?? () from /usr/lib/apache2/modules/mod_mpm_event.so #11 0x00007f51f19501ea in ?? () from /usr/lib/apache2/modules/mod_mpm_event.so #12 0x00007f51f1b66ac3 in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:442 #13 0x00007f51f1bf8850 in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81 ``` + Here is also the OOPS collected from the same machine: https://errors.ubuntu.com/oops/b7bdd23a-f5c3-11ef-98c6-fa163e171f02 - * This bug is already fixed upstream, and is described in the following issue and related links: https://github.com/grisha/mod_python/issues/100 - The backtrace is more or less the same, but points to the same line (mod_python.c:306), giving me confidence it's the same issue, thus the same fix. + * This bug is already fixed upstream, and is described in the following issue and related links: https://github.com/grisha/mod_python/issues/100 + The backtrace is more or less the same, but points to the same line (mod_python.c:306), giving me confidence it's the same issue, thus the same fix. [ Test Plan ] - * Install the new version on the new server of Merge O Matic (or deploy everything locally, as you prefer). - * Load https://merges.ubuntu.com/new_mom_py3/main.html a few times, the page should always come in full. + * Install the new version on the new server of Merge O Matic (or deploy everything locally, as you prefer). + * Load https://merges.ubuntu.com/new_mom_py3/main.html a few times, the page should always come in full. [ Where problems could occur ] The patch touches a part of mod_python related to the release of the interpreter, and just skips calling PyThreadState_Clear(). I expect this could lead to some resource loss if the patch is wrong, but I don't expect other big outage, as there is not much logic change. Besides, since the thread is usually short lived, there shouldn't be too much long-lasting effects, but we could probably observe some particular pages failing to be processed by mod_python. - [ Other Info ] - * Since the patch is coming from upstream, it's actually already + * Since the patch is coming from upstream, it's actually already applied on Noble and onward. Only Jammy fell into the issue by combining the right Python version and an unpatched mod_python. ** Changed in: libapache2-mod-python (Ubuntu) Status: New => In Progress ** Also affects: libapache2-mod-python (Ubuntu Jammy) Importance: Undecided Status: New ** Changed in: libapache2-mod-python (Ubuntu Jammy) Status: New => In Progress ** Changed in: libapache2-mod-python (Ubuntu Jammy) Importance: Undecided => High ** Changed in: libapache2-mod-python (Ubuntu) Status: In Progress => Invalid ** Changed in: libapache2-mod-python (Ubuntu Jammy) Assignee: (unassigned) => Skia (hyask) -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/2100590 Title: Segfault when releasing a thread with Python 3.9+ To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/libapache2-mod-python/+bug/2100590/+subscriptions -- ubuntu-bugs mailing list [email protected] https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
