Hello. I am running Apache 2.4.43 with mod_wsgi 4.71 compiled against
Python 3.8.3 (all manually compiled, not part of the RHEL 8.1 distro).
Apache MPM is "event".
The application running in one of my virtual hosts will occasionally crash,
but randomly. Repeating the same request immediately will usually
succeed. The application may continue working for hours before randomly
crashing again.
This started happening recently with an upgrade of the Flask and SQLAlchemy
modules. (again, all manually installed via pip)
A crash is reported this way in the vhost's error_log:
[Mon Aug 31 11:11:07.504787 2020] [wsgi:error] [pid 35980:tid
140546546816768] [client 10.83.210.200:47266] Truncated or oversized
response headers received from daemon process 'sgm-prod':
/apps/www/sgm/wsgi/SGM/run.py
[Mon Aug 31 11:11:07.504806 2020] [wsgi:error] [pid 35980:tid
140546571994880] [client 10.31.82.142:35810] Truncated or oversized
response headers received from daemon process 'sgm-prod':
/apps/www/sgm/wsgi/SGM/run.py
[Mon Aug 31 11:11:08.512262 2020] [wsgi:info] [pid 40601:tid
140547110185856] mod_wsgi (pid=40601): Attach interpreter ''.
[Mon Aug 31 11:11:08.514415 2020] [wsgi:info] [pid 40601:tid
140547110185856] mod_wsgi (pid=40601): Adding '/apps/www/sgm/wsgi/SGM' to
path.
[Mon Aug 31 11:11:08.514526 2020] [wsgi:info] [pid 40601:tid
140547110185856] mod_wsgi (pid=40601): Adding
'/apps/vmscan/.virtualenvs/sgm/lib/python3.8/site-packages' to path.
[Mon Aug 31 11:11:08.515520 2020] [wsgi:debug] [pid 40601:tid
140546715096832] src/server/mod_wsgi.c(9118): mod_wsgi (pid=40601): Started
thread 0 in daemon process 'sgm-prod'.
[Mon Aug 31 11:11:08.515575 2020] [wsgi:debug] [pid 40601:tid
140546706704128] src/server/mod_wsgi.c(9118): mod_wsgi (pid=40601): Started
thread 1 in daemon process 'sgm-prod'.
[Mon Aug 31 11:11:08.515629 2020] [wsgi:debug] [pid 40601:tid
140546698311424] src/server/mod_wsgi.c(9118): mod_wsgi (pid=40601): Started
thread 2 in daemon process 'sgm-prod'.
[Mon Aug 31 11:11:08.515702 2020] [wsgi:debug] [pid 40601:tid
140546689918720] src/server/mod_wsgi.c(9118): mod_wsgi (pid=40601): Started
thread 3 in daemon process 'sgm-prod'.
[Mon Aug 31 11:11:08.515737 2020] [wsgi:debug] [pid 40601:tid
140546681526016] src/server/mod_wsgi.c(9118): mod_wsgi (pid=40601): Started
thread 4 in daemon process 'sgm-prod'.
[Mon Aug 31 11:11:08.515766 2020] [wsgi:debug] [pid 40601:tid
140546673133312] src/server/mod_wsgi.c(9118): mod_wsgi (pid=40601): Started
thread 5 in daemon process 'sgm-prod'.
[Mon Aug 31 11:11:08.515795 2020] [wsgi:debug] [pid 40601:tid
140546664740608] src/server/mod_wsgi.c(9118): mod_wsgi (pid=40601): Started
thread 6 in daemon process 'sgm-prod'.
[Mon Aug 31 11:11:08.515821 2020] [wsgi:debug] [pid 40601:tid
140546656347904] src/server/mod_wsgi.c(9118): mod_wsgi (pid=40601): Started
thread 7 in daemon process 'sgm-prod'.
[Mon Aug 31 11:11:08.515853 2020] [wsgi:debug] [pid 40601:tid
140546647955200] src/server/mod_wsgi.c(9118): mod_wsgi (pid=40601): Started
thread 8 in daemon process 'sgm-prod'.
[Mon Aug 31 11:11:08.515921 2020] [wsgi:debug] [pid 40601:tid
140546639562496] src/server/mod_wsgi.c(9118): mod_wsgi (pid=40601): Started
thread 9 in daemon process 'sgm-prod'.
[Mon Aug 31 11:11:08.515948 2020] [wsgi:debug] [pid 40601:tid
140546631169792] src/server/mod_wsgi.c(9118): mod_wsgi (pid=40601): Started
thread 10 in daemon process 'sgm-prod'.
[Mon Aug 31 11:11:08.515972 2020] [wsgi:debug] [pid 40601:tid
140546622777088] src/server/mod_wsgi.c(9118): mod_wsgi (pid=40601): Started
thread 11 in daemon process 'sgm-prod'.
[Mon Aug 31 11:11:08.516002 2020] [wsgi:debug] [pid 40601:tid
140546614384384] src/server/mod_wsgi.c(9118): mod_wsgi (pid=40601): Started
thread 12 in daemon process 'sgm-prod'.
[Mon Aug 31 11:11:08.516036 2020] [wsgi:debug] [pid 40601:tid
140546605991680] src/server/mod_wsgi.c(9118): mod_wsgi (pid=40601): Started
thread 13 in daemon process 'sgm-prod'.
[Mon Aug 31 11:11:08.516061 2020] [wsgi:debug] [pid 40601:tid
140546597598976] src/server/mod_wsgi.c(9118): mod_wsgi (pid=40601): Started
thread 14 in daemon process 'sgm-prod'.
The WSGI portion of the configuration for the vhost in Apache looks like
this:
WSGIPassAuthorization On
LogLevel info wsgi:trace6
SetEnv SGM_PRODUCTION 1
SetEnv SGM_USE_ORACLE 1
WSGIDaemonProcess sgm-prod user=sgmuser group=sgmgroup threads=15
python-home=/apps/sgmuser/.virtualenvs/sgm
python-path=/apps/www/sgm/wsgi/SGM:/apps/sgmuser/.virtualenvs/sgm/lib/python3.8/site-packages
socket-timeout=6000
WSGIScriptAlias / /apps/www/sgm/wsgi/SGM/run.py
<Directory /apps/www/sgm/wsgi/SGM>
Require all granted
AllowOverride AuthConfig
WSGIProcessGroup sgm-prod
WSGIApplicationGroup %{GLOBAL}
</Directory>
The main Apache error log shows nothing relevant, and the
application-specific logs (with Python logging messages) show only that the
request was made. There is no exception traceback data being logged (I'm
guessing the app is crashing before that can happen.)
Given how transitory this error is, I'm not sure how else to configure
Apache or the SGM app itself to get more details about why the "sgm-prod"
process seems to be crashing.
Any help would be appreciated! Thanks.
--
You received this message because you are subscribed to the Google Groups
"modwsgi" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/modwsgi/8df00cf3-abeb-43a8-b14a-6666af322a9an%40googlegroups.com.
