Adding to my recent post:
It seems that the `request-timeout=30` is effective, after all: The error
messages have changed, but there is no stack trace.
For completeness, here is the site config and error log excerpt:
Site config:
<VirtualHost *:80>
ServerAdmin webmaster@localhost
DocumentRoot /var/www/html
# Available loglevels: trace8, ..., trace1, debug, info, notice, warn,
# error, crit, alert, emerg.
# It is also possible to configure the loglevel for particular
# modules, e.g.
#LogLevel info ssl:warn
LogLevel info
ErrorLog ${APACHE_LOG_DIR}/error.log
CustomLog ${APACHE_LOG_DIR}/access.log combined
# Siehe https://bz.apache.org/bugzilla/show_bug.cgi?id=45023
# Hierfür muss mod_headers aktiv sein: sudo a2enmod headers
RequestHeader edit "If-None-Match" '^"((.*)-gzip)"$' '"$1", "$2"'
Alias /static/ /var/www/HallCam-static/
<Directory /var/www/HallCam-static>
Require all granted
</Directory>
Alias /media/ /var/www/HallCam-media/
<Directory /var/www/HallCam-media>
Require all granted
</Directory>
WSGIDaemonProcess cf-hallcam-site request-timeout=30 user=carsten
group=carsten processes=2 display-name=%{GROUP}
python-home=/home/carsten/.virtualenvs/HallCam-web
python-path=/home/carsten/HallCam/web
WSGIProcessGroup cf-hallcam-site
# WSGIApplicationGroup %{GLOBAL}
WSGIScriptAlias / /home/carsten/HallCam/web/HallCam/wsgi.py
<Directory /home/carsten/HallCam/web/HallCam>
<Files wsgi.py>
Require all granted
</Files>
</Directory>
</VirtualHost>
>From `/var/log/apache2/error.log`:
[Sun Feb 05 17:18:55.470532 2023] [mpm_event:notice] [pid 635:tid
140277991982976] AH00489: Apache/2.4.52 (Ubuntu) mod_wsgi/4.9.0 Python/3.10
configured -- resuming normal operations
[Sun Feb 05 17:18:55.690430 2023] [core:notice] [pid 635:tid
140277991982976] AH00094: Command line: '/usr/sbin/apache2'
[Sun Feb 05 17:18:56.338443 2023] [wsgi:info] [pid 638:tid 140277991982976]
mod_wsgi (pid=638): Attach interpreter ''.
[Sun Feb 05 17:18:56.342866 2023] [wsgi:info] [pid 636:tid 140277991982976]
mod_wsgi (pid=636): Attach interpreter ''.
[Sun Feb 05 17:18:56.366079 2023] [wsgi:info] [pid 636:tid 140277991982976]
mod_wsgi (pid=636): Adding '/home/carsten/HallCam/web' to path.
[Sun Feb 05 17:18:56.371777 2023] [wsgi:info] [pid 638:tid 140277991982976]
mod_wsgi (pid=638): Adding '/home/carsten/HallCam/web' to path.
[Sun Feb 05 17:19:24.823860 2023] [wsgi:info] [pid 636:tid 140277959865920]
mod_wsgi (pid=636): Create interpreter '192.168.1.222:32228|'.
[Sun Feb 05 17:19:24.837012 2023] [wsgi:info] [pid 636:tid 140277959865920]
mod_wsgi (pid=636): Adding '/home/carsten/HallCam/web' to path.
[Sun Feb 05 17:19:24.837752 2023] [wsgi:info] [pid 636:tid 140277959865920]
[remote 88.75.25.178:50558] mod_wsgi (pid=636, process='cf-hallcam-site',
application='192.168.1.222:32228|'): Loading Python script file
'/home/carsten/HallCam/web/HallCam/wsgi.py'.
[Sun Feb 05 17:19:43.356895 2023] [wsgi:info] [pid 638:tid 140277867546176]
mod_wsgi (pid=638): Create interpreter '192.168.1.222:32228|'.
[Sun Feb 05 17:19:43.369787 2023] [wsgi:info] [pid 638:tid 140277867546176]
mod_wsgi (pid=638): Adding '/home/carsten/HallCam/web' to path.
[Sun Feb 05 17:19:43.370532 2023] [wsgi:info] [pid 638:tid 140277867546176]
[remote 88.75.25.178:48210] mod_wsgi (pid=638, process='cf-hallcam-site',
application='192.168.1.222:32228|'): Loading Python script file
'/home/carsten/HallCam/web/HallCam/wsgi.py'.
[Sun Feb 05 17:23:21.631286 2023] [wsgi:info] [pid 636:tid 140277976651328]
mod_wsgi (pid=636): Daemon process request time limit exceeded, stopping
process 'cf-hallcam-site'.
[Sun Feb 05 17:23:21.631405 2023] [wsgi:info] [pid 636:tid 140277991982976]
mod_wsgi (pid=636): Shutdown requested 'cf-hallcam-site'.
[Sun Feb 05 17:23:26.631650 2023] [wsgi:info] [pid 636:tid 140277591307840]
mod_wsgi (pid=636): Aborting process 'cf-hallcam-site'.
[Sun Feb 05 17:23:26.631699 2023] [wsgi:info] [pid 636:tid 140277591307840]
mod_wsgi (pid=636): Exiting process 'cf-hallcam-site'.
[Sun Feb 05 17:23:26.714616 2023] [wsgi:error] [pid 639:tid
140277725460032] [client 88.75.25.178:48224] Truncated or oversized
response headers received from daemon process 'cf-hallcam-site':
/home/carsten/HallCam/web/HallCam/wsgi.py, referer:
http://vdzuggmrroo5k7e9.myfritz.net:32228/upload/
[Sun Feb 05 17:23:26.714952 2023] [wsgi:error] [pid 639:tid
140277959865920] (104)Connection reset by peer: [client 88.75.25.178:50558]
mod_wsgi (pid=639): Failed to proxy response from daemon., referer:
http://vdzuggmrroo5k7e9.myfritz.net:32228/upload/
[Sun Feb 05 17:23:26.983714 2023] [wsgi:info] [pid 885:tid 140277991982976]
mod_wsgi (pid=885): Attach interpreter ''.
[Sun Feb 05 17:23:26.994980 2023] [wsgi:info] [pid 885:tid 140277991982976]
mod_wsgi (pid=885): Adding '/home/carsten/HallCam/web' to path.
[Sun Feb 05 17:23:37.641476 2023] [wsgi:info] [pid 638:tid 140277976651328]
mod_wsgi (pid=638): Daemon process request time limit exceeded, stopping
process 'cf-hallcam-site'.
[Sun Feb 05 17:23:37.641609 2023] [wsgi:info] [pid 638:tid 140277991982976]
mod_wsgi (pid=638): Shutdown requested 'cf-hallcam-site'.
[Sun Feb 05 17:23:42.641863 2023] [wsgi:info] [pid 638:tid 140277591307840]
mod_wsgi (pid=638): Aborting process 'cf-hallcam-site'.
[Sun Feb 05 17:23:42.641910 2023] [wsgi:info] [pid 638:tid 140277591307840]
mod_wsgi (pid=638): Exiting process 'cf-hallcam-site'.
[Sun Feb 05 17:23:42.648248 2023] [wsgi:error] [pid 639:tid
140277717067328] [client 37.81.109.237:49336] Truncated or oversized
response headers received from daemon process 'cf-hallcam-site':
/home/carsten/HallCam/web/HallCam/wsgi.py
[Sun Feb 05 17:23:42.648624 2023] [wsgi:error] [pid 639:tid
140277733852736] (104)Connection reset by peer: [client 88.75.25.178:48210]
mod_wsgi (pid=639): Failed to proxy response from daemon., referer:
http://vdzuggmrroo5k7e9.myfritz.net:32228/
[Sun Feb 05 17:23:42.997522 2023] [wsgi:info] [pid 906:tid 140277991982976]
mod_wsgi (pid=906): Attach interpreter ''.
[Sun Feb 05 17:23:43.039478 2023] [wsgi:info] [pid 906:tid 140277991982976]
mod_wsgi (pid=906): Adding '/home/carsten/HallCam/web' to path.
[Sun Feb 05 17:25:01.761551 2023] [wsgi:info] [pid 885:tid 140277959865920]
mod_wsgi (pid=885): Create interpreter '192.168.1.222:32228|'.
[Sun Feb 05 17:25:01.774488 2023] [wsgi:info] [pid 885:tid 140277959865920]
mod_wsgi (pid=885): Adding '/home/carsten/HallCam/web' to path.
[Sun Feb 05 17:25:01.775225 2023] [wsgi:info] [pid 885:tid 140277959865920]
[remote 37.81.109.237:49338] mod_wsgi (pid=885, process='cf-hallcam-site',
application='192.168.1.222:32228|'): Loading Python script file
'/home/carsten/HallCam/web/HallCam/wsgi.py'.
[Sun Feb 05 17:25:03.155261 2023] [wsgi:error] [pid 885:tid
140277959865920] [remote 37.81.109.237:49338] Picture saved to
/var/www/HallCam-media/pictures/camera-1/pic_20230205_172500_4.jpg
(image/jpeg, 896262 bytes, camera camera-1)
[Sun Feb 05 17:30:03.323110 2023] [wsgi:info] [pid 885:tid 140277976651328]
mod_wsgi (pid=885): Daemon process deadlock timer expired, stopping process
'cf-hallcam-site'.
[Sun Feb 05 17:30:03.323253 2023] [wsgi:info] [pid 885:tid 140277991982976]
mod_wsgi (pid=885): Shutdown requested 'cf-hallcam-site'.
Carsten Fuchs schrieb am Sonntag, 5. Februar 2023 um 17:23:29 UTC+1:
> Dear Graham,
>
> I experienced the same timeout errors after having upgraded a server from
> Ubuntu 20.04 LTS to Ubuntu 22.04 LTS, thereby also upgrading from Python
> 3.8 to Python 3.10. The application is a relatively simple Django project
> that used to work well until the upgrade. After the upgrade, I deleted the
> old virtualenv and built a new one, using `pip install --no-cache-dir -r
> requirements.txt` to install it. However, I experience the same problem.
>
> Adding `WSGIApplicationGroup %{GLOBAL}` solved the problem, but I am still
> concerned because the site worked well with the older Ubuntu 20.04 LTS and
> I would prefer to not mask a potential problem and rather find its root.
>
> Therefore, I added the `request-timeout=30` option to `WSGIDaemonProcess`
> (and temporarily commented `WSGIApplicationGroup` out again) in order to
> get a stack trace, however it doesn't seem to have any effect: Requests
> time out only much later than 30 seconds.
>
> Can you please advise what may have caused the problem when upgrading from
> Ubuntu 20.04 LTS to Ubuntu 22.04 LTS and why `request-timeout=30` may not
> have any effect?
>
> Best regards,
> Carsten
>
> A schrieb am Dienstag, 24. Januar 2023 um 10:39:40 UTC+1:
>
>> Dear Graham,
>>
>> It was a matter of adding that line and everything fell into place!
>>
>>
>> Thanks a lot,
>> A
>>
>> On Monday, January 23, 2023 at 8:21:44 PM UTC+1 Graham Dumpleton wrote:
>>
>>> Your configuration means only a single request can be handled at a time
>>> by the daemon process, so if you have a very long running request, any
>>> other requests would block waiting.
>>>
>>> Even if not a long running request, a problem may be that you are using
>>> a Python package that isn't designed to work in a Python sub interpreter.
>>> This could cause that package to hang and so everything blocks up (same as
>>> long running request at that point) for that request.
>>>
>>> Application Issues — mod_wsgi 4.9.4 documentation
>>> <https://modwsgi.readthedocs.io/en/master/user-guides/application-issues.html#python-simplified-gil-state-api>
>>> modwsgi.readthedocs.io
>>> <https://modwsgi.readthedocs.io/en/master/user-guides/application-issues.html#python-simplified-gil-state-api>
>>> [image: favicon.ico]
>>> <https://modwsgi.readthedocs.io/en/master/user-guides/application-issues.html#python-simplified-gil-state-api>
>>>
>>> <https://modwsgi.readthedocs.io/en/master/user-guides/application-issues.html#python-simplified-gil-state-api>
>>>
>>> Try the solution described in the docs of forcing the use of the main
>>> Python interpreter by adding:
>>>
>>> WSGIApplicationGroup %{GLOBAL}
>>>
>>> If that doesn't help you will need to work out where your code is
>>> blocking. For that try adding:
>>>
>>> request-timeout=30
>>>
>>> option to WSGIDaemonProcess directive.
>>>
>>> This will cause daemon process in your case to restart after 30 seconds
>>> when it blocks and in doing that will attempt to log to error log file the
>>> stack traces of where your code was blocked.
>>>
>>> Graham
>>>
>>> On 24 Jan 2023, at 3:27 am, A <[email protected]> wrote:
>>>
>>> It keeps timing out and I've been trying to solve it to no avail.
>>> Here my modwsgi.conf
>>>
>>> <VirtualHost *:80>
>>> ServerName localhost
>>> ServerAlias ----------.com *.----------.com
>>>
>>> Define project_name ----------
>>> Define user -------------
>>>
>>> Define project_path /srv/http/fosware
>>> Define wsgi_path /srv/http/fosware/fosware
>>> Define environment_path /srv/http/fosware/venv
>>>
>>> WSGIDaemonProcess ${user}-${project_name} user=${user}
>>> group=${user} processes=1 threads=1 python-eggs=/tmp/python-eggs/
>>> python-path=${project_path}:${environment_path}/lib/python2.7/site-packages
>>> WSGIProcessGroup ${user}-${project_name}
>>>
>>> WSGIScriptAlias / ${wsgi_path}/wsgi.py
>>>
>>> <Directory ${project_path}>
>>> <IfVersion < 2.3 >
>>> Order allow,deny
>>> Allow from all
>>> </IfVersion>
>>> <IfVersion >= 2.3>
>>> Require all granted
>>> </IfVersion>
>>> </Directory>
>>>
>>>
>>> Alias /static ${project_path}/static
>>> <Directory ${project_path}/static>
>>> Require all granted
>>> SetHandler None
>>> FileETag none
>>> Options FollowSymLinks
>>> </Directory>
>>>
>>> Alias /media ${project_path}/media
>>> <Directory ${project_path}/media>
>>> Require all granted
>>> SetHandler None
>>> FileETag none
>>> Options FollowSymLinks
>>> ErrorDocument 404 /error404
>>> </Directory>
>>>
>>> ErrorLog /var/log/httpd/${user}-${project_name}-error.log
>>> LogLevel info
>>> CustomLog /var/log/httpd/${user}-${project_name}-access.log
>>> combined
>>> </VirtualHost>
>>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "modwsgi" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to [email protected].
>>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/modwsgi/02fb2171-ca59-4053-9be3-8ff75e9cf9edn%40googlegroups.com
>>>
>>> <https://groups.google.com/d/msgid/modwsgi/02fb2171-ca59-4053-9be3-8ff75e9cf9edn%40googlegroups.com?utm_medium=email&utm_source=footer>
>>> .
>>>
>>>
>>>
--
You received this message because you are subscribed to the Google Groups
"modwsgi" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/modwsgi/a3ede6cc-dfd2-423a-a8e0-55068433afcen%40googlegroups.com.
