Bug#1057126: Bug#1067104: Acknowledgement (server stalls: AH00046: child process 2876749 still did not exit, sending a SIGKILL)

2024-03-21 Thread Peter Krefting 

2024-03-21 13:12 skrev Yaroslav Halchenko:


FWIW here is a dirty workaround script I just crafted with chatgpt to
monitor/restart apache2 as soon as it starts happening


My workaround is simpler, I have this line in root's crontab:

 5 * * * * curl --silent --max-time 5 --output /dev/null 
http://localhost/trac/ || systemctl restart apache2


It seems to restart Apache once every 5-8 days, according to the notices 
I see from Zabbix. The frequency might very well be related to the 
number of accesses to the server.


--
\\// Peter - http://www.softwolves.pp.se/

Bug#1057126: Bug#1067104: Acknowledgement (server stalls: AH00046: child process 2876749 still did not exit, sending a SIGKILL)

2024-03-21 Thread Yaroslav Halchenko 
"All ingenious is simple" -- thanks for sharing.  I might redo following
your example but to check more frequently.

On Thu, 21 Mar 2024, Peter Krefting wrote:
> My workaround is simpler, I have this line in root's crontab:

>  5 * * * * curl --silent --max-time 5 --output /dev/null
> http://localhost/trac/ || systemctl restart apache2

-- 
Yaroslav O. Halchenko
Center for Open Neuroscience http://centerforopenneuroscience.org
Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
WWW:   http://www.linkedin.com/in/yarik

Bug#1057126: Bug#1067104: Acknowledgement (server stalls: AH00046: child process 2876749 still did not exit, sending a SIGKILL)

2024-03-21 Thread Yaroslav Halchenko 
I think "my" https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1067104
is a duplicate of this one.  I blame mod_wsgi since this I believe
started to happen after I started to use it.

FWIW here is a dirty workaround script I just crafted with chatgpt to
monitor/restart apache2 as soon as it starts happening (doesn't happen upon
every maintenance event for me I believe). Let me know if I should gather any
additional information.

#!/bin/bash

set -eu

# Define the lock file and log directory
lock_file="/var/log/apache-scoreboard-restart/lock.lck"
log_dir="/var/log/apache-scoreboard-restart/"

# Ensure the log directory exists
mkdir -p "$log_dir"

# Attempt to acquire a lock
exec 200>"$lock_file"
if ! flock -n 200 ; then
echo "Another instance is running."
exit 0
fi

# Function to perform actions when the specified log line is found
handle_scoreboard_full() {
local timestamp=$(date --iso-8601=seconds)
local log_file="${log_dir}${timestamp}.log"

echo "Logging system information to $log_file."
{ ps auxw -H; echo "---"; lsof; } > "$log_file"

echo "Reloading Apache." >> "$log_file"
service apache2 reload

echo "Sleeping for a minute." >> "$log_file"
sleep 60
}

# Monitor the Apache error log
while true; do
tail --follow=name /var/log/apache2/error.log | while read line 
; do
if echo "$line" | grep -q "AH03490: scoreboard is full, 
not at MaxRequestWorkers.Increase ServerLimit." ; then
handle_scoreboard_full
break  # so we start with a fresh tail
fi
done
done

-- 
Yaroslav O. Halchenko
Center for Open Neuroscience http://centerforopenneuroscience.org
Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
WWW:   http://www.linkedin.com/in/yarik



signature.asc
Description: PGP signature