Thanks for the report, Peter. I'm sure it'll help with tracking this problem
down.

A few thoughts come to mind (first thing in the morning, so I hope I'm awake!):

Restarting qmail (via qmailctl restart or service qmail restart, they're the
same thing) does not restart *everything*. It only recycles qmail-smtp and
qmail-send. To recycle everything (including clamav), you should
.) qmailctl stop
.) qmailctl status
I'm not positive that this will show if there are zombies or not. You should
probably do a few 'ps' commands to see if you can find any stragglers. If
there are still any processes still running, especially zombies, you should
hunt them down and kill them.
.) qmailctl start

You seem to have disabled clamav effectively for the time being, so email is
flowing but not being scanned by clamav. I wouldn't want to run in that
state for an extended period. If anyone needs
clamav-toaster-0.88.7-1.3.7.src.rpm to restore to for the time being, let me
know and I'll make it available.

clamdscan appears to be getting into an infinite-loop kind of state. Peter,
can you verify from your logs whether or not any emails were scanned to
completion? I'd like to know if this is happening on all emails or just some
of them.

My thinking at this point is that the clamd log messages that are doubling
up are likly pointing to the problem. The doubling messages are not normal,
and if there are indeed two processes running somehow, they might be
stepping on each other, causing the lock-up.

I'll continue to try and figure this out.

Peter Peltonen wrote:
> At 12:50 the situation is: 8 clamdscan processes eating 100 % cpu
> 
> I notice that freshclam is not running:
> 
> # service freshclam status
> freshclam dead but subsys locked
> 
> All I see in freshclam log are messages like this:
> 
> Received signal: wake up
> ClamAV update process started at Mon Feb 26 10:40:41 2007
> main.cvd is up to date (version: 42, sigs: 83951, f-level: 10, builder:
> tkojm)
> daily.inc is up to date (version: 2654, sigs: 10790, f-level: 13,
> builder: ccordes)
> 
> I restarted it:
> 
> # service freshclam stop
> Stopping freshclam:                                        [FAILED]
> 
> # service freshclam start
> Starting freshclam:                                        [  OK  ]
> 
> And the log says:
> 
> freshclam daemon 0.90 (OS: linux-gnu, ARCH: i386, CPU: i386)
> ClamAV update process started at Mon Feb 26 13:01:47 2007
> main.cvd is up to date (version: 42, sigs: 83951, f-level: 10, builder:
> tkojm)
> Downloading daily-2655.cdiff [100%]
> daily.inc updated (version: 2655, sigs: 10791, f-level: 13, builder:
> ccordes)
> Database updated (94742 signatures) from db.fi.clamav.net (IP:
> 62.236.254.228)
> 
> [EMAIL PROTECTED] root]# service freshclam status
> freshclam (pid 11365) is running...
> 
> Any effect on clamdscan? Nope:
> 
> At 13:05 the situation is: 9 clamdscan processes eating 100 % cpu
> 
> After that I tried a testing procedure that I found at
> http://wiki.clamav.net/Main/SimScan
> 
> which did not go quite well:
> 
> # QMAILQUEUE=/var/qmail/bin/simscan SIMSCAN_DEBUG=2 \
> /var/qmail/bin/qmail-inject [EMAIL PROTECTED] < testmail.txt
> qmail-inject: fatal: qq crashed (#4.3.0)
> 
> I found out that no email is going through again... I also see now 19
> clamscan
> processes.
> 
> When looking at the log I see that a reload has been issued before my
> test (because of freshclam restart I suppose?):
> 
> 2007-02-26 13:09:04.310381500 SelfCheck: Database modification detected.
> Forcing
> reload.
> 2007-02-26 13:09:04.310385500 SelfCheck: Database modification detected.
> Forcing
> reload.
> 2007-02-26 13:09:04.310387500 Reading databases from /usr/share/clamav
> 2007-02-26 13:09:04.310389500 Reading databases from /usr/share/clamav
> 2007-02-26 13:09:11.251511500 Database correctly reloaded (94742
> signatures)
> 2007-02-26 13:09:11.251580500 Database correctly reloaded (94742
> signatures)
> 
> I disable clam for all domains in /var/qmail/control/simcontrol and
> rebuild the
> cdb:
> 
> [EMAIL PROTECTED] root]# service qmail cdb
> 
> Then I restart clamd:
> 
> root]# svc -h /var/qmail/supervise/clamd
> 
> The logs say:
> 
> 2007-02-26 13:25:27.830833500 SIGHUP caught: re-opening log file.
> 2007-02-26 13:25:27.830879500 SIGHUP caught: re-opening log file.
> 
> Email seems to go through now. No effect on the clamscan processes
> though. Ok,
> let's shutdown clamav totally:
> 
> svc -d /var/qmail/supervise/clamd
> svc -t /var/qmail/supervise/clamd
> 
> Nothing happens, the processes still stay there, nothing in the logs.
> `service
> qmail stat` says:
> 
> clamd: up (pid 26992) 6896 seconds, want down
> 
> After a few minutes this is still the issue. There are now 21
> clamdscan processes
> spawned by qmail-smtpd and simscan, at least pstree reports me the
> processes like
> this:
> 
> |-qmail-smtpd(31206)---simscan(31209)---clamdscan(31223)
> |-qmail-smtpd(1462)---simscan(1465)---clamdscan(1468)
> 
> I kill the clamdscan processes with `kill -9` and the processes
> disappear (also the
> related qmail-smtpd processes). Clamd still won't shut itself down, I
> have to kill
> it also.
> 
> Mail seems to go through normally so I suppose I do not have to have the
> clamd
> process running when no domains use it.
> 
> As this is a production server I'll stop my testing here... I hope it
> gets resolved. I tried
> to peek at ClamAV bugtracker / mailing list, but did not see any
> similiar reports.
> 
> Regards,
> Peter
> 


-- 
-Eric 'shubes'

---------------------------------------------------------------------
     QmailToaster hosted by: VR Hosted <http://www.vr.org>
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Reply via email to