Off the cuff guesses: Maybe queue_only_load could help because if the load jumps too high, it will start queueing instead of forking to call procmail. Ending up with a large queue is not really desirable though.
Obviously your system benefits from file system caches. Maybe adding into your exim init script a simple 'find /path/to/maildirs -type d -maxdepth 1 -mindepth 1 &>/dev/null' would warm the caches enough so that you can achieve that low load you're looking for. On Tue, Sep 21, 2010 at 7:55 AM, Stanczak Slawomir <[email protected]> wrote: > Hello, > > Sorry for the very long letter. > > I have a problem with the connection of exim with procmail. Without > procmail exim works *very* good. When I send a test mail over the script > for one thousand users, the avarage load is about 2.50 - 3.00. > > #!/bin/bash > > MAIL=$HOME/Maildir/ > export MAIL > > if [ -f ./message -a -s ./message ] > then > echo "Sending mail for all users..." > for u in $(cat ./mail-users); do > mailx -s "Test" $u < ./message > done > exit > > There are a few Exim processes during the delivery: > > ara:~# ps -ef | grep exim > 100 3375 1 0 14:02 ? 00:00:00 /usr/sbin/exim4 -bd -q30m > root 13849 1 0 14:18 ? 00:00:00 /usr/sbin/exim4 -Mc > 1Oy1nb-0003bM-G8 > xdori 13852 13849 0 14:18 ? 00:00:00 [exim4] <defunct> > root 13858 12220 0 14:18 pts/2 00:00:00 grep exim > > This is o.k. > > Exim + Procmail (router and transport sections) > ----------------------------------------------- > procmail: > debug_print = "R: procmail for $local_p...@$domain" > driver = accept > domains = +local_domains > check_local_user > transport = procmail_pipe > require_files = ${local_part}:+${home}:+/usr/bin/procmail > no_verify > no_expn > > procmail_pipe: > debug_print = "T: procmail_pipe for $local_p...@$domain" > driver = pipe > path = "/bin:/usr/bin:/usr/sbin" > command = "/usr/bin/procmail -d $local_part" > return_path_add > delivery_date_add > envelope_to_add > > When I include this sections and I run this script many procmail > processes appear and within a few seconds load average reaches value > 400-500. > > What interesting if after that I run this script second time again, > the avarage load is low (2.5 - 3) and the number of procmail processes > is low. > > I tested it many times. It shows allways during the first run of the > script after system reboot. > > I did another test. I sent from my other mail host 60-100 letters > on aliass to all useres (60x1000 messages) on this server but the max load > never exceeded value 50.0. > > I did another test. I sent from my other mail host 100 letters to aliass > for all useres (100x1000 messages) on this server but the max load never > exceeded value 55.0. > > I checked variables: > > cat /etc/profile: > [...] > MAIL=$HOME/Maildir/ > MAILDIR=$HOME/Maildir/ > export MAIL MAILDIR > > cat /etc/login.defs > MAIL_DIR Maildir/ > > I changed commands in scripts ("mailx" to "mutt"). > > I checked procmail files: > > Main file: > ----------------- > cat /etc/procmail > MAIL=$HOME/Maildir/ > PATH=/usr/bin:/bin:/usr/sbin:$HOME/bin:. > MAILDIR=$HOME/Maildir/ > DEFAULT=$HOME/Maildir/ > LOGFILE=$HOME/Maildir/procmail.log > #VERBOSE=yes > #LOGABSTRACT=all > # Use a system-wide /etc/procmailrc (ignore $HOME/.procmailrc) > DROPPRIVS=no > ------------ > > and users file; > --------------- > PATH=$HOME/bin:/usr/bin:/bin:/usr/local/bin:. > MAILDIR=$HOME/Maildir/ > DEFAULT=$HOME/Maildir/ > LOGFILE=$HOME/Maildir/procmail.log > LOCKFILE=$HOME/Maildir/.lockmail > BOGOFILTER_DIR=$HOME/.bogofilter > TRASH=$HOME/Maildir/.Quarantine/ > > :0 H: > * < 200000 > * 0^0 > * 1^1 ^(From|To|Content-Type|Subject).*koi8-r > * 1^1 ^(From|To|Content-Type|Subject).*(gb2312|big5) > * 1^1 ^(From|To|Content-Type|Subject).*ks_c_5601\-1987 > * 1^1 ^(From|To|Content-Type|Subject).*(euc|EUC).(kr|KR) > * 1^1 ^(From|To|Content-Type|Subject).*(iso|ISO).2022.(jp|JP|kr|KR) > $TRASH > > :0 B: > * < 200000 > * .*ks_c_5601-1987 > * .*charset=[.]EUC(-|_)KR > * .*charset=[.]euc(-|_)kr > $TRASH > > :0 H: > * < 200000 > * > ^(From|To|Reply-To|Return-Path).*\.(cc|cn|hk|in|iq|ir|kp|kr|th|tr|tw|vn)\.com > * ^(From|To|Reply-To|Return-Path).*\.(cc|cn|hk|in|iq|ir|kp|kr|th|tr|tw|vn) > $TRASH > > :0fw > * < 200000 > | bogofilter -p -e -v > > :0e > { EXITCODE=75 HOST } > > :0: > * < 200000 > * ^Subject:.\*\*\*SPAM\*\*\* > $TRASH > > :0: > * < 200000 > * ^Subject:.\?\?\?UNSURE\?\?\? > $MAILDIR > > :0 > $MAILDIR > > These changes did not produce any effect. > ------------------------------------------------------------------ > > This is *slow* machine (HP360G5). I tested it on the faster platform > (8 core, 32 GB RAM). Within a few seconds the avarage load reached > value 950 and the system crashed. > > Info from the message log: > > Sep 4 09:21:59 ara kernel: [ 8973.522891] procmail D > 0000000000000000 0 8067 8066 > Sep 4 09:21:59 ara kernel: [ 8973.571796] ffff81076a3a5d48 > 0000000000000086 0000000000000000 0000000000000000 > Sep 4 09:21:59 ara kernel: [ 8973.618943] ffff81080712c0c0 > ffff81082c9514f0 ffff81080712c348 00000003a022fc43 > Sep 4 09:21:59 ara kernel: [ 8973.641745] 0000000100000001 > 0000000000000000 00000000ffffffff 0000000000000000 > Sep 4 09:21:59 ara kernel: [ 8973.646562] Call Trace: > Sep 4 09:21:59 ara kernel: [ 8973.774846] [<ffffffff802710a6>] > sync_page+0x0/0x41 > Sep 4 09:21:59 ara kernel: [ 8973.803572] [<ffffffff80429437>] > io_schedule+0x5c/0x9e > Sep 4 09:21:59 ara kernel: [ 8973.847019] [<ffffffff802710e2>] > sync_page+0x3c/0x41 > Sep 4 09:21:59 sci kernel: [ 8973.884952] [<ffffffff80429692>] > __wait_on_bit+0x40/0x6e > Sep 4 09:21:59 ara kernel: [ 8973.933370] [<ffffffff8027132d>] > wait_on_page_bit+0x6b/0x71 > Sep 4 09:21:59 ara kernel: [ 8973.980036] [<ffffffff8024619f>] > wake_bit_function+0x0/0x23 > Sep 4 09:21:59 ara kernel: [ 8974.004037] [<ffffffff80278dfa>] > pagevec_lookup_tag+0x1a/0x21 > Sep 4 09:21:59 ara kernel: [ 8974.046710] [<ffffffff80271844>] > wait_on_page_writeback_range+0x66/0x113 > Sep 4 09:21:59 ara kernel: [ 8974.070277] [<ffffffff80246171>] > autoremove_wake_function+0x0/0x2e > Sep 4 09:21:59 ara kernel: [ 8974.113929] [<ffffffffa0248d65>] > :xfs:xfs_fsync+0x3a/0x183 > Sep 4 09:21:59 ara kernel: [ 8974.158211] [<ffffffffa02501d8>] > :xfs:xfs_file_fsync+0x41/0x49 > Sep 4 09:21:59 ara kernel: [ 8974.181369] [<ffffffff802b971a>] > do_fsync+0x52/0xa4 > Sep 4 09:21:59 ara kernel: [ 8974.228046] [<ffffffff802b978f>] > __do_fsync+0x23/0x36 > Sep 4 09:21:59 ara kernel: [ 8974.250611] [<ffffffff8020beda>] > system_call_after_swapgs+0x8a/0x8f > Sep 4 09:21:59 ara kernel: [ 8974.276945] > > I changed the kernel (2.6.32) and file system from xfs on ext3 > but no effect. > > Thank you in advance for any suggestions > > Regards > > Slawomir Stanczak > > -- > ## List details at http://lists.exim.org/mailman/listinfo/exim-users > ## Exim details at http://www.exim.org/ > ## Please use the Wiki with this list - http://wiki.exim.org/ > -- Regards... Todd I seek the truth...it is only persistence in self-delusion and ignorance that does harm. -- Marcus Aurealius -- ## List details at http://lists.exim.org/mailman/listinfo/exim-users ## Exim details at http://www.exim.org/ ## Please use the Wiki with this list - http://wiki.exim.org/
