|
Moray, You are running into the limitations of mbx formatted boxes - especially when they are >2GB. Everytime a client requests a listing of the contents of the mail box - the server must perform a complete scan of an mbx file. In addition, when a message is appended to an mbx file another scan is performed. There simply is no way to get around the fact that mbx is basically a modified serial text file, no random access/insert is possible and when the size runs into the multi gigabyte level it just takes a lot of I/O to manage it. I suspect that you will find your system is struggling with high levels of I/O wait. >From personal experience I can say that the solution to your problem is to convert to mix mailbox format. I realize that the conversion of all users' mail stores on an established mail server is not trivial, especially on a live system so, in an effort to help, I have attached a script that I wrote and used to perform the conversion on our mail severs that are Sun Solaris systems. The script should be easily adapted to a linux system. It does all of the necessary checks, descends the user's mail folder tree (correctly determining whether it is mbox, mbx or a combination of mbx and mix folders) and does it's best to do the 'right' thing, avoiding re-converting any already existing mix format mail folders. It also preserves the original mail folder in case something goes wrong. You can convert a single user or all users in the passwd file. It won't touch uids <500 or >60000. It also won't create mail stores for a user if none already exist. I offer this script without any support or warranty, express or implied. I suggest running it against a test user account to make sure that you are satisfied with the results. I hope that it will be helpful to anyone struggling with the type of problem that you are having with overloaded mbx files. In addition to the conversion you will need to change the default operation of your uw-imap to use mix format folders and you should also convert to using tmail as the local delivery agent. We did this when we compiled uw-imap. You don't have to use tmail, but I highly recommend it. Good luck. --
Guy Dawson wrote: On 30/07/2010 11:34, Moray Henderson wrote:
| ||||||
#!/bin/sh # # -- Convert mailboxes to mix format for every user # # -- Author: Bob Atkins, DigiLink Network Services # # -- First Release : 2/16/08 # # -- Revised: # 8/25/09 - Added options to convert a single user. Added checks for already # converted mix format boxes so that they are not re-converted. # 9/9/09 - Create user INBOX after the conversion if it doesn't already exist. # # Copyright (C) 2008, Bob Atkins # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version.
# This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # You should have received a copy of the GNU General Public License # along with this program. If not, see <http://www.gnu.org/licenses/> # PROGNAME=`basename $0` USAGE="Usage: $PROGNAME [-u username] [-x]" SIGNAL=-TERM MIXCVT=mixcvt MAILUTIL=mailutil OSVER=`uname -r` SHUTDOWN=y umask 0077 usage () { echo $USAGE cat << "-EOF-" Converts existing mail stores of all users in the system passwd file to UW-Imap mix format. Where: -u Specifiy a single user to convert -x Do NOT shutdown imap/pop mail services. Use with CAUTION! mail2mix.sh Copyright (C) 2008, Bob Atkins This program comes with ABSOLUTELY NO WARRANTY. This is free software, and you are welcome to redistribute it under certain conditions. -EOF- } # -- Process any args if [ $# -ne 0 ]; then while getopts h\?u:x flags; do case $flags in u) USERNAME="${OPTARG}";; # Pickup arg to append to the package name x) SHUTDOWN=n;; ? | h) usage; exit 4;; esac done if [ -z "$USERNAME" ]; then echo "$PROGNAME -- Invalid arg/no username specified" usage; exit 4 fi else echo "\n!!!! WARNING !!!!\n" echo "This script will convert existing mail stores of all users in the system" echo "passwd file to UW-Imap mix format.\n" echo "Are you sure that you want to proceed [n] ? \c" read ans if [ "$ans" != "y" ]; then exit 1 fi fi # -- Shutdown all imap and pop mail services if [ "$SHUTDOWN" = "y" -a -z "$USERNAME" ]; then echo "$PROGNAME -- Shutting down system wide client mail services...\c" if [ "$OSVER" = "5.10" ]; then svcadm disable pop2 pop3 pop3s imap imaps elif [ "$OSVER" = "5.8" ]; then rm -f /tmp/inetd.conf sed -e 's/^pop/#pop/g s/^imap/#imap/g' /home/bob/tmp/inetd.conf > /tmp/inetd.conf mv /etc/inetd.conf /etc/inetd.conf.sav mv /tmp/inetd.conf /etc/inetd.conf pkill -HUP inetd pkill imapd pkill pop fi # - Make sure that all imapd and pop services have stopped before continuing nimapd=`ps -ef | grep imap | wc -l` npop=`ps -ef | grep pop | wc -l` nproc=`expr $nimapd + $npop` while [ nproc -gt 0 ] do sleep 1 echo ".\c" nimapd=`ps -ef | grep imap | wc -l` npop=`ps -ef | grep pop | wc -l` nproc=`expr $nimapd + $npop` done fi echo "Done!" if [ ! -z "$USERNAME" ]; then grep "$USERNAME:" /etc/passwd else cat /etc/passwd fi | while read line do username=`echo $line|cut -d: -f1` uid=`echo $line|cut -d: -f3` gid=`echo $line|cut -d: -f4` homedir=`echo $line|cut -d: -f6` maildir=$homedir/mail premix=".premix" mixmaildir=$homedir/mail.mix #echo $line if [ $uid -ge 100 -a $uid -lt 500 -o $username = "guest" ]; then echo "\n\n$PROGNAME -- Processing User '$username' mail files..." #echo "User = '$username'" #echo "Maildir = '$maildir'" if [ -d "$maildir" -a ! -d "${maildir}${premix}" ]; then if [ "$SHUTDOWN" = "y" -a ! -z "$USERNAME" ]; then echo "$PROGNAME -- Shutting down user client mail services...\c" pkill -u $username imap pkill -u $username pop nimapd=`ps -u $username | grep imap | wc -l` npop=`ps -u $username | grep pop | wc -l` nproc=`expr $nimapd + $npop` while [ nproc -gt 0 ] do sleep 1 echo ".\c" nimapd=`ps -u $username | grep imap | wc -l` npop=`ps -u $username | grep pop | wc -l` nproc=`expr $nimapd + $npop` done echo "Done!" fi find "$maildir" -name ".*" -prune -o -print | grep -v ".lock" | while read mailfile do if [ "$maildir" != "$mailfile" ]; then newmailfile=`echo $mailfile|sed -e "s^$maildir^$mixmaildir^g"` smailfile=`echo $mailfile|sed -e "s^$maildir^^g"` echo "'$smailfile'...\c" # -- Is this already a mix format directory box? if [ ! -f "$mailfile/.mixindex" ]; then # -- Create the destination directory if it doesn't exist newmailfiled="`dirname "$newmailfile"`" if [ ! -d "$newmailfiled" ]; then mkdir -p "$newmailfiled" fi # -- Convert to mix format #message=`$MIXCVT "$mailfile" "$newmailfile" 2>&1` $MIXCVT "$mailfile" "$newmailfile" if [ $? -eq 0 ]; then echo "Done!" else message=`echo $message|cut -d: -f2` echo "$message" fi else echo "Skipped - already mix format!" mv "$mailfile" "`dirname "$newmailfile"`" fi fi done if [ -d "${maildir}${premix}" -a ! -d "$mixmaildir" ]; then # -- There were no folders converted (empty maildir) so no mix dir was created mkdir "$mixmaildir" fi # -- If there is no INBOX then create it if [ ! -d "$mixmaildir/INBOX" ]; then echo "$PROGNAME -- Creating '$mixmaildir/INBOX'" $MAILUTIL create "#driver.mix/$mixmaildir/INBOX" fi # -- Make sure that everything in destination is properly owned by the user chown -R $uid:$gid "$mixmaildir" mv "$maildir" "${maildir}${premix}" mv "$mixmaildir" "$maildir" elif [ -d "${maildir}${premix}" ]; then echo "User '$username' - already converted - skipped!" else echo "No '$username' maildir found - skipped!" fi fi done # - Restore the client mail services if [ "$SHUTDOWN" = "y" -a -z "$USERNAME" ]; then echo "\n$PROGNAME -- Re-enabling system wide client mail services...\c" if [ "$OSVER" = "5.10" ]; then svcadm enable pop2 pop3 pop3s imap imaps elif [ "$OSVER" = "5.8" ]; then mv /etc/inetd.conf.sav /etc/inetd.conf pkill -HUP inetd fi fi echo "Done!" exit
_______________________________________________ Imap-uw mailing list [email protected] http://mailman2.u.washington.edu/mailman/listinfo/imap-uw
