Re: rooling replication and many pop3d
On 22/11/2011 15:52, Andy Bennett wrote: Hi, I enabled replication between two servers with version 2.4.10 cyrus. I set the option for the rolling replication, and it works fine but obviously I have a high CPU load. Unfortunately after 10 minutes of running processes pop3d increasing from 50 to over 200, making the server unusable for customers. Can you tell me why this increase is abnormal? Can you use something like 'top' to work out which processes are consuming most of the CPU time? This screenshot of top before : http://www.digicolor.net/cyrus/img1.jpg and after 20 minutes of rooling replication : http://www.digicolor.net/cyrus/img5.jpg Can you use something like 'vmstat 1' to show us how much I/O there is on the system? This screenshot of top before : http://www.digicolor.net/cyrus/img3.jpg and after 20 minutes of rooling replication : http://www.digicolor.net/cyrus/img6.jpg Are most of the pop3d processes sleeping in iowait? Do you use any other servers such as the impad? Yes I have imapd This is screenshot of pstree before : http://www.digicolor.net/cyrus/img2.jpg This is screenshot of pstree after : http://www.digicolor.net/cyrus/img4.jpg Regards, Obe. -- Oberdan Albertoni --- DIGICOLOR.NET s.r.l. Via Ala Ponzone, 30 - 26100 CREMONA (CR) Tel. +39 (0372) 30669 - 1965101 - 1951509 Fax +39 (0372) 464840 E-mail i...@digicolor.net - Website http://www.digicolor.net C.F / P.IVA 01227520192 - R.E.A. CR n. 153382 - Capitale Sociale € 10.400 i.v. Filiale di Codogno Via Galilei, 17 – 26845 CODOGNO (LO) Tel. +39 (0377) 741006 – 431931 Fax +39 (0372) 464840 E-mail info.codo...@digicolor.net - Website http://www.digicolor.net --- Ai sensi del D.lgs n.196 del 30.06.03 (Codice Privacy) si precisa che le informazioni contenute in questo messaggio sono riservate e ad uso esclusivo del destinatario. Qualora il messaggio in parola Le fosse pervenuto per errore, La preghiamo di eliminarlo senza copiarlo e di non inoltrarlo a terzi, dandocene gentilmente comunicazione. Grazie This message, for the D.lgs n.196 / 30.06.03 (Privacy Code), may contain confidential and/or privileged information. If you are not the addressee or authorized to receive this for the addressee, you must not use, copy, disclose or take any action based on this message or any information herein. If you have received this message in error, please advise the sender immediately by reply e-mail and delete this message. Thank you for your cooperation. --- Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Huge load after update from 2.3.16 to 2.4.12
Hi,
I just updated Cyrus Imapd on our somewhat old mailsever (dual-Xeon, but
8 years now, running gentoo linux) from 2.3.16 to 2.4.12
(options USE=berkdb kerberos pam replication sieve snmp sqlite ssl tcpd
-afs -mysql -nntp -postgres)
There is no murder and I also could drop sqlite, if better.
After the flawless update, restarting generated a huge load. I believe I
never had a load of 50 any time before.
It even gets /var/imap/socket/lmtp] timed out while sending MAIL FROM
on local delivery of mails.
This is cyrus.conf:
START {
# do not delete this entry!
recover cmd=ctl_cyrusdb -r
# this is only necessary if using idled for IMAP IDLE
# idledcmd=idled
}
# UNIX sockets start with a slash and are put into /var/imap/socket
SERVICES {
# add or remove based on preferences
imap cmd=imapd listen=127.0.0.1:imap2 prefork=2
#pop3 cmd=pop3d listen=pop-3 prefork=0
# Don't forget to generate the needed keys for SSL or TLS
# (see doc/html/install-configure.html)
imaps cmd=imapd -s listen=imaps prefork=15
#pop3scmd=pop3d -s listen=pop3s prefork=0
sieve cmd=timsieved listen=sieve prefork=15
# at least one LMTP is required for delivery
lmtp cmd=lmtpd -a -D listen=127.0.0.1:lmtp prefork=1
lmtpunix cmd=lmtpd listen=/var/imap/socket/lmtp prefork=2
# this is only necessary if using notifications
notifycmd=notifyd listen=/var/imap/socket/notify
proto=udp prefork=1
}
EVENTS {
# this is required
checkpointcmd=ctl_cyrusdb -c period=5
# this is only necessary if using duplicate delivery suppression
# delprune cmd=ctl_deliver -E 3 period=1440
delprune cmd=cyr_expire -E 3 -x period=1440
# this is only necessary if caching TLS sessions
tlsprune cmd=tls_prune period=1440 at=0600
# squatter indiziert die Boxen fuer die Suche
squatter cmd=squatter -r -s user period=1440 at=0200
}
and imapd.conf:
proxyservers: cyrus
configdirectory:/var/imap
partition-default: /var/imap/part
sievedir: /var/imap/sieve
sieve_sasl_mech_list: PLAIN
tls_cert_file: /etc/cyrusimapd/server2009.pem
tls_key_file: /etc/cyrusimapd/server2009.pem
tls_ca_file:/etc/cyrusimapd/server2009.pem
duplicate_db: skiplist
mboxlist_db: skiplist
quota_db: quotalegacy
tlscache_db:skiplist
admins: cyrus
hashimapspool: yes
allowanonymouslogin:no
allowplaintext: yes
allowusermoves: yes
# Use this if sieve-scripts could be in ~user/.sieve.
#sieveusehomedir: yes
# Use saslauthd if you want to use pam for imap.
# But be warned: login with DIGEST-MD5 or CRAM-MD5
# is not possible using pam.
sasl_pwcheck_method:saslauthd
sasl_mech_list: LOGIN PLAIN
autocreatequota: 1500
altnamespace: yes
sasl_auto_transition: no
quotawarn: 90
tls_cipher_list: TLSv1:SSLv3:SSLv2:!NULL:!EXPORT:!DES:!LOW:@STRENGTH
allowallsubscribe: no
unixhierarchysep: yes
lmtp_downcase_rcpt: yes
annotation_db: skiplist
seenstate_db: flat
notifysocket: /var/imap/socket/notify
I have backups of the old packages, but I'man not shure if this will
work anymore.
Any ideas please ? My bloodpressure can't stand this...
CU
Michael
Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: Huge load after update from 2.3.16 to 2.4.12
On 23/11/11 19:18 +0100, Michael Plate wrote:
Hi,
I just updated Cyrus Imapd on our somewhat old mailsever (dual-Xeon, but
8 years now, running gentoo linux) from 2.3.16 to 2.4.12
(options USE=berkdb kerberos pam replication sieve snmp sqlite ssl tcpd
-afs -mysql -nntp -postgres)
There is no murder and I also could drop sqlite, if better.
After the flawless update, restarting generated a huge load. I believe I
never had a load of 50 any time before.
It even gets /var/imap/socket/lmtp] timed out while sending MAIL FROM
on local delivery of mails.
This is cyrus.conf:
START {
# do not delete this entry!
recover cmd=ctl_cyrusdb -r
# this is only necessary if using idled for IMAP IDLE
# idledcmd=idled
}
# UNIX sockets start with a slash and are put into /var/imap/socket
SERVICES {
# add or remove based on preferences
imap cmd=imapd listen=127.0.0.1:imap2 prefork=2
#pop3 cmd=pop3d listen=pop-3 prefork=0
# Don't forget to generate the needed keys for SSL or TLS
# (see doc/html/install-configure.html)
imaps cmd=imapd -s listen=imaps prefork=15
#pop3scmd=pop3d -s listen=pop3s prefork=0
sieve cmd=timsieved listen=sieve prefork=15
# at least one LMTP is required for delivery
lmtp cmd=lmtpd -a -D listen=127.0.0.1:lmtp prefork=1
lmtpunix cmd=lmtpd listen=/var/imap/socket/lmtp prefork=2
# this is only necessary if using notifications
notifycmd=notifyd listen=/var/imap/socket/notify
proto=udp prefork=1
}
EVENTS {
# this is required
checkpointcmd=ctl_cyrusdb -c period=5
# this is only necessary if using duplicate delivery suppression
# delprune cmd=ctl_deliver -E 3 period=1440
delprune cmd=cyr_expire -E 3 -x period=1440
# this is only necessary if caching TLS sessions
tlsprune cmd=tls_prune period=1440 at=0600
# squatter indiziert die Boxen fuer die Suche
squatter cmd=squatter -r -s user period=1440 at=0200
}
and imapd.conf:
proxyservers: cyrus
configdirectory:/var/imap
partition-default: /var/imap/part
sievedir: /var/imap/sieve
sieve_sasl_mech_list: PLAIN
tls_cert_file: /etc/cyrusimapd/server2009.pem
tls_key_file: /etc/cyrusimapd/server2009.pem
tls_ca_file:/etc/cyrusimapd/server2009.pem
duplicate_db: skiplist
mboxlist_db: skiplist
quota_db: quotalegacy
tlscache_db:skiplist
admins: cyrus
hashimapspool: yes
allowanonymouslogin:no
allowplaintext: yes
allowusermoves: yes
# Use this if sieve-scripts could be in ~user/.sieve.
#sieveusehomedir: yes
# Use saslauthd if you want to use pam for imap.
# But be warned: login with DIGEST-MD5 or CRAM-MD5
# is not possible using pam.
sasl_pwcheck_method:saslauthd
sasl_mech_list: LOGIN PLAIN
autocreatequota: 1500
altnamespace: yes
sasl_auto_transition: no
quotawarn: 90
tls_cipher_list: TLSv1:SSLv3:SSLv2:!NULL:!EXPORT:!DES:!LOW:@STRENGTH
allowallsubscribe: no
unixhierarchysep: yes
lmtp_downcase_rcpt: yes
annotation_db: skiplist
seenstate_db: flat
notifysocket: /var/imap/socket/notify
I have backups of the old packages, but I'man not shure if this will
work anymore.
Any ideas please ? My bloodpressure can't stand this...
When performing an upgrade from earlier than 2.4.3, all messages are
reparsed, and your cache files are re-generated, when a mailbox is first
opened. You may be able to increase you syslogging levels to get a better
view into what's going on:
http://www.cyrusimap.org/docs/cyrus-imapd/2.4.12/install-configure.php
For other possible upgrade issues, see
http://www.cyrusimap.org/docs/cyrus-imapd/2.4.12/install-upgrade.php
--
Dan White
Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: Huge load after update from 2.3.16 to 2.4.12
On 11/23/2011 02:18 PM, Michael Plate wrote:
Hi,
I just updated Cyrus Imapd on our somewhat old mailsever (dual-Xeon, but
8 years now, running gentoo linux) from 2.3.16 to 2.4.12
(options USE=berkdb kerberos pam replication sieve snmp sqlite ssl tcpd
-afs -mysql -nntp -postgres)
There is no murder and I also could drop sqlite, if better.
After the flawless update, restarting generated a huge load. I believe I
never had a load of 50 any time before.
It even gets /var/imap/socket/lmtp] timed out while sending MAIL FROM
on local delivery of mails.
This is cyrus.conf:
START {
# do not delete this entry!
recover cmd=ctl_cyrusdb -r
# this is only necessary if using idled for IMAP IDLE
# idledcmd=idled
}
# UNIX sockets start with a slash and are put into /var/imap/socket
SERVICES {
# add or remove based on preferences
imap cmd=imapd listen=127.0.0.1:imap2 prefork=2
#pop3 cmd=pop3d listen=pop-3 prefork=0
# Don't forget to generate the needed keys for SSL or TLS
# (see doc/html/install-configure.html)
imaps cmd=imapd -s listen=imaps prefork=15
#pop3scmd=pop3d -s listen=pop3s prefork=0
sieve cmd=timsieved listen=sieve prefork=15
# at least one LMTP is required for delivery
lmtp cmd=lmtpd -a -D listen=127.0.0.1:lmtp prefork=1
lmtpunix cmd=lmtpd listen=/var/imap/socket/lmtp prefork=2
# this is only necessary if using notifications
notifycmd=notifyd listen=/var/imap/socket/notify
proto=udp prefork=1
}
EVENTS {
# this is required
checkpointcmd=ctl_cyrusdb -c period=5
# this is only necessary if using duplicate delivery suppression
# delprune cmd=ctl_deliver -E 3 period=1440
delprune cmd=cyr_expire -E 3 -x period=1440
# this is only necessary if caching TLS sessions
tlsprune cmd=tls_prune period=1440 at=0600
# squatter indiziert die Boxen fuer die Suche
squatter cmd=squatter -r -s user period=1440 at=0200
}
and imapd.conf:
proxyservers: cyrus
configdirectory:/var/imap
partition-default: /var/imap/part
sievedir: /var/imap/sieve
sieve_sasl_mech_list: PLAIN
tls_cert_file: /etc/cyrusimapd/server2009.pem
tls_key_file: /etc/cyrusimapd/server2009.pem
tls_ca_file:/etc/cyrusimapd/server2009.pem
duplicate_db: skiplist
mboxlist_db: skiplist
quota_db: quotalegacy
tlscache_db:skiplist
admins: cyrus
hashimapspool: yes
allowanonymouslogin:no
allowplaintext: yes
allowusermoves: yes
# Use this if sieve-scripts could be in ~user/.sieve.
#sieveusehomedir: yes
# Use saslauthd if you want to use pam for imap.
# But be warned: login with DIGEST-MD5 or CRAM-MD5
# is not possible using pam.
sasl_pwcheck_method:saslauthd
sasl_mech_list: LOGIN PLAIN
autocreatequota: 1500
altnamespace: yes
sasl_auto_transition: no
quotawarn: 90
tls_cipher_list: TLSv1:SSLv3:SSLv2:!NULL:!EXPORT:!DES:!LOW:@STRENGTH
allowallsubscribe: no
unixhierarchysep: yes
lmtp_downcase_rcpt: yes
annotation_db: skiplist
seenstate_db: flat
notifysocket: /var/imap/socket/notify
I have backups of the old packages, but I'man not shure if this will
work anymore.
Any ideas please ? My bloodpressure can't stand this...
CU
Michael
How many mailboxes on the system? Took us about 2 hours to come down
from loads over 500 after the upgrade with 10,000 active user accounts.
Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
attachment: boutilpj.vcf
Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: Huge load after update from 2.3.16 to 2.4.12
[...] When performing an upgrade from earlier than 2.4.3, all messages are reparsed, and your cache files are re-generated, when a mailbox is first opened. You may be able to increase you syslogging levels to get a better view into what's going on: [...] Thanx, four weeks ago I made a dump and tried a new installation on a test machine, but I had no problems. CU Michael Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: Huge load after update from 2.3.16 to 2.4.12
Hi, [...] How many mailboxes on the system? Took us about 2 hours to come down from loads over 500 after the upgrade with 10,000 active user accounts. just around 200 ;-) but its our Backup MX an mailman is running too, so a mail to every user produces that load. The machine is a bit...old, just 2 GB of memory and a RAID5. CU Michael Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: rooling replication and many pop3d
Hi, I enabled replication between two servers with version 2.4.10 cyrus. I set the option for the rolling replication, and it works fine but obviously I have a high CPU load. Unfortunately after 10 minutes of running processes pop3d increasing from 50 to over 200, making the server unusable for customers. Can you tell me why this increase is abnormal? Can you use something like 'top' to work out which processes are consuming most of the CPU time? Thanks for the screenshots. This screenshot of top before : http://www.digicolor.net/cyrus/img1.jpg This shows a load average of around 1. That means that, at any given point in time over the past 1 and 5 minutes, 1 process has been waiting in the run queue, ready to go. This therefore not an entirely idle machine. I see you're running a nameserver and a few other things: it looks they've been busy on the CPU but not excessively so. What's more worrying is the 4.5% of CPU time spent waiting. This time is accrued when processes are unable to run due to outstanding IO. and after 20 minutes of rooling replication : http://www.digicolor.net/cyrus/img5.jpg This shows a load average of around 7 and 30% of CPU time spent in iowait. This machine does not seem to be managing well with the IO load of rolling replication. Can you use something like 'vmstat 1' to show us how much I/O there is on the system? This screenshot of top before : http://www.digicolor.net/cyrus/img3.jpg This shows a system that is not reading anything from disk (bi). A small number of blocks are being written out to disk (bo). Each line represents activity for a period of 1 second, as specified by the parameter to 'vmstat'. and after 20 minutes of rooling replication : http://www.digicolor.net/cyrus/img6.jpg This system is writing to disk but it's very choppy. Sometimes it's getting 7,000 blocks out per second and other times it's only 1,000. Depending on your block size, this probably represents only a few MB per second. The last column shows iowait CPU percentage.. and it's rather large. What IO subsystem do you have on this machine? What filesystem are you using? The IO on this machine appears to be struggling significantly. I did a quick test on my laptop. I have a 2.5, 7,200rpm 200GB disk. I ran this in my home directory to cause every file to be read from disk: - $ find -type f | xargs cat /dev/null - 'vmstat 1' gives lines like this: - procs ---memory-- ---swap-- -io -system-- cpu r b swpd free buff cache si sobibo in cs us sy id wa 1 1 1680 30600272 257568800 12376 576 2141 3424 3 6 47 43 1 1 1680 32396272 257673600 29988 0 1882 3632 5 7 47 41 0 1 1680 33868272 257666000 46820 0 2304 4443 4 8 48 39 1 0 1680 33416272 257860000 36716 0 2067 3733 3 7 48 42 0 1 1680 34000272 258194400 50432 0 1164 2983 3 6 50 42 0 1 1680 31876272 258532000 4646464 1223 2964 3 8 49 40 1 1 1680 30288272 258867200 51712 0 1380 3658 3 7 46 43 0 1 1680 29836272 259055200 59776 0 1288 3549 4 7 47 42 0 1 1680 30324272 259294800 58368 0 1287 3568 2 7 49 41 1 1 1680 30308272 259310800 1280018 917 1673 2 2 49 46 - They're an order of magnitude greater than what you're seeing. As you can see, I drop a few bi when I start to do bo but that's because I've only got a single spindle. Please can you run the same test? Can you track the source of all those writes in img3? Please can you tell us more about the type of machine you are trying to run this on? Thanks for the info and screenshots so far. Are most of the pop3d processes sleeping in iowait? Do you use any other servers such as the impad? Yes I have imapd This is screenshot of pstree before : http://www.digicolor.net/cyrus/img2.jpg This is screenshot of pstree after : http://www.digicolor.net/cyrus/img4.jpg Regards, @ndy -- andy...@ashurst.eu.org http://www.ashurst.eu.org/ 0x7EBA75FF Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Not able to delete mails in 2.4.6
Hello I had upgraded my imap server to 2.4.6 Some users keep complaining that delete is not working. Even If I use raw Imap commands and delete the message .. the delete flag is not being set As a fix , If I remove cyrus.* from all the folders in the users directory and reconstruct then the problem gets solved But why does this happen ? How can I debug ? I have noticed that these users have a line NonJunk in the cyrus.header file , Is that any indication of anything ? Thanks Ram Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
