moving mail from one folder to another automatically
Hello, I need some way to move mail older than X days from one folder to another. IOW something like ipurge -fd 30 somefolder except move it elsewhere instead of delete. Is there any way to do this? Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
defer (110): Connection timed out
Hello, in my exim logs I can see a lot of lines like R=mailbox_cyrus T=mailbox_lmtp defer (110): Connection timed out I have set a maximum of 30 concurrent deliveries using lmtp over tcp. What is the limit for lmtpd? How can I increase it? What else could have caused this message? Regards Marten Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Mail forwarding to a dynamic IP address.
I have a Cyrus installation with postfix and open xchange. I get the mails from catchall account with procmail and distribute with the mail server. Now I have a new site office where couples of staff members are going to be relocated. Since we have only an ADSL line on site office if there any way I could forward the mails for these staff members to another Mail Server with dynamic IP address? Any input will be really appreciated. Thanks Rajeev R. Veedu. Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Mail forwarding to a dynamic IP address.
Rajeev R Veedu schrieb: Since we have only an ADSL line on site office if there any way I could forward the mails for these staff members to another Mail Server with dynamic IP address? Use a dyndns service [1] to make the remote ADSL IP resolvable by name and forward directly to that, then forward port 25 in Your firewall setup at the remote site to the mailserver inside the remote intranet (guessing at Your setup here). e.g. have Your distributing mail server's virtual-maps state something like this [EMAIL PROTECTED] [EMAIL PROTECTED] NB: After automatic disconnects of the ADSL line until Your dyndns-client updates the IP You will get timeouts in Your logs but those are temporary and should be retried anyway. To make things look nice You can create an explicit MX record for the dyndns domain. Remember to set the remote mailserver to accept mails addressed to the dyndns domain. Even nicer: Use TLS between the two servers, maybe even with certs. [1] https://www.dyndns.com/services/dns/dyndns/features.html have fun Martin Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Mail forwarding to a dynamic IP address.
Hi, Quoting Rajeev R Veedu [EMAIL PROTECTED]: I have a Cyrus installation with postfix and open xchange. I get the mails from catchall account with procmail and distribute with the mail server. Now I have a new site office where couples of staff members are going to be relocated. Since we have only an ADSL line on site office if there any way I could forward the mails for these staff members to another Mail Server with dynamic IP address? You can set up an DDNS Server (Dynamic Domain Name Service) or use a free DDNS and tell the DDNS the new IP every time you connect via DSL. Or you use fetchmail or imapsync to download the E-Mails from the Main Mailserver Any input will be really appreciated. Thanks Rajeev R. Veedu. M.Menge Tel.: (49) 7071/29-70316 Universitaet Tuebingen Fax.: (49) 7071/29-5912 Zentrum fuer Datenverarbeitung mail: [EMAIL PROTECTED] Waechterstrasse 76 72074 Tuebingen smime.p7s Description: S/MIME krytographische Unterschrift Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: renaming a mailbox fails
On Tuesday 17 October 2006 18:34, Marten Lehmann wrote: Set 'allowusermoves' to '1' in imapd.conf? However, I still get renamemailbox: Operation is not supported on mailbox as soon as I try to rename out of the user/ prefix. Is there Did you restart cyrus after changing the config? wt -- Warren Turkal, Research Associate III/Systems Administrator Colorado State University, Dept. of Atmospheric Science Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
troubleshooting an unified config
Hi all,
I have an almost running unified-configuration murder installation with 3
machines:
* backbox: backend
* unibox: fronend/backend in unified mode
* mupbox: mupdate master
It all looks like it's fine. when accessing unibox with an imap client, it
serves the mailbox if it is at unibox or proxies it if it's at backbox. So,
the unified config looks like it works, in terms of accessing the mailboxes.
But we are trying to be able to move mailboxes from backbox to unibox and
from unibox to backbox as well. Right now, if I issue a xfer command to
transfer a mailbox from unibox to backbox, it works flawlessly. But not
the other way around. If we try to move from backbox to unibox, it fails
with an error like:
backbox xfer [EMAIL PROTECTED] unibox
xfermailbox: The remote Server(s) denied the operation
Sniffing the connection, there is a connect to the mupdate master as follows:
* AUTH CRAM-MD5 DIGEST-MD5 LOGIN PLAIN
* PARTIAL-UPDATE
* OK MUPDATE mupbox Cyrus Murder v2.3.7 (master)
A01 AUTHENTICATE PLAIN {32+}
PASSWORD=
A01 OK Authenticated
X0 DEACTIVATE {23+}
domain.com!user.andy {13+}
backbox!default
X0 OK done
X1 ACTIVATE {23+}
domain.com!user.andy {13+}
backbox!default {31+}
[EMAIL PROTECTED]
X1 OK done
and then, there's this connection to unibox' imap port:
* OK [CAPABILITY IMAP4 IMAP4rev1 LITERAL+ ID MUPDATE=mupdate://mupbox/
AUTH=DIGEST-MD5 AUTH=CRAM-MD5 SASL-IR] unibox Cyrus IMAP4 (Murder) v2.3.7
server ready
C01 CAPABILITY
* CAPABILITY IMAP4 IMAP4rev1 LITERAL+ ID MUPDATE=mupdate://mupbox/
AUTH=DIGEST-MD5 AUTH=CRAM-MD5 SASL-IR ACL RIGHTS=kxte QUOTA NAMESPACE UIDPLUS
NO_ATOMIC_RENAME UNSELECT CHILDREN MULTIAPPEND BINARY SORT SORT=MODSEQ
THREAD=ORDEREDSUBJECT THREAD=REFERENCES ANNOTATEMORE CATENATE CONDSTORE IDLE
URLAUTH
C01 OK Completed
A01 AUTHENTICATE PLAIN PASSWORD==
A01 OK [CAPABILITY IMAP4 IMAP4rev1 LITERAL+ ID MUPDATE=mupdate://mupbox/
LOGINDISABLED ACL RIGHTS=kxte QUOTA NAMESPACE UIDPLUS NO_ATOMIC_RENAME
UNSELECT CHILDREN MULTIAPPEND BINARY SORT SORT=MODSEQ THREAD=ORDEREDSUBJECT
THREAD=REFERENCES ANNOTATEMORE CATENATE CONDSTORE IDLE URLAUTH] Success (no
protection)
LC1 LOCALCREATE {23+}
[EMAIL PROTECTED]
LC1 NO Mailbox already exists
Q01 LOGOUT
* BYE LOGOUT received
Q01 OK Completed
The thing is that it is failing at the LOCALCREATE command because the mailbox
looks like it *exist* to cyrus, although it is not at that server
(unibox).
Having a look at the boxes, I've noticed that the mailboxes.db files contains
info as follows:
* mupbox[mupdate master] - contains all the mailboxes from backbox and unibox
* unibox(unified) - contains all the mailboxes from backbox and unibox
* backbox - contains only the mailboxes at backbox.
I guess this is the right way to do it because unibox is running an unified
config and it is running a slave mupdate process for its frontend section and
needs to know all this data, right? So then, why is it failing to move the
mailbox?
I think everything is right in terms of permissions. backbox logs in to
unibox as an admin, so there should be no problem with permissions at all,
i guess.
Any ideas? I am running out of them at this point.
Configuration files follows:
backbox -- cyrus.conf
+
START {
recover cmd=ctl_cyrusdb -r
mupdatepush cmd=ctl_mboxlist -m
}
# UNIX sockets start with a slash and are put into /var/imap/socket
SERVICES {
# add or remove based on preferences
#mupdate cmd=/opt/cyrus-imapd-2.3.7/bin/mupdate listen=3905 prefork=1
imap cmd=imapd listen=imap prefork=0
imaps cmd=imapd -s listen=imaps prefork=0
pop3 cmd=pop3d listen=pop3 prefork=0
pop3s cmd=pop3d -s listen=pop3s prefork=0
sieve cmd=timsieved listen=sieve prefork=0
lmtp cmd=lmtpd listen=lmtp prefork=0
}
EVENTS {
checkpointcmd=ctl_cyrusdb -c period=30
}
backbox -- imapd.conf
+
configdirectory: /var/imap
partition-default: /var/spool/imap
sasl_pwcheck_method: auxprop
sasl_mech_list: plain login CRAM-MD5 DIGEST-MD5
sasldb_path: /etc/sasldb2
sieve_admins: cyrus, golan
sievedir: /var/spool/sieve
virtdomains: yes
altnamespace: yes
allowallsubscribe: yes
allowusermoves: 1
admins: golan murder [EMAIL PROTECTED] [EMAIL PROTECTED]
# How to get to mupdate master server
mupdate_username: backend1
mupdate_password: Password
mupdate_server: mupbox
mupdate_port: 2004
mupdate_authname: backend1
mupdate_config: standard
unibox_mechs: PLAIN
proxy_authname: murder
unibox_password: Password
mupbox_mechs: PLAIN
mupbox_password: Password
force_sasl_client_mech: PLAIN
proxyservers: murder
allowpaintext: 1
unibox -- cyrus
+
START {
mboxlist cmd=ctl_cyrusdb -r
mupdatepush cmd=ctl_mboxlist -m
}
SERVICES {
mupdate cmd=/home/golan/cyrus-imapd-2.3.7/bin/mupdate
Re: delayed expunge and subfolders
I've understood that this is supposed to mean that if delayed expuge is used and folder annotations does not specify anything else then messages are supposed to be deleted from all folders after two days. More information. I tried to run the cyr_expire from shell and it segfaulted: $ cyr_expire -E 3 -X 2 -v expunging deleted messages in mailboxes older than 2 days Segmentation Fault But if define expunge delay to zero days then everything works ok and no segmentation faults: $ cyr_expire -E 3 -X 0 -v expunging deleted messages in mailboxes older than 0 days expunged ... $ -- Leena Heino University of Tampere / Computer Centre ( liinu at uta.fi ) ( http://www.uta.fi/laitokset/tkk ) Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: renaming a mailbox fails
Hello, Set 'allowusermoves' to '1' in imapd.conf? However, I still get renamemailbox: Operation is not supported on mailbox as soon as I try to rename out of the user/ prefix. Is there Did you restart cyrus after changing the config? yes. Renaming users works, but only within user/, e.g. from user/[EMAIL PROTECTED] to user/[EMAIL PROTECTED] But what doesn't work is to rename it completely like from user/[EMAIL PROTECTED] to backup/[EMAIL PROTECTED] Regards Marten Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
very slow syncing, any ideas?
Hello, I'm about to migrate several thousand mailboxes from Maildir to Cyrus using the tool imapsync. It does its job very well and when I tested the migration on a small development server it was very fast. But now on the production system the synchronisation is very slow with a maximum of one message per second (and we have gigabytes of messages in the storage, partically 10,000 messages per mailbox!). The general load of the system isn't very high, maybe a load average of 30. I disabled the duplicate message suppression. The mailboxes.db is about 8 megabytes big with approx. 13,000 mailboxes and 4 default folders each (Drafts, Junk, Sent, Trash). I have the following entries in my configuration which should provide a better hierarchie and balance of directories than if they were all in one main directory: altnamespace: true hashimapspool: true unixhierarchysep: true virtdomains: userid I also tried to move the old Maildirs to a different server, so that getting messages from the old mailbox and putting it to the new mailbox through IMAP doesn't come up with reads and writes on the same server. But the performance benefit was minimal. But in the end, syncing is still really slow. It would take weeks to sync all mailboxes that way. How else could we move them to the new storage if doing it through IMAP is too slow? On the other hand we would like to keep all flags so I guess syncing it with IMAP is the only choice? What could be the reason to be that slow? Is it the big mailboxes.db? Regards Marten Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Database usage/backup/restores
I have some questions for those of you gurus who truly know the cyrus system. I have a XSAN Mac cluster and we are running mail on it. Cyrus 2.2.12 if I remember correctly. Anyways, the questions are these: (1) To back up the cyrus database, we shut down the mail server and back up the database and then restart the mail server. To restore the database, we save the seen files and we copy over a backup copy of the database. So the question is, is this the right way to backup and restore the database? Is putting an old database in place going to break anything? (2) What is in the mailboxes.db? Is it the list of mailboxes for everyone on the server? (3) After restoring should we always kick off a reconstruct? Thank you David -- David Korpiewski Phone: 413-545-4319 Software Specialist IFax: 413-577-2285 Department of Computer Science ICQ: 7565766 University of Massachusetts Amherst Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: very slow syncing, any ideas?
On Thu, 2006-10-19 at 20:10 +0200, Marten Lehmann wrote: Hello, I'm about to migrate several thousand mailboxes from Maildir to Cyrus using the tool imapsync. It does its job very well and when I tested the migration on a small development server it was very fast. But now on the production system the synchronisation is very slow with a maximum of one message per second (and we have gigabytes of messages in the storage, partically 10,000 messages per mailbox!). The general load of the system isn't very high, maybe a load average of 30. I disabled the duplicate message suppression. The mailboxes.db is about 8 megabytes big with approx. 13,000 mailboxes and 4 default folders each (Drafts, Junk, Sent, Trash). What is I/O load on the new server? on the old server? Was your test set representative of your production system? You could be running into any number of bottlenecks that you've not described: 1) disk I/O 2) network I/O 3) message size 4) file system format etc. etc. Find out what those are, then find out which you're hitting, then come back. I also tried to move the old Maildirs to a different server, so that getting messages from the old mailbox and putting it to the new mailbox through IMAP doesn't come up with reads and writes on the same server. But the performance benefit was minimal. So both the maildir server and the Cyrus server are on the same box? Do their spools share the disks? Controllers? How many users? How many connections? How much mail? You need to better understand the limits of your system. Yes your development system was fast, but it sounds like you didn't try to replicate the load of doing everything at once on it, so your expectations have been set artifically. Z Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: very slow syncing, any ideas?
--On October 19, 2006 8:10:13 PM +0200 Marten Lehmann [EMAIL PROTECTED] wrote: Hello, I'm about to migrate several thousand mailboxes from Maildir to Cyrus using the tool imapsync. It does its job very well and when I tested the migration on a small development server it was very fast. But now on the production system the synchronisation is very slow with a maximum of one message per second (and we have gigabytes of messages in the storage, partically 10,000 messages per mailbox!). The general load of the system isn't very high, maybe a load average of 30. I disabled the duplicate message suppression. The mailboxes.db is about 8 megabytes big with approx. 13,000 mailboxes and 4 default folders each (Drafts, Junk, Sent, Trash). Uhm... LA of 30 is very high. What OS? I assume Linux, vmstat 5 will tell you where you're hitting the wall, but unless you've got an 8 CPU machine LA 30 is rather quite high. Linux LA is a measurement of processes blocked on I/O, processes running and processes waiting to run on a CPU. -- Genius might be described as a supreme capacity for getting its possessors into trouble of all kinds. -- Samuel Butler Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: very slow syncing, any ideas?
Hi, What is I/O load on the new server? on the old server? Was your test set representative of your production system? the test system has much older hardware than the pretty new production server (Xeon 3.4 with RAID5 and SCSI HDs). Old and new server are the same from the hardware side, we have just changed the software. You could be running into any number of bottlenecks that you've not described: 1) disk I/O 2) network I/O Even syncing from an IMAP server in the internet to my local server was faster. 3) message size 4) file system format I guess its none of them. uptime shows 21:04:59 up 1 day, 5:46, 7 users, load average: 29.10, 43.29, 41.94 I can easily copy large amounts of files within the system, so it is definetely not a problem of the filesystem. Syncing is always slow, of course large mails take a bit longer, but that isn't the bottleneck. The question is: What is Cyrus doing in the background when doing an APPEND in IMAP (I guess thats the way new messages are added)? When I can copy 20 files within a second in the filesystem, but only 1 in three seconds through IMAP, whats wrong then? So both the maildir server and the Cyrus server are on the same box? We tried both: syncing from the same box or from one to another. It was slow in both cases. Do their spools share the disks? Controllers? How many users? How many connections? How much mail? At this moment it is evening here, so there aren't many deliveries (about 90 per minute). I guess that the bottleneck is somewhere in Cyrus. We are using cyrus-imapd-2.2.12 from RHEL4. Is there a problem with berkeley db with large mailboxes.db? What is Cyrus doing during an IMAP APPEND? Is it always looking for the folder in mailboxes.db? Is it sync'ing the harddisk after each APPEND? You need to better understand the limits of your system. Yes your development system was fast, but it sounds like you didn't try to replicate the load of doing everything at once on it, so your expectations have been set artifically. Our old way to deliver mails sort of broke down, so we urgently needed a replacement. We worked on Cyrus for some weeks so we knew enough to build the right configuration, but we didn't had the time to do a stresstest. Regards Marten Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
migrating mailboxes by copying files?
Hello, since imapsync sessions take to long for our several thousand mailbox setup (although it's still running in the background while I'm writing this) I'm asking myself, if there isn't another way to migrate the messages. Maildir does store each message in a separate file and same does cyrus. The accounts in cyrus are all created. Couldn't I just copy all messages from a Maildir to the according cyrus-folder and run a rebuild-tool for the mailbox after I moved and renamed the message-files? Which scheme does the numbering of the files follow in cyrus? Is it always increasing (1., 2., 3., and so on)? If there is such a tool, what is the name and how can I use it? I guess cyrus is rebuilding the cyrus.* files and updating the quota-usage? Regards Marten Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: very slow syncing, any ideas?
Hello, Uhm... LA of 30 is very high. What OS? I assume Linux, vmstat 5 will tell you where you're hitting the wall, but unless you've got an 8 CPU machine LA 30 is rather quite high. Linux LA is a measurement of processes blocked on I/O, processes running and processes waiting to run on a CPU. yes, Linux (2.6.9, RHEL4): procs ---memory-- ---swap-- -io --system-- cpu r b swpd free buff cache si sobibo incs us sy id wa 15 1 34896 800696 11788 4399685 595 670 2763 34 6 14 46 32 2 34896 792796 11332 4627200 1138 1124 1851 1427 83 8 1 9 11 3 34852 794696 9264 4392470 1194 1087 1896 1546 81 8 1 9 33 1 34852 791752 8660 4478800 902 1059 1933 1672 81 9 1 9 13 2 34852 792652 7004 4150400 1220 1037 1863 1476 81 8 2 9 30 2 34852 795584 7572 4093600 423 1222 1876 1484 82 7 2 9 16 3 34852 796836 8816 4411200 817 1268 1927 1613 83 8 1 8 12 1 34852 794000 9888 4434000 249 1135 1821 1458 82 8 3 8 9 2 34852 793328 9876 4617200 1098 976 1816 1091 86 7 2 6 7 5 34852 790188 10332 4831600 1100 1274 1785 959 85 7 2 6 26 1 34852 789504 7656 5255200 1837 1087 1991 1444 81 8 2 9 20 0 34852 794252 6900 4706800 440 1246 1932 1461 83 8 2 7 Does this tell you anything? I have two imapsync processes which cause some load. Regards Marten Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: very slow syncing, any ideas?
On Thu, 2006-10-19 at 21:59 +0200, Marten Lehmann wrote: Hi, What is I/O load on the new server? on the old server? Was your test set representative of your production system? the test system has much older hardware than the pretty new production server (Xeon 3.4 with RAID5 and SCSI HDs). Old and new server are the same from the hardware side, we have just changed the software. So your old mailserver (maildir) and your new cyrus server are on two different machines? Was the test server also running RAID5 for the spool? Even syncing from an IMAP server in the internet to my local server was faster. To which local server, your Cyrus server or your Maildir server? I guess its none of them. uptime shows 21:04:59 up 1 day, 5:46, 7 users, load average: 29.10, 43.29, 41.94 What is the load a result of? Are you doing av checks on the new server? You've still not answered the fundmental question, what is causing this load? I can easily copy large amounts of files within the system, so it is definetely not a problem of the filesystem. Syncing is always slow, of course large mails take a bit longer, but that isn't the bottleneck. Large being *greater* than the fs cache? What fs are you using? Is the same as the one on your old Maildir system? The question is: What is Cyrus doing in the background when doing an APPEND in IMAP (I guess thats the way new messages are added)? When I can copy 20 files within a second in the filesystem, but only 1 in three seconds through IMAP, whats wrong then? You've not told us whether or not the load is a result of Cyrus processes. Is it? Which Cyrus process? Authenication? IMAP? what? We tried both: syncing from the same box or from one to another. It was slow in both cases. I don't understand what you mean, do you mean you tried running imapsync on the old mailserver, then tried running off the new mailserver? At this moment it is evening here, so there aren't many deliveries (about 90 per minute). Argh. How much mail do you need to move, # messages and GB? I guess that the bottleneck is somewhere in Cyrus. We are using cyrus-imapd-2.2.12 from RHEL4. Is there a problem with berkeley db with large mailboxes.db? What is Cyrus doing during an IMAP APPEND? Is it always looking for the folder in mailboxes.db? Is it sync'ing the harddisk after each APPEND? You need to pinpoint the bottleneck. What does vmstat look like while trying to sync? What filesystem are you running and with what options is it mounted? Are those options the same/different than your test/old servers? I have 60GB of email in 860,000 messages and my mailboxes.db is 323KB. Our old way to deliver mails sort of broke down, so we urgently needed a replacement. We worked on Cyrus for some weeks so we knew enough to build the right configuration, but we didn't had the time to do a stresstest. I can understand that, but you're shooting in the dark here, and you've given ZERO information that might help anyone on the list help you. Start by finding the bottleneck, don't assume that it's Cyrus. Especially since you didn't see this behavior on your old slower test system, I can't understand why you'd immediately assume it was Cyrus. Don't arbitrarily rule out possible causes for your problem (as it looks like you have). Sometimes it is something as stupid as a dns issue. Or a bad hardware. Or a bad nic. Or a ... Z Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
outlook as a client
I have several accounts whose mailboxes on a cyrus-imap server are connected by Outlook. However, they are able to download mail from the server, but they cannot create emails from the outlook client on a remote machine and be read and connected to the cyrus-imap server for processing. A friend says he has tested sendmail and the cyrus installation and all seems correct. How should I proceed? Thanks Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: very slow syncing, any ideas?
--On October 19, 2006 11:25:07 PM +0200 Marten Lehmann [EMAIL PROTECTED] wrote: Hello, Uhm... LA of 30 is very high. What OS? I assume Linux, vmstat 5 will tell you where you're hitting the wall, but unless you've got an 8 CPU machine LA 30 is rather quite high. Linux LA is a measurement of processes blocked on I/O, processes running and processes waiting to run on a CPU. yes, Linux (2.6.9, RHEL4): Looking at taht i'd say you're VERY badly CPU bound. a simple dd/cp doesn't do anything to the mail but IMAP ops will require some CPU workCyrus also will probably be forcing syncs but your I/O load doesn't look that high (my mfe's run more I/O and they're not storing any mail, just logs and temporary files for virus/spam scanning heh, and they only have a little IDE HDD each) the numebrs in the procs-r column indicate you've got a lot of processes vying for CPU time. the b column indicates you've got a little bit of blocking going on. Processes in the 'D' state may show up in ps or top output (doubtfully top unless you change the default sort). I'd guess you're being CPU bound and cyrus is probably running (usually does by default) at a bit lower priority than your interactive cp operations. 2.6 kernel scheduler does a lot of weird stuff if it determines you're interactive, and then tends to give you priority over daemons. it's very non-deterministic, but I don't think you're running into this. quite likely some inefficiencies in the cyrus code (or intentional backoffs?) but mostly just being CPU bound. I know our MBEs can generally scp or rsync quite a bit faster than they can perform a XFER/RENAME command that moves between mail store servers (IE MURDER host to MURDER host). Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: very slow syncing, any ideas?
Hello, Looking at taht i'd say you're VERY badly CPU bound. a simple dd/cp doesn't do anything to the mail but IMAP ops will require some CPU workCyrus also will probably be forcing syncs but your I/O load doesn't look that high (my mfe's run more I/O and they're not storing any mail, just logs and temporary files for virus/spam scanning heh, and they only have a little IDE HDD each) I found the option berkeley_cachesize which is way too low by default. Transactions just reading or writing db-entries (i.e. changes of flags) are much faster now. Indead, the imapsync processes are causing much load: PID USER PR NI VIRT RES SHR S %CPU %MEMTIME+ COMMAND 20461 root 25 0 79988 36m 2488 R 63 3.6 13:24.10 imapsync 20925 root 25 0 14908 8196 2664 R 57 0.8 0:31.49 imapsync But what does them make so cpu-expensive? I'd guess you're being CPU bound What would you do if you would need to migrate from Maildir to cyrus? It is not important for me to keep the flags. I would be happy if I could move all message files to the appropriate cyrus folder. But will cyrus detect them automatically? Or is there any rebuild-command required? Regards Marten Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
CREATE INBOX/attention: NO Invalid mailbox name
Hello, I have the combination of the following in my imapd.conf: altnamespace: true unixhierarchysep: true That way folders can be created on the same level as INBOX. But I didn't understood this the way, that one cannot have folders below INBOX then. 173 CREATE INBOX/attention 173 NO Invalid mailbox name Thats a bit tricky because we are migrating from dovecot, and dovecot allowed INBOX.attention as well as attention. Is there any option to still allow this? Regards Marten Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Stupid question
I am attempting to learn all I can about Cyrus for a work project but I am unable to find documentation about what exactly the /var/imap/db directory and the enclosed files are for. Any assistance or direction on where to find this information is greatly appreciated. __ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
RE: very slow syncing, any ideas?
Marten, What would you do if you would need to migrate from Maildir to cyrus? It is not important for me to keep the flags. I would be happy if I could move all message files to the appropriate cyrus folder. But will cyrus detect them automatically? Or is there any rebuild-command required? I had the same problem when I did a migration. I have a fast mbox to cyrus migration script which you could convert to do maildir easily enough if you have any shell programming knowledge (and if not, time to learn!). You will lose all information though - the messages will be flagged as new. Instead of using an IMAP sync, you feed the email message in as a brand new message to the deliver program. It's fast and effective. Regards, Sarah Walters Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: very slow syncing, any ideas?
On Thu, 19 Oct 2006, Michael Loftis wrote: --On October 19, 2006 11:25:07 PM +0200 Marten Lehmann [EMAIL PROTECTED] wrote: Hello, Uhm... LA of 30 is very high. What OS? I assume Linux, vmstat 5 will tell you where you're hitting the wall, but unless you've got an 8 CPU machine LA 30 is rather quite high. Linux LA is a measurement of processes blocked on I/O, processes running and processes waiting to run on a CPU. yes, Linux (2.6.9, RHEL4): Looking at taht i'd say you're VERY badly CPU bound. a simple dd/cp doesn't do anything to the mail but IMAP ops will require some CPU workCyrus also will probably be forcing syncs but your I/O load doesn't look that high (my mfe's run more I/O and they're not storing any mail, just logs and temporary files for virus/spam scanning heh, and they only have a little IDE HDD each) the numebrs in the procs-r column indicate you've got a lot of processes vying for CPU time. the b column indicates you've got a little bit of blocking going on. Processes in the 'D' state may show up in ps or top output (doubtfully top unless you change the default sort). I'd guess you're being CPU bound and cyrus is probably running (usually does by default) at a bit lower priority than your interactive cp operations. 2.6 kernel scheduler does a lot of weird stuff if it determines you're interactive, and then tends to give you priority over daemons. it's very non-deterministic, but I don't think you're running into this. quite likely some inefficiencies in the cyrus code (or intentional backoffs?) but mostly just being CPU bound. I know our MBEs can generally scp or rsync quite a bit faster than they can perform a XFER/RENAME command that moves between mail store servers (IE MURDER host to MURDER host). No clue if it matters in this imapsync case, but if you are using the 2.6 kernel, you should set the I/O scheduler to deadline rather than the default anticipatory. You can set this in the boot loader, such as the following for lilo.conf: append=elevator=deadline or you can change the scheduler on the fly by: echo deadline /sys/block/sdb/queue/scheduler Replace sdb with the block device you store your cyrus mail spool on. Andy Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: CREATE INBOX/attention: NO Invalid mailbox name
On Fri, 20 Oct 2006, Marten Lehmann wrote: Hello, I have the combination of the following in my imapd.conf: altnamespace: true unixhierarchysep: true That way folders can be created on the same level as INBOX. But I didn't understood this the way, that one cannot have folders below INBOX then. 173 CREATE INBOX/attention 173 NO Invalid mailbox name Thats a bit tricky because we are migrating from dovecot, and dovecot allowed INBOX.attention as well as attention. Is there any option to still allow this? I think it's one way or the other, not both. With altnamespace: 0, all folders are under INBOX. With altnamespace: 1, no folders are under INBOX. Andy Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: CREATE INBOX/attention: NO Invalid mailbox name
Hello, I think it's one way or the other, not both. With altnamespace: 0, all folders are under INBOX. With altnamespace: 1, no folders are under INBOX. ok, then I have to rename them manually. Regards Marten Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: CREATE INBOX/attention: NO Invalid mailbox name
Marten Lehmann wrote: Hello, I think it's one way or the other, not both. With altnamespace: 0, all folders are under INBOX. With altnamespace: 1, no folders are under INBOX. ok, then I have to rename them manually. Rename what manually? Seriously, are you considering the problem correctly? -- Jo Rhett Network/Software Engineer Net Consonance Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
