Re: [Dovecot] Creating an IMAP repo for ~100 users need some advice
On 3/17/2012 4:24 PM, Kaya Saman wrote: Long story but we don't have any control over our mail server which is handled by the parent company abroad and is on MS Exchange. To use an IMAP storage solution is the only way to get rid of pesky MS .pst files which have been causing everyone grief and havoc. It's been many years since I used, or supported, MS Outlook. That said, for the 10+ years I did support it, ~1996-2006, the corporate version of Outlook, not to be confused with Outlook Express, did not store any mail in local .PST files unless specifically configured to do so. By default it keeps all mail in the user account in the Exchange server store. Thus I would assume these Outlook clients have been manually configured to use .PST files to keep copies of mail locally, for faster access and to keep inefficient MS Exchange (MAPI) traffic off the WAN link? Is your problem with the PST files themselves, or merely the fact they're stored on the local PC, probably in the users' roaming profiles, thus creating the problem of large data movement during logon/off? If the problem isn't with the .PST format for storing the emails, why not simply setup a local Samba server and configure the Outlook clients to store users' PSTs on Samba shares? Better yet, if you already have a file server for home directories, simply use a folder redirection policy to put the PST files in folders in users' home directories. This is an extremely common practice in the MS world because all Microsoft Windows apps store everything in the user profile directory by default, which again, causes big problems with roaming profiles, which many/most enterprises use. -- Stan
Re: [Dovecot] Creating an IMAP repo for ~100 users need some advice
On 03/18/2012 09:16 AM, Stan Hoeppner wrote: On 3/17/2012 4:24 PM, Kaya Saman wrote: Long story but we don't have any control over our mail server which is handled by the parent company abroad and is on MS Exchange. To use an IMAP storage solution is the only way to get rid of pesky MS .pst files which have been causing everyone grief and havoc. It's been many years since I used, or supported, MS Outlook. That said, for the 10+ years I did support it, ~1996-2006, the corporate version of Outlook, not to be confused with Outlook Express, did not store any mail in local .PST files unless specifically configured to do so. By default it keeps all mail in the user account in the Exchange server store. Thus I would assume these Outlook clients have been manually configured to use .PST files to keep copies of mail locally, for faster access and to keep inefficient MS Exchange (MAPI) traffic off the WAN link? Is your problem with the PST files themselves, or merely the fact they're stored on the local PC, probably in the users' roaming profiles, thus creating the problem of large data movement during logon/off? If the problem isn't with the .PST format for storing the emails, why not simply setup a local Samba server and configure the Outlook clients to store users' PSTs on Samba shares? Better yet, if you already have a file server for home directories, simply use a folder redirection policy to put the PST files in folders in users' home directories. This is an extremely common practice in the MS world because all Microsoft Windows apps store everything in the user profile directory by default, which again, causes big problems with roaming profiles, which many/most enterprises use. Thanks Stan for this, currently our users have about 270MB space located on the Exchange server which we have no control over. Therefor users are currently manually backing up their information to .pst. Since all contact/calendar/other information is already stored on the server the IMAP solution is a better one. It also means that a more UNIX/Linux centric approach is being ask for/tolerated which is where I come in being the only full-bread UNIX engineer on site. I couldn't comment on the MS side of things as I have never really used MS stuff before but my plan using ZFS and FreeBSD should be the best. Again it's going to be **scalable** storage which is perfect! -Also easy to maintain; otherwise I don't think there would be anyone left who will be able to admin the SMB/local directory method (I can't as I don't understand MS) - additionally there isn't much space available and bandwidth either meaning we would purchase a dedicated server or build a dedicated server for this (well I would :-) ). Regards, Kaya
Re: [Dovecot] Creating an IMAP repo for ~100 users need some advice
On 18.3.2012, at 0.28, Sven Hartge wrote: mbox has big problems with concurrent writes, the bigger the mbox is, the more problems you get. This is mainly caused by the meta-data of a message (meaning flags, status, etc.) which is stored inside the mbox file itself. Flagging a message as read or replied causes the whole mbox file to be rewritten. Dovecot moves only minimal amount of data within mbox. A flag change writes only a few bytes to mbox file, it doesn't rewrite it the whole file. Only time when the entire mbox file is rewritten is when you expunge the first message.
Re: [Dovecot] Creating an IMAP repo for ~100 users need some advice
Timo Sirainen t...@iki.fi wrote: On 18.3.2012, at 0.28, Sven Hartge wrote: mbox has big problems with concurrent writes, the bigger the mbox is, the more problems you get. This is mainly caused by the meta-data of a message (meaning flags, status, etc.) which is stored inside the mbox file itself. Flagging a message as read or replied causes the whole mbox file to be rewritten. Dovecot moves only minimal amount of data within mbox. A flag change writes only a few bytes to mbox file, it doesn't rewrite it the whole file. Only time when the entire mbox file is rewritten is when you expunge the first message. OK, then my memory was clouded by my exposure to the brokenness of uw-imapd and uw-popd, who rewrite an mbox file very often, resulting in abysmal performance. Grüße, Sven. -- Sigmentation fault. Core dumped.
Re: [Dovecot] Creating an IMAP repo for ~100 users need some advice
On 2012-03-18 5:16 AM, Stan Hoeppner s...@hardwarefreak.com wrote: Is your problem with the PST files themselves, or merely the fact they're stored on the local PC, probably in the users' roaming profiles, thus creating the problem of large data movement during logon/off? If so, using redirected folders (if you're not using them, you should be) would alleviate this problem nicely, even in a large environment. -- Best regards, Charles
Re: [Dovecot] Creating an IMAP repo for ~100 users need some advice
On 2012-03-18 11:15 AM, Timo Sirainen t...@iki.fi wrote: Only time when the entire mbox file is rewritten is when you expunge the first message. Hmmm... wonder if there would be a way to add some kind of 'dummy' first message that dovecot would simply ignore (not show to the user), that would prevent that bevaior? Although I have no desire to use mbox (planning on using mdbox), so it isn't important to me... ;) -- Best regards, Charles
Re: [Dovecot] Creating an IMAP repo for ~100 users need some advice
On Sun, Mar 18, 2012 at 11:36:25AM -0400, Charles Marcus wrote: Hmmm... wonder if there would be a way to add some kind of 'dummy' first message that dovecot would simply ignore (not show to the user), that would prevent that bevaior? That's what uw-imap does. It creates a message with the subject DON'T DELETE THIS MESSAGE -- FOLDER INTERNAL DATA, which is very annoying if your users has direct access to the mbox's... http://www.washington.edu/imap/IMAP-FAQs/index.html#6.14 -jf
Re: [Dovecot] Creating an IMAP repo for ~100 users need some advice
On 18.3.2012, at 17.36, Charles Marcus wrote: On 2012-03-18 11:15 AM, Timo Sirainen t...@iki.fi wrote: Only time when the entire mbox file is rewritten is when you expunge the first message. Hmmm... wonder if there would be a way to add some kind of 'dummy' first message that dovecot would simply ignore (not show to the user), that would prevent that bevaior? Dovecot has such a dummy first message exactly like UW-IMAP. But what I meant is that if you expunge the first message and you want to free the space used by it, there's no other choice than to rewrite all of the messages after it. (And of course the first message isn't special in any way, there's just about as much of rewriting if you delete the 2nd or 3rd or other messages from the beginning of the mbox.)
[Dovecot] mdbox and filesystem quota
http://wiki2.dovecot.org/MailboxFormat/dbox Expunging a message only decreases the message's refcount. The space is later freed in purge step. This is typically done in a nightly cronjob when there's less disk I/O activity. What happens if there is filesystem hard quota that is exceeded? Will dovecot allow to delete mails to free space without a need to wait for cronjob to do the purge? -- Arkadiusz MiśkiewiczPLD/Linux Team arekm / maven.plhttp://ftp.pld-linux.org/
Re: [Dovecot] mdbox and filesystem quota
On 18.3.2012, at 23.00, Arkadiusz Miśkiewicz wrote: http://wiki2.dovecot.org/MailboxFormat/dbox Expunging a message only decreases the message's refcount. The space is later freed in purge step. This is typically done in a nightly cronjob when there's less disk I/O activity. What happens if there is filesystem hard quota that is exceeded? Will dovecot allow to delete mails to free space without a need to wait for cronjob to do the purge? No. Also the purging itself won't work, because it needs to write new data first before it can delete old data. Don't run out of disk space!
Re: [Dovecot] mdbox and filesystem quota
On Sunday 18 of March 2012, Timo Sirainen wrote: On 18.3.2012, at 23.00, Arkadiusz Miśkiewicz wrote: http://wiki2.dovecot.org/MailboxFormat/dbox Expunging a message only decreases the message's refcount. The space is later freed in purge step. This is typically done in a nightly cronjob when there's less disk I/O activity. What happens if there is filesystem hard quota that is exceeded? Will dovecot allow to delete mails to free space without a need to wait for cronjob to do the purge? No. Also the purging itself won't work, because it needs to write new data first before it can delete old data. Don't run out of disk space! Can dovecot treat soft quota like hard quota for user then? Or better enforce quota based on filesystem quot information. With xfs I can set quota but turn enforcement off. All fs quota counters work but no enforcement is being made by xfs itself. -- Arkadiusz MiśkiewiczPLD/Linux Team arekm / maven.plhttp://ftp.pld-linux.org/
Re: [Dovecot] mdbox and filesystem quota
On 18.3.2012, at 23.52, Arkadiusz Miśkiewicz wrote: Expunging a message only decreases the message's refcount. The space is later freed in purge step. This is typically done in a nightly cronjob when there's less disk I/O activity. What happens if there is filesystem hard quota that is exceeded? Will dovecot allow to delete mails to free space without a need to wait for cronjob to do the purge? No. Also the purging itself won't work, because it needs to write new data first before it can delete old data. Don't run out of disk space! Can dovecot treat soft quota like hard quota for user then? Or better enforce quota based on filesystem quot information. With xfs I can set quota but turn enforcement off. All fs quota counters work but no enforcement is being made by xfs itself. Yes, Dovecot does the quota enforcement itself. I'm not entirely sure if it uses soft or hard quota though.
