Re: [Dovecot] Max IMAP fodlers
On Dec 8, 2009, at 12:41 AM, Steffen Kaiser wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Mon, 7 Dec 2009, Jeffrey Nikoletich wrote: Also is there is hard limit on the amount of folders dovecot can handle? I have a user who has over 3000 folder (including subfolders) that is having issues with delays in loading, moving, and deleting messages. I cannot believe that ext3 chokes on just 3000 directory entries in such situation. I would think that one would notice the same with 20,000 messages per mailbox. Yeah... like i said... it is really weird How about really use another mail client for testing, as Marcus suggested, e.g. some brain-dead MUA, that does not try to be smart, for example Alpine or Mutt :) I have tried... macmail. outlook, thunderbird, entourage... I havent tried mutt but I will now. Also, try to log the session, maybe the user watches all 3000 folders for changes. So they all will be selected in a (short?) period of time again and again. Maybe a firewall in between is casuing a throttle, because of too many connections or something? I have an strace running on the process now just as Timo suggested. Regards, - -- Steffen Kaiser -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.6 (GNU/Linux) iQEVAwUBSx4RKnWSIuGy1ktrAQJF4Af/diS3UqhWWiJONM8SntptVn3UH0rLvT+p NoSYfS2/0aVTstuU6vGhntGPsGCPXmSRVuGbW+ERS4U/6C7V9JQuo8Fdf2aYjGKe 2U/LxaodeKBc/c7cZdf4Uthu0z8NScWZJU8nG23EK31e9nlcte0D/eHF132i2zPZ mw+ZxFdgveTA+V4hnRVbUO49mYu3EmePkdbCMwhPOGo9oX5tlgsLzWIuZ5aQ9eph 3bjhH9TPgpWDphEab79pDBjV3WeS5je27L5+58kwW7g3cGneL02BaNztu96w8BPd 9eF+YXVBX3QkbhG3xEwCDdSTrKBYjPF6w4hExMl6ifFHihMNpDi9sg== =NK/w -END PGP SIGNATURE-
Re: [Dovecot] Max IMAP fodlers
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Mon, 7 Dec 2009, Jeffrey Nikoletich wrote: Also is there is hard limit on the amount of folders dovecot can handle? I have a user who has over 3000 folder (including subfolders) that is having issues with delays in loading, moving, and deleting messages. I cannot believe that ext3 chokes on just 3000 directory entries in such situation. I would think that one would notice the same with 20,000 messages per mailbox. How about really use another mail client for testing, as Marcus suggested, e.g. some brain-dead MUA, that does not try to be smart, for example Alpine or Mutt :) Also, try to log the session, maybe the user watches all 3000 folders for changes. So they all will be selected in a (short?) period of time again and again. Maybe a firewall in between is casuing a throttle, because of too many connections or something? Regards, - -- Steffen Kaiser -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.6 (GNU/Linux) iQEVAwUBSx4RKnWSIuGy1ktrAQJF4Af/diS3UqhWWiJONM8SntptVn3UH0rLvT+p NoSYfS2/0aVTstuU6vGhntGPsGCPXmSRVuGbW+ERS4U/6C7V9JQuo8Fdf2aYjGKe 2U/LxaodeKBc/c7cZdf4Uthu0z8NScWZJU8nG23EK31e9nlcte0D/eHF132i2zPZ mw+ZxFdgveTA+V4hnRVbUO49mYu3EmePkdbCMwhPOGo9oX5tlgsLzWIuZ5aQ9eph 3bjhH9TPgpWDphEab79pDBjV3WeS5je27L5+58kwW7g3cGneL02BaNztu96w8BPd 9eF+YXVBX3QkbhG3xEwCDdSTrKBYjPF6w4hExMl6ifFHihMNpDi9sg== =NK/w -END PGP SIGNATURE-
Re: [Dovecot] Max IMAP fodlers
On Mon, 7 Dec 2009 13:28:03 -0800, Jeffrey Nikoletich jeff...@nikoletich.com wrote: Thanks. Is there plans for folder quotas in the future? Also how would i change the layout of an existing maildir? # v1.1+ only: mail_location = maildir:~/Maildir:LAYOUT=fs Can I just change for that user and it will convert it? or does the folders have to by physically moved as well. I don't think the convert plugin would do this, but another alternative would be a multi-account MUA like Thunderbird. Create a new account with the LAYOUT=fs and verify it's setup properly. Then you can drag-n-drop from the old account to the new. Then, depending on your daring, can simply delete the old account and rename the base folder of the new account back to the old name. Just don't drag over too many folders at once... -- Daniel
Re: [Dovecot] Max IMAP fodlers
On Tue, 2009-12-08 at 10:53 -0800, dmil...@amfes.com wrote: # v1.1+ only: mail_location = maildir:~/Maildir:LAYOUT=fs Can I just change for that user and it will convert it? or does the folders have to by physically moved as well. I don't think the convert plugin would do this, With v2.0 dsync could do this :) dsync convert 'maildir:~/Maildir-fs:LAYOUT=fs' I think convert-tool could also do it, but it wouldn't preserve UIDs. And I'm not sure if it would actually copy or hard link the files. In any case neither would do just simple renaming.. signature.asc Description: This is a digitally signed message part
Re: [Dovecot] Max IMAP fodlers
On Dec 8, 2009, at 2:26 PM, Timo Sirainen wrote: # v1.1+ only: mail_location = maildir:~/Maildir:LAYOUT=fs Can I just change for that user and it will convert it? or does the folders have to by physically moved as well. I don't think the convert plugin would do this, With v2.0 dsync could do this :) dsync convert 'maildir:~/Maildir-fs:LAYOUT=fs' I think convert-tool could also do it, but it wouldn't preserve UIDs. And I'm not sure if it would actually copy or hard link the files. In any case neither would do just simple renaming.. Would this just be a matter of creating subdirs and moving the files? If there are no other concerns, a simple awk script could to that pretty easily. -Dave -- Dave McGuire Port Charlotte, FL
Re: [Dovecot] Max IMAP fodlers
On Tue, 2009-12-08 at 14:29 -0500, Dave McGuire wrote: dsync convert 'maildir:~/Maildir-fs:LAYOUT=fs' I think convert-tool could also do it, but it wouldn't preserve UIDs. And I'm not sure if it would actually copy or hard link the files. In any case neither would do just simple renaming.. Would this just be a matter of creating subdirs and moving the files? If there are no other concerns, a simple awk script could to that pretty easily. Yes, it should be pretty easy to create a script that just renames/mkdirs as necessary. signature.asc Description: This is a digitally signed message part
[Dovecot] Max IMAP fodlers
Is there a way to limit the amount of maildir folders a user can have? Also is there is hard limit on the amount of folders dovecot can handle? I have a user who has over 3000 folder (including subfolders) that is having issues with delays in loading, moving, and deleting messages. Thanks in advance Jeff N.
Re: [Dovecot] Max IMAP fodlers
dunno about how DC does its quotas, but Berkely O/S filesystem quotas as implemented on IBM's AIX controls not only how much storage you can have but also how many inodes, with 1 inode per file or folder Jeffrey Nikoletich wrote: Is there a way to limit the amount of maildir folders a user can have? Also is there is hard limit on the amount of folders dovecot can handle? I have a user who has over 3000 folder (including subfolders) that is having issues with delays in loading, moving, and deleting messages. Thanks in advance Jeff N. -- Once upon a time, the Internet was a friendly, neighbors-helping-neighbors small town, and no one locked their doors. Now it's like an apartment in Bed-Stuy: you need three heavy duty pick-proof locks, one of those braces that goes from the lock to the floor, and bars on the windows Stewart Dean, Unix System Admin, Bard College, New York 12504 sd...@bard.edu voice: 845-758-7475, fax: 845-758-7035
Re: [Dovecot] Max IMAP fodlers
On 2009-12-07 11:24:20 -0800, Jeffrey Nikoletich wrote: Is there a way to limit the amount of maildir folders a user can have? Also is there is hard limit on the amount of folders dovecot can handle? I have a user who has over 3000 folder (including subfolders) that is having issues with delays in loading, moving, and deleting messages. are you sure it is not just a client problem? -- openSUSE - SUSE Linux is my linux openSUSE is good for you www.opensuse.org
Re: [Dovecot] Max IMAP fodlers
Yes, We have reinstalled multiple times (OS and Client) with the same results. On Dec 7, 2009, at 11:57 AM, Marcus Rueckert wrote: On 2009-12-07 11:24:20 -0800, Jeffrey Nikoletich wrote: Is there a way to limit the amount of maildir folders a user can have? Also is there is hard limit on the amount of folders dovecot can handle? I have a user who has over 3000 folder (including subfolders) that is having issues with delays in loading, moving, and deleting messages. are you sure it is not just a client problem? -- openSUSE - SUSE Linux is my linux openSUSE is good for you www.opensuse.org
Re: [Dovecot] Max IMAP fodlers
On Mon, 2009-12-07 at 11:24 -0800, Jeffrey Nikoletich wrote: Is there a way to limit the amount of maildir folders a user can have? No. Also is there is hard limit on the amount of folders dovecot can handle? I have a user who has over 3000 folder (including subfolders) that is having issues with delays in loading, moving, and deleting messages. I guess you're using Maildir++, so all of the folders are in the one Maildir root directory? That's probably the main reason why it's slow. With v1.1+ you could switch to fs layout, so that the subfolders would no longer be in the root directory. signature.asc Description: This is a digitally signed message part
Re: [Dovecot] Max IMAP fodlers
Thanks. Is there plans for folder quotas in the future? Also how would i change the layout of an existing maildir? # v1.1+ only: mail_location = maildir:~/Maildir:LAYOUT=fs Can I just change for that user and it will convert it? or does the folders have to by physically moved as well. Thanks in advance, Jeff N. On Dec 7, 2009, at 12:12 PM, Timo Sirainen wrote: On Mon, 2009-12-07 at 11:24 -0800, Jeffrey Nikoletich wrote: Is there a way to limit the amount of maildir folders a user can have? No. Also is there is hard limit on the amount of folders dovecot can handle? I have a user who has over 3000 folder (including subfolders) that is having issues with delays in loading, moving, and deleting messages. I guess you're using Maildir++, so all of the folders are in the one Maildir root directory? That's probably the main reason why it's slow. With v1.1+ you could switch to fs layout, so that the subfolders would no longer be in the root directory.
Re: [Dovecot] Max IMAP fodlers
On Mon, 2009-12-07 at 13:28 -0800, Jeffrey Nikoletich wrote: Thanks. Is there plans for folder quotas in the future? I suppose some day I should add some limits to avoid DoSing. But for now wouldn't it be easier to just get that single problem user to use fewer folders? :) Also how would i change the layout of an existing maildir? # v1.1+ only: mail_location = maildir:~/Maildir:LAYOUT=fs Can I just change for that user and it will convert it? or does the folders have to by physically moved as well. There's no automatic way to do it. You'd have to create some script to rename the directories. signature.asc Description: This is a digitally signed message part
Re: [Dovecot] Max IMAP fodlers
Timo Sirainen wrote: On Mon, 2009-12-07 at 11:24 -0800, Jeffrey Nikoletich wrote: Is there a way to limit the amount of maildir folders a user can have? No. Also is there is hard limit on the amount of folders dovecot can handle? I have a user who has over 3000 folder (including subfolders) that is having issues with delays in loading, moving, and deleting messages. I guess you're using Maildir++, so all of the folders are in the one Maildir root directory? That's probably the main reason why it's slow. With v1.1+ you could switch to fs layout, so that the subfolders would no longer be in the root directory. Then, from a performance perspective, should fs layout be the recommended configuration for Maildir++ ? M.A.
Re: [Dovecot] Max IMAP fodlers
On Mon, 2009-12-07 at 16:46 -0500, Mario Antonio wrote: Then, from a performance perspective, should fs layout be the recommended configuration for Maildir++ ? I suppose. Although it won't be Maildir++ then anymore. :) signature.asc Description: This is a digitally signed message part
Re: [Dovecot] Max IMAP fodlers
On Mon, 2009-12-07 at 11:24 -0800, Jeffrey Nikoletich wrote: Also is there is hard limit on the amount of folders dovecot can handle? I have a user who has over 3000 folder (including subfolders) that is having issues with delays in loading, moving, and deleting messages. Are you sure those problems are even caused by the number of folders? There are only two issues I can think of: 1) Client LISTs all folders, which can take a while to download. Clients don't usually do this very often. 2) Filesystem is being slow. If you're using filesystem with directory indexes (and most are nowadays indexed) I don't think 2) should be much of a problem. And I think even without indexes OS would have cached the directory entries in a way that accessing the same directory would always be fast. So have you looked at other potential problems, like if deleting messages is slow and deletion is done by move-to-Trash, does the Trash have tens of thousands of messages? And at least ext3 already limits the number of folders that can be created with Maildir++ to 31998. That's probably good enough as a folder quota. :) signature.asc Description: This is a digitally signed message part
Re: [Dovecot] Max IMAP fodlers
I have been working with user to have them cut down on their folder list. the fs is ext3 and even when moving the user to own server, etc I get the slowdowns and email hangs. Thought it was a connection issue so I brought the users machine in to the DC with the same issues. It is just weird Jeff N. On Dec 7, 2009, at 2:00 PM, Timo Sirainen wrote: On Mon, 2009-12-07 at 11:24 -0800, Jeffrey Nikoletich wrote: Also is there is hard limit on the amount of folders dovecot can handle? I have a user who has over 3000 folder (including subfolders) that is having issues with delays in loading, moving, and deleting messages. Are you sure those problems are even caused by the number of folders? There are only two issues I can think of: 1) Client LISTs all folders, which can take a while to download. Clients don't usually do this very often. 2) Filesystem is being slow. If you're using filesystem with directory indexes (and most are nowadays indexed) I don't think 2) should be much of a problem. And I think even without indexes OS would have cached the directory entries in a way that accessing the same directory would always be fast. So have you looked at other potential problems, like if deleting messages is slow and deletion is done by move-to-Trash, does the Trash have tens of thousands of messages? And at least ext3 already limits the number of folders that can be created with Maildir++ to 31998. That's probably good enough as a folder quota. :)
Re: [Dovecot] Max IMAP fodlers
On Mon, 2009-12-07 at 15:11 -0800, Jeffrey Nikoletich wrote: I have been working with user to have them cut down on their folder list. the fs is ext3 and even when moving the user to own server, etc I get the slowdowns and email hangs. Thought it was a connection issue so I brought the users machine in to the DC with the same issues. It is just weird Looking at strace -tt output of the hanging imap process could show something interesting. http://wiki.dovecot.org/Debugging/ProcessTracing signature.asc Description: This is a digitally signed message part
