RE: [Imap-uw] slow IMAP connection
On Tue, 16 Mar 2010, Troy Campbell wrote: Thanks Mabry, it sounds like we need to move to a newer file format. Is mix supported by other imap servers as well i.e., is it a standard format or something "UW" specific? mix is specific to UW IMAP (imap-2006 and later) and Panda IMAP. Any application which uses the c-client library (e.g., Alpine) is also mix-capable. The new Messaging Architects mail store uses a superset of mix. It is possible to take mailbox data from UW IMAP or Panda IMAP and place it into an MA mail store mailbox. In fact, that is how I created my test data. -- Mark -- http://panda.com/mrc Democracy is two wolves and a sheep deciding what to eat for lunch. Liberty is a well-armed sheep contesting the vote. ___ Imap-uw mailing list Imap-uw@u.washington.edu http://mailman2.u.washington.edu/mailman/listinfo/imap-uw
RE: [Imap-uw] slow IMAP connection
Thanks Mabry, it sounds like we need to move to a newer file format. Is mix supported by other imap servers as well i.e., is it a standard format or something "UW" specific? Thanks again, Troy -Original Message- From: imap-uw-boun...@mailman2.u.washington.edu on behalf of Mabry Tyson Sent: Tue 3/16/2010 2:17 AM To: imap-uw@u.washington.edu Subject: Re: [Imap-uw] slow IMAP connection Mark may correct me, but here's how I understand what we've experienced. It may bear some resemblance to your situation. You will also see the Connection Reset messages if your users are using multiple mail clients (for instance, iPhone, home machine, work machine) simultaneously and they have mbox formatted mailboxes. Especially if their inboxes are large, users will notice delays. In such a case, the imap server is trying to rewrite the inbox (in response to, say, reads of new messages). It has a lock for the duration of the writing (so the larger the file, the longer the time to rewrite). Other mail clients for the same user will eventually lose their lock and then opportunistically grab the lock later, which means the mail client currently in use by the user will have to wait for the lock (resulting in reports of slow access). The high I/O wait time is consistent with writing large mbox files repeatedly. If that's your situation, changing the mailbox format is the easiest way to improve the situation. We moved users to mbx format. It also helps to educate your users to avoid multiple simultaneous clients and to keep the size of their mailboxes down. It helps to have the mail files on the IMAP server (rather than mounted remotely, which only makes the I/O wait time worse). On 3/15/10 3:48 PM, Mark Crispin wrote: > On Mon, 15 Mar 2010, Troy Campbell wrote: >> I recently installed the 2007e of UW IMAP and seeing high I/O wait on >> the server. > > What, specifically, do you mean by "high I/O wait"? > >> I have about 50 users. > > That is a very small facility that even modest server hardware should > handle with aplomb. > >> Mar 15 14:42:47 request1 imapd[1445]: Connection reset by peer, while >> reading line user=x host=f.q.d.n[11.22.33.44] > > This message simply means that a client disconnected from the server > without first issuing a proper LOGOUT command, and that this was detected > while the server was reading a new IMAP command line. > > The only time this message is at all interesting is if a user complains > about being disconnected from the server. This message is evidence that, > from the server's viewpoint, the client disconnected. This means that you > can dismiss the server as a possible cause of the problem, and look > elsewhere. When both client and server point fingers at each other, a > common cause is an intermediate router (especially a NAT). > >> Mar 15 14:42:49 request1 imapd[3250]: imap service init from >> 192.123.45.77 >> Mar 15 14:42:49 request1 imapd[3250]: Login user= host=x.y.z >> [192.31.97.142] >> Mar 15 14:42:52 request1 imapd[30847]: Logout user= host=c.d.e >> [1.2.3.4] > > None of these messages are remarkable. They are all indications of normal > operation. > > Your /etc/xinetd.d/imap file looks OK as well > >> My initial build was like this: >> make lr5 SSLTYPE=none > > Why are you using SSLTYPE=none? I hope that this is strictly for in-house > use, and not exported over the Internet; as it will result in > confidential > traffic (including passwords) being sent in the clear. > > -- Mark -- > > http://panda.com/mrc > Democracy is two wolves and a sheep deciding what to eat for lunch. > Liberty is a well-armed sheep contesting the vote. > ___ > Imap-uw mailing list > Imap-uw@u.washington.edu > http://mailman2.u.washington.edu/mailman/listinfo/imap-uw ___ Imap-uw mailing list Imap-uw@u.washington.edu http://mailman2.u.washington.edu/mailman/listinfo/imap-uw ___ Imap-uw mailing list Imap-uw@u.washington.edu http://mailman2.u.washington.edu/mailman/listinfo/imap-uw
RE: [Imap-uw] slow IMAP connection
I decided to fall back to the old server until I've figured out a game plan to move forward. Appreciate all the help from people on this list (esp Mark). :) Thanks, Troy -Original Message- From: m...@hsinghsing.panda.com on behalf of Mark Crispin Sent: Tue 3/16/2010 7:36 PM To: Troy Campbell Cc: UW IMAP Software Interest List Subject: RE: [Imap-uw] slow IMAP connection I would have to see the dumps in question to comment intelligently on what they contain and whether or not there is a problem. Unfortunately I don't have the time to do that. However, I can tell you that a STATUS command internally opens the destination mailbox. Certain broken clients think that they should do a LIST of all possible names, SUBSCRIBE all those names, and then issue a STATUS of those names at each synchronization. If you use traditional UNIX mailbox format (mbox format) then you'll certainly have performance problems with clients that do that. mix format will help you a lot with that. On Tue, 16 Mar 2010, Troy Campbell wrote: > > My other concern is that there are certain folders showing up empty. > When I do a tcpdump from the server watching the client just as > they click on the folder and capture the traffic I'm seeing > IMAP STATUS commands of other folders. It's like the pointers > are screwed up somehow or I'm seeing delayed data from her > "scrolling" down (kind of like seeing the "ls" data in a > directory prior to the "cat" command of the file). -- Mark -- http://panda.com/mrc Democracy is two wolves and a sheep deciding what to eat for lunch. Liberty is a well-armed sheep contesting the vote. ___ Imap-uw mailing list Imap-uw@u.washington.edu http://mailman2.u.washington.edu/mailman/listinfo/imap-uw
RE: [Imap-uw] slow IMAP connection
I would have to see the dumps in question to comment intelligently on what they contain and whether or not there is a problem. Unfortunately I don't have the time to do that. However, I can tell you that a STATUS command internally opens the destination mailbox. Certain broken clients think that they should do a LIST of all possible names, SUBSCRIBE all those names, and then issue a STATUS of those names at each synchronization. If you use traditional UNIX mailbox format (mbox format) then you'll certainly have performance problems with clients that do that. mix format will help you a lot with that. On Tue, 16 Mar 2010, Troy Campbell wrote: My other concern is that there are certain folders showing up empty. When I do a tcpdump from the server watching the client just as they click on the folder and capture the traffic I'm seeing IMAP STATUS commands of other folders. It's like the pointers are screwed up somehow or I'm seeing delayed data from her "scrolling" down (kind of like seeing the "ls" data in a directory prior to the "cat" command of the file). -- Mark -- http://panda.com/mrc Democracy is two wolves and a sheep deciding what to eat for lunch. Liberty is a well-armed sheep contesting the vote. ___ Imap-uw mailing list Imap-uw@u.washington.edu http://mailman2.u.washington.edu/mailman/listinfo/imap-uw
RE: [Imap-uw] slow IMAP connection
On Tue, 16 Mar 2010, Troy Campbell wrote: What authentication should I put in to support Outlook? Since you're using Red Hat Enterprise, simply make lr5 should be good enough. If you're using Kerberos, use make lr5 EXTRAAUTHENTICATORS=gss The build command would be even better. :) It also needs to support other types of clients like mozilla. Outlook, Mozilla, etc. all support LOGIN and the PLAIN SASL authenticator which is built by default. Any modern client should also handle LOGINDISABLED which disables these password-based authenticators until either SSL or TLS encryption is first negotiated. I have a valid certificate file. Just follow the instructions in the BUILD and SSLBUILD file. On RHE, your public key will go into /etc/pki/tls/certs and your private key in /etc/pki/tls/private -- Mark -- http://panda.com/mrc Democracy is two wolves and a sheep deciding what to eat for lunch. Liberty is a well-armed sheep contesting the vote. ___ Imap-uw mailing list Imap-uw@u.washington.edu http://mailman2.u.washington.edu/mailman/listinfo/imap-uw
Re: [Imap-uw] slow IMAP connection
On Tue, 16 Mar 2010, Jim O'Leary wrote: The mix format dug us out of an approaching performance catastrophe. Three cheers to Mark for developing mix in the nick of time to save our necks! Thanks everybody for the nice comments. It means a lot to me to know that the efforts were appreciated, especially in light of subsequent events! Not everybody at UW was happy that I fixed the IMAP servers... If you like mix, and are considering using commercial products, I'd like to mention what I've been doing at Messaging Architects. We recently released version 2009.1 of M+Guardian: http://www.messagingarchitects.com/products/m-guardian-email-security.html 2009.1 was a major update, nearly a complete rewrite. It is now the first of our products to use the "foundation" technology that will presently underlie our other products. One of my big efforts for foundation's was the mail store, which uses a greatly expanded version of mix; adding clustering (no more requirement that the IMAP server and mailbox files be located on the same CPU), mailbox metadata, message metadata, and...stubbing!! Currently, we use stubbing (my term for it is "virtual mailboxes") to implement the quarantine in M+Guardian; all quarantined messages go to a mailbox in the global quarantine. The per-user quarantines are virtual mailboxes containing pointers to the data in the global quarantine. However, the stubbing facility is far more general than this; among other things it can be nested (a virtual mailbox can have a pointer into a virtual mailbox). I'll leave it to your imagination how this will be used in our other products... The clustering is also way cool. Between the IMAP server and the mix store is a stateless URI-based protocol (albeit with features to support stateful protocols such as IMAP and POP) and a clustered filesystem. I had my paws in the design of the stateless protocol and was the first to beat(!) hard(!!) on the clustered filesystem. I also wrote a completely new IMAP server that (unlike UW and Panda IMAP) supports ACL. Obviously, this is not just for M+Guardian (an anti-malware, anti-spam, content filtering, and data leak prevention appliance). If you look at our web site, it's not hard to guess how we'll be using foundation in those products. This is all stuff that our customers are using now. There's a lot more neat stuff coming up in the future. My teammates have been quite busy... -- Mark -- http://panda.com/mrc Democracy is two wolves and a sheep deciding what to eat for lunch. Liberty is a well-armed sheep contesting the vote. ___ Imap-uw mailing list Imap-uw@u.washington.edu http://mailman2.u.washington.edu/mailman/listinfo/imap-uw
Re: [Imap-uw] slow IMAP connection
On Tue, 16 Mar 2010, David B Funk wrote: It's also kinda fun to talk to the Exchange admins and watch them cringe when I talk about our users who have 5GB inboxes with 30K messages in them. ;) Hah! I suppose that you know that UW jumped on the Exchange bandwagon. That is, for the privileged few, such as their million-dollar plus president; plebes, proles, and students are to be outsourced to "the cloud". Then again, I guess that Exchange is an upgrade from GroupWise... -- Mark -- http://panda.com/mrc Democracy is two wolves and a sheep deciding what to eat for lunch. Liberty is a well-armed sheep contesting the vote. ___ Imap-uw mailing list Imap-uw@u.washington.edu http://mailman2.u.washington.edu/mailman/listinfo/imap-uw
Re: [Imap-uw] slow IMAP connection
On Mar 16, 2010, at 3:39 PM, Jim O'Leary wrote: > I'll second everything that Bob said plus the BIG advantage > of improved backup performance. Now our incremental backups > actually buy us something. No more "incremental is almost same > as full because each of those huge mbox/mbx inboxes got one new > message" syndrome. > > It's also kinda fun to talk to the Exchange admins and watch them > cringe when I talk about our users who have 5GB inboxes with > 30K messages in them. ;) > > I'll third everything. The mix format dug us out of an approaching > performance catastrophe. Three cheers to Mark for developing mix in the > nick of time to save our necks! ...and a fourth here. mix was a HUGE boon for us. We were using MBOX previously, and all the performance metrics we were tracking -- disk i/o, storage CPU consumption, etc -- dropped by 90% after converting to mix. That is not a typo: 90% REDUCTION in performance-related metrics. -- Andrew LaurenceOffice of Information Technology atlau...@uci.edu University of California, Irvine ___ Imap-uw mailing list Imap-uw@u.washington.edu http://mailman2.u.washington.edu/mailman/listinfo/imap-uw
Re: [Imap-uw] slow IMAP connection
> > I'll second everything that Bob said plus the BIG advantage > of improved backup performance. Now our incremental backups > actually buy us something. No more "incremental is almost same > as full because each of those huge mbox/mbx inboxes got one new > message" syndrome. > > It's also kinda fun to talk to the Exchange admins and watch them > cringe when I talk about our users who have 5GB inboxes with > 30K messages in them. ;) > I'll third everything. The mix format dug us out of an approaching performance catastrophe. Three cheers to Mark for developing mix in the nick of time to save our necks! ___ Imap-uw mailing list Imap-uw@u.washington.edu http://mailman2.u.washington.edu/mailman/listinfo/imap-uw
Re: [Imap-uw] slow IMAP connection
On Tue, 16 Mar 2010, Bob Atkins wrote:
> Yes, that is correct. The mix format consists of many files in a
[snip..]
> status info which makes access /_very_/ fast. FYI, we implemented our
> mix solution using 10MB files rather than the default 1MB size because
> we felt that would be a more efficient file size and it has proven to
> work very well. As those who went down the maildir path (with another
> imap server) have painfully discovered - directories with too many
> inodes are very slow to search. mix is a great balance between not too
> many inodes and keeping file sizes and linear search runs to a minimum.
I'll second everything that Bob said plus the BIG advantage
of improved backup performance. Now our incremental backups
actually buy us something. No more "incremental is almost same
as full because each of those huge mbox/mbx inboxes got one new
message" syndrome.
It's also kinda fun to talk to the Exchange admins and watch them
cringe when I talk about our users who have 5GB inboxes with
30K messages in them. ;)
--
Dave Funk University of Iowa
College of Engineering
319/335-5751 FAX: 319/384-0549 1256 Seamans Center
Sys_admin/Postmaster/cell_adminIowa City, IA 52242-1527
#include
Better is not better, 'standard' is better. B{
___
Imap-uw mailing list
Imap-uw@u.washington.edu
http://mailman2.u.washington.edu/mailman/listinfo/imap-uw
Re: [Imap-uw] slow IMAP connection
In regard to: Re: [Imap-uw] slow IMAP connection, Mark Crispin said (at...: If SMTP is a suitable mechanism to transmit 2GB hunks of data, that begs the question of why we need bittorrent... ;) Based on how I see people use email these days, there's frankly some truth to that. I dearly wish our campus had deployed some kind of decent "digital drop box" software and then trained our customers in how to use it. It's too late for that now; people have already become accustomed to using email as the sole method for digital file distribution. Tim -- Tim Mooney moo...@dogbert.cc.ndsu.nodak.edu Enterprise Computing & Infrastructure 701-231-1076 (Voice) Room 242-J6, IACC Building 701-231-8541 (Fax) North Dakota State University, Fargo, ND 58105-5164 ___ Imap-uw mailing list Imap-uw@u.washington.edu http://mailman2.u.washington.edu/mailman/listinfo/imap-uw
RE: [Imap-uw] slow IMAP connection
My other concern is that there are certain folders showing up empty. When I do a tcpdump from the server watching the client just as they click on the folder and capture the traffic I'm seeing IMAP STATUS commands of other folders. It's like the pointers are screwed up somehow or I'm seeing delayed data from her "scrolling" down (kind of like seeing the "ls" data in a directory prior to the "cat" command of the file). Troy -Original Message- From: imap-uw-boun...@mailman2.u.washington.edu on behalf of Troy Campbell Sent: Tue 3/16/2010 3:35 PM To: Mark Crispin; Bob Atkins Cc: Tim Mooney; UW IMAP Software Interest List Subject: RE: [Imap-uw] slow IMAP connection What authentication should I put in to support Outlook? The build command would be even better. :) It also needs to support other types of clients like mozilla. I have a valid certificate file. Regards, Troy -Original Message- From: imap-uw-boun...@mailman2.u.washington.edu on behalf of Mark Crispin Sent: Tue 3/16/2010 1:53 PM To: Bob Atkins Cc: Tim Mooney; UW IMAP Software Interest List Subject: Re: [Imap-uw] slow IMAP connection On Tue, 16 Mar 2010, Bob Atkins wrote: > As those who went down the maildir path (with another > imap server) have painfully discovered - directories with too many > inodes are very slow to search. mix is a great balance between not too > many inodes and keeping file sizes and linear search runs to a minimum. Interesting. I have had acolytes of the maildir church scream that I don't know what I am talking about when I said that there was a scaling issue with directories with too many inodes. FWIW, the same issue happens with mh and Cyrus. -- Mark -- http://panda.com/mrc Democracy is two wolves and a sheep deciding what to eat for lunch. Liberty is a well-armed sheep contesting the vote. ___ Imap-uw mailing list Imap-uw@u.washington.edu http://mailman2.u.washington.edu/mailman/listinfo/imap-uw ___ Imap-uw mailing list Imap-uw@u.washington.edu http://mailman2.u.washington.edu/mailman/listinfo/imap-uw ___ Imap-uw mailing list Imap-uw@u.washington.edu http://mailman2.u.washington.edu/mailman/listinfo/imap-uw
RE: [Imap-uw] slow IMAP connection
What authentication should I put in to support Outlook? The build command would be even better. :) It also needs to support other types of clients like mozilla. I have a valid certificate file. Regards, Troy -Original Message- From: imap-uw-boun...@mailman2.u.washington.edu on behalf of Mark Crispin Sent: Tue 3/16/2010 1:53 PM To: Bob Atkins Cc: Tim Mooney; UW IMAP Software Interest List Subject: Re: [Imap-uw] slow IMAP connection On Tue, 16 Mar 2010, Bob Atkins wrote: > As those who went down the maildir path (with another > imap server) have painfully discovered - directories with too many > inodes are very slow to search. mix is a great balance between not too > many inodes and keeping file sizes and linear search runs to a minimum. Interesting. I have had acolytes of the maildir church scream that I don't know what I am talking about when I said that there was a scaling issue with directories with too many inodes. FWIW, the same issue happens with mh and Cyrus. -- Mark -- http://panda.com/mrc Democracy is two wolves and a sheep deciding what to eat for lunch. Liberty is a well-armed sheep contesting the vote. ___ Imap-uw mailing list Imap-uw@u.washington.edu http://mailman2.u.washington.edu/mailman/listinfo/imap-uw ___ Imap-uw mailing list Imap-uw@u.washington.edu http://mailman2.u.washington.edu/mailman/listinfo/imap-uw
Re: [Imap-uw] slow IMAP connection
On Tue, 16 Mar 2010, Bob Atkins wrote: As those who went down the maildir path (with another imap server) have painfully discovered - directories with too many inodes are very slow to search. mix is a great balance between not too many inodes and keeping file sizes and linear search runs to a minimum. Interesting. I have had acolytes of the maildir church scream that I don't know what I am talking about when I said that there was a scaling issue with directories with too many inodes. FWIW, the same issue happens with mh and Cyrus. -- Mark -- http://panda.com/mrc Democracy is two wolves and a sheep deciding what to eat for lunch. Liberty is a well-armed sheep contesting the vote. ___ Imap-uw mailing list Imap-uw@u.washington.edu http://mailman2.u.washington.edu/mailman/listinfo/imap-uw
RE: [Imap-uw] slow IMAP connection
On Tue, 16 Mar 2010, Troy Campbell wrote: I'll take a look at the size of the INBOXES to see if that's the issue. What would be the tool/process for converting from mbx to mix? mixcvt is the best tool to use. There is an updated Panda version of mixcvt, but UW mixcvt should be alright for most purposes. The tool that you want to avoid is UW mixrbld. UW's code is badly broken and will lose messages. Panda mixrbld fixes these issues. -- Mark -- http://panda.com/mrc Democracy is two wolves and a sheep deciding what to eat for lunch. Liberty is a well-armed sheep contesting the vote. ___ Imap-uw mailing list Imap-uw@u.washington.edu http://mailman2.u.washington.edu/mailman/listinfo/imap-uw
Re: [Imap-uw] slow IMAP connection
Tim, Tim Mooney wrote: In regard to: Re: [Imap-uw] slow IMAP connection, Bob Atkins said (at...: The only way to fix the slow I/O problem, especially with large mail boxes is to convert to mix format. mbx format is only marginally better than mbox from a performance standpoint. This is from my personal experience of having mail boxes with 40,000+ messages and >2GB in size. This brings up a question I've had: both traditional format and MBX run into issues when they cross the 2 GiB boundary, because of signedness issues in the UW code. I realize a MIX folder is composed of multiple files, but does that mean that the 2 GiB is not a problem for MIX? Yes, that is correct. The mix format consists of many files in a directory. A refinement of the maildir concept which was flawed because it created a separate file for /_each_/ message. mix creates files that contain multiple messages and a mix file will never get to >2GB in size so that 2GB boundary problem is solved. I believe that the default file size for mix is 1MB. Any message larger than 1MB will be the only message in a file however, the many smaller messages will fill up to 1 MB file before another file is started. All of the messages are indexed into an index file(s) which contains all of the subject headings and status info which makes access /_very_/ fast. FYI, we implemented our mix solution using 10MB files rather than the default 1MB size because we felt that would be a more efficient file size and it has proven to work very well. As those who went down the maildir path (with another imap server) have painfully discovered - directories with too many inodes are very slow to search. mix is a great balance between not too many inodes and keeping file sizes and linear search runs to a minimum. --- Bob ___ Imap-uw mailing list Imap-uw@u.washington.edu http://mailman2.u.washington.edu/mailman/listinfo/imap-uw
Re: [Imap-uw] slow IMAP connection
On Tue, 16 Mar 2010, Tim Mooney wrote: Not until/unless you start dealing with single messages that are larger than 2GB. It will be a while before people start wanting to do that. :) We're still not allowing individual messages larger than 25 MiB, so we're safe for now... Yeah, even Gmail limits to 35MB. Even if the promise of universal 50Mbit/s broadband comes to pass, a 2GB Godzillagram would take a minimum of several minutes to transmit at (unobtainable) optimum performance with no other traffic. If SMTP is a suitable mechanism to transmit 2GB hunks of data, that begs the question of why we need bittorrent... ;) -- Mark -- http://panda.com/mrc Democracy is two wolves and a sheep deciding what to eat for lunch. Liberty is a well-armed sheep contesting the vote. ___ Imap-uw mailing list Imap-uw@u.washington.edu http://mailman2.u.washington.edu/mailman/listinfo/imap-uw
Re: [Imap-uw] slow IMAP connection
In regard to: Re: [Imap-uw] slow IMAP connection, Mark Crispin said (at...: I realize a MIX folder is composed of multiple files, but does that mean that the 2 GiB is not a problem for MIX? Not until/unless you start dealing with single messages that are larger than 2GB. It will be a while before people start wanting to do that. :) We're still not allowing individual messages larger than 25 MiB, so we're safe for now... Thanks for the info Mark. Tim -- Tim Mooney moo...@dogbert.cc.ndsu.nodak.edu Enterprise Computing & Infrastructure 701-231-1076 (Voice) Room 242-J6, IACC Building 701-231-8541 (Fax) North Dakota State University, Fargo, ND 58105-5164 ___ Imap-uw mailing list Imap-uw@u.washington.edu http://mailman2.u.washington.edu/mailman/listinfo/imap-uw
Re: [Imap-uw] slow IMAP connection
On Tue, 16 Mar 2010, Tim Mooney wrote: This brings up a question I've had: both traditional format and MBX run into issues when they cross the 2 GiB boundary, because of signedness issues in the UW code. Correct. It's not the code so much as it is the system calls; there are separate system calls that have to be used if you want to cross 2GB filesize on a 32-bit OS. It doesn't make much sense to fix because flat files perform so poorly long before you get to that size. I realize a MIX folder is composed of multiple files, but does that mean that the 2 GiB is not a problem for MIX? Not until/unless you start dealing with single messages that are larger than 2GB. It will be a while before people start wanting to do that. :) -- Mark -- http://panda.com/mrc Democracy is two wolves and a sheep deciding what to eat for lunch. Liberty is a well-armed sheep contesting the vote. ___ Imap-uw mailing list Imap-uw@u.washington.edu http://mailman2.u.washington.edu/mailman/listinfo/imap-uw
RE: [Imap-uw] slow IMAP connection
Sorry I haven't been able to reply as our system has been very slow with lots of calls going on right now. I'll take a look at the size of the INBOXES to see if that's the issue. What would be the tool/process for converting from mbx to mix? Thanks again for all the replies, Troy -Original Message- From: imap-uw-boun...@mailman2.u.washington.edu on behalf of Mark Crispin Sent: Tue 3/16/2010 12:34 PM To: Bob Atkins Cc: Mabry Tyson; UW IMAP Software Interest List Subject: Re: [Imap-uw] slow IMAP connection Although I agree with Bob's conclusion (use mix format), I think that it overstates the issue to say that "mbx format is only marginally better" than mbox". mbx is a major jump up from mbox, and is handled much better than mbox. mbx scaled very well for the demands of 1996 when it was designed; it's just that we no longer live in that world. mix was designed in 2006 to be as much of a jump from mbx as mbx is a jump from mbox. mix is the ONLY choice for a mailbox >2GB in size; mbx and mbox will not work at all. Although mbx will work with a mailbox of 40,000+ message, it would be painful; and mbox would be unusable. I think that a way of looking at it is that if mbox is a "1", then mbx is a "10" and mix is a "100". On Tue, 16 Mar 2010, Bob Atkins wrote: > The only way to fix the slow I/O problem, especially with large mail > boxes is to convert to mix format. mbx format is only marginally better > than mbox from a performance standpoint. This is from my personal > experience of having mail boxes with 40,000+ messages and >2GB in size. > Today's users have no idea how much space their mail is consuming, nor > would they care if they did. Our only option as administrators is to > implement the fastest and most scalable solution that we can. We > converted our systems to mix format about a year ago and oh what a > different it made. Much happier users and even happier admins ;-) -- Mark -- http://panda.com/mrc Democracy is two wolves and a sheep deciding what to eat for lunch. Liberty is a well-armed sheep contesting the vote. ___ Imap-uw mailing list Imap-uw@u.washington.edu http://mailman2.u.washington.edu/mailman/listinfo/imap-uw ___ Imap-uw mailing list Imap-uw@u.washington.edu http://mailman2.u.washington.edu/mailman/listinfo/imap-uw
Re: [Imap-uw] slow IMAP connection
i still use MH format, and it's painful, but it means i can also use MH tools. ___ Imap-uw mailing list Imap-uw@u.washington.edu http://mailman2.u.washington.edu/mailman/listinfo/imap-uw
Re: [Imap-uw] slow IMAP connection
In regard to: Re: [Imap-uw] slow IMAP connection, Bob Atkins said (at...: The only way to fix the slow I/O problem, especially with large mail boxes is to convert to mix format. mbx format is only marginally better than mbox from a performance standpoint. This is from my personal experience of having mail boxes with 40,000+ messages and >2GB in size. This brings up a question I've had: both traditional format and MBX run into issues when they cross the 2 GiB boundary, because of signedness issues in the UW code. I realize a MIX folder is composed of multiple files, but does that mean that the 2 GiB is not a problem for MIX? Tim -- Tim Mooney moo...@dogbert.cc.ndsu.nodak.edu Enterprise Computing & Infrastructure 701-231-1076 (Voice) Room 242-J6, IACC Building 701-231-8541 (Fax) North Dakota State University, Fargo, ND 58105-5164 ___ Imap-uw mailing list Imap-uw@u.washington.edu http://mailman2.u.washington.edu/mailman/listinfo/imap-uw
Re: [Imap-uw] slow IMAP connection
Although I agree with Bob's conclusion (use mix format), I think that it overstates the issue to say that "mbx format is only marginally better" than mbox". mbx is a major jump up from mbox, and is handled much better than mbox. mbx scaled very well for the demands of 1996 when it was designed; it's just that we no longer live in that world. mix was designed in 2006 to be as much of a jump from mbx as mbx is a jump from mbox. mix is the ONLY choice for a mailbox >2GB in size; mbx and mbox will not work at all. Although mbx will work with a mailbox of 40,000+ message, it would be painful; and mbox would be unusable. I think that a way of looking at it is that if mbox is a "1", then mbx is a "10" and mix is a "100". On Tue, 16 Mar 2010, Bob Atkins wrote: The only way to fix the slow I/O problem, especially with large mail boxes is to convert to mix format. mbx format is only marginally better than mbox from a performance standpoint. This is from my personal experience of having mail boxes with 40,000+ messages and >2GB in size. Today's users have no idea how much space their mail is consuming, nor would they care if they did. Our only option as administrators is to implement the fastest and most scalable solution that we can. We converted our systems to mix format about a year ago and oh what a different it made. Much happier users and even happier admins ;-) -- Mark -- http://panda.com/mrc Democracy is two wolves and a sheep deciding what to eat for lunch. Liberty is a well-armed sheep contesting the vote. ___ Imap-uw mailing list Imap-uw@u.washington.edu http://mailman2.u.washington.edu/mailman/listinfo/imap-uw
Re: [Imap-uw] slow IMAP connection
The issues that Mabry identifies here is definitely a problem, but the normal log message for this condition is "Killed (lost mailbox lock)" and not "Connection Reset". "Connection Reset", particularly with Outlook, is more commonly associated with Outlook's timeout set very small. Outlook treats IMAP as if it was HTTP and times out very aggressively; this worsens any situation with heavy load and/or large mailbox. The fix for Outlook is to look in the advanced options and set the timeout to the maximum (usually, this is a slider that should be moved all the way to the right). Changing the mailbox format from something other than traditional UNIX format (mbox format) is also a very good idea. I will deal with this in more detail in my next reply. On Tue, 16 Mar 2010, Mabry Tyson wrote: Mark may correct me, but here's how I understand what we've experienced. It may bear some resemblance to your situation. You will also see the Connection Reset messages if your users are using multiple mail clients (for instance, iPhone, home machine, work machine) simultaneously and they have mbox formatted mailboxes. Especially if their inboxes are large, users will notice delays. In such a case, the imap server is trying to rewrite the inbox (in response to, say, reads of new messages). It has a lock for the duration of the writing (so the larger the file, the longer the time to rewrite). Other mail clients for the same user will eventually lose their lock and then opportunistically grab the lock later, which means the mail client currently in use by the user will have to wait for the lock (resulting in reports of slow access). The high I/O wait time is consistent with writing large mbox files repeatedly. If that's your situation, changing the mailbox format is the easiest way to improve the situation. We moved users to mbx format. It also helps to educate your users to avoid multiple simultaneous clients and to keep the size of their mailboxes down. It helps to have the mail files on the IMAP server (rather than mounted remotely, which only makes the I/O wait time worse). -- Mark -- http://panda.com/mrc Democracy is two wolves and a sheep deciding what to eat for lunch. Liberty is a well-armed sheep contesting the vote. ___ Imap-uw mailing list Imap-uw@u.washington.edu http://mailman2.u.washington.edu/mailman/listinfo/imap-uw
Re: [Imap-uw] slow IMAP connection
The only way to fix the slow I/O problem, especially with large mail boxes is to convert to mix format. mbx format is only marginally better than mbox from a performance standpoint. This is from my personal experience of having mail boxes with 40,000+ messages and >2GB in size. Today's users have no idea how much space their mail is consuming, nor would they care if they did. Our only option as administrators is to implement the fastest and most scalable solution that we can. We converted our systems to mix format about a year ago and oh what a different it made. Much happier users and even happier admins ;-) --- Bob Mabry Tyson wrote: Mark may correct me, but here's how I understand what we've experienced. It may bear some resemblance to your situation. You will also see the Connection Reset messages if your users are using multiple mail clients (for instance, iPhone, home machine, work machine) simultaneously and they have mbox formatted mailboxes. Especially if their inboxes are large, users will notice delays. In such a case, the imap server is trying to rewrite the inbox (in response to, say, reads of new messages). It has a lock for the duration of the writing (so the larger the file, the longer the time to rewrite). Other mail clients for the same user will eventually lose their lock and then opportunistically grab the lock later, which means the mail client currently in use by the user will have to wait for the lock (resulting in reports of slow access). The high I/O wait time is consistent with writing large mbox files repeatedly. If that's your situation, changing the mailbox format is the easiest way to improve the situation. We moved users to mbx format. It also helps to educate your users to avoid multiple simultaneous clients and to keep the size of their mailboxes down. It helps to have the mail files on the IMAP server (rather than mounted remotely, which only makes the I/O wait time worse). On 3/15/10 3:48 PM, Mark Crispin wrote: On Mon, 15 Mar 2010, Troy Campbell wrote: I recently installed the 2007e of UW IMAP and seeing high I/O wait on the server. What, specifically, do you mean by "high I/O wait"? I have about 50 users. That is a very small facility that even modest server hardware should handle with aplomb. Mar 15 14:42:47 request1 imapd[1445]: Connection reset by peer, while reading line user=x host=f.q.d.n[11.22.33.44] This message simply means that a client disconnected from the server without first issuing a proper LOGOUT command, and that this was detected while the server was reading a new IMAP command line. The only time this message is at all interesting is if a user complains about being disconnected from the server. This message is evidence that, from the server's viewpoint, the client disconnected. This means that you can dismiss the server as a possible cause of the problem, and look elsewhere. When both client and server point fingers at each other, a common cause is an intermediate router (especially a NAT). Mar 15 14:42:49 request1 imapd[3250]: imap service init from 192.123.45.77 Mar 15 14:42:49 request1 imapd[3250]: Login user= host=x.y.z [192.31.97.142] Mar 15 14:42:52 request1 imapd[30847]: Logout user= host=c.d.e [1.2.3.4] None of these messages are remarkable. They are all indications of normal operation. Your /etc/xinetd.d/imap file looks OK as well My initial build was like this: make lr5 SSLTYPE=none Why are you using SSLTYPE=none? I hope that this is strictly for in-house use, and not exported over the Internet; as it will result in confidential traffic (including passwords) being sent in the clear. -- Mark -- http://panda.com/mrc Democracy is two wolves and a sheep deciding what to eat for lunch. Liberty is a well-armed sheep contesting the vote. ___ Imap-uw mailing list Imap-uw@u.washington.edu http://mailman2.u.washington.edu/mailman/listinfo/imap-uw ___ Imap-uw mailing list Imap-uw@u.washington.edu http://mailman2.u.washington.edu/mailman/listinfo/imap-uw ___ Imap-uw mailing list Imap-uw@u.washington.edu http://mailman2.u.washington.edu/mailman/listinfo/imap-uw
Re: [Imap-uw] slow IMAP connection
Mark may correct me, but here's how I understand what we've experienced. It may bear some resemblance to your situation. You will also see the Connection Reset messages if your users are using multiple mail clients (for instance, iPhone, home machine, work machine) simultaneously and they have mbox formatted mailboxes. Especially if their inboxes are large, users will notice delays. In such a case, the imap server is trying to rewrite the inbox (in response to, say, reads of new messages). It has a lock for the duration of the writing (so the larger the file, the longer the time to rewrite). Other mail clients for the same user will eventually lose their lock and then opportunistically grab the lock later, which means the mail client currently in use by the user will have to wait for the lock (resulting in reports of slow access). The high I/O wait time is consistent with writing large mbox files repeatedly. If that's your situation, changing the mailbox format is the easiest way to improve the situation. We moved users to mbx format. It also helps to educate your users to avoid multiple simultaneous clients and to keep the size of their mailboxes down. It helps to have the mail files on the IMAP server (rather than mounted remotely, which only makes the I/O wait time worse). On 3/15/10 3:48 PM, Mark Crispin wrote: On Mon, 15 Mar 2010, Troy Campbell wrote: I recently installed the 2007e of UW IMAP and seeing high I/O wait on the server. What, specifically, do you mean by "high I/O wait"? I have about 50 users. That is a very small facility that even modest server hardware should handle with aplomb. Mar 15 14:42:47 request1 imapd[1445]: Connection reset by peer, while reading line user=x host=f.q.d.n[11.22.33.44] This message simply means that a client disconnected from the server without first issuing a proper LOGOUT command, and that this was detected while the server was reading a new IMAP command line. The only time this message is at all interesting is if a user complains about being disconnected from the server. This message is evidence that, from the server's viewpoint, the client disconnected. This means that you can dismiss the server as a possible cause of the problem, and look elsewhere. When both client and server point fingers at each other, a common cause is an intermediate router (especially a NAT). Mar 15 14:42:49 request1 imapd[3250]: imap service init from 192.123.45.77 Mar 15 14:42:49 request1 imapd[3250]: Login user= host=x.y.z [192.31.97.142] Mar 15 14:42:52 request1 imapd[30847]: Logout user= host=c.d.e [1.2.3.4] None of these messages are remarkable. They are all indications of normal operation. Your /etc/xinetd.d/imap file looks OK as well My initial build was like this: make lr5 SSLTYPE=none Why are you using SSLTYPE=none? I hope that this is strictly for in-house use, and not exported over the Internet; as it will result in confidential traffic (including passwords) being sent in the clear. -- Mark -- http://panda.com/mrc Democracy is two wolves and a sheep deciding what to eat for lunch. Liberty is a well-armed sheep contesting the vote. ___ Imap-uw mailing list Imap-uw@u.washington.edu http://mailman2.u.washington.edu/mailman/listinfo/imap-uw ___ Imap-uw mailing list Imap-uw@u.washington.edu http://mailman2.u.washington.edu/mailman/listinfo/imap-uw
Re: [Imap-uw] slow IMAP connection
On Mon, 15 Mar 2010, Troy Campbell wrote: I recently installed the 2007e of UW IMAP and seeing high I/O wait on the server. What, specifically, do you mean by "high I/O wait"? I have about 50 users. That is a very small facility that even modest server hardware should handle with aplomb. Mar 15 14:42:47 request1 imapd[1445]: Connection reset by peer, while reading line user=x host=f.q.d.n[11.22.33.44] This message simply means that a client disconnected from the server without first issuing a proper LOGOUT command, and that this was detected while the server was reading a new IMAP command line. The only time this message is at all interesting is if a user complains about being disconnected from the server. This message is evidence that, from the server's viewpoint, the client disconnected. This means that you can dismiss the server as a possible cause of the problem, and look elsewhere. When both client and server point fingers at each other, a common cause is an intermediate router (especially a NAT). Mar 15 14:42:49 request1 imapd[3250]: imap service init from 192.123.45.77 Mar 15 14:42:49 request1 imapd[3250]: Login user= host=x.y.z [192.31.97.142] Mar 15 14:42:52 request1 imapd[30847]: Logout user= host=c.d.e [1.2.3.4] None of these messages are remarkable. They are all indications of normal operation. Your /etc/xinetd.d/imap file looks OK as well My initial build was like this: make lr5 SSLTYPE=none Why are you using SSLTYPE=none? I hope that this is strictly for in-house use, and not exported over the Internet; as it will result in confidential traffic (including passwords) being sent in the clear. -- Mark -- http://panda.com/mrc Democracy is two wolves and a sheep deciding what to eat for lunch. Liberty is a well-armed sheep contesting the vote. ___ Imap-uw mailing list Imap-uw@u.washington.edu http://mailman2.u.washington.edu/mailman/listinfo/imap-uw
[Imap-uw] slow IMAP connection
Hello,
I recently installed the 2007e of UW IMAP and seeing high I/O wait on the
server.
I have about 50 users. I'm seeing the following in the log files (hosts and
IP's are concocted):
Mar 15 14:42:47 request1 imapd[1445]: Connection reset by peer, while reading
line user=x host=f.q.d.n[11.22.33.44]
Mar 15 14:42:49 request1 imapd[3250]: imap service init from 192.123.45.77
Mar 15 14:42:49 request1 imapd[3250]: Login user= host=x.y.z [192.31.97.142]
Mar 15 14:42:52 request1 imapd[30847]: Logout user= host=c.d.e [1.2.3.4]
[r...@request1 xinetd.d]# cat /etc/xinetd.d/imap
# default: off
# description: The IMAP service allows remote users to access their mail using \
# an IMAP client such as Mutt, Pine, fetchmail, or Netscape \
# Communicator.
service imap
{
disable = no
socket_type = stream
wait= no
user= root
server = /usr/sbin/imapd
log_on_success += HOST DURATION
log_on_failure += HOST
instances = 500
}
My initial build was like this:
make lr5 SSLTYPE=none
Any help with the performance or log file errors is appreciated,
Troy
___
Imap-uw mailing list
Imap-uw@u.washington.edu
http://mailman2.u.washington.edu/mailman/listinfo/imap-uw
