Re: What does CLIENT BUG in result of STATUS operation mean?
On 04/16/2005 07:44:18 PM, Mike Schmidt wrote: Thanks Mark. When I look at the server logs for Thunderbird or Outlook, I never see more that one IMAP logon at a time. That also goes for all the users on our production server, although they are nearly all either Outlook or Thunderbird, with maybe an occasional Eudora thrown in for good measure. In my own case, on one account, I have maybe 30 or more mail folders, on another maybe 6 or 7. Yet I never see these IMAP clients logging on more than once simultaneously. Now, they don't use c-client as far as I can tell. Does this mean they are operating in a different way ? Something roughly equivalent to 1 connection, multiple mailstreams? There are many clients that open several concurrent connections. Mozilla/Thunderbird allows limiting number of concurrent IMAP connections (Server Settings/Advanced) and often uses just one by chosing to SELECT mailboxes frequently and doing extensive client-side caching. I guess the reason for this kind of model was that mozilla was designed to work efficiently in offline imap mode which obviously is not able to take advantage of many concurrent connections anyway. As Mark wrote, it's a tradoff. If you have relatively short-lived connections or your server's has some database-based backend with very fast SELECT operation, opening one/very few connections may be a good idea. If you need to interactively watch several mailboxes for modifications, keeping separate connections open may be better. (BTW, some imap servers allegedly limit number of concurrent connections for the same user). For the moment, I expect to use just an inbox and a sent folder, so I only have two mailboxes to monitor/operate, and that may not even be simultaneously. But of course I'm just getting started. I 'd rather not be obliged to re-design my system later, so I need to at least consider the implications. If this becomes an issue, is it possible/useful/interesting to separate the connection from the mailstream? IIRC, each c-client mailstream opens a separate connection. Pawel Or is uw- imap effective in this environment? (BTW, I have no intention of changing my mail server under any circumstances) unless forced to, and that would be kicking and screaming. Mark Crispin wrote: On Sat, 16 Apr 2005, Mike Schmidt wrote: Now I have an additional quick question, more a design question than anything else: if I understand correctly, I should have 1 mailstream open to an imap server, not one per mailbox. I should reuse this mailstream when switching mailboxes instead of making a new one, is that correct? What happens to the message cache in this case? It's a bit more complicated than that. You should have one MAILSTREAM open for each mailbox which you want to be open *simultaneously*. If you want to "switch" from one open mailbox to another, it is better to reuse the mailstream than to close the existing mailstream and open a new mailstream on the new name. With any of these operations, there are certain costs: Opening a new connection has the cost of TCP/IP session initiation, encryption negotiation, and authentication negotiation. Selecting a mailbox has the cost of server overhead to load/process the mailbox, and the protocol overload of loading the client cache. Closing a mailstream has the cost of discarding the message cache for the old name and closing the TCP/IP session. Opening a new mailstream has both the "opening a new connection" and the "selecting a mailbox" cost. Recycling a mailstream has the cost of discarding the message cache for the old name and "selecting a mailbox", but avoids the costs of closing the TCP/IP session and "opening a new connection". Keeping multiple sessions open allows you to monitor multiple mailboxes simultaneously, but there is a practical limit of how many mailboxes that you should have open at a time, no more than a handful. You want to do this if you want immediate real-time access to the mailbox and its messages, as opposed to passive monitoring. -- Mark -- http://staff.washington.edu/mrc Science does not emerge from voting, party politics, or public debate. Si vis pacem, para bellum.
Re: What does CLIENT BUG in result of STATUS operation mean?
Thanks Mark. When I look at the server logs for Thunderbird or Outlook, I never see more that one IMAP logon at a time. That also goes for all the users on our production server, although they are nearly all either Outlook or Thunderbird, with maybe an occasional Eudora thrown in for good measure. In my own case, on one account, I have maybe 30 or more mail folders, on another maybe 6 or 7. Yet I never see these IMAP clients logging on more than once simultaneously. Now, they don't use c-client as far as I can tell. Does this mean they are operating in a different way ? Something roughly equivalent to 1 connection, multiple mailstreams? For the moment, I expect to use just an inbox and a sent folder, so I only have two mailboxes to monitor/operate, and that may not even be simultaneously. But of course I'm just getting started. I 'd rather not be obliged to re-design my system later, so I need to at least consider the implications. If this becomes an issue, is it possible/useful/interesting to separate the connection from the mailstream? Or is uw-imap effective in this environment? (BTW, I have no intention of changing my mail server under any circumstances) unless forced to, and that would be kicking and screaming. Mark Crispin wrote: On Sat, 16 Apr 2005, Mike Schmidt wrote: Now I have an additional quick question, more a design question than anything else: if I understand correctly, I should have 1 mailstream open to an imap server, not one per mailbox. I should reuse this mailstream when switching mailboxes instead of making a new one, is that correct? What happens to the message cache in this case? It's a bit more complicated than that. You should have one MAILSTREAM open for each mailbox which you want to be open *simultaneously*. If you want to "switch" from one open mailbox to another, it is better to reuse the mailstream than to close the existing mailstream and open a new mailstream on the new name. With any of these operations, there are certain costs: Opening a new connection has the cost of TCP/IP session initiation, encryption negotiation, and authentication negotiation. Selecting a mailbox has the cost of server overhead to load/process the mailbox, and the protocol overload of loading the client cache. Closing a mailstream has the cost of discarding the message cache for the old name and closing the TCP/IP session. Opening a new mailstream has both the "opening a new connection" and the "selecting a mailbox" cost. Recycling a mailstream has the cost of discarding the message cache for the old name and "selecting a mailbox", but avoids the costs of closing the TCP/IP session and "opening a new connection". Keeping multiple sessions open allows you to monitor multiple mailboxes simultaneously, but there is a practical limit of how many mailboxes that you should have open at a time, no more than a handful. You want to do this if you want immediate real-time access to the mailbox and its messages, as opposed to passive monitoring. -- Mark -- http://staff.washington.edu/mrc Science does not emerge from voting, party politics, or public debate. Si vis pacem, para bellum.
Re: What does CLIENT BUG in result of STATUS operation mean?
On Sat, 16 Apr 2005, Mike Schmidt wrote: Now I have an additional quick question, more a design question than anything else: if I understand correctly, I should have 1 mailstream open to an imap server, not one per mailbox. I should reuse this mailstream when switching mailboxes instead of making a new one, is that correct? What happens to the message cache in this case? It's a bit more complicated than that. You should have one MAILSTREAM open for each mailbox which you want to be open *simultaneously*. If you want to "switch" from one open mailbox to another, it is better to reuse the mailstream than to close the existing mailstream and open a new mailstream on the new name. With any of these operations, there are certain costs: Opening a new connection has the cost of TCP/IP session initiation, encryption negotiation, and authentication negotiation. Selecting a mailbox has the cost of server overhead to load/process the mailbox, and the protocol overload of loading the client cache. Closing a mailstream has the cost of discarding the message cache for the old name and closing the TCP/IP session. Opening a new mailstream has both the "opening a new connection" and the "selecting a mailbox" cost. Recycling a mailstream has the cost of discarding the message cache for the old name and "selecting a mailbox", but avoids the costs of closing the TCP/IP session and "opening a new connection". Keeping multiple sessions open allows you to monitor multiple mailboxes simultaneously, but there is a practical limit of how many mailboxes that you should have open at a time, no more than a handful. You want to do this if you want immediate real-time access to the mailbox and its messages, as opposed to passive monitoring. -- Mark -- http://staff.washington.edu/mrc Science does not emerge from voting, party politics, or public debate. Si vis pacem, para bellum.
Re: What does CLIENT BUG in result of STATUS operation mean?
Thank you very much, Mark. This is exactly the information I was looking for. I am just learning to understand c-client, and, although I have read the various doc files many times, especially the internal.txt one, there are many aspects of the operational use of c-client that I do not yet understand. Your help is great, and I greatly appreciate your clear responses. Many of the commands I am issuing at this time are exploratory; since I don't yet understand the proper use of imap (I was only a user and server jock with respect to imapd), I am still exploring. I appreciate the easy callbacks that c-client provides for this kind of information because it means I can see (and display) all the imap command traffic in both directions. Hence the questions. I will continue asking questions because I am not capable of writing code I don't understand the why and wherefore of. I noticed there are a number of functions exposed in mail.h that are not documented in internal.txt. I don't want to bother you unnecessarily, but you may get questions about some of these functions from time to time, if they seem to be useful to my approach. Now I have an additional quick question, more a design question than anything else: if I understand correctly, I should have 1 mailstream open to an imap server, not one per mailbox. I should reuse this mailstream when switching mailboxes instead of making a new one, is that correct? What happens to the message cache in this case? Thanks for your quick response. You help is invaluable to me. Mike Mark Crispin wrote: On Sat, 16 Apr 2005, Mike Schmidt wrote: Can someone tell me what the CLIENT BUG references mean in the status report? It means that you did a STATUS command on the selected (opened) mailbox; a completely unnecessary and wasteful operation which could also have additional severe negative consequences. When you have a mailbox selected, the IMAP state already has all the data that a STATUS command could return. The STATUS command is to the be used only to get this data from a mailbox which is *NOT* selected. This has been shown to be an area which many novices fail to understand; many seem to believe (incorrectly) that you have to do a STATUS to get updated mailbox state. Learning how IMAP really works in this case is likely to help client authors understand IMAP in other cases, hence this is a mistake that I felt is important to correct. This is a transcript of the IMAP status operation. The client is based on c-client, running in Windows XP, and the server is uw-imapd running over a TLS connection (on Linux) If the client was written using c-client, then you did a mail_status() call on a mailbox name when you already had a perfectly good MAILSTREAM with all the data you need from a mail_open() call. The correct procedure is to get the data from the MAILSTREAM. For example, in your case of getting: (MESSAGES RECENT UNSEEN UIDNEXT UIDVALIDITY) you should use stream->nmsgs stream->recent either do a mail_search() for unseen and count the number of responses coming back, or if all message metadata is in c-client's cache just count the number of messages which have !mail_elt(stream,i)->seen, for i=1 to nmsgs. stream->uid_last+1 stream->uid_validity The simplest way to get the RECENT count is: mail_fetch_fast (stream,"1:*",NIL); for (i = 1, j = 0; i <= stream->nmsgs; ++i) if (!mail_elt (stream,i)->seen) ++j; However that mail_fetch_fast() should only be done once in any session. If you've already gotten all the message metadata once you don't need to get it again. IMAP automatically updates it for you. A STATUS command forces the IMAP server to do its open mail_open() call get the data from the resulting MAILSTREAM, then close it. But it already has done a mail_open() on that mailbox, as has your client. Hence the waste. The multiple opens can also cause other problems. Thanks very much. This certainly looks pretty confusing to me, but I' m just getting started writing a client using c-client. The message is obnoxious for a reason; it's to get the client author's attention and get him to ask "what's going on?" (and hopefully the explanation will convince him to do the right thing instead of abusing STATUS). So, in this case, it worked as designed. If there's anything unclear about any of the above, please ask. I'm not sure if the above answer adequately explains "why" as opposed to "what". Also, please don't feel bad; you have lots of company in having made this mistake. You responded intelligently as well (ask and find out what's wrong); too many people just try to hide the message and never fix what their client is doing wrong... :-( -- Mark -- http://staff.washington.edu/mrc Science does not emerge from voting, party politics, or public debate. Si vis pacem, para bellum.
Re: What does CLIENT BUG in result of STATUS operation mean?
On Sat, 16 Apr 2005, Mike Schmidt wrote: Can someone tell me what the CLIENT BUG references mean in the status report? It means that you did a STATUS command on the selected (opened) mailbox; a completely unnecessary and wasteful operation which could also have additional severe negative consequences. When you have a mailbox selected, the IMAP state already has all the data that a STATUS command could return. The STATUS command is to the be used only to get this data from a mailbox which is *NOT* selected. This has been shown to be an area which many novices fail to understand; many seem to believe (incorrectly) that you have to do a STATUS to get updated mailbox state. Learning how IMAP really works in this case is likely to help client authors understand IMAP in other cases, hence this is a mistake that I felt is important to correct. This is a transcript of the IMAP status operation. The client is based on c-client, running in Windows XP, and the server is uw-imapd running over a TLS connection (on Linux) If the client was written using c-client, then you did a mail_status() call on a mailbox name when you already had a perfectly good MAILSTREAM with all the data you need from a mail_open() call. The correct procedure is to get the data from the MAILSTREAM. For example, in your case of getting: (MESSAGES RECENT UNSEEN UIDNEXT UIDVALIDITY) you should use stream->nmsgs stream->recent either do a mail_search() for unseen and count the number of responses coming back, or if all message metadata is in c-client's cache just count the number of messages which have !mail_elt(stream,i)->seen, for i=1 to nmsgs. stream->uid_last+1 stream->uid_validity The simplest way to get the RECENT count is: mail_fetch_fast (stream,"1:*",NIL); for (i = 1, j = 0; i <= stream->nmsgs; ++i) if (!mail_elt (stream,i)->seen) ++j; However that mail_fetch_fast() should only be done once in any session. If you've already gotten all the message metadata once you don't need to get it again. IMAP automatically updates it for you. A STATUS command forces the IMAP server to do its open mail_open() call get the data from the resulting MAILSTREAM, then close it. But it already has done a mail_open() on that mailbox, as has your client. Hence the waste. The multiple opens can also cause other problems. Thanks very much. This certainly looks pretty confusing to me, but I' m just getting started writing a client using c-client. The message is obnoxious for a reason; it's to get the client author's attention and get him to ask "what's going on?" (and hopefully the explanation will convince him to do the right thing instead of abusing STATUS). So, in this case, it worked as designed. If there's anything unclear about any of the above, please ask. I'm not sure if the above answer adequately explains "why" as opposed to "what". Also, please don't feel bad; you have lots of company in having made this mistake. You responded intelligently as well (ask and find out what's wrong); too many people just try to hide the message and never fix what their client is doing wrong... :-( -- Mark -- http://staff.washington.edu/mrc Science does not emerge from voting, party politics, or public debate. Si vis pacem, para bellum.
What does CLIENT BUG in result of STATUS operation mean?
Hi, Can someone tell me what the CLIENT BUG references mean in the status report? This is a transcript of the IMAP status operation. The client is based on c-client, running in Windows XP, and the server is uw-imapd running over a TLS connection (on Linux): 4/16/2005 10:55:05 AM: 0005 STATUS INBOX (MESSAGES RECENT UNSEEN UIDNEXT UIDVALIDITY) 4/16/2005 10:55:12 AM: * NO CLIENT BUG DETECTED: STATUS on selected mailbox: INBOX 4/16/2005 10:55:21 AM: WARN - CLIENT BUG DETECTED: STATUS on selected mailbox: INBOX 4/16/2005 10:55:30 AM: * STATUS INBOX (MESSAGES 150 RECENT 0 UNSEEN 0 UIDNEXT 247 UIDVALIDITY 1108742917) 4/16/2005 10:56:53 AM: 0005 OK STATUS completed Thanks very much. This certainly looks pretty confusing to me, but I' m just getting started writing a client using c-client. Mike -- -- For information about this mailing list, and its archives, see: http://www.washington.edu/imap/c-client-list.html --
