Re: [Imap-uw] How to manage c-client memory management?
Joel Reicher wrote: Shawn Walker writes: I would say that Nokia N800 does not store all of you message on your phone. I can access my 15,000 IMAP folder with my iPhone with 16 GB of memory, but the iPhone does not does download all 15,000 messages, just the first 100 or whatever I have it configured it for. Why are you downloading any? http://tools.ietf.org/html/rfc1733 "No mailbox data are kept on the client; the client retrieves data from the server as is needed." Did you mean to say "caching"? Yes, caching and to allow the users to work/read their messages while offline. It's just not messages, but events, tasks, contacts, etc. And for anybody to be able to read their messages from the server, it has to be downloaded unless I'm missing something about how to get the messages from the server to the client. The users that we are dealing with will and always will download all 30,000, 100,000 1,000,000 messages to their computer. We cannot control how the user want to use the p roduct. Unless we really cripple how the product work. When Jon Postel said "...be liberal in what you accept from others" he probably did not have this kind of protocol abuse in mind. Force these non-IMAP users to use POP; that's what they're already doing. POP won't give the ability for the users use the IMAP server as the data storage for their messages. I'm not running a server for these users, just providing them a application to be able to groupware functionality of their data on the IMAP server. I also cannot tell users how they managed their data on the server. That's the admin's job that is maintaining those IMAP servers and set quotas to limit how much data the user can have on the server. And there are some admin that doesn't set a quota so the user has unlimited space to store their messages. Cheers, - Joel ___ Imap-uw mailing list Imap-uw@u.washington.edu http://mailman2.u.washington.edu/mailman/listinfo/imap-uw -- Shawn Walker Senior Software Developer Bynari, Inc. 6220 Gaston Ave, Suite 403 Dallas, Tx 75214 http://www.bynari.net swal...@bynari.net (800) 241-1086 (214) 350-5772 X29 (214) 352-3530 fax ___ Imap-uw mailing list Imap-uw@u.washington.edu http://mailman2.u.washington.edu/mailman/listinfo/imap-uw
Re: [Imap-uw] How to manage c-client memory management?
Shawn Walker writes: > I would say that Nokia N800 does not store all of you message on your phone. > I can access my 15,000 > IMAP folder with my iPhone with 16 GB of memory, but the iPhone does not does > download all 15,000 > messages, just the first 100 or whatever I have it configured it for. Why are you downloading any? http://tools.ietf.org/html/rfc1733 "No mailbox data are kept on the client; the client retrieves data from the server as is needed." Did you mean to say "caching"? > The users that we are dealing with will and always will download all 30,000, > 100,000 1,000,000 > messages to their computer. We cannot control how the user want to use the p > roduct. Unless we > really cripple how the product work. When Jon Postel said "...be liberal in what you accept from others" he probably did not have this kind of protocol abuse in mind. Force these non-IMAP users to use POP; that's what they're already doing. Cheers, - Joel ___ Imap-uw mailing list Imap-uw@u.washington.edu http://mailman2.u.washington.edu/mailman/listinfo/imap-uw
Re: [Imap-uw] How to manage c-client memory management?
On Tue, 14 Jul 2009, Shawn Walker wrote: There are companies that will not upgrade the computer regardless how cheap memory is today. I can't force the customers to buy more memory. Their choice is stark and brutal. Either they buy modern equipment to handle modern requirements, or they pay much more for the software work that will do a mediocre job of getting dinosaur equipment to do a half-assed job of the requirements. A 486 with 16MB memory and Windows 95 is not a suitable platform for dealing with 100,000 message mailboxes. And no, Outlook doesn't handle 100,000 message mailboxes on Windows 95. 14 year old computers are suitable for tasks of 14 years ago. They are not suitable for modern tasks. At today's netbook prices, there is no excuse to continue using an obsolete, power-wasting, dinosaur. Try saying that to the customers that won't budge on upgrading. We are only forced to have to deal with what the customers are using. Nothing can change that. If they are stupid enough to expect 100,000 message mailbox support on a 16MB Windows 95 machine, then they are stupid enough to pay large sums of money forever for software that will never work. 4GB of memory costs less than an espresso at Starbucks. However, you should not need anywhere near this. I regularly play with large mailboxes on a Nokia N800 which only has 128MB. I would say that Nokia N800 does not store all of you message on your phone. It is not a phone, it is an Internet tablet; and this is using Alpine. I can access my 15,000 IMAP folder with my iPhone with 16 GB of memory, but the iPhone does not does download all 15,000 messages, just the first 100 or whatever I have it configured it for. I am not talking about the iPhone client, or any other client written by people who insist upon doing things their way because they don't understand how IMAP should be used. I can access my 35,000 message IMAP mailbox with a Nokia N800 with 128MB RAM, with full scrolling threaded view and access to all messages and not just 100 messages. Think about the story of Columbus' egg. From your use of the word "download", you are not using the c-client library effectively. If you were, nobody would ever download 30,000 messages in a session, much less 100,000 messages. The users that we are dealing with will and always will download all 30,000, 100,000 1,000,000 messages to their computer. We cannot control how the user want to use the product. Unless we really cripple how the product work. Users don't think in terms of "downloading" messages. They think in terms of reading messages. Client authors think in terms of "downloading", mostly because they don't understand how else to enable a user to read their mail. The only reason that users need to "download" is if the client author is unable to envision any other technique for enabling the user to read mail. Downloading is most suitable for saving content (such as attachments) on the local machine. There are other, more suitable, techniques for other tasks. If the only tool that you ever use is a screwdriver, it may seem that all tasks are solved by using screwdrivers. Look at the Alpine source code for an example of how it is done properly. Using Alpine source to how we need to use IMAP is comparing apples and oranges. If we could control how the application is running then we could model after Alpine. Unfortunately, you are making your life much harder for yourself. I have given you as many clues as I can. Unfortunately, there is a limit to my free advice. -- Mark -- http://panda.com/mrc Science does not emerge from voting, party politics, or public debate. Si vis pacem, para bellum. ___ Imap-uw mailing list Imap-uw@u.washington.edu http://mailman2.u.washington.edu/mailman/listinfo/imap-uw
Re: [Imap-uw] How to manage c-client memory management?
Mark Crispin wrote: On Tue, 14 Jul 2009, Shawn Walker wrote: I'm trying OP_SHORTCACHE, but this seems to slow the download messages since now each message has to be fetched from the server. Indeed it does. If you want to use early 1990s memory models, you must suffer early 1990s performance problems. there are countries that have very low bandwidth (Australia has 180k) and to download huge files can take a long time. That is why you should cache. Aggressively. Memory is cheaper than bandwidth. There are companies that will not upgrade the computer regardless how cheap memory is today. I can't force the customers to buy more memory. Hence your quote above about 1990s memory models. The issue with calling mail_free_cache is that the "nmsgs" get reset back to 0 when I'm trying to get the header, envelope, body or attachment as the message is being downloaded or the user took some action that I need to get the text of the message or attachment. I did not see that you were using mail_free_cache(). That is an internal function that you should never call. The appropriate function, which is documented in internal.txt, is mail_gc(). I'll try that out. The issue is how the users is storing their messages on the server, some users has 30,000, 100,000 or some ridiculous amount of messages in a folder. Caching all of those messages consume the memory. Memory is cheaper than bandwidth. I routinely deal with mailboxes of that size. I would not think of slowing things down to save a few pennies of bandwidth. There are companies that will not upgrade the computer regardless how cheap memory is today. I can't force the customers to buy more memory. Hence your quote above about 1990s memory models. And not everybody has a computer that has 4 GB of memory, some people in eastern Europe is still using computers from the mid nineties running Windows 95! 14 year old computers are suitable for tasks of 14 years ago. They are not suitable for modern tasks. At today's netbook prices, there is no excuse to continue using an obsolete, power-wasting, dinosaur. Try saying that to the customers that won't budge on upgrading. We are only forced to have to deal with what the customers are using. Nothing can change that. Try running Outlook on that 100,000 message mailbox on that Windows 95 machine and see how well it does. 4GB of memory costs less than an espresso at Starbucks. However, you should not need anywhere near this. I regularly play with large mailboxes on a Nokia N800 which only has 128MB. I would say that Nokia N800 does not store all of you message on your phone. I can access my 15,000 IMAP folder with my iPhone with 16 GB of memory, but the iPhone does not does download all 15,000 messages, just the first 100 or whatever I have it configured it for. From your use of the word "download", you are not using the c-client library effectively. If you were, nobody would ever download 30,000 messages in a session, much less 100,000 messages. The users that we are dealing with will and always will download all 30,000, 100,000 1,000,000 messages to their computer. We cannot control how the user want to use the product. Unless we really cripple how the product work. Look at the Alpine source code for an example of how it is done properly. Using Alpine source to how we need to use IMAP is comparing apples and oranges. If we could control how the application is running then we could model after Alpine. -- Mark -- http://panda.com/mrc Science does not emerge from voting, party politics, or public debate. Si vis pacem, para bellum. -- Shawn Walker Senior Software Developer Bynari, Inc. 6220 Gaston Ave, Suite 403 Dallas, Tx 75214 http://www.bynari.net swal...@bynari.net (800) 241-1086 (214) 350-5772 X29 (214) 352-3530 fax ___ Imap-uw mailing list Imap-uw@u.washington.edu http://mailman2.u.washington.edu/mailman/listinfo/imap-uw
Re: [Imap-uw] How to manage c-client memory management?
On Tue, 14 Jul 2009, Shawn Walker wrote: I'm trying OP_SHORTCACHE, but this seems to slow the download messages since now each message has to be fetched from the server. Indeed it does. If you want to use early 1990s memory models, you must suffer early 1990s performance problems. there are countries that have very low bandwidth (Australia has 180k) and to download huge files can take a long time. That is why you should cache. Aggressively. Memory is cheaper than bandwidth. The issue with calling mail_free_cache is that the "nmsgs" get reset back to 0 when I'm trying to get the header, envelope, body or attachment as the message is being downloaded or the user took some action that I need to get the text of the message or attachment. I did not see that you were using mail_free_cache(). That is an internal function that you should never call. The appropriate function, which is documented in internal.txt, is mail_gc(). The issue is how the users is storing their messages on the server, some users has 30,000, 100,000 or some ridiculous amount of messages in a folder. Caching all of those messages consume the memory. Memory is cheaper than bandwidth. I routinely deal with mailboxes of that size. I would not think of slowing things down to save a few pennies of bandwidth. And not everybody has a computer that has 4 GB of memory, some people in eastern Europe is still using computers from the mid nineties running Windows 95! 14 year old computers are suitable for tasks of 14 years ago. They are not suitable for modern tasks. At today's netbook prices, there is no excuse to continue using an obsolete, power-wasting, dinosaur. Try running Outlook on that 100,000 message mailbox on that Windows 95 machine and see how well it does. 4GB of memory costs less than an espresso at Starbucks. However, you should not need anywhere near this. I regularly play with large mailboxes on a Nokia N800 which only has 128MB. From your use of the word "download", you are not using the c-client library effectively. If you were, nobody would ever download 30,000 messages in a session, much less 100,000 messages. Look at the Alpine source code for an example of how it is done properly. -- Mark -- http://panda.com/mrc Science does not emerge from voting, party politics, or public debate. Si vis pacem, para bellum. ___ Imap-uw mailing list Imap-uw@u.washington.edu http://mailman2.u.washington.edu/mailman/listinfo/imap-uw
Re: [Imap-uw] How to manage c-client memory management?
Mark Crispin wrote: On Mon, 13 Jul 2009, Shawn Walker wrote: I have an issue that I would like to tell c-client to flush it's cache since I don't want it to keep headers, body of the messages, etc for 20,000 messages since it take up a lot of memory. Either mail_free_cache() or use short caching (OP_SHORTCACHE mode of open). I'm trying OP_SHORTCACHE, but this seems to slow the download messages since now each message has to be fetched from the server. What I would like c-client to do is that it knows there are 30,000 messages in a folder. I request the first 100 messages, either that would be just the envelope or the entire message (depending on the user's configuration). After I have processed those 100 messages, I want c-client to release the memory of those messages. The reason why we have a configuration to allow users to do 'download headers only', 'only download message of the body', 'don't download attachments' and 'only download attachments if they are less than X bytes' is because there are countries that have very low bandwidth (Australia has 180k) and to download huge files can take a long time. mail_free_cache wouldn't be the ideal function to call since that wipe out the list of known messages on the server and cannot download any known messages that I want to download (headers only or just the body of the message or the attachments). I have no idea what you mean by this. I have repeatedly read what you wrote, and I can not make any sense of that combination of words. The issue with calling mail_free_cache is that the "nmsgs" get reset back to 0 when I'm trying to get the header, envelope, body or attachment as the message is being downloaded or the user took some action that I need to get the text of the message or attachment. Would be ideal that once I have what I wanted downloaded I want c-client to free all of the memory for that message before I go on to the next message. That sounds like short caching, but unless you are running on MS-DOS in 640K, short caching is a very bad idea. UW stopped using short caching about 10 years ago since it is so horribly inefficient. If you want to download without going through caching, you can set up a mailgets function for that purpose. If you just want to download messages to a local copy, why are you using IMAP at all? POP would be better for download. If you want to do interactive access, then you want full caching. The issue is how the users is storing their messages on the server, some users has 30,000, 100,000 or some ridiculous amount of messages in a folder. Caching all of those messages consume the memory. It would be nice if the users would managed their folders better, but we can't be dictating our customers how they should managed their folders. And not everybody has a computer that has 4 GB of memory, some people in eastern Europe is still using computers from the mid nineties running Windows 95! -- Mark -- http://panda.com/mrc Science does not emerge from voting, party politics, or public debate. Si vis pacem, para bellum. ___ Imap-uw mailing list Imap-uw@u.washington.edu http://mailman2.u.washington.edu/mailman/listinfo/imap-uw
Re: [Imap-uw] How to manage c-client memory management?
On Mon, 13 Jul 2009, Shawn Walker wrote: I have an issue that I would like to tell c-client to flush it's cache since I don't want it to keep headers, body of the messages, etc for 20,000 messages since it take up a lot of memory. Either mail_free_cache() or use short caching (OP_SHORTCACHE mode of open). mail_free_cache wouldn't be the ideal function to call since that wipe out the list of known messages on the server and cannot download any known messages that I want to download (headers only or just the body of the message or the attachments). I have no idea what you mean by this. I have repeatedly read what you wrote, and I can not make any sense of that combination of words. Would be ideal that once I have what I wanted downloaded I want c-client to free all of the memory for that message before I go on to the next message. That sounds like short caching, but unless you are running on MS-DOS in 640K, short caching is a very bad idea. UW stopped using short caching about 10 years ago since it is so horribly inefficient. If you want to download without going through caching, you can set up a mailgets function for that purpose. If you just want to download messages to a local copy, why are you using IMAP at all? POP would be better for download. If you want to do interactive access, then you want full caching. -- Mark -- http://panda.com/mrc Science does not emerge from voting, party politics, or public debate. Si vis pacem, para bellum. ___ Imap-uw mailing list Imap-uw@u.washington.edu http://mailman2.u.washington.edu/mailman/listinfo/imap-uw
[Imap-uw] How to manage c-client memory management?
I have an issue that I would like to tell c-client to flush it's cache since I don't want it to keep headers, body of the messages, etc for 20,000 messages since it take up a lot of memory. mail_free_cache wouldn't be the ideal function to call since that wipe out the list of known messages on the server and cannot download any known messages that I want to download (headers only or just the body of the message or the attachments). Would be ideal that once I have what I wanted downloaded I want c-client to free all of the memory for that message before I go on to the next message. Regards, Shawn ___ Imap-uw mailing list Imap-uw@u.washington.edu http://mailman2.u.washington.edu/mailman/listinfo/imap-uw
