Re: [courier-users] Server side sorting
Alessandro Vesely writes: Their deficiencies notwithstanding (e.g. FAMPending: timeout), file systems are way more mature than DBMS. Even writing in PHP or Python at times requires to consider the brand of the underlying database, let alone C/C++. And DBs make automating installation even harder than programming, IME. How much does that state of affairs condition current development? Well, you've made my argument for the IMAP server to be little more than a translator between IMAP and the underlying filesystem. For that task, the current state of affairs is that the IMAP server is doing a pretty good job. pgpcstTP02k_l.pgp Description: PGP signature -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot___ courier-users mailing list courier-users@lists.sourceforge.net Unsubscribe: https://lists.sourceforge.net/lists/listinfo/courier-users
Re: [courier-users] Server side sorting
On Wed 22/Mar/2017 18:47:10 +0100 Sam Varshavchik wrote: > Alessandro Vesely writes: > >> Some IMAP servers use indexed files too. Courier does not. What is the >> rationale behind that design choice? > > I expected – as I said – for clients to handle their own caching and indexing. > Indexing adds complexity. More code, more opportunities for bugs. Furthermore, > there is no preset recipe for indexing. IMAP allows the client to request, and > search, on any mail header, and on anything in the body of the email. There's > nothing obvious to index. One could take the approach of indexing common mail > headers; only to discover that one's own mail client doesn't search or request > them. One could take the approach of indexing all headers only to get a client > that caches everything itself, and thus never requests the same message twice; > so now you're doing a lot of work creating an index that will never be used. Yes. AFAIK, databases are not yet so smart as to cache required indexes on demand. They require well designed schemata to work with. > Another factor that the fact that maildirs are open to anyone. Anyone can come > in and add or remove messages from a maildir. Allowing for this immediately > increases the complexity of any indexing solution. It's one thing for an IMAP > server that maintains its own private mail store, and all access to the mail > has to go through the IMAP server. That makes it much easier to implement some > kind of indexing. It's no longer as straightforward when anyone can come in > and > simply delete the message, that you previously indexed. This means that even > if > you have an index, you still have to go and check that the message still > exists, before returning search results to the client. That, pretty much, > takes > back a good chunk one expected to gain, from indexing. That factor sounds questionable considering IMAP keyword implementation. A full-fledged addition of messages to a maildir had better depend on a proper MDA. Their deficiencies notwithstanding (e.g. FAMPending: timeout), file systems are way more mature than DBMS. Even writing in PHP or Python at times requires to consider the brand of the underlying database, let alone C/C++. And DBs make automating installation even harder than programming, IME. How much does that state of affairs condition current development? Ale -- signature.asc Description: OpenPGP digital signature -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot___ courier-users mailing list courier-users@lists.sourceforge.net Unsubscribe: https://lists.sourceforge.net/lists/listinfo/courier-users
Re: [courier-users] Server side sorting
Alessandro Vesely writes: Some IMAP servers use indexed files too. Courier does not. What is the rationale behind that design choice? I expected – as I said – for clients to handle their own caching and indexing. Indexing adds complexity. More code, more opportunities for bugs. Furthermore, there is no preset recipe for indexing. IMAP allows the client to request, and search, on any mail header, and on anything in the body of the email. There's nothing obvious to index. One could take the approach of indexing common mail headers; only to discover that one's own mail client doesn't search or request them. One could take the approach of indexing all headers only to get a client that caches everything itself, and thus never requests the same message twice; so now you're doing a lot of work creating an index that will never be used. Another factor that the fact that maildirs are open to anyone. Anyone can come in and add or remove messages from a maildir. Allowing for this immediately increases the complexity of any indexing solution. It's one thing for an IMAP server that maintains its own private mail store, and all access to the mail has to go through the IMAP server. That makes it much easier to implement some kind of indexing. It's no longer as straightforward when anyone can come in and simply delete the message, that you previously indexed. This means that even if you have an index, you still have to go and check that the message still exists, before returning search results to the client. That, pretty much, takes back a good chunk one expected to gain, from indexing. pgp5be9KjI78Q.pgp Description: PGP signature -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot___ courier-users mailing list courier-users@lists.sourceforge.net Unsubscribe: https://lists.sourceforge.net/lists/listinfo/courier-users
Re: [courier-users] Server side sorting
On Wed 22/Mar/2017 16:55:09 +0100 Sam Varshavchik wrote: > > A server is a shared resource. It never made any sense to me to offload as > much > processing as possible to the server. It makes more sense for most of the > processing to be done on the client side, with the server's role limited to > feeding the raw data to the client. Thunderbird client (apparently) features server-side searches. I never found a check-box to try server-side threading. Anyway, IME, client-side searches became reliable after the client sticked with maintaining an indexed database of messages. Some IMAP servers use indexed files too. Courier does not. What is the rationale behind that design choice? > There are more clients than there are servers. Clients, collectively have > more shared processing power. A CPU currently busy sorting some > knucklehead's ten year mail archive can't do anything else, for other > clients. That never made any sense, but that's how IMAP is overall designed, > to push as much processing to the server. Sometimes it makes sense to use disposable clients... > And, of course, it's much easier for some hacked-together IMAP-over-web client > to send a single command and parse the response, than to do the job by itself. Yes. Ale -- signature.asc Description: OpenPGP digital signature -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot___ courier-users mailing list courier-users@lists.sourceforge.net Unsubscribe: https://lists.sourceforge.net/lists/listinfo/courier-users
Re: [courier-users] Server side sorting
Michelle Konzack writes: Hello Sam, On 2017-03-22 06:35:16 Sam Varshavchik hacked into the keyboard: > The IMAP command is THREAD REFERENCES. OK, this is fine since I can setup ANY commands. > The response consists of > message numbers, with parenthesis indicating various threads and > subthreads: > > a THREAD REFERENCES UTF-8 ALL > * THREAD (1)(2)(5 3 4)(6 7)(8)(9)(10 11 12)(13)(14)(15 16 17)(18)(19) > (20 22)(21)(23 (24)(25))(26)(27)(28)(29)(30)(31)(32)(33)((34)(37)(39)) > (35)(36)(38)(40)(41)((42)(43))(44)(45 46)(47 48)(49 50 ((51)(52))) > a OK THREAD done. Hmm, question: If I understand it right, the a) (1) mean a singel messages but what does b) (5 3 4) mean? This would look like the message 5 came before 3 and 4 and then Message numbers are assigned to messages the first time they're seen. If the IMAP server hasn't logged on for a while and is now seeing a bunch of messages for the first time, the order in which the files get read from the directory may not necessarily match the order they were delivered to. So the server may see a reply before the original message, and the REFERENCES sort will rearrange them in chronological order. c) (23 (24)(25)) d) ((34)(37)(39)) e) (49 50 ((51)(52))) which look very courious to me. > The complete specification is a somewhat of a big pill to swallow. > See https://tools.ietf.org/html/rfc5256 It seems I have to suck it! Yes. This is one of the more …involved parts of IMAP. There's a lot of history and legacy involved. It's my understanding that some of the original actors have suffered health problems in recent past; so I don't want to say anything on that account. But I'll say this. I believe that server-side sorting was a mistake. The most sensible usage model for IMAP is for the client to sync and cache with the server. An IMAP client should sort and thread messages using its cached message metadata and don't hassle the server with it. A server is a shared resource. It never made any sense to me to offload as much processing as possible to the server. It makes more sense for most of the processing to be done on the client side, with the server's role limited to feeding the raw data to the client. There are more clients than there are servers. Clients, collectively have more shared processing power. A CPU currently busy sorting some knucklehead's ten year mail archive can't do anything else, for other clients. That never made any sense, but that's how IMAP is overall designed, to push as much processing to the server. And, of course, it's much easier for some hacked-together IMAP-over-web client to send a single command and parse the response, than to do the job by itself. pgptBsBdKg4mw.pgp Description: PGP signature -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot___ courier-users mailing list courier-users@lists.sourceforge.net Unsubscribe: https://lists.sourceforge.net/lists/listinfo/courier-users
Re: [courier-users] Server side sorting
Hello Sam, On 2017-03-22 06:35:16 Sam Varshavchik hacked into the keyboard: > The IMAP command is THREAD REFERENCES. OK, this is fine since I can setup ANY commands. > The response consists of > message numbers, with parenthesis indicating various threads and > subthreads: > > a THREAD REFERENCES UTF-8 ALL > * THREAD (1)(2)(5 3 4)(6 7)(8)(9)(10 11 12)(13)(14)(15 16 17)(18)(19) > (20 22)(21)(23 (24)(25))(26)(27)(28)(29)(30)(31)(32)(33)((34)(37)(39)) > (35)(36)(38)(40)(41)((42)(43))(44)(45 46)(47 48)(49 50 ((51)(52))) > a OK THREAD done. Hmm, question: If I understand it right, the a) (1) mean a singel messages but what does b) (5 3 4) mean? This would look like the message 5 came before 3 and 4 and then c) (23 (24)(25)) d) ((34)(37)(39)) e) (49 50 ((51)(52))) which look very courious to me. > The complete specification is a somewhat of a big pill to swallow. > See https://tools.ietf.org/html/rfc5256 It seems I have to suck it! -- Michelle KonzackMiila ITSystems @ TDnet GNU/Linux Developer 00372-54541400 signature.asc Description: Digital signature -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot___ courier-users mailing list courier-users@lists.sourceforge.net Unsubscribe: https://lists.sourceforge.net/lists/listinfo/courier-users
Re: [courier-users] Server side sorting
Michelle Konzack writes: But how can I do thread sorting? The squirrelmail sourcecode is really weird and I do not understand how it works. Question: Can I use for this the "IMAP CAPABILITY" server side sorting? I access the imap account with php5 and php-imap. The IMAP command is THREAD REFERENCES. The response consists of message numbers, with parenthesis indicating various threads and subthreads: a THREAD REFERENCES UTF-8 ALL * THREAD (1)(2)(5 3 4)(6 7)(8)(9)(10 11 12)(13)(14)(15 16 17)(18)(19)(20 22) (21)(23 (24)(25))(26)(27)(28)(29)(30)(31)(32)(33)((34)(37)(39))(35)(36)(38) (40)(41)((42)(43))(44)(45 46)(47 48)(49 50 ((51)(52))) a OK THREAD done. The complete specification is a somewhat of a big pill to swallow. See https://tools.ietf.org/html/rfc5256 pgp7tWVIXdk_9.pgp Description: PGP signature -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot___ courier-users mailing list courier-users@lists.sourceforge.net Unsubscribe: https://lists.sourceforge.net/lists/listinfo/courier-users
Re: [courier-users] Server side sorting [SOLVED]
Am Samstag, 10. März 2007 02:38 schrieb Sam Varshavchik: Each IMAP message has two timestamps associated with it: the message arrival time, and the contents of the Date: header. A mail client can request either one to be used as a sort key. If Squirrelmail sorts messages using the Date: header when sorting it by itself, it should also request that the IMAP server do the same. Otherwise, report this as a Squirrelmail bug. Good to know! Thanks, Arne -- Dipl.-Inform. Arne Schmitz Phone +49 (0)241 80-21817 Computer Graphics Group Fax +49 (0)241 80-22899 RWTH Aachen University http://www.rwth-graphics.de Ahornstrasse 55, 52074 Aachen, Germany pgpaYtd73Aq62.pgp Description: PGP signature - Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT business topics through brief surveys-and earn cash http://www.techsay.com/default.php?page=join.phpp=sourceforgeCID=DEVDEV___ courier-users mailing list courier-users@lists.sourceforge.net Unsubscribe: https://lists.sourceforge.net/lists/listinfo/courier-users
Re: [courier-users] Server side sorting
Am Dienstag, 6. März 2007 11:37 schrieb Arne Schmitz: I am using courier-imap 4.1.x and I am having problems with server side sorting. In squirrelmail mails are sorted wrong if I sort them by date. If squirrelmail sorts the mails by itself, it works, but server side sorting fails. What could be the problem? No ideas for this problem? Arne -- Dipl.-Inform. Arne Schmitz Phone +49 (0)241 80-21817 Computer Graphics Group Fax +49 (0)241 80-22899 RWTH Aachen University http://www.rwth-graphics.de Ahornstrasse 55, 52074 Aachen, Germany pgp8lIXQ7ajID.pgp Description: PGP signature - Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT business topics through brief surveys-and earn cash http://www.techsay.com/default.php?page=join.phpp=sourceforgeCID=DEVDEV___ courier-users mailing list courier-users@lists.sourceforge.net Unsubscribe: https://lists.sourceforge.net/lists/listinfo/courier-users
Re: [courier-users] Server side sorting [SOLVED]
Am Freitag, 9. März 2007 09:26 schrieb Arne Schmitz: Am Dienstag, 6. März 2007 11:37 schrieb Arne Schmitz: I am using courier-imap 4.1.x and I am having problems with server side sorting. In squirrelmail mails are sorted wrong if I sort them by date. If squirrelmail sorts the mails by itself, it works, but server side sorting fails. What could be the problem? No ideas for this problem? Now I know -- Courier uses the file modification time for sorting. I have migrated emails to Courier, so now they have new file modifications times. Because of that, the sorting is garbled. Nice to know! Also, new emails get sorted correctly. :^) Cheers, Arne -- Dipl.-Inform. Arne Schmitz Phone +49 (0)241 80-21817 Computer Graphics Group Fax +49 (0)241 80-22899 RWTH Aachen University http://www.rwth-graphics.de Ahornstrasse 55, 52074 Aachen, Germany pgpqOioYafCsU.pgp Description: PGP signature - Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT business topics through brief surveys-and earn cash http://www.techsay.com/default.php?page=join.phpp=sourceforgeCID=DEVDEV___ courier-users mailing list courier-users@lists.sourceforge.net Unsubscribe: https://lists.sourceforge.net/lists/listinfo/courier-users
Re: [courier-users] Server side sorting [SOLVED]
Am Freitag, 9. März 2007 15:41 schrieben Sie: Now I know -- Courier uses the file modification time for sorting. I have migrated emails to Courier, so now they have new file modifications times. The file modification times shouldn't have changed in the first place. Yes, but I copied from one IMAP server to another, using my Email program. So I did not have access to the original files, only via IMAP. Arne -- Dipl.-Inform. Arne Schmitz Phone +49 (0)241 80-21817 Computer Graphics Group Fax +49 (0)241 80-22899 RWTH Aachen University http://www.rwth-graphics.de Ahornstrasse 55, 52074 Aachen, Germany pgpo3tmfFfzKw.pgp Description: PGP signature - Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT business topics through brief surveys-and earn cash http://www.techsay.com/default.php?page=join.phpp=sourceforgeCID=DEVDEV___ courier-users mailing list courier-users@lists.sourceforge.net Unsubscribe: https://lists.sourceforge.net/lists/listinfo/courier-users
Re: [courier-users] Server side sorting [SOLVED]
Am Freitag, 9. März 2007 15:40 schrieben Sie: just curious: what happens if you migrate the emails as follows: cp -p or cd /old tar cf - . | ( cd /new | tar xvf - ) ? Haven't tested, but it should preserve the modification time. As I said in another post, I did not have access to the original files when migrating. I fetched them via IMAP. Arne -- Dipl.-Inform. Arne Schmitz Phone +49 (0)241 80-21817 Computer Graphics Group Fax +49 (0)241 80-22899 RWTH Aachen University http://www.rwth-graphics.de Ahornstrasse 55, 52074 Aachen, Germany pgpsBgOsjc2rC.pgp Description: PGP signature - Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT business topics through brief surveys-and earn cash http://www.techsay.com/default.php?page=join.phpp=sourceforgeCID=DEVDEV___ courier-users mailing list courier-users@lists.sourceforge.net Unsubscribe: https://lists.sourceforge.net/lists/listinfo/courier-users
Re: [courier-users] Server side sorting [SOLVED]
Arne Schmitz wrote: Now I know -- Courier uses the file modification time for sorting. I have migrated emails to Courier, so now they have new file modifications times. Because of that, the sorting is garbled. Nice to know! Also, new emails get sorted correctly. :^) If it bugs you, a handful of scripts circulated, a couple of years ago, that could fix the mtime of maildir files. I put one here: http://phantom.dragonsdawn.net/~gordon/scripts/fix-maildir-mtime.py Google groups could probably find the others. - Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT business topics through brief surveys-and earn cash http://www.techsay.com/default.php?page=join.phpp=sourceforgeCID=DEVDEV ___ courier-users mailing list courier-users@lists.sourceforge.net Unsubscribe: https://lists.sourceforge.net/lists/listinfo/courier-users
Re: [courier-users] Server side sorting [SOLVED]
Arne Schmitz writes: Am Freitag, 9. März 2007 15:41 schrieben Sie: Now I know -- Courier uses the file modification time for sorting. I have migrated emails to Courier, so now they have new file modifications times. The file modification times shouldn't have changed in the first place. Yes, but I copied from one IMAP server to another, using my Email program. So Your E-mail program is broken. Report it as a bug. I did not have access to the original files, only via IMAP. Irrelevant. Your E-mail program, when copying each individual message, should preserve the message's arrival timestamp. This has been fully supported in IMAP, right from the beginning. pgpw1TsuC1wxl.pgp Description: PGP signature - Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT business topics through brief surveys-and earn cash http://www.techsay.com/default.php?page=join.phpp=sourceforgeCID=DEVDEV___ courier-users mailing list courier-users@lists.sourceforge.net Unsubscribe: https://lists.sourceforge.net/lists/listinfo/courier-users
Re: [courier-users] Server side sorting [SOLVED]
Arne Schmitz writes: Am Freitag, 9. März 2007 09:26 schrieb Arne Schmitz: Am Dienstag, 6. März 2007 11:37 schrieb Arne Schmitz: I am using courier-imap 4.1.x and I am having problems with server side sorting. In squirrelmail mails are sorted wrong if I sort them by date. If squirrelmail sorts the mails by itself, it works, but server side sorting fails. What could be the problem? No ideas for this problem? Now I know -- Courier uses the file modification time for sorting. No, it doesn't. Courier uses whatever the mail client asks it to use: either the file modification time, or the contents of the Date: header. Each IMAP message has two timestamps associated with it: the message arrival time, and the contents of the Date: header. A mail client can request either one to be used as a sort key. If Squirrelmail sorts messages using the Date: header when sorting it by itself, it should also request that the IMAP server do the same. Otherwise, report this as a Squirrelmail bug. pgpDV3qCa3DzQ.pgp Description: PGP signature - Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT business topics through brief surveys-and earn cash http://www.techsay.com/default.php?page=join.phpp=sourceforgeCID=DEVDEV___ courier-users mailing list courier-users@lists.sourceforge.net Unsubscribe: https://lists.sourceforge.net/lists/listinfo/courier-users
