Re: [courier-users] maildirsize race condition and removal of the maildirsize file
Sam Varshavchik mr...@courier-mta.com (Sat Mar 12 17:57:45 2011): 1. If maildirsize *does not exist*, or if its size is at least 5120 bytes, recalculate it using the procedure defined above, and use the recalculated numbers. Otherwise, read the contents of maildirsize, and add up the totals. Therefore, after maildirsize is removed, it gets recreated from scratch at the next opportunity to do so. Because it does not exist. Another thing, note that if the maildirsize does not exist, this means that no quota is defined for that maildir, so an implementation cannot really remove the maildirsize for a prolonged interval, because that disables the quota mechanism completely. This is fine, as stated the quota mechanism is not 100% foolproof, but a tradeoff to keep the overall overhead to an absolute minimum, while doing a reasonable guesstimate most of the time. I do not see an interoperability issue with either approach, whether the maildirsize file gets removed, then recreated from scratch, or whether it is recreated without being removed. I'll add a note that either alternative is acceptable. I agree, if the remover knows the first line, it can be recreated later, by the remover. If I understood the exim implementation, it just removed and that's it. The quota for a later recreation comes from the configuration file. Having no quota for a limited frame of time would be no problem, but loosing the quota at all, definitivly is a problem. Ok, if I understand well, Courier always relies on an existing quota file and never ever will exit w/o creating a new one. But if no quota file is found, no quota will apply. The userdb option quota is not used at all, is it? -- Heiko signature.asc Description: Digital signature -- Colocation vs. Managed Hosting A question and answer guide to determining the best fit for your organization - today and in the future. http://p.sf.net/sfu/internap-sfd2d___ courier-users mailing list courier-users@lists.sourceforge.net Unsubscribe: https://lists.sourceforge.net/lists/listinfo/courier-users
Re: [courier-users] maildirsize race condition and removal of the maildirsize file
Heiko Schlittermann writes: If I understood the exim implementation, it just removed and that's it. The quota for a later recreation comes from the configuration file. Having no quota for a limited frame of time would be no problem, but loosing the quota at all, definitivly is a problem. Since exim is the one that's removing the file, and exim is the one that's doing the delivery, then after removing the file the next delivery attempt will result in exim recalculate the quota, from scratch. There's nothing wrong with that approach. Ok, if I understand well, Courier always relies on an existing quota file and never ever will exit w/o creating a new one. But if no quota file is found, no quota will apply. The userdb option quota is not used at all, is it? It's there to be used. The standalone build of maildrop, if the courier- authlib hook is used, maildrop should end up retrieving the set quota from courier-authlib. The fallback is to use MAILDIRQUOTA from the environment. Courier itself will use the quota, whether it comes from userdb or from somewhere else. If Courier runs an external mail delivery agent, it sets MAILDIRQUOTA and leaves it up to the external mail delivery agent to use it. pgpAkIkkLQap2.pgp Description: PGP signature -- Colocation vs. Managed Hosting A question and answer guide to determining the best fit for your organization - today and in the future. http://p.sf.net/sfu/internap-sfd2d___ courier-users mailing list courier-users@lists.sourceforge.net Unsubscribe: https://lists.sourceforge.net/lists/listinfo/courier-users
Re: [courier-users] maildirsize race condition and removal of the maildirsize file
Sam Varshavchik mr...@courier-mta.com (Sun Mar 13 18:00:53 2011): Heiko Schlittermann writes: If I understood the exim implementation, it just removed and that's it. The quota for a later recreation comes from the configuration file. Having no quota for a limited frame of time would be no problem, but loosing the quota at all, definitivly is a problem. Since exim is the one that's removing the file, and exim is the one that's doing the delivery, then after removing the file the next delivery attempt will result in exim recalculate the quota, from scratch. There's nothing wrong with that approach. You're right. If It wasn't the last delivery for a long period of time AND if the mailbox filled mainly by the delivery agent. But partly we have other use patterns - usage of the mailbox mainly for IMAP STORE, IMAP COPY. Thus, if the maildirsize file is lost, than no quota is applied anymore. Ok, if I understand well, Courier always relies on an existing quota file and never ever will exit w/o creating a new one. But if no quota file is found, no quota will apply. The userdb option quota is not used at all, is it? It's there to be used. The standalone build of maildrop, if the courier-authlib hook is used, maildrop should end up retrieving the set quota from courier-authlib. The fallback is to use MAILDIRQUOTA from the environment. Ah, ok. But I'm not talking about the delivery (it's done by exim for various reasons), I'm talking about the IMAP/POP3 part of Courier. Courier itself will use the quota, whether it comes from userdb or from somewhere else. If Courier runs an external mail delivery agent, it sets MAILDIRQUOTA and leaves it up to the external mail delivery agent to use it. It looks as if the imapd/popd never gets passed the quota setting from the userdb. Seems to be the locical conclusion, if we suppose, that Courier will find the current active quota settings in the first line of the maildirsize file. -- Heiko :: dresden : linux : SCHLITTERMANN.de GPG Key 48D0359B : 3061 CFBF 2D88 F034 E8D2 7E92 EE4E AC98 48D0 359B signature.asc Description: Digital signature -- Colocation vs. Managed Hosting A question and answer guide to determining the best fit for your organization - today and in the future. http://p.sf.net/sfu/internap-sfd2d___ courier-users mailing list courier-users@lists.sourceforge.net Unsubscribe: https://lists.sourceforge.net/lists/listinfo/courier-users
Re: [courier-users] maildirsize race condition and removal of the maildirsize file
Heiko Schlittermann writes: Courier itself will use the quota, whether it comes from userdb or from somewhere else. If Courier runs an external mail delivery agent, it sets MAILDIRQUOTA and leaves it up to the external mail delivery agent to use it. It looks as if the imapd/popd never gets passed the quota setting from the userdb. Seems to be the locical conclusion, if we suppose, that Courier will find the current active quota settings in the first line of the maildirsize file. That does look like the issue here. There is code in imapd that reads and uses MAILDIRQUOTA from the environment, if it's set, but there's nothing that explicitly sets it; it predates the current authentication architecture where this was done differently. But I do see that the quota data should be available to imaplogin. This should be fixable, I'll look into it. pgpGjNzq8u4eu.pgp Description: PGP signature -- Colocation vs. Managed Hosting A question and answer guide to determining the best fit for your organization - today and in the future. http://p.sf.net/sfu/internap-sfd2d___ courier-users mailing list courier-users@lists.sourceforge.net Unsubscribe: https://lists.sourceforge.net/lists/listinfo/courier-users
Re: [courier-users] maildirsize race condition and removal of the maildirsize file
Sam Varshavchik mr...@courier-mta.com (Sun Mar 13 20:34:33 2011):
Heiko Schlittermann writes:
Courier itself will use the quota, whether it comes from userdb or
from somewhere else. If Courier runs an external mail delivery
agent, it sets MAILDIRQUOTA and leaves it up to the external mail
delivery agent to use it.
It looks as if the imapd/popd never gets passed the quota setting from
the userdb. Seems to be the locical conclusion, if we suppose, that
Courier will find the current active quota settings in the first line of
the maildirsize file.
That does look like the issue here. There is code in imapd that
reads and uses MAILDIRQUOTA from the environment, if it's set, but
there's nothing that explicitly sets it; it predates the current
authentication architecture where this was done differently.
But I do see that the quota data should be available to imaplogin.
This should be fixable, I'll look into it.
I did it, but unfortunely restructred the source a bit…
diff --git a/imap/imaplogin.c b/imap/imaplogin.c
--- a/imap/imaplogin.c
+++ b/imap/imaplogin.c
@@ -211,36 +211,33 @@
{
p=malloc(sizeof(OPTIONS=) + strlen(ainfo-options ?
ainfo-options:));
+ if (!p) return(rc);
+ strcat(strcpy(p, OPTIONS=), ainfo-options ?
ainfo-options:);
+ putenv(p);
- if (p)
- {
- strcat(strcpy(p, OPTIONS=),
- ainfo-options ? ainfo-options:);
+ p=malloc(sizeof(IMAPLOGINTAG=) + strlen(tag));
+ if (!p) return(rc);
+ strcat(strcpy(p, IMAPLOGINTAG=), tag);
+ putenv(p);
+
+ p=malloc(sizeof(AUTHENTICATED=) + strlen(ainfo-address));
+ if (!p) return(rc);
+ strcat(strcpy(p, AUTHENTICATED=), ainfo-address);
+ putenv(p);
+
+ if (ainfo-quota) {
+ p=malloc(sizeof(MAILDIRQUOTA=) +
strlen(ainfo-quota));
+ if (!p) return(rc);
+ strcat(strcpy(p, MAILDIRQUOTA=), ainfo-quota);
putenv(p);
+ }
- p=malloc(sizeof(IMAPLOGINTAG=)+strlen(tag));
- if (p)
- {
- strcat(strcpy(p, IMAPLOGINTAG=), tag);
- putenv(p);
-
- p=malloc(sizeof(AUTHENTICATED=)+
-strlen(ainfo-address));
- if (p)
- {
- strcat(strcpy(p, AUTHENTICATED=),
- ainfo-address);
- putenv(p);
- alarm(0);
- execl(imapd, imapd,
- ainfo-maildir ?
- ainfo-maildir:defaultmaildir,
- NULL);
- fprintf(stderr, ERR: exec(%s)
failed!!\n,
-imapd);
- }
- }
- }
+ alarm(0);
+ execl(imapd, imapd,
+ ainfo-maildir ?
+ ainfo-maildir:defaultmaildir,
+ NULL);
+ fprintf(stderr, ERR: exec(%s) failed!!\n, imapd);
}
return(rc);
diff --git a/imap/pop3login.c b/imap/pop3login.c
--- a/imap/pop3login.c
+++ b/imap/pop3login.c
@@ -192,32 +192,31 @@
if (rc == 0)
{
- char *p=malloc(sizeof(OPTIONS=) + strlen(ainfo-options ?
- ainfo-options:));
+ char *p;
- if (p)
- {
- strcat(strcpy(p, OPTIONS=),
- ainfo-options ? ainfo-options:);
+ p = malloc(sizeof(OPTIONS=) + strlen(ainfo-options ?
ainfo-options:));
+ if (!p) return(rc);
+ strcat(strcpy(p, OPTIONS=), ainfo-options ?
ainfo-options:);
+ putenv(p);
+
+ p = malloc(sizeof(AUTHENTICATED=) + strlen(ainfo-address));
+ if (!p) return(rc);
+ strcat(strcpy(p, AUTHENTICATED=), ainfo-address);
+ putenv(p);
+
+ if (ainfo-quota) {
+ p = malloc(sizeof(MAILDIRQUOTA=) +
strlen(ainfo-quota));
+ if (!p) return(rc);
+ strcat(strcpy(p, MAILDIRQUOTA=), ainfo-quota);
putenv(p);
+ }
-
[courier-users] (fwd)
This is a development build of _courier_, _courier-imap_, _sqwebmail_, _maildrop_, and _cone_ packages. Download: http://www.courier-mta.org/download.php The custom implementation of unicode-based character set mapping tables has been replaced by an iconv(3)-based implementation. This release also includes a major upgrade to sqwebmail that includes better support for flowed-format text, and a new search function. == Backwards compatibility and upgrade notes * The format of the autoreplies has changed. Autoreply content must now use flowed-text markup. This affects mailbot and the equivalent autoreply feature in sqwebmail. Existing mailbot autoreply files need to be edit and reformatted as flowed text. Ditto for autoreplies set up in sqwebmail. They need to be opened, and edited so that the only linebreaks occur between paragraphs. Additionally, the autoreply text in sqwebmail is now saved in UTF-8; existing autoreplies may need to be transcoded manually. * When using sqwebmail to generate HTML text, the markup for generating text headings has slightly changed. * Saved message signatures are now also saved in the UTF-8 character set, to get more consistent behavior when switching browser codings. Existing message signatures may have to be recoded. * Site-specific message footer files, in the html/LANG directory, must now be coded in UTF-8, and use the format=flowed, delsp=yes text format, as per RFC 3676. * Slight semantical difference in processing of signatures and footers. A signature, if specified, will no longer appear in the editable text area, but get automatically added to the message, upon submission. The signature will also be shown when previewing the message. * A site-specific message footer file, if installed, will also be shown when previewing the message. == Other, general changes: * sqwebmail and Cone can now process mail coded in any character set that's supported by the system iconv(3) library, not just a few dozen character sets that used to be built into the codebase. This is true, of course, only if the browser or the terminal display supports UTF-8. Similarly Courier- IMAP now accepts search queries encoded in any character set known to iconv(3). * This is due to the replacement of the internal character set and unicode library, that predates the standardization of iconv(3). The new iconv(3)- based API is completely different than the previous library API. The only remaining bits of code are mapping function that case-convert character data, as well as compute character width (there's no analogous functionality in iconv(3)). This is augmented by new implementation of unicode grapheme breaking and line breaking algorithms. * Translating and mapping between different character sets is core functionality; from converting email content, in sqwebmail and Cone, to client's character set, to parsing i18n data from various mail headers (which have several different character set and language encoding formats as well), to searching and sorting messages, by sqwebmail, Cone, or the IMAP server. Most the impact from the rewrite should hopefully go unnoticed, aside from the larger character set universe known to sqwebmail, Cone, and Courier-IMAP. * There's a new search box at the bottom of the folder index page. sqwebmail searches the messages in the order of their appearance on the folder index screen. The search starts with the first message on the folder index screen. If the folder index screen does not start with the first message in the folder, the previous messages are not searched, and the search stops when enough messages with the search string are found to fill one folder index screen. In this manner, a poorly chosen search string that finds a hit in a lot of messages will limit itself. If necessary, the search can be repeated after navigating the folder index screen to the starting point for the second search. * Up to four occurrences of the search string in each message are shown on the search results page. * Word wrapping rules in sqwebmail now use unicode line break rules, rather than simple word wrapping on whitespace. This should improve sqwebmail's word wrap with non-Western languages. * Suppress logging the contents of a failed AUTH command in syslog, in case it includes encoded passwords. * Fix requirements of rpm spec files, to depend on /etc/cron.hourly, rather than a specific cron package. * Adjust indentation to avoid parts of DSNs unintentionally interpreted as quoted content. * mailbot now generates flowed text-formatted replies. -- pgpifw22R6efa.pgp Description: PGP signature -- Colocation vs. Managed Hosting A question and answer guide to determining the best fit for your organization - today and in the future.
[courier-users] New development builds of Courier packages
This is a development build of _courier_, _courier-imap_, _sqwebmail_, _maildrop_, and _cone_ packages. Download: http://www.courier-mta.org/download.php The custom implementation of unicode-based character set mapping tables has been replaced by an iconv(3)-based implementation. This release also includes a major upgrade to sqwebmail that includes better support for flowed-format text, and a new search function. == Backwards compatibility and upgrade notes * The format of the autoreplies has changed. Autoreply content must now use flowed-text markup. This affects mailbot and the equivalent autoreply feature in sqwebmail. Existing mailbot autoreply files need to be edit and reformatted as flowed text. Ditto for autoreplies set up in sqwebmail. They need to be opened, and edited so that the only linebreaks occur between paragraphs. Additionally, the autoreply text in sqwebmail is now saved in UTF-8; existing autoreplies may need to be transcoded manually. * When using sqwebmail to generate HTML text, the markup for generating text headings has slightly changed. * Saved message signatures are now also saved in the UTF-8 character set, to get more consistent behavior when switching browser codings. Existing message signatures may have to be recoded. * Site-specific message footer files, in the html/LANG directory, must now be coded in UTF-8, and use the format=flowed, delsp=yes text format, as per RFC 3676. * Slight semantical difference in processing of signatures and footers. A signature, if specified, will no longer appear in the editable text area, but get automatically added to the message, upon submission. The signature will also be shown when previewing the message. * A site-specific message footer file, if installed, will also be shown when previewing the message. == Other, general changes: * sqwebmail and Cone can now process mail coded in any character set that's supported by the system iconv(3) library, not just a few dozen character sets that used to be built into the codebase. This is true, of course, only if the browser or the terminal display supports UTF-8. Similarly Courier- IMAP now accepts search queries encoded in any character set known to iconv(3). * This is due to the replacement of the internal character set and unicode library, that predates the standardization of iconv(3). The new iconv(3)- based API is completely different than the previous library API. The only remaining bits of code are mapping function that case-convert character data, as well as compute character width (there's no analogous functionality in iconv(3)). This is augmented by new implementation of unicode grapheme breaking and line breaking algorithms. * Translating and mapping between different character sets is core functionality; from converting email content, in sqwebmail and Cone, to client's character set, to parsing i18n data from various mail headers (which have several different character set and language encoding formats as well), to searching and sorting messages, by sqwebmail, Cone, or the IMAP server. Most the impact from the rewrite should hopefully go unnoticed, aside from the larger character set universe known to sqwebmail, Cone, and Courier-IMAP. * There's a new search box at the bottom of the folder index page. sqwebmail searches the messages in the order of their appearance on the folder index screen. The search starts with the first message on the folder index screen. If the folder index screen does not start with the first message in the folder, the previous messages are not searched, and the search stops when enough messages with the search string are found to fill one folder index screen. In this manner, a poorly chosen search string that finds a hit in a lot of messages will limit itself. If necessary, the search can be repeated after navigating the folder index screen to the starting point for the second search. * Up to four occurrences of the search string in each message are shown on the search results page. * Word wrapping rules in sqwebmail now use unicode line break rules, rather than simple word wrapping on whitespace. This should improve sqwebmail's word wrap with non-Western languages. * Suppress logging the contents of a failed AUTH command in syslog, in case it includes encoded passwords. * Fix requirements of rpm spec files, to depend on /etc/cron.hourly, rather than a specific cron package. * Adjust indentation to avoid parts of DSNs unintentionally interpreted as quoted content. * mailbot now generates flowed text-formatted replies. -- -- pgpQOadVSwgUm.pgp Description: PGP signature -- Colocation vs. Managed Hosting A question and answer guide to determining the best fit for your organization - today and in the future.
Re: [courier-users] maildirsize race condition and removal of the maildirsize file
Heiko Schlittermann writes: Sam Varshavchik mr...@courier-mta.com (Sun Mar 13 20:34:33 2011): Heiko Schlittermann writes: Courier itself will use the quota, whether it comes from userdb or from somewhere else. If Courier runs an external mail delivery agent, it sets MAILDIRQUOTA and leaves it up to the external mail delivery agent to use it. It looks as if the imapd/popd never gets passed the quota setting from the userdb. Seems to be the locical conclusion, if we suppose, that Courier will find the current active quota settings in the first line of the maildirsize file. That does look like the issue here. There is code in imapd that reads and uses MAILDIRQUOTA from the environment, if it's set, but there's nothing that explicitly sets it; it predates the current authentication architecture where this was done differently. But I do see that the quota data should be available to imaplogin. This should be fixable, I'll look into it. I did it, but unfortunely restructred the source a bit… I was thinking along the same lines. However, there's some duplicated code here that, upon further review, needs to go into the courier-authlib package... -- pgp2YmspaEOoy.pgp Description: PGP signature -- Colocation vs. Managed Hosting A question and answer guide to determining the best fit for your organization - today and in the future. http://p.sf.net/sfu/internap-sfd2d___ courier-users mailing list courier-users@lists.sourceforge.net Unsubscribe: https://lists.sourceforge.net/lists/listinfo/courier-users
