Re: [Zope3-dev] Re: ZOPE3 Maildir implementation
On Sat, Apr 22, 2006 at 01:27:36PM +0300, Marius Gedminas wrote: > Actually, the DirectMailDelivery utility also aborts emails if the > transaction is aborted. The reason for using Maildirs was to speed up > the transaction commit -- if you send 100 emails in a transaction, you > do not want to wait for 100 SMTP connections while the user is sitting > in front of his browser waiting. Creating a 100 files in the outgoing > mail queue was supposed to be faster. Thanks for fixing it Marius! I think your XP approach is fine ;-). I agree you don't want blocking. Nevertheless should we not store it in the ZODB instead - at least when ZODB is around? I don't know what the policy is with ZOPE3, but I would think that we should minimise deployment dependencies. Locating a directory is such a dependency - and people using, for example, NFS may get into trouble (though I think maildir tried to address that with that naming and file moving scheme - it is ugly, arguably fragile and slow!). Correct me if I am wrong. Someone wrote that you can't pass the ZODB access from a web server thread to a service/utility thread - "The mailer utility is global and has no convenient access to the ZODB." Is that really true? Is there no way to pass this info? Pj. ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] Re: ZOPE3 Maildir implementation
Hi, folks! On Fri, Apr 07, 2006 at 05:23:06PM +0200, Philipp von Weitershausen wrote: > > The Maildir step could have been skipped, I suppose, just the > > connection to a mail server directly over the network interface would > > suffice. Is there a specific reason for the Maildir implementation? > > The reason for queued delivery (as opposed to direct delivery) is that > you might send an email that is later aborted. For example, consider a > change notification email whereas the change itself is aborted. The > email would be lying about the state of the application. With queued > delivery, the email isn't sent when the transaction is aborted. > As Gary pointed out, the reason for using Maildir is a practical one. We > don't want to block. Actually, the DirectMailDelivery utility also aborts emails if the transaction is aborted. The reason for using Maildirs was to speed up the transaction commit -- if you send 100 emails in a transaction, you do not want to wait for 100 SMTP connections while the user is sitting in front of his browser waiting. Creating a 100 files in the outgoing mail queue was supposed to be faster. It didn't work due to the extreme brain-deadness of the Maildir code. IIRC I wrote it (a number of years ago, when I was young and foolish) and I cannot fathom why I didn't realize that 1 email per second would be an extremely inconvenient. Maybe I was too enamored with Extreme Programming ideas at the time (let's do the simplest thing that could possibly work, and when it turns out to be not enough, we can just fix it). Well, the time to fix it has finally come. There were also some bugs in the transaction commit/abort logic that could lead to multiple copies of the emails being sent on ZODB ConflictErrors. Thankfully all this is now fixed in Zope 3 trunk, and in the 3.2 branch. The issue was http://www.zope.org/Collectors/Zope3-dev/590 > > 2. The current implementation can only handle between 30-60 E-mail's a > > minute. This is mostly due to the naming convention used for maildir > > files (one a second - actually the Maildir naming allows in between > > numbers, but you don't use that). > > Actually, the Maildir spec (http://cr.yp.to/proto/maildir.html) now > suggests "new fashion" Maildir names that are a lot "more unique" than > old fashion ones. Every major Maildir implementation I've come across in > the last two years uses new fashion Maildir names. I didn't know about this. Yesterday I just changed the unique name generator to include a large random number in the filename. In the wrong place, so it no longer matches the Maildir spec. *sigh* Any volunteers for making it match the spec? Marius Gedminas -- #define QUESTION ((bb) || !(bb)) /* Shakespeare */ signature.asc Description: Digital signature ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
[Zope3-dev] Re: ZOPE3 Maildir implementation
Pjotr Prins wrote: > I have a few questions regarding the ZOPE mailer implementation: > > 1. Why did you go for a file system implementation - as ZODB objects > are being made into a mail queue anyway? The mail queue doesn't create persistent objects. > The Maildir step could have been skipped, I suppose, just the > connection to a mail server directly over the network interface would > suffice. Is there a specific reason for the Maildir implementation? The reason for queued delivery (as opposed to direct delivery) is that you might send an email that is later aborted. For example, consider a change notification email whereas the change itself is aborted. The email would be lying about the state of the application. With queued delivery, the email isn't sent when the transaction is aborted. As Gary pointed out, the reason for using Maildir is a practical one. We don't want to block. > 2. The current implementation can only handle between 30-60 E-mail's a > minute. This is mostly due to the naming convention used for maildir > files (one a second - actually the Maildir naming allows in between > numbers, but you don't use that). Actually, the Maildir spec (http://cr.yp.to/proto/maildir.html) now suggests "new fashion" Maildir names that are a lot "more unique" than old fashion ones. Every major Maildir implementation I've come across in the last two years uses new fashion Maildir names. > The reason I am asking is not so much an interest in fixing it for > mail, Again, as Gary pointed out, it'd be great if you could help fix this. Philipp ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
