Re: [Zope3-dev] Re: ZOPE3 Maildir implementation

2006-04-22 Thread Marius Gedminas
Hi, folks!

On Fri, Apr 07, 2006 at 05:23:06PM +0200, Philipp von Weitershausen wrote:
  The Maildir step could have been skipped, I suppose, just the
  connection to a mail server directly over the network interface would
  suffice. Is there a specific reason for the Maildir implementation?
 
 The reason for queued delivery (as opposed to direct delivery) is that
 you might send an email that is later aborted. For example, consider a
 change notification email whereas the change itself is aborted. The
 email would be lying about the state of the application. With queued
 delivery, the email isn't sent when the transaction is aborted.
 As Gary pointed out, the reason for using Maildir is a practical one. We
 don't want to block.

Actually, the DirectMailDelivery utility also aborts emails if the
transaction is aborted.  The reason for using Maildirs was to speed up
the transaction commit -- if you send 100 emails in a transaction, you
do not want to wait for 100 SMTP connections while the user is sitting
in front of his browser waiting.  Creating a 100 files in the outgoing
mail queue was supposed to be faster.

It didn't work due to the extreme brain-deadness of the Maildir code.
IIRC I wrote it (a number of years ago, when I was young and foolish)
and I cannot fathom why I didn't realize that 1 email per second would
be an extremely inconvenient.  Maybe I was too enamored with Extreme
Programming ideas at the time (let's do the simplest thing that could
possibly work, and when it turns out to be not enough, we can just fix
it).  Well, the time to fix it has finally come.

There were also some bugs in the transaction commit/abort logic that
could lead to multiple copies of the emails being sent on ZODB
ConflictErrors.

Thankfully all this is now fixed in Zope 3 trunk, and in the 3.2 branch.
The issue was http://www.zope.org/Collectors/Zope3-dev/590

  2. The current implementation can only handle between 30-60 E-mail's a
  minute. This is mostly due to the naming convention used for maildir
  files (one a second - actually the Maildir naming allows in between
  numbers, but you don't use that).
 
 Actually, the Maildir spec (http://cr.yp.to/proto/maildir.html) now
 suggests new fashion Maildir names that are a lot more unique than
 old fashion ones. Every major Maildir implementation I've come across in
 the last two years uses new fashion Maildir names.

I didn't know about this.  Yesterday I just changed the unique name
generator to include a large random number in the filename.  In the
wrong place, so it no longer matches the Maildir spec.  *sigh*

Any volunteers for making it match the spec?

Marius Gedminas
-- 
#define QUESTION ((bb) || !(bb)) /* Shakespeare */


signature.asc
Description: Digital signature
___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



Re: [Zope3-dev] Re: ZOPE3 Maildir implementation

2006-04-22 Thread Pjotr Prins
On Sat, Apr 22, 2006 at 01:27:36PM +0300, Marius Gedminas wrote:

 Actually, the DirectMailDelivery utility also aborts emails if the
 transaction is aborted.  The reason for using Maildirs was to speed up
 the transaction commit -- if you send 100 emails in a transaction, you
 do not want to wait for 100 SMTP connections while the user is sitting
 in front of his browser waiting.  Creating a 100 files in the outgoing
 mail queue was supposed to be faster.

Thanks for fixing it Marius! I think your XP approach is fine ;-).

I agree you don't want blocking. Nevertheless should we not store it
in the ZODB instead - at least when ZODB is around?

I don't know what the policy is with ZOPE3, but I would think that we
should minimise deployment dependencies. Locating a directory is
such a dependency - and people using, for example, NFS may get into
trouble (though I think maildir tried to address that with that naming
and file moving scheme - it is ugly, arguably fragile and slow!).

Correct me if I am wrong.

Someone wrote that you can't pass the ZODB access from a web server
thread to a service/utility thread - The mailer utility is global and
has no convenient access to the ZODB. Is that really true? Is there
no way to pass this info?

Pj.
___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com