Hi,
Trying to learn more, I read http://www.ietf.org/rfc/rfc1892.txt and
have a few questions:
- Could you sketch a simple use case of ESMTP-DSN (sorry to ask, I don't
know anything about it).
- Is DSN for MTA-to-MTA only or also for Client-to-MTA ?
- Do you know if thunderbird 3 support DSN?
(http://forums.mozillazine.org/viewtopic.php?f=29&t=780365 I may ask th
ml, but maybe you've got experience with it)
Tks,
Eric
On 23/01/2011 17:16, Stefano Bagnara wrote:
I'm not sure the complexity introduced by ESMTP-DSN extension worth
its implementation or not. I use DSN so I'd like to see it being
implemented in JAMES and some of the big smtp servers support it
(sendmail, postfix, other?).
I thought a lot at this issue in past but I never found a good and
easy solution to support DSN in James. Here is a review of what DSN is
and what it means supporting it, just to share some thoughts.
DSN is an ESMTP extension that introduces 4 new SMTP keywords.
- ENVID and RET are attributes of the whole "envelope" and are issued
as "MAIL FROM" attributes. ENVID is an identifier for the message for
the originating system, RET instead says if we want full message, just
headers or nothing attached in case of bounce.
- ORCPT and NOTIFY are "per recipient" attributes and issued in the
"RCPT TO" command. ORCPT is an identifier for the recipient in the
originating system (usually the original "recipient" email), NOTIFY
says what kind of bounces we want for this message wrt this recipient
(SUCCESS, FAILURE, DELAY).
In order to create a DSN compliant MTA we have to be both compliant
servers and compliant client.
- A compliant client is easy to write once we have the informations
because sun Javamail is easily extended to support the above keywords
(I'm already using it as client with javamail)
- A compliant server have to "remember" the attributes received for
the message and correctly deal with them (propagate to other DSN
compliant servers, issue appropriate DSN/bounces)
So, to support DSN in James we have to:
- keep track of the 2 envelope attributes (ENVID/RET) and keep track
of the 2 recipient attributes (ORCPT/NOTIFY). In past I simply stored
a custom object as mail attribute but we now moved to string only
attributes and also having them as attribute makes it difficult to
keep track and make sure users don't break them. So we probably have
to bring them at the "Mail" object specification making them "first
class citizen" in the mailet api, but maybe we can also work around
this with something container specific.
- be able to track what mailets do with the mail/recipient. This is
the hard task because from the container we usually don't have
knowledge of what mailets do with our Mail. Most "final" mailets set
the message to "GHOST" but we don't know if the message has been
delivered, has failed (and why) or has been relayed, expanded or any
other action, so we can't issue a notification for that mail.
DSN actions
- DSN allow notifications not only for "bounces" but also for
succesfull delivery. In particular DSN defines the following
"actions": "failed", "delayed", "delivered", "relayed", "expanded".
failed and delivered are "terminal" actions, so once you receive one
of them you don't expect to receive more notifications, instead other
actions may be "temporary" notifications and be followed by more
notifications.
In addition to the action a DSN contains some more informations like
the status code (X.X.X code that is a "category" for the error) and a
Diagnostic-Code that is an explanation string for the error. Most
times "delivered", "relayed" and "expanded" have not so much
information in the status and diagnostic code, e.g:
------------
action: delivered
status: 2.1.5
diagnostic-code: delivered to mailbox
action: relayed
status: 2.0.0
diagnostic-code: succesfully relayed
action: expanded
status: 2.0.0
diagnostic-code: alias succesfully expanded
--------------
There is not much value in providing alternative "status" values in
such DSN, but sometimes could be "good" to provide additional
informations in the diagnostic-code (e.g: a "delivered" mail may have
more details, like "delivered to maildir", "expanded to mailing list",
"processed" as sometimes we simply process a message so it is
succesfully "delivered" but we didn't put it in an inbox).
Instead when we return a "failed" or "delayed" action then the status
and diagnostic-code are much more useful and needed.
In order to keep the api simple one idea that came to mind was to use
"conventional" states for the actions. So, instead of ghosting
(setState(GHOST)) a mail you could mark it as "delivered", "relayed",
"failed" and so on and each "action" processor would care of issuing
the correct DSN based on the DSN attributes provided during the ESMTP
conversation: this have a couple of big limits:
1) This doesn't support "status" or "diagnostic-code" so it is really "limited"
2) This doesn't let a mailet to mark some recipient as delivered and
some other recipient as failed.
We can try to fix #1 by saying that a mailet may raise a
"DeliveryException" and the exception includes the 3 informations
(action/status/diagnostic-code) so that a correct DSN can be issued by
the container.
We can try to fix #2 by introducing a new "type" of mailet: e.g: each
mailet that needs to deal separately with each recipient of a multi
recipient Mail may "tag" itself as @SingleRecipient mailet. When the
container finds such a mailet it "split" the incoming multi-recipients
mail into multiple single-recipient emails. This has the advantage of
moving "splitting" to the container (like we already do with Matcher)
and make most mailets easier to write, on the other hand we have the
disadvantage of splitting every mail just because we have a
@SingleRecipient mailet in the processor.
But, let's say a mailet takes a 2 recipients mail and alter it
replacing the 2 recipients with 3 new recipients: what happened? The
container have no idea and cannot decide what to do with DSN
attributes for the original recipient. The container doens't know if
the 2 original recipients have been delivered, relayed, expanded,
failed or what else. TO let the container track this we have to
"protect" the recipient list.
A mailet should not be allowed to simply replace the recipients of a
Mail without explanation, instead we should add more specific methods
to let the container understand what's going on.
- in order to remove a recipient you have to tell the container WHY
you remove it (you delivered to that recipient or you instead know the
recipient is invalid?)
- expanding a recipient to 2 recipients is different from adding 2 new
recipients and removing the original, so we should probably add some
method to "alias" a recipient or to "expand" a recipient to multiple
recipients.
Maybe some scenario help you see the complexity. Let's take into
consideration only the easy case of single recipients email (multiple
recipients are compositions of the others).
A) the message is delivered to inbox: the mailet uses
setState('delivered') and the container looks at the NOTIFY for the
recipient and if SUCCESS is there issue a success notification to the
sender.
B) the user is unknown: the mailet raise a new
DeliveryException("failed","5.1.1","User unknown") and the container
looks at the NOTIFY and if FAILURE is there issue a failure
notification to the sender.
C) the recipient is "aliased" to 1 other recipient: the mailet have to
replace the recipient but we can't allow mailets to remove/add
recipients. So we probably need a "replace" or "alias" method to do
this in a recipient list. When you replace one address with another we
have to mantain the DSN attributes for the original recipient and
associate them to the new recipient and we don't have to issue DSNs.
So if DSN attributes are in the "recipient" object then we simply have
to replace the recipient-address part of the object and leave
everything else as is.
D) the recipient is replaced with multiple recipients. The DSN RFC
specify a few different cases:
D1) additional notification addresses: we want to add new addresses
to the mail (e.g: in order to support vacation auto responders or to
put a copy into a "monitoring" address. We add each additional
recipient using NOTIFY=NEVER so that this address won't issue DSN and
instead only the original recipient will work as if no address was
added.
D2) multiple alias expansion: unlike the mailing list case in this
case we want to keep the original sender. In this case the RFC ask us
to issue an "expanded" DSN for the original recipient (only if NOTIFY
included SUCCESS) and then use the same DSN attributes for new
recipients but removing SUCCESS from NOTIFY (so we issue an expanded
but we don't want that every expanded recipient issue a
delivered/relayed instead we simply want to notify about errors in
expanded aliases). For this use case we probably need a specific
method "expand" that deals with the specific notification option.
D3) mailing list expansion: the RFC says we have to deal with it
like a "delivery to inbox". So I expect the mailet to simply issue new
sendmail (with a new sender!) and at the end mark the original mail as
delivered.
D4) relaying to multiple aliases: RFC also allows a case in the
middle between mailing list expansion and multiple alias expansion. In
this case the final messages keep the original sender (like the alias
expansion) but the DSN attributes must not be propagated. So this is
like the mailing list expansion, but the new sent emails use the same
sender of the original message. The result is "relayed" instead of
"delivered". I think we can leave this to the mailet writer (we can't
track this details).
You can see that as long as you want to keep the original DSN
attributes you have to work on the recipient list and simply mark
recipients as failed, aliased, expanded, delivered, relayed instead if
you want to add recipients that are "hidden" to the original sender
then you use MailetContext.sendMail. It's up to the mailet writer to
correctly set the sender of the new sent mails depending on the type
of mail being sent.
One important difference between adding a new recipient and sending a
"cloned" email to the new recipient is that when you send an email
using sendMail it starts the spooling from "root" as a new incoming
message, instead adding recipients to the current mail simply will
continue the processing from the next matcher/mailet in the current
processor: they are already very different ways to do things.
Unless we add @SingleRecipient there is no way for a mailet to "split"
an incoming Mail and keep every splitted mail at the current
processing stage. Another possibility would be to allow mailets to
"split" a mail using the container so that the container know 1 mail
entered and 2 mail will exit from that mailet.
Also, even if we go for the @SingleRecipient annotation we probably
have to let advanced mailets to deal with each recipient status
separately and still support DSN so marking some of them as failed,
some delivered, leave some "as is" for processing in the following
mailets and so on.
I will think more about this in the next days/weeks and keep you
updated if I have new ideas, in the mean time I'm open to
suggestions/hints/critics/opinions.
Stefano
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
