Hi Stefano, first of thank you for taking the time to write down all this stuff. Very informative.
So after reading the email and thinking a bit more about this I think we have 2 options: 1) use Collections.unmodifiableCollection(..) when return the Recipient list via getRecipients() and remove the setRecipients(..) method. This would also need a few more methods in the MailetContext implementation to add/remove/alias/expand recipients. 2) Return some special class when for getRecipients() which can handle all this stuff and remove the setRecipients(..) method. I think I would prefer 2) as it is more clean. Beside this I also agree about the different states we would need and about the @SingleRecipient annotation. Bye, Norman 2011/1/23 Stefano Bagnara <[email protected]>: > I'm not sure the complexity introduced by ESMTP-DSN extension worth > its implementation or not. I use DSN so I'd like to see it being > implemented in JAMES and some of the big smtp servers support it > (sendmail, postfix, other?). > I thought a lot at this issue in past but I never found a good and > easy solution to support DSN in James. Here is a review of what DSN is > and what it means supporting it, just to share some thoughts. I'm not aware of any other "big mta" which support it. > > DSN is an ESMTP extension that introduces 4 new SMTP keywords. > - ENVID and RET are attributes of the whole "envelope" and are issued > as "MAIL FROM" attributes. ENVID is an identifier for the message for > the originating system, RET instead says if we want full message, just > headers or nothing attached in case of bounce. > - ORCPT and NOTIFY are "per recipient" attributes and issued in the > "RCPT TO" command. ORCPT is an identifier for the recipient in the > originating system (usually the original "recipient" email), NOTIFY > says what kind of bounces we want for this message wrt this recipient > (SUCCESS, FAILURE, DELAY). > > In order to create a DSN compliant MTA we have to be both compliant > servers and compliant client. > - A compliant client is easy to write once we have the informations > because sun Javamail is easily extended to support the above keywords > (I'm already using it as client with javamail) > - A compliant server have to "remember" the attributes received for > the message and correctly deal with them (propagate to other DSN > compliant servers, issue appropriate DSN/bounces) > > So, to support DSN in James we have to: > - keep track of the 2 envelope attributes (ENVID/RET) and keep track > of the 2 recipient attributes (ORCPT/NOTIFY). In past I simply stored > a custom object as mail attribute but we now moved to string only > attributes and also having them as attribute makes it difficult to > keep track and make sure users don't break them. So we probably have > to bring them at the "Mail" object specification making them "first > class citizen" in the mailet api, but maybe we can also work around > this with something container specific. > - be able to track what mailets do with the mail/recipient. This is > the hard task because from the container we usually don't have > knowledge of what mailets do with our Mail. Most "final" mailets set > the message to "GHOST" but we don't know if the message has been > delivered, has failed (and why) or has been relayed, expanded or any > other action, so we can't issue a notification for that mail. > > DSN actions > - DSN allow notifications not only for "bounces" but also for > succesfull delivery. In particular DSN defines the following > "actions": "failed", "delayed", "delivered", "relayed", "expanded". > failed and delivered are "terminal" actions, so once you receive one > of them you don't expect to receive more notifications, instead other > actions may be "temporary" notifications and be followed by more > notifications. > > In addition to the action a DSN contains some more informations like > the status code (X.X.X code that is a "category" for the error) and a > Diagnostic-Code that is an explanation string for the error. Most > times "delivered", "relayed" and "expanded" have not so much > information in the status and diagnostic code, e.g: > ------------ > action: delivered > status: 2.1.5 > diagnostic-code: delivered to mailbox > > action: relayed > status: 2.0.0 > diagnostic-code: succesfully relayed > > action: expanded > status: 2.0.0 > diagnostic-code: alias succesfully expanded > -------------- > There is not much value in providing alternative "status" values in > such DSN, but sometimes could be "good" to provide additional > informations in the diagnostic-code (e.g: a "delivered" mail may have > more details, like "delivered to maildir", "expanded to mailing list", > "processed" as sometimes we simply process a message so it is > succesfully "delivered" but we didn't put it in an inbox). > Instead when we return a "failed" or "delayed" action then the status > and diagnostic-code are much more useful and needed. > > In order to keep the api simple one idea that came to mind was to use > "conventional" states for the actions. So, instead of ghosting > (setState(GHOST)) a mail you could mark it as "delivered", "relayed", > "failed" and so on and each "action" processor would care of issuing > the correct DSN based on the DSN attributes provided during the ESMTP > conversation: this have a couple of big limits: > 1) This doesn't support "status" or "diagnostic-code" so it is really > "limited" > 2) This doesn't let a mailet to mark some recipient as delivered and > some other recipient as failed. > > We can try to fix #1 by saying that a mailet may raise a > "DeliveryException" and the exception includes the 3 informations > (action/status/diagnostic-code) so that a correct DSN can be issued by > the container. > We can try to fix #2 by introducing a new "type" of mailet: e.g: each > mailet that needs to deal separately with each recipient of a multi > recipient Mail may "tag" itself as @SingleRecipient mailet. When the > container finds such a mailet it "split" the incoming multi-recipients > mail into multiple single-recipient emails. This has the advantage of > moving "splitting" to the container (like we already do with Matcher) > and make most mailets easier to write, on the other hand we have the > disadvantage of splitting every mail just because we have a > @SingleRecipient mailet in the processor. > > But, let's say a mailet takes a 2 recipients mail and alter it > replacing the 2 recipients with 3 new recipients: what happened? The > container have no idea and cannot decide what to do with DSN > attributes for the original recipient. The container doens't know if > the 2 original recipients have been delivered, relayed, expanded, > failed or what else. TO let the container track this we have to > "protect" the recipient list. > A mailet should not be allowed to simply replace the recipients of a > Mail without explanation, instead we should add more specific methods > to let the container understand what's going on. > - in order to remove a recipient you have to tell the container WHY > you remove it (you delivered to that recipient or you instead know the > recipient is invalid?) > - expanding a recipient to 2 recipients is different from adding 2 new > recipients and removing the original, so we should probably add some > method to "alias" a recipient or to "expand" a recipient to multiple > recipients. > > Maybe some scenario help you see the complexity. Let's take into > consideration only the easy case of single recipients email (multiple > recipients are compositions of the others). > A) the message is delivered to inbox: the mailet uses > setState('delivered') and the container looks at the NOTIFY for the > recipient and if SUCCESS is there issue a success notification to the > sender. > B) the user is unknown: the mailet raise a new > DeliveryException("failed","5.1.1","User unknown") and the container > looks at the NOTIFY and if FAILURE is there issue a failure > notification to the sender. > C) the recipient is "aliased" to 1 other recipient: the mailet have to > replace the recipient but we can't allow mailets to remove/add > recipients. So we probably need a "replace" or "alias" method to do > this in a recipient list. When you replace one address with another we > have to mantain the DSN attributes for the original recipient and > associate them to the new recipient and we don't have to issue DSNs. > So if DSN attributes are in the "recipient" object then we simply have > to replace the recipient-address part of the object and leave > everything else as is. > D) the recipient is replaced with multiple recipients. The DSN RFC > specify a few different cases: > D1) additional notification addresses: we want to add new addresses > to the mail (e.g: in order to support vacation auto responders or to > put a copy into a "monitoring" address. We add each additional > recipient using NOTIFY=NEVER so that this address won't issue DSN and > instead only the original recipient will work as if no address was > added. > D2) multiple alias expansion: unlike the mailing list case in this > case we want to keep the original sender. In this case the RFC ask us > to issue an "expanded" DSN for the original recipient (only if NOTIFY > included SUCCESS) and then use the same DSN attributes for new > recipients but removing SUCCESS from NOTIFY (so we issue an expanded > but we don't want that every expanded recipient issue a > delivered/relayed instead we simply want to notify about errors in > expanded aliases). For this use case we probably need a specific > method "expand" that deals with the specific notification option. > D3) mailing list expansion: the RFC says we have to deal with it > like a "delivery to inbox". So I expect the mailet to simply issue new > sendmail (with a new sender!) and at the end mark the original mail as > delivered. > D4) relaying to multiple aliases: RFC also allows a case in the > middle between mailing list expansion and multiple alias expansion. In > this case the final messages keep the original sender (like the alias > expansion) but the DSN attributes must not be propagated. So this is > like the mailing list expansion, but the new sent emails use the same > sender of the original message. The result is "relayed" instead of > "delivered". I think we can leave this to the mailet writer (we can't > track this details). > > You can see that as long as you want to keep the original DSN > attributes you have to work on the recipient list and simply mark > recipients as failed, aliased, expanded, delivered, relayed instead if > you want to add recipients that are "hidden" to the original sender > then you use MailetContext.sendMail. It's up to the mailet writer to > correctly set the sender of the new sent mails depending on the type > of mail being sent. > > One important difference between adding a new recipient and sending a > "cloned" email to the new recipient is that when you send an email > using sendMail it starts the spooling from "root" as a new incoming > message, instead adding recipients to the current mail simply will > continue the processing from the next matcher/mailet in the current > processor: they are already very different ways to do things. > > Unless we add @SingleRecipient there is no way for a mailet to "split" > an incoming Mail and keep every splitted mail at the current > processing stage. Another possibility would be to allow mailets to > "split" a mail using the container so that the container know 1 mail > entered and 2 mail will exit from that mailet. > > Also, even if we go for the @SingleRecipient annotation we probably > have to let advanced mailets to deal with each recipient status > separately and still support DSN so marking some of them as failed, > some delivered, leave some "as is" for processing in the following > mailets and so on. > > I will think more about this in the next days/weeks and keep you > updated if I have new ideas, in the mean time I'm open to > suggestions/hints/critics/opinions. > > Stefano > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > > --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
