Re: virtualdomains vs. VERP and Delivered-To
On 8 Aug 2001, John R. Levine wrote:
Like I said:
It's true, qmail doesn't work the way you might first have guessed it
does. That doesn't mean it's wrong.
The fact qmail--or any other piece of software--does something does not
mean it is correct.
Executive summary: qmail breaks VERP under certain circumstances.
Let H be a host running qmail, A and B users at H, and V a virtual domain
redirected to B@H. Let X@V, i.e. B-X@H, be forwarded to some other, maybe
remote, address, say K@L. Now, let's assume A uses
QMAILINJECT=r qmail-inject X@V
to send a VERPed message M to X@V. M is forwarded to K@L. Now, let's
assume the delivery to K@L fails and the message is bounced back to A.
Well, it should be bounced to A-X=V@M, shouldn't it? After all, A sent the
message to X@V, and VERP is supposed to preserve the *original* recipient
address. Indeed, qmail-inject's manpage says:
r Use a per-recipient VERP. qmail-inject will append each
recipient address to the envelope sender of the copy going
to that recipient.
Unfortunately, the return address in the scenario described above is
A-B-X=V@M
Is A supposed to know B- is superflous (if and only if the domain is V!)
and should be removed? Is A supposed to analyze qmail's configurations
files in order to fix something that should have never been broken? (BTW:
I cannot find any code analyzing virtualdomains in ezmlm. Am I blind?)
A does not care what is recorded in Delivered-To or what a program run
from ~B/.qmail-V-... sees in its environment. A cares what qmail does when
he sends a message and asks qmail to use VERP.
Is this scenario purely artifical? Not at all. It is easy to imagine a
host run by some ISP-like company hosting both an email forwarding service
implemented as a virtual domain and some mailing lists.
Surprisingly, the fix appears to be almost a one-liner if we use the same
approach qmail-send itself uses to fix recipient addresses when it
generates a bounce. A *completely untested* patch is here:
--- qmail-send.c~ Mon Jun 15 12:53:16 1998
+++ qmail-send.cWed Aug 8 16:47:15 2001
@@ -162,6 +162,8 @@
return 2;
}
+char *stripvdomprepend(char *recip);
+
void senderadd(sa,sender,recip)
stralloc *sa;
char *sender;
@@ -175,6 +177,7 @@
if (i = 4)
if (str_equal(sender + i - 4,-@[]))
{
+ recip = stripvdomprepend(recip);
j = byte_rchr(sender,i - 4,'@');
k = str_rchr(recip,'@');
if (recip[k] (j + 5 = i))
I repeat: There is absolutely no guarantee this patch will work as
expected. It might cause syntax errors, it might transport all mail it
gets its hands on to the nearest blackhole, and it might even massacre all
your family with a chainsaw. Use at your own risk.
P.S. I wonder whether we will see any reaction from DJB himself.
--Pavel Kankovsky aka Peak [ Boycott Microsoft--http://www.vcnet.com/bms ]
Resistance is futile. Open your source code and prepare for assimilation.
Re: virtualdomains vs. VERP and Delivered-To
Executive summary: qmail breaks VERP under certain circumstances. Revised executive summary: qmail's VERP works fine, but some people are more than a little unclear on the way virtual domains work. Let H be a host running qmail, A and B users at H, and V a virtual domain redirected to B@H. Let X@V, i.e. B-X@H, be forwarded to some other, maybe remote, address, say K@L. Now, let's assume A uses QMAILINJECT=r qmail-inject X@V to send a VERPed message M to X@V. M is forwarded to K@L. Now, let's assume the delivery to K@L fails and the message is bounced back to A. Well, it should be bounced to A-X=V@M, shouldn't it? Well, actually, it should be bounced to A-X=V@H, and that's exactly where it goes since that's the address that VERP creates. (I presume M was a typo for H there.) ... Unfortunately, the return address in the scenario described above is A-B-X=V@M No, it's not. Qmail rewrites target virtual domain addresses at the time they're delivered, and virtual domain handling doesn't rewrite return addresses at all, ever. ... A *completely untested* patch is here: Too bad you didn't test it, you could have avoided wasting a lot of time. I misunderstood what you were arguing last time. The only time you might have to consult control/virtualdomains to handle a VERP is if the domain sending the VERP'ed mail is itself a virtual domain. I happen to have a bunch of mailing lists in virtual domains, and they have bounce handlers. I can assure you from experience that all addresses on the mailing lists are handled the same, and it makes no difference whatsoever if an address to which VERP mail is sent is local, remote, virtual, or anything else. -- John R. Levine, IECC, POB 727, Trumansburg NY 14886 +1 607 387 6869 [EMAIL PROTECTED], Village Trustee and Sewer Commissioner, http://iecc.com/johnl, Member, Provisional board, Coalition Against Unsolicited Commercial E-mail
Re: virtualdomains vs. VERP and Delivered-To
On 8 Aug 2001, John R. Levine wrote: Well, actually, it should be bounced to A-X=V@H, and that's exactly where it goes since that's the address that VERP creates. (I presume M was a typo for H there.) Oops. Yes, it should read A-X=V@H. Unfortunately, the return address in the scenario described above is A-B-X=V@M No, it's not. Qmail rewrites target virtual domain addresses at the time they're delivered, and virtual domain handling doesn't rewrite return addresses at all, ever. *Yes, it is* (when M is replaced with H). I tested it (I did not test the patch). I consulted the source code. The problem lies in the fact the message is injected with a proto-VERP (*), with A-@H-@[] or something like that instead of a real return path. This pattern is recognized by qmail-send and transformed into a real VERP path during delivery (see senderadd()). But recipient addresses are rewritten according to controls/virtualdomains before the transformation is done and their *rewritten* forms are used both as a destination address and a value for the construction of VERPs (see comm_write()). Of course, this is bad whenever the rewritten form is different from the original one. ... A *completely untested* patch is here: Too bad you didn't test it, you could have avoided wasting a lot of time. I guess I am wasting much more time trying to explain the problem. :P I misunderstood what you were arguing last time. The only time you might have to consult control/virtualdomains to handle a VERP is if the domain sending the VERP'ed mail is itself a virtual domain. No, the problem is when the domain *receiving* it is virtual and local, and when the message is injected with a proto-VERP. Of course, hardcoded VERPs are not mangled by qmail but a mailing list software would have to inject one copy per recipient into the queue in order to be able to use them and this would be quite inefficient (**). If it uses proto-VERPs (qmail-inject with QMAILINJECT=r uses them, ezmlm uses them too), it can inject a single multi-recipient message only...assuming qmail-send won't do anything stupid. Is it clear now? (*) This is no official (DJB's) terminology, just an ad hoc name invented to label the concept in question in this text. (**) Arguments against multi-RCPT SMTP are not applicable here. --Pavel Kankovsky aka Peak [ Boycott Microsoft--http://www.vcnet.com/bms ] Resistance is futile. Open your source code and prepare for assimilation.
Re: virtualdomains vs. VERP and Delivered-To
On 6 Aug 2001, John R. Levine wrote: Is it really that overwhelmingly difficult to have whatever configures your bounce handler look in /var/qmail/control/virtualdomains to see what prefix to strip off the local part of the VERP address? I suspect either of us could do it in about four lines of perl. You can turn the question upside-down: Is is really that overwhelmingly difficult to add or change about four lines of C and make qmail behave in a sane way and eliminate the need to add such a twisted piece of code to every program using VERP on this planet? If I (DJB) want to keep my program (qmail) as small and clean as possible to avoid bugs etc., I should not force other people to make their programs bloated, should I? --Pavel Kankovsky aka Peak [ Boycott Microsoft--http://www.vcnet.com/bms ] Resistance is futile. Open your source code and prepare for assimilation.
Re: virtualdomains vs. VERP and Delivered-To
John R. Levine: It's true, qmail doesn't work the way you might first have guessed it does. That doesn't mean it's wrong. Well, qmail-send does rewrite the envelope recipient for local deliveries. That's not a very good thing. /filip
Re: virtualdomains vs. VERP and Delivered-To
Is it really that overwhelmingly difficult to have whatever configures your bounce handler look in /var/qmail/control/virtualdomains to see what prefix to strip off the local part of the VERP address? I suspect either of us could do it in about four lines of perl. You can turn the question upside-down: Is is really that overwhelmingly difficult to add or change about four lines of C and make qmail behave in a sane way and eliminate the need to add such a twisted piece of code to every program using VERP on this planet? Difficult? Of course not, if you want to change that, you have the source. But just because it's easy doesn't mean it's not a good idea. If I (DJB) want to keep my program (qmail) as small and clean as possible to avoid bugs etc., I should not force other people to make their programs bloated, should I? Of course. That's why it works the way it does. The Delivered-To: contents is actually $RECIPIENT which is $LOCAL@$HOST. When qmail delivers a message, it finds the longest prefix of $LOCAL in the users database and uses that to set the user/group IDs and home directory for the delivery. Then the rest of $LOCAL is the extension and is used to pick the appropriate .qmail file and is available as $EXT and so forth. This is what happens regardless of whether the message was originally addressed to a local domain or a virtual one. If it was to a virtual domain, there was a preprocessing step that put the virtual domain's prefix on the front of $LOCAL, but delivery code doesn't have to worry about that. In the particular case where a program run from .qmail does VERP bounce processing, it has to de-prefix $LOCAL, but for other purposes, $LOCAL shows the address that the message is delivered to and that's what delivery scripts need. Now let's look at your plan. If a message is addressed to a virtual domain, qmail looks it up, finds the prefix and does, um, something with it. Does it change $LOCAL? Or does it concoct $REALLYLOCAL or the like? Do .qmail scripts see the unprefixed $LOCAL or the prefixed one? Since $LOCAL no longer is the actual delivery address for virtual domains, to work reliably scripts that deliver mail that might have been sent to a virtual domain have to look at $HOST and do one thing if it's a local domain and another if it's virtual. Sounds pretty bloated to me, particularly since there are generally far more deliveries to virtual addresses, which want the prefixed address, than VERP bounces, which don't. Like I said: It's true, qmail doesn't work the way you might first have guessed it does. That doesn't mean it's wrong. -- John R. Levine, IECC, POB 727, Trumansburg NY 14886 +1 607 387 6869 [EMAIL PROTECTED], Village Trustee and Sewer Commissioner, http://iecc.com/johnl, Member, Provisional board, Coalition Against Unsolicited Commercial E-mail
Re: virtualdomains vs. VERP and Delivered-To
Charles M. Hannum writes: Delivered-To: [EMAIL PROTECTED] ... This seems very wrong. The Delivered-To: address here isn't even correct; it should be something the actually exists -- either `[EMAIL PROTECTED]' or `[EMAIL PROTECTED]'. Don't think of Delivered-To: as an address. Think of it as a unique magic cookie derived from email delivery path. You can always reconstruct the address if you know something about the delivery path, and sometimes you may indeed have to. I don't need to be taught the religion, thanks. I'm already well aware of it. And I don't buy it in this case. What if `[EMAIL PROTECTED]' *was* a valid, different address? It could falsely detect loops. Maybe that wouldn't make sense in this particular case, but I'm sure you can construct a more palatable case with little effort. Also, that doesn't resolve my VERP problem.
Re: virtualdomains vs. VERP and Delivered-To
Charles M. Hannum writes: Charles M. Hannum writes: Delivered-To: [EMAIL PROTECTED] ... This seems very wrong. The Delivered-To: address here isn't even correct; it should be something the actually exists -- either `[EMAIL PROTECTED]' or `[EMAIL PROTECTED]'. Don't think of Delivered-To: as an address. Think of it as a unique magic cookie derived from email delivery path. You can always reconstruct the address if you know something about the delivery path, and sometimes you may indeed have to. I don't need to be taught the religion, thanks. I'm already well aware of it. But there are other people who are not. I didn't write you a private reply, did I? And I don't buy it in this case. What if `[EMAIL PROTECTED]' *was* a valid, different address? It could falsely detect loops. Maybe that wouldn't make sense in this particular case, but I'm sure you can construct a more palatable case with little effort. Then use a character in virtualdomains which is not legal in an email address. I thought you didn't need to be taught the religion? Repent, sinner! Also, that doesn't resolve my VERP problem. Sorry, I thought it did. Why doesn't it? -- -russ nelson [EMAIL PROTECTED] http://russnelson.com Crynwr sells support for free software | PGPok | 521 Pleasant Valley Rd. | +1 315 268 1925 voice | #exclude windows.h Potsdam, NY 13676-3213 | +1 315 268 9201 FAX |
Re: virtualdomains vs. VERP and Delivered-To
Also, that doesn't resolve my VERP problem. Sorry, I thought it did. Why doesn't it? Uhhh, did you *read* my first piece of email? If I get a VERP address of `[EMAIL PROTECTED]', how pray tell is my mailing list software supposed to know that the mail was actually sent to `[EMAIL PROTECTED]'? It doesn't have a prayer -- unless it knows about qmail's virtualdomains, which would be a major abstraction violation. VERP and virtualdomains just don't work properly together in a stock qmail.
Re: virtualdomains vs. VERP and Delivered-To
Charles M. Hannum writes: Also, that doesn't resolve my VERP problem. Sorry, I thought it did. Why doesn't it? Uhhh, did you *read* my first piece of email? If I get a VERP address of `[EMAIL PROTECTED]', how pray tell is my mailing list software supposed to know that the mail was actually sent to `[EMAIL PROTECTED]'? It's supposed to strip off the foo-owner-mycroft- prefix and the @netbsd.org suffix, and change the rightmost = into an @. Were you expecting me to write the script for you? VERP and virtualdomains just don't work properly together in a stock qmail. Yup. -- -russ nelson [EMAIL PROTECTED] http://russnelson.com Crynwr sells support for free software | PGPok | 521 Pleasant Valley Rd. | +1 315 268 1925 voice | #exclude windows.h Potsdam, NY 13676-3213 | +1 315 268 9201 FAX |
Re: virtualdomains vs. VERP and Delivered-To
On Mon, 6 Aug 2001, Russell Nelson wrote: Charles M. Hannum writes: Uhhh, did you *read* my first piece of email? If I get a VERP address of `[EMAIL PROTECTED]', how pray tell is my mailing list software supposed to know that the mail was actually sent to `[EMAIL PROTECTED]'? It's supposed to strip off the foo-owner-mycroft- prefix and the @netbsd.org suffix, and change the rightmost = into an @. Were you expecting me to write the script for you? Oops. The result would be [EMAIL PROTECTED] rather than [EMAIL PROTECTED]. Of course, some people's favourite scripting languages might include an implementation of an oracle able to figure out spamalicious- should be stripped off as well... --Pavel Kankovsky aka Peak [ Boycott Microsoft--http://www.vcnet.com/bms ] Resistance is futile. Open your source code and prepare for assimilation.
Re: virtualdomains vs. VERP and Delivered-To
On Mon, 6 Aug 2001, Russell Nelson wrote: Charles M. Hannum writes: Uhhh, did you *read* my first piece of email? If I get a VERP address of `[EMAIL PROTECTED]', how pray tell is my mailing list software supposed to know that the mail was actually sent to `[EMAIL PROTECTED]'? A better question is: Why is the the envelope return path getting munged? Or: What does VERP have to do with Delivered-To? It's supposed to strip off the foo-owner-mycroft- prefix and the @netbsd.org suffix, and change the rightmost = into an @. Were you expecting me to write the script for you? Why strip mycroft-? -Dave
Re: virtualdomains vs. VERP and Delivered-To
Charles M. Hannum writes: There is no way for the mailing list software to get from `[EMAIL PROTECTED]' to `[EMAIL PROTECTED]' without having knowledge of virtualdomains. That's not an acceptable solution. Why not? -- -russ nelson [EMAIL PROTECTED] http://russnelson.com Crynwr sells support for free software | PGPok | 521 Pleasant Valley Rd. | +1 315 268 1925 voice | #exclude windows.h Potsdam, NY 13676-3213 | +1 315 268 9201 FAX |
Re: virtualdomains vs. VERP and Delivered-To
There is no way for the mailing list software to get from `[EMAIL PROTECTED]' to `[EMAIL PROTECTED]' without having knowledge of virtualdomains. That's not an acceptable solution. Is it really that overwhelmingly difficult to have whatever configures your bounce handler look in /var/qmail/control/virtualdomains to see what prefix to strip off the local part of the VERP address? I suspect either of us could do it in about four lines of perl. It's true, qmail doesn't work the way you might first have guessed it does. That doesn't mean it's wrong. -- John R. Levine, IECC, POB 727, Trumansburg NY 14886 +1 607 387 6869 [EMAIL PROTECTED], Village Trustee and Sewer Commissioner, http://iecc.com/johnl, Member, Provisional board, Coalition Against Unsolicited Commercial E-mail
Re: virtualdomains vs. VERP and Delivered-To
On Thu, Aug 02, 2001 at 09:39:07PM +, Charles M. Hannum wrote: In virtualdomains, I have: spamalicious.com:mycroft-spamalicious .spamalicious.com:mycroft-spamalicious When mail is sent to `[EMAIL PROTECTED]' -- e.g. from majordomo (please spare me the majordomo vs. ezmlm flames) -- I see: From [EMAIL PROTECTED] Thu Aug 02 20:13:30 2001 ... Delivered-To: [EMAIL PROTECTED] ... This seems very wrong. The Delivered-To: address here isn't even correct; it should be something the actually exists -- either `[EMAIL PROTECTED]' or `[EMAIL PROTECTED]'. Delivered-To: is considered opaque, which is satisfactory for its role as a loop inhibitor -- as long as the Delivered-To: line for a given address is the same, a loop will be detected. For VERP to be useful, the VERP address needs to be the latter; otherwise my mailing list manager won't be able to handle the bounces correctly, since it will have the wrong address. This is a (the?) bug with qmail, though I cannot find the patch at the moment.
virtualdomains vs. VERP and Delivered-To
I have a mail host -- call it netbsd.org -- that's been running qmail 1.03 for rather a long time. It uses VERP heavily to do automatic bounce handling for mailing lists. It also uses virtualdomains to serve a couple of personal vanity domains. In virtualdomains, I have: spamalicious.com:mycroft-spamalicious .spamalicious.com:mycroft-spamalicious When mail is sent to `[EMAIL PROTECTED]' -- e.g. from majordomo (please spare me the majordomo vs. ezmlm flames) -- I see: From [EMAIL PROTECTED] Thu Aug 02 20:13:30 2001 ... Delivered-To: [EMAIL PROTECTED] ... This seems very wrong. The Delivered-To: address here isn't even correct; it should be something the actually exists -- either `[EMAIL PROTECTED]' or `[EMAIL PROTECTED]'. For VERP to be useful, the VERP address needs to be the latter; otherwise my mailing list manager won't be able to handle the bounces correctly, since it will have the wrong address. Has anyone fixed this problem already?
