Re: Replacing Message-Id for SASL authenticated senders

[mouss]

Marc Patermann a écrit :
> Hi,
> 
> Bastian Blank schrieb:
>> On Sun, Feb 08, 2009 at 03:38:22AM -0500, Sahil Tandon wrote:
>>> This works as I'd expect, but will it break anything else?
>>
>> Yes. It will break the complete mail handling of the client. _Never_
>> ever touch a message id.
> Not all users are dumb. ;)
> 
> Sender:I'm missing your response, didn't you get my mail?
> Recipient: Can you tell me the message-id?
> Sender: Yes, *looks it up in "Sent"-folder* it is
> 012...@domain.tld.
> Recipient: One Moment please, *searches his mailbox* no, it's not
> here ...
> 
> Just one of the cases you might break what user would expect.
> 

This "case" is a bit artificial. People would search for the recipient
and the subject. and even if they find the message in the Sent folder,
it doesn't help much. MTA logs are more helpful.

The real problem is breaking threads. so more testing is needed.

anyway, my opinion (at this time, as I didn't yet find a sufficient
counter argument) is that the message-id should be generated by the MSA:
- it is easier to guarantee unicity, because in general, MSAs know their
"public" hostname
- it is easier to guarantee well-formed Message-IDs. MUAs (and I am
including web applications and the like) have a habit of implementing
broken behaviour
- it simplifies the MUA task. admins would (some day:) spend less time
configure the many MUAs in their network (there are obviously fewer MSAs).

note that I am not using "hide internal infos" or "ease backscatter
detection" as arguments, even though these look "interesting" to customers.

Re: Replacing Message-Id for SASL authenticated senders

[Marc Patermann]

Hi,

Bastian Blank schrieb:

On Sun, Feb 08, 2009 at 03:38:22AM -0500, Sahil Tandon wrote:

This works as I'd expect, but will it break anything else?


Yes. It will break the complete mail handling of the client. _Never_
ever touch a message id.

Not all users are dumb. ;)

Sender: I'm missing your response, didn't you get my mail?
Recipient:  Can you tell me the message-id?
Sender: Yes, *looks it up in "Sent"-folder* it is
012...@domain.tld.
Recipient:  One Moment please, *searches his mailbox* no, it's not
here ...

Just one of the cases you might break what user would expect.


Marc

Re: Replacing Message-Id for SASL authenticated senders

[Jorey Bump]

Victor Duchovni wrote, at 02/08/2009 03:37 PM:
> On Sun, Feb 08, 2009 at 09:08:32PM +0100, mouss wrote:
> 
>> No, I was referring to the "Sent" folder, populated by the MUA, either
>> in a local disk or using IMAP.
> 
> I know some people clever-enough to set "Sent == Inbox", yes this is not
> very common.

I do this, and also use the Thunderbird feature "Place replies in the
folder of the message being replied to" to keep the entire thread in a
single folder. This makes it a lot easier to review the thread in
progress and then properly archive it.

> I personally have rules that tag outgoing mail into non-default Fcc
> folders, replies are moved there too, and correct threading is expected.
> 
> Still, clearly this will do only modest harm if any for some sets of users.

Some MUAs are better than others at organizing threads. Nonetheless, I'd
be more than a little miffed if an admin broke threading and justified
it because most users are unaware of the feature.

Re: Replacing Message-Id for SASL authenticated senders

[mouss]

Victor Duchovni a écrit :
> On Sun, Feb 08, 2009 at 09:08:32PM +0100, mouss wrote:
> 
>> No, I was referring to the "Sent" folder, populated by the MUA, either
>> in a local disk or using IMAP.
> 
> I know some people clever-enough to set "Sent == Inbox", yes this is not
> very common.
> 
> I personally have rules that tag outgoing mail into non-default Fcc
> folders, replies are moved there too, and correct threading is expected.
> 
> Still, clearly this will do only modest harm if any for some sets of users.
> 

I just enabled this here. I tested and it looks like it works (threads
not broken) on
- Thunderbird
- kmail
- Claws

but are broken on Evolution.

PS. In the message-id, claws uses the hostname, while TB and kmail use
the sender domain.
Note that the test used a test folder which only contains the test
messages. I have no idea if threads will still be correctly detected
with a folder full of mail.

I would conjecture that it "works" because the response comes from a
user listed in the recipients of the original mail. so threads would
break if a message is sent to a group alias (or a list) or if the
recipient responds with a different address. I might test this...

Re: Replacing Message-Id for SASL authenticated senders

[Victor Duchovni]

On Sun, Feb 08, 2009 at 09:08:32PM +0100, mouss wrote:

> No, I was referring to the "Sent" folder, populated by the MUA, either
> in a local disk or using IMAP.

I know some people clever-enough to set "Sent == Inbox", yes this is not
very common.

I personally have rules that tag outgoing mail into non-default Fcc
folders, replies are moved there too, and correct threading is expected.

Still, clearly this will do only modest harm if any for some sets of users.

-- 
Viktor.

Disclaimer: off-list followups get on-list replies or get ignored.
Please do not ignore the "Reply-To" header.

To unsubscribe from the postfix-users list, visit
http://www.postfix.org/lists.html or click the link below:


If my response solves your problem, the best way to thank me is to not
send an "it worked, thanks" follow-up. If you must respond, please put
"It worked, thanks" in the "Subject" so I can delete these quickly.

Re: Replacing Message-Id for SASL authenticated senders

[mouss]

M. Fioretti a écrit :
> On Sun, Feb 08, 2009 18:22:17 PM +0100, mouss wrote:
>>> I mean replacing or deleting already set Message-Id headers. And
>>> it will break MUA driven thread handling
>> - very few people put their Sent mail in the same folders as
>> - received mail even then, MUAs have heuristics to cope with such
>> - situations.
> 
> ??? Maybe I'm missing something, but having sent and received messages
> in the same folder is exactly what happens whenever you have a folder
> receiving or storing mailing list messages, isn't it?

No, I was referring to the "Sent" folder, populated by the MUA, either
in a local disk or using IMAP.

I am not talking about mail you get via the MTA (which is the case for
mail forwarded by a list server).

> 
> And if I participate to a thread my sent email goes right in the
> middle of many received emails and I do want to see to which ones I
> replied and who replied directly to me. Ditto for almost everybody I
> know who uses email. Why are you saying this is a rare case?
> 

under my Thunderbird, when I send a message, a copy is put in a folder
named "Sent" (actually "Envoyés" here). The messages in this folder
don't have much infos (no Received headers, no X-Spam-* headers, ... etc).

In the case of list mail, such copies are not really useful regarding
threading, since I also have the copy of my message that was remailed by
the list server. (the copies in "Sent" may be useful for other purposes).

The only case where there would be a problem is if:

(1) I configure my MUA to save copies of my "sent" mail in the same
folder(s) as received mail is stored.

_and_

(2) I use a threaded view

Note that I am not saying that it is ok to "punish" a minority of users.

Re: Replacing Message-Id for SASL authenticated senders

[M. Fioretti]

On Sun, Feb 08, 2009 18:22:17 PM +0100, mouss wrote:
> > I mean replacing or deleting already set Message-Id headers. And
> > it will break MUA driven thread handling
> 
> - very few people put their Sent mail in the same folders as
> - received mail even then, MUAs have heuristics to cope with such
> - situations.

??? Maybe I'm missing something, but having sent and received messages
in the same folder is exactly what happens whenever you have a folder
receiving or storing mailing list messages, isn't it?

And if I participate to a thread my sent email goes right in the
middle of many received emails and I do want to see to which ones I
replied and who replied directly to me. Ditto for almost everybody I
know who uses email. Why are you saying this is a rare case?

Thanks,
Marco
-- 
Your own civil rights and the quality of your life heavily depend on how
software is used *around* you:http://digifreedom.net/node/84

Re: Replacing Message-Id for SASL authenticated senders

[Sahil Tandon]

On Feb 8, 2009, at 1:02 PM, mouss  wrote:


Victor Duchovni a écrit :

On Sun, Feb 08, 2009 at 06:22:17PM +0100, mouss wrote:

I mean replacing or deleting already set Message-Id headers. And  
it will

break MUA driven thread handling
- very few people put their Sent mail in the same folders as  
received mail

- even then, MUAs have heuristics to cope with such situations.


Why break message-id threading for those (few) people?



it would be interesting to see if that would break. I may test with a
few MUAs.

As for the argument, the common one I heard is "hiding internal
information". sure, this is not a very satisfactory argument. but if a
customer says so and they "have no users who use a threaded view", I  
can

hardly find a counter-argument.


Exactly.

Re: Replacing Message-Id for SASL authenticated senders

[Sahil Tandon]

On Feb 8, 2009, at 12:01 PM, Bastian Blank > wrote:



On Sun, Feb 08, 2009 at 11:13:53AM -0500, Sahil Tandon wrote:

On Sun, 08 Feb 2009, Bastian Blank wrote:

Yes. It will break the complete mail handling of the client. _Never_
ever touch a message id.
Do explain how adding/replacing a valid Message-ID only to  
submitted mail

will "break the complete mail handling of the client".


I mean replacing or deleting already set Message-Id headers. And it  
will

break MUA driven thread handling and similar things which is based on
the original submitted ID.


   And how do you
reconcile your last sentence with section 8.3 of RFC 4409?


Please read the RFC again. This sentence only speaks about _adding_  
this

header, not about replacing or deleting it. There is however no other
point which allows arbitrary modifications to the messages.


There *is* mention of replacing and the replacing is not arbitrary.   
The breaking of threading is interesting but  irrelevant in my  
particular case.  Thanks everyone for the input.


- Sahil

Re: Replacing Message-Id for SASL authenticated senders

[mouss]

Victor Duchovni a écrit :
> On Sun, Feb 08, 2009 at 06:22:17PM +0100, mouss wrote:
> 
>>> I mean replacing or deleting already set Message-Id headers. And it will
>>> break MUA driven thread handling 
>> - very few people put their Sent mail in the same folders as received mail
>> - even then, MUAs have heuristics to cope with such situations.
> 
> Why break message-id threading for those (few) people?
> 

it would be interesting to see if that would break. I may test with a
few MUAs.

As for the argument, the common one I heard is "hiding internal
information". sure, this is not a very satisfactory argument. but if a
customer says so and they "have no users who use a threaded view", I can
hardly find a counter-argument.

Re: Replacing Message-Id for SASL authenticated senders

[Victor Duchovni]

On Sun, Feb 08, 2009 at 06:22:17PM +0100, mouss wrote:

> > I mean replacing or deleting already set Message-Id headers. And it will
> > break MUA driven thread handling 
> 
> - very few people put their Sent mail in the same folders as received mail
> - even then, MUAs have heuristics to cope with such situations.

Why break message-id threading for those (few) people?

-- 
Viktor.

Disclaimer: off-list followups get on-list replies or get ignored.
Please do not ignore the "Reply-To" header.

To unsubscribe from the postfix-users list, visit
http://www.postfix.org/lists.html or click the link below:


If my response solves your problem, the best way to thank me is to not
send an "it worked, thanks" follow-up. If you must respond, please put
"It worked, thanks" in the "Subject" so I can delete these quickly.

Re: Replacing Message-Id for SASL authenticated senders

[rafa]

mouss wrote:

and if a spam filter blocks/discards/quarantines mail because of this,
it is the filter that should be blamed.



I use this setup for detecting Backscatter. Until now without problems, 
but it's difficult to know.

Re: Replacing Message-Id for SASL authenticated senders

[mouss]

Bastian Blank a écrit :
> On Sun, Feb 08, 2009 at 11:13:53AM -0500, Sahil Tandon wrote:
>> On Sun, 08 Feb 2009, Bastian Blank wrote:
>>> Yes. It will break the complete mail handling of the client. _Never_
>>> ever touch a message id.
>> Do explain how adding/replacing a valid Message-ID only to submitted mail 
>> will "break the complete mail handling of the client".
> 
> I mean replacing or deleting already set Message-Id headers. And it will
> break MUA driven thread handling 

- very few people put their Sent mail in the same folders as received mail
- even then, MUAs have heuristics to cope with such situations.

> and similar things which is based on
> the original submitted ID.
> 

other than spam filters that try to detect forged mail, the original
message-id, if replaced by the MSA, is irrelevant.

and if a spam filter blocks/discards/quarantines mail because of this,
it is the filter that should be blamed.

after all, the practice of hiding private infos is not new. whether it
is called "security by obscurity" or "an additional layer of security",
it's there.

I might give this a try and see what happens...

>> And how do you
>> reconcile your last sentence with section 8.3 of RFC 4409?
> 
> Please read the RFC again. This sentence only speaks about _adding_ this
> header, not about replacing or deleting it. There is however no other
> point which allows arbitrary modifications to the messages.
> 

correction: The rfc speaks about _replacing_ the message-id if it is
malformed. (and what Sahil wants to do is replace the MUA generated
message-id with one generated by postfix).

while this doesn't say anything about replacing a well formed
message-id, it indirectly acknowledges the fact that message-id's may be
replaced. and such things do happen.

In case you missed it: Sahil wants to _replace_ the message-id generated
by the MUA by one generated by postfix. He uses the fact that postfix
will add a message-id if one is missing.

Re: Replacing Message-Id for SASL authenticated senders

[Bastian Blank]

On Sun, Feb 08, 2009 at 11:13:53AM -0500, Sahil Tandon wrote:
> On Sun, 08 Feb 2009, Bastian Blank wrote:
> > Yes. It will break the complete mail handling of the client. _Never_
> > ever touch a message id.
> Do explain how adding/replacing a valid Message-ID only to submitted mail 
> will "break the complete mail handling of the client".

I mean replacing or deleting already set Message-Id headers. And it will
break MUA driven thread handling and similar things which is based on
the original submitted ID.

> And how do you
> reconcile your last sentence with section 8.3 of RFC 4409?

Please read the RFC again. This sentence only speaks about _adding_ this
header, not about replacing or deleting it. There is however no other
point which allows arbitrary modifications to the messages.

Bastian

-- 
If a man had a child who'd gone anti-social, killed perhaps, he'd still
tend to protect that child.
-- McCoy, "The Ultimate Computer", stardate 4731.3

Re: Replacing Message-Id for SASL authenticated senders

[Sahil Tandon]

On Sun, 08 Feb 2009, Bastian Blank wrote:

> On Sun, Feb 08, 2009 at 03:38:22AM -0500, Sahil Tandon wrote:
> > This works as I'd expect, but will it break anything else?
> 
> Yes. It will break the complete mail handling of the client. _Never_
> ever touch a message id.

Do explain how adding/replacing a valid Message-ID only to submitted mail 
will "break the complete mail handling of the client".  And how do you
reconcile your last sentence with section 8.3 of RFC 4409?

-- 
Sahil Tandon 

Re: Replacing Message-Id for SASL authenticated senders

[mouss]

Sahil Tandon a écrit :
> I have been asked to replace the MUA Message-ID of SASL senders with a
> Postfix-generated ID.  The Message-ID of incoming mail which arrives via the
> same Postfix instance, but does not originate from a SASL authenticated
> sender, should not be touched. The submission service runs on port 587.  Are 
> there any unintended consequences with (or more efficient alternatives to)
> the following implementation?
> 
> In master.cf:
> 
> - create cleanup clone called "special" with its own header_checks setting.
> - add "-o cleanup_service_name=special" under the submission service
> 
> In header_checks.submit, which is referenced by the cleanup clone:
> /^Message-/ IGNORE
> 
> This works as I'd expect, but will it break anything else?
> 

as long as you only modify "submitted" mail, it should be ok. after all,
LookOut no more generates a message-id. and the fact that the generated
message-id will differ from the one in the Sent folder is moot.

While I am in, RFC 4409, 8.3, says:

   The MSA SHOULD add or replace the 'Message-ID' field, if it lacks it,
   or it is not valid syntax (as defined by [MESSAGE-FORMAT]).  Note
   that a number of clients still do not generate Message-ID fields.

Re: Replacing Message-Id for SASL authenticated senders

[Bastian Blank]

On Sun, Feb 08, 2009 at 03:38:22AM -0500, Sahil Tandon wrote:
> This works as I'd expect, but will it break anything else?

Yes. It will break the complete mail handling of the client. _Never_
ever touch a message id.

Bastian

-- 
Fascinating, a totally parochial attitude.
-- Spock, "Metamorphosis", stardate 3219.8

Replacing Message-Id for SASL authenticated senders

[Sahil Tandon]

I have been asked to replace the MUA Message-ID of SASL senders with a
Postfix-generated ID.  The Message-ID of incoming mail which arrives via the
same Postfix instance, but does not originate from a SASL authenticated
sender, should not be touched. The submission service runs on port 587.  Are 
there any unintended consequences with (or more efficient alternatives to)
the following implementation?

In master.cf:

- create cleanup clone called "special" with its own header_checks setting.
- add "-o cleanup_service_name=special" under the submission service

In header_checks.submit, which is referenced by the cleanup clone:
/^Message-/ IGNORE

This works as I'd expect, but will it break anything else?

-- 
Sahil Tandon