Re: Removing trace records on submission MSA

[@lbutlr]

On 2018-05-02 (14:57 MDT), Philip Paeps  wrote:
> When it comes in over submission from authenticated users, I consider the 
> HELO hostname, the IP address and the reverse lookup of the IP address 
> sensitive. 


Hmm. OK, I do not see my home IP address in any mail header (and yes, I use 
submission and that is the only way to send mail from my server), nor do i see 
the helo hostname.

These are the received headers from a message I sent to an iCloud account I 
have:

Received: from mr28p00im-smtpin039.mac.com ([17.110.71.38])
 by ms20521.mac.com (Oracle Communications Messaging Server 8.0.1.3.20170906
 64bit (built Sep  6 2017)) with ESMTP id <0p85003x1u2a9...@ms20521.mac.com>
 for *user*@icloud.com; Thu, 03 May 2018 16:06:10 + (GMT)
Original-recipient: rfc822;*user*@icloud.com
Received: from mail.covisp.net (www.covisp.net [65.121.55.45])
 by mr28p00im-smtpin039.me.com
 (Oracle Communications Messaging Server 8.0.1.2.20170607 64bit (built Jun  7
 2017)) with ESMTPS id <0p8500evtu27d...@mr28p00im-smtpin039.me.com> for
 *user*@icloud.com (ORCPT *user*@icloud.com); Thu, 03 May 2018 16:06:10 + 
(GMT)
Received-SPF: pass (mr21p00im-spfmilter008.me.com: domain of kr...@kreme.com
 designates 65.121.55.45 as permitted sender)
 receiver=mr21p00im-spfmilter008.me.com; client-ip=65.121.55.45;
 helo=mail.covisp.net; envelope-from=kr...@kreme.com

I also tried it with a local domain to another local domain, sending to a gmail 
address, and sending from an iCloud account to gmail (don't think this would 
involve postfix at all). In none of these where my home IP or helo name or user 
login name included in the headers.

Now I wonder why?

-- 
"Is that a star?" "Nah, that's Ted Danson."

Re: Removing trace records on submission MSA

[Wietse Venema]

Philip Paeps:
> On 2018-05-02 20:52:46 (+0200), @lbutlr wrote:
> > On 2018-05-01 (04:02 MDT), Philip Paeps  wrote:
> >> I wonder if it wouldn't be easier to add a configuration option to 
> >> smtpd to suitably expurgate Received: headers of sensitive 
> >> information.
> >
> > What information in the Received header do you consider sensitive?
> 
> When it comes in over submission from authenticated users, I consider 
> the HELO hostname, the IP address and the reverse lookup of the IP 
> address sensitive.  Those data allow the user to be tracked around the 
> internet based on where they send email from.
> 
> The queue id, the date and the sasl username are sufficient trace 
> information to grep in logfiles if something needs to be debugged.
> 
> Note that I'm only talking about submission.  The trace headers added on 
> mail being relayed are perfectly fine.
> 
> I'm not sure if there's a tidy way to implement this as an option.  The 
> hairy header_checks hack also "just works".  My mind just rebels against 
> something so conceptually simple requiring such a crazy regular 
> expresion. :)

Instead of saying what to remove from headers, it would be more
natural to say what should be in headers. Configurable headers (not
just From: or Recived:) are on the wishlist for as long as Postfix
exists.

Wietse

Re: Removing trace records on submission MSA

[Philip Paeps]

On 2018-05-02 20:52:46 (+0200), @lbutlr wrote:

On 2018-05-01 (04:02 MDT), Philip Paeps  wrote:
I wonder if it wouldn't be easier to add a configuration option to 
smtpd to suitably expurgate Received: headers of sensitive 
information.


What information in the Received header do you consider sensitive?


When it comes in over submission from authenticated users, I consider 
the HELO hostname, the IP address and the reverse lookup of the IP 
address sensitive.  Those data allow the user to be tracked around the 
internet based on where they send email from.


The queue id, the date and the sasl username are sufficient trace 
information to grep in logfiles if something needs to be debugged.


Note that I'm only talking about submission.  The trace headers added on 
mail being relayed are perfectly fine.


I'm not sure if there's a tidy way to implement this as an option.  The 
hairy header_checks hack also "just works".  My mind just rebels against 
something so conceptually simple requiring such a crazy regular 
expresion. :)


Philip

--
Philip Paeps
Senior Reality Engineer
Ministry of Information

Re: Removing trace records on submission MSA

[@lbutlr]

On 2018-05-01 (04:02 MDT), Philip Paeps  wrote:
> 
> I wonder if it wouldn't be easier to add a configuration option to smtpd to 
> suitably expurgate Received: headers of sensitive information.

What information in the Received header do you consider sensitive?

-- 
"You see, in this world there's two kinds of people, my friend: Those
with loaded guns and those who dig. You dig."

Re: Removing trace records on submission MSA

[Philip Paeps]

On 2018-03-10 22:01:01 (+0100), J Doe wrote:
I have a question in regards to removing some trace records when 
providing submission on Postfix 3.1.x and later.


Apologies for resurrecting an old thread.

I had some time to kill yesterday and I came up with this PCRE monster:

/^Received:.*([\n]).*sender: (.+?)\).*(by.+?)\).*(id \w+).*(;.*)/
  REPLACE Received: ${3}, authenticated sender ${2})${1} ${4}${5}

It's a bit hairy but it makes the Received: header of a submission user 
look a lot like the Received: header added by local delivery:


Received: by rincewind.trouble.is (Postfix, authenticated sender 
philip)

  id X; Tue, 1 May 2018 09:56:20 + (UTC)

I wonder if it wouldn't be easier to add a configuration option to smtpd 
to suitably expurgate Received: headers of sensitive information.


This is working for me though.  It's ugly but it seems to work for all 
my users and the exotic devices they use.


Philip

--
Philip Paeps
Senior Reality Engineer
Ministry of Information

Re: Removing trace records on submission MSA

[J Doe]

Hi Viktor,

> On Apr 7, 2018, at 1:32 PM, Viktor Dukhovni  
> wrote:
> 
> It is now portable POSIX.  For the record, in email the allowed whitespace is 
> more narrow than
> is recognized by [[:space:]], you're not likely to run into any false 
> positives.  The email
> header whitespace consists of just SPACE, TAB, CR and LF.  VT and FF 
> (vertical tab and form feed)
> are not valid whitespace in email headers.

Ok, great!  Thank you for those observations about whitespace

Thanks to everyone else on this thread who also provided examples and 
suggestions.  Going forward I will take the advice to install PCRE support.

- J

Re: Removing trace records on submission MSA

[Viktor Dukhovni]

> On Apr 7, 2018, at 3:05 AM, J Doe  wrote:
> 
> Thank you for your reply.
> 
> It occurred to me that I could side-step the issue of GNU extensions and 
> whether they’re supported by converting the string I e-mailed a couple of 
> e-mails back to full POSIX regex (in this case removing the \s).  I ended up 
> with:
> 
> 
> /^(Received:[[:space:]]from)[^;]+(;[[:space:]][A-Z]{1}[a-z]{2,3},)[[:space:]]+([[:digit:]]{1,2}[^\n]+)/
> REPLACE $1 [127.0.0.1] (localhost [127.0.0.1]) by myserver.com$2 $3
> 
> …and it works.
> 
> Have I missed anything else that needs to be converted so that the regular 
> expression is POSIX only ?

It is now portable POSIX.  For the record, in email the allowed whitespace is 
more narrow than
is recognized by [[:space:]], you're not likely to run into any false 
positives.  The email
header whitespace consists of just SPACE, TAB, CR and LF.  VT and FF (vertical 
tab and form feed)
are not valid whitespace in email headers.

-- 
Viktor.

Re: Removing trace records on submission MSA

[J Doe]

Hi Viktor and Dominic,

> On Apr 7, 2018, at 2:46 AM, Dominic Raferd  wrote:
> 
> On 7 April 2018 at 07:39, J Doe  > wrote:
> Hi Viktor and Dominic,
> 
> If I do the following on Ubuntu 16.04 LTS:
> 
> $ echo "1 2" | egrep '[[:digit:]]\s[[:digit:]]’
> 1 2
> 
> … where “1 2” are highlighted in bash
> 
> Am I correct that since this POSIX regex for the digits AND the \s is still 
> being interpreted, my system must support the GNU regex extensions ?
> 
> ​It is standard Ubuntu it will support GNU regex extensions, but why not use 
> pcre? It is more powerful, more standardised, and - my impression​ - more 
> widely used for Postfix tables. Just do:
> $ sudo apt-get install postfix-pcre

Thank you for your reply.

It occurred to me that I could side-step the issue of GNU extensions and 
whether they’re supported by converting the string I e-mailed a couple of 
e-mails back to full POSIX regex (in this case removing the \s).  I ended up 
with:


/^(Received:[[:space:]]from)[^;]+(;[[:space:]][A-Z]{1}[a-z]{2,3},)[[:space:]]+([[:digit:]]{1,2}[^\n]+)/
REPLACE $1 [127.0.0.1] (localhost [127.0.0.1]) by myserver.com$2 $3

…and it works.

Have I missed anything else that needs to be converted so that the regular 
expression is POSIX only ?

- J

Re: Removing trace records on submission MSA

[Dominic Raferd]

On 7 April 2018 at 07:39, J Doe  wrote:

> Hi Viktor and Dominic,
>
> If I do the following on Ubuntu 16.04 LTS:
>
> $ echo "1 2" | egrep '[[:digit:]]\s[[:digit:]]’
> 1 2
>
> … where “1 2” are highlighted in bash
>
> Am I correct that since this POSIX regex for the digits AND the \s is
> still being interpreted, my system must support the GNU regex extensions ?


​It is standard Ubuntu it will support GNU regex extensions, but why not
use pcre? It is more powerful, more standardised, and - my impression​ -
more widely used for Postfix tables. Just do:
$ sudo apt-get install postfix-pcre

Re: Removing trace records on submission MSA

[J Doe]

Hi Viktor and Dominic,

If I do the following on Ubuntu 16.04 LTS:

$ echo "1 2" | egrep '[[:digit:]]\s[[:digit:]]’
1 2

… where “1 2” are highlighted in bash

Am I correct that since this POSIX regex for the digits AND the \s is still 
being interpreted, my system must support the GNU regex extensions ?

Thanks,

- J

Re: Removing trace records on submission MSA

[J Doe]

Hi Viktor,

> On Apr 7, 2018, at 2:04 AM, Viktor Dukhovni  
> wrote:
> 
> FreeBSD 11 (POSIX):
> 
>  $ echo "1 b" | egrep '\d\s\w'
>  $
> 
> MacOS High Sierra (POSIX with GNU or similar extensions):
> 
>  $ echo "1 b" | egrep '\d\s\w'
>  1 b
>  $
> 
> Your Ubuntu system most likely will match the MacOS results.  Which means 
> that your regexp table is not portable, but happens to work on your system.

I can confirm the expected result for MacOS High Sierra . . . but interestingly 
enough, I get the FreeBSD 11 results on Ubuntu 16.04 LTS.

Are there any other tests I can try to confirm the existence of GNU extensions 
to POSIX regular expressions ?

Thanks,

- J

Re: Removing trace records on submission MSA

[Dominic Raferd]

On 7 April 2018 at 07:04, Viktor Dukhovni 
wrote:

>
>
> > On Apr 7, 2018, at 1:59 AM, J Doe  wrote:
> >
> > Ah, interesting - that must be it, then.
> >
> > This is on an Ubuntu 16.04 LTS server.  I can see the dependencies
> compiled in from Ubuntu’s page [1] and GNU libc is listed.  [2] seems to
> suggest that regular expressions are part of GNU libc.
> >
> > Is there another way I can confirm that ?
>
> FreeBSD 11 (POSIX):
>
>   $ echo "1 b" | egrep '\d\s\w'
>   $
>
> MacOS High Sierra (POSIX with GNU or similar extensions):
>
>   $ echo "1 b" | egrep '\d\s\w'
>   1 b
>   $
>
> Your Ubuntu system most likely will match the MacOS results.  Which means
> that your regexp table is not portable, but happens to work on your system.


On my Ubuntu 16.04.4:
$ echo "1 b" | grep -E '\d\s\w'
$ echo "1 b" | grep -E '[0-9]\s\w'
1 b
​
But I agree OP should switch to using pcre tables, it is supported under
Ubuntu.

Re: Removing trace records on submission MSA

[Viktor Dukhovni]

> On Apr 7, 2018, at 1:59 AM, J Doe  wrote:
> 
> Ah, interesting - that must be it, then.
> 
> This is on an Ubuntu 16.04 LTS server.  I can see the dependencies compiled 
> in from Ubuntu’s page [1] and GNU libc is listed.  [2] seems to suggest that 
> regular expressions are part of GNU libc.
> 
> Is there another way I can confirm that ?

FreeBSD 11 (POSIX):

  $ echo "1 b" | egrep '\d\s\w'
  $

MacOS High Sierra (POSIX with GNU or similar extensions):

  $ echo "1 b" | egrep '\d\s\w'
  1 b
  $

Your Ubuntu system most likely will match the MacOS results.  Which means that 
your regexp table is not portable, but happens to work on your system.

-- 
Viktor.

Re: Removing trace records on submission MSA

[J Doe]

Hi Viktor,

> On Apr 7, 2018, at 1:50 AM, Viktor Dukhovni  
> wrote:
> 
>> On Apr 7, 2018, at 1:34 AM, J Doe  wrote:
>> 
>> mmm.  I just sent a test message via submission to a Gmail account and 
>> checked the headers and the replacement works.
>> 
>> According to the site [1]   \s is shorthand for POSIX regular expressions.
>> 
>> Perhaps the POSIX regex library compiled with Postfix now supports this ?
> 
> No "\s" is not a POSIX feature, it is however a GNU extension, so you may have
> a GNU regexp library, that supports "\s" (outside bracket expressions):
> 
>   https://www.regular-expressions.info/gnu.html

Ah, interesting - that must be it, then.

This is on an Ubuntu 16.04 LTS server.  I can see the dependencies compiled in 
from Ubuntu’s page [1] and GNU libc is listed.  [2] seems to suggest that 
regular expressions are part of GNU libc.

Is there another way I can confirm that ?

Thanks,

- J

Sources:

[1] https://packages.ubuntu.com/xenial/postfix 

[2] https://www.gnu.org/software/libc/manual/html_node/Regular-Expressions.html 

Re: Removing trace records on submission MSA

[Viktor Dukhovni]

> On Apr 7, 2018, at 1:34 AM, J Doe  wrote:
> 
> mmm.  I just sent a test message via submission to a Gmail account and 
> checked the headers and the replacement works.
> 
> According to the site [1]   \s is shorthand for POSIX regular expressions.
> 
> Perhaps the POSIX regex library compiled with Postfix now supports this ?

No "\s" is not a POSIX feature, it is however a GNU extension, so you may have
a GNU regexp library, that supports "\s" (outside bracket expressions):

   https://www.regular-expressions.info/gnu.html

-- 
Viktor.

Re: Removing trace records on submission MSA

[J Doe]

Hi Viktor,

> On Apr 7, 2018, at 1:26 AM, Viktor Dukhovni  
> wrote:
>> On Apr 7, 2018, at 1:23 AM, J Doe  wrote:
>> 
>> I did some Googling for doing PCRE to POSIX regular expressions and updated 
>> the string:
>> 
>>   
>> /^(Received:\sfrom)[^;]+(;\s[A-Z]{1}[a-z]{2,3},)\s+([[:digit:]]{1,2}[^\n]+)/ 
>> REPLACE $1 [127.0.0.1] (localhost [127.0.0.1]) by myserver.com$2 $3
>> 
>> … and it works!
> 
> It can't, the above is still PCRE, "\s" for whitespace is PCRE, not "regexp". 
>  Perhaps that's no the string you're using.
> 
> My advice is to ditch regexp and use PCRE.  Install the package that adds 
> PCRE support.

Hmmm.  I just sent a test message via submission to a Gmail account and checked 
the headers and the replacement works.

According to the site [1]   \s is shorthand for POSIX regular expressions.

Perhaps the POSIX regex library compiled with Postfix now supports this ?

Sources:

[1] https://www.regular-expressions.info/posixbrackets.html 

Re: Removing trace records on submission MSA

[Viktor Dukhovni]

> On Apr 7, 2018, at 1:23 AM, J Doe  wrote:
> 
> I did some Googling for doing PCRE to POSIX regular expressions and updated 
> the string:
> 
>
> /^(Received:\sfrom)[^;]+(;\s[A-Z]{1}[a-z]{2,3},)\s+([[:digit:]]{1,2}[^\n]+)/ 
> REPLACE $1 [127.0.0.1] (localhost [127.0.0.1]) by myserver.com$2 $3
> 
> … and it works!

It can't, the above is still PCRE, "\s" for whitespace is PCRE, not "regexp".  
Perhaps that's no the string you're using.

My advice is to ditch regexp and use PCRE.  Install the package that adds PCRE 
support.

-- 
Viktor.

Re: Removing trace records on submission MSA

[J Doe]

Hi Viktor,

> On Apr 7, 2018, at 12:36 AM, Viktor Dukhovni  
> wrote:
> 
> That's PCRE syntax.
> 
>> Does anyone know what I’m doing wrong and/or is there a way to make Postfix 
>> provide more debug output for a regexp: operation ?
> 
> You're using a "regexp" table, those don't support PCRE.

Thanks for your response and that’s definitely the problem I had.

I did some Googling for doing PCRE to POSIX regular expressions and updated the 
string:


/^(Received:\sfrom)[^;]+(;\s[A-Z]{1}[a-z]{2,3},)\s+([[:digit:]]{1,2}[^\n]+)/ 
REPLACE $1 [127.0.0.1] (localhost [127.0.0.1]) by myserver.com$2 $3

… and it works!

Not sure why I had it in my head that regexp: was PCRE (especially when there’s 
another dictionary type that explicitly says pcre:), but at least I’ve learned 
something.

- J

Re: Removing trace records on submission MSA

[Viktor Dukhovni]

> On Apr 7, 2018, at 12:34 AM, J Doe  wrote:
> 
> I tried the following with a visual regex program (to make checking captures 
> easier):
> 
> /etc/postfix/submission_privacy_header
> 
> /(Received\:\s*from)[^\;]+(\;\s[A-Z]{1}[a-z]{2,3}\,)\s+(\d{1,2}[^\n]+)/
> REPLACE $1 [127.0.0.1] (localhost [127.0.0.1]) by myserver.com$2 $3

That's PCRE syntax.

> Does anyone know what I’m doing wrong and/or is there a way to make Postfix 
> provide more debug output for a regexp: operation ?

You're using a "regexp" table, those don't support PCRE.

-- 
Viktor.

Re: Removing trace records on submission MSA

[J Doe]

Hi Karol,

> I am using this:
> 
> /^(Received:) from.*]\).*(.{2}by mail\.nimitz\.pl.*Postfix.*) (with
> [E]{0,1}SMTP[S]{0,1}[A]{0,1}) (.*)/ REPLACE $1 from mail.nimitz.pl
> (localhost [127.0.0.1])$2 with SMTP $4
> 
> Just change 'mail.nimitz.pl' with FQDN of your server. This expression
> works for me and also removes information about the connection, which in
> my case can tell if the mail was sent from webmail (unencrypted
> connection from webmail host to postfix host) or client's MUA
> (encrypted).
> 
> It can probably fail on some systems due to .* matching, which is
> greedy, but I wrote it many years ago and it works, so I am not fixing
> it.


Thanks for this.  I’m looking to mask out the DDNS name of a xDSL connection.

I tried the following with a visual regex program (to make checking captures 
easier):

/etc/postfix/submission_privacy_header
/(Received\:\s*from)[^\;]+(\;\s[A-Z]{1}[a-z]{2,3}\,)\s+(\d{1,2}[^\n]+)/ 
   REPLACE $1 [127.0.0.1] (localhost [127.0.0.1]) by myserver.com$2 $3

…however this does not match when Postfix evaluates it.  I am attempting three 
captures:

1. Received: from
2. The first part of the date from the ; to the , after the day (eg: “Fri”)
3. The last part of the date from the numerical day number to the end

The reason for the two part handling of the date is I want to strip out 
whitespace between “Day,  6 Apr . . .”.  If I don’t strip this out it puts a 
line break inline in the Received: header that breaks the date over two lines.

Does anyone know what I’m doing wrong and/or is there a way to make Postfix 
provide more debug output for a regexp: operation ?

Thanks,

- J

Re: Removing trace records on submission MSA

[Karol Augustin]

On 06/04/2018 09:27 PM, J Doe wrote:
> Hi Philip,
> 
>>> Thank you for your reply.
>>>
>>> I currently use DKIM and as per the RFC for DKIM, I don’t include trace 
>>> headers in the message hash that makes up the DKIM signature.  I am under 
>>> the impression that my DKIM signatures should be correct in this case if I 
>>> use your solution and it re-writes the first trace header - is that true or 
>>> are there any other DKIM issues I might run into ?
>>
>> Unless you have specifically configured your DKIM setup to include trace 
>> headers in the hash (which you should not do according to the RFC), your 
>> DKIM signatures will continue to be correct if you anonymise the first trace 
>> header like I do.
> 
> Thank you for your reply.
> 
> I configured master.cf and created the regular expression lookup
> table, but my installation of Postfix (3.1.0), does not appear to
> support PCRE as placing “pcre:” as the dictionary type in master.cf
> generated an error that "this dictionary type is unsupported".
> 
> Some Googling revealed that I may be able to install support for that,
> but rather than install something else I switched to “regexp:”. 
> Unfortunately, regexp stated there was an error in the regular
> expression string (the error indicated the line but not the character
> in the regexp that it did not like).
> 
> My regular expression skills are rusty, so I went with an unoptimized
> search string:
> 
> /etc/postifx/submission_header_rules
> 
> /Received: from/ REPLACE Received: from [127.0.0.1]
> (localhost [127.0.0.1]) by server.com 
> 
> … where server.com  is the FQDN for my mail server.
> 
> As I have this configured for submission, I then tested sending e-mail
> to Gmail and can confirm that my DKIM is still valid (as expected - I
> don’t include Received: headers in the DKIM hash, as the DKIM RFC
> recommends), and this is not doing any unwanted edits on mail over
> port 25.
> 
> I figured this was sufficient but further reading indicates that some
> anti spam software pays attention to the Received: headers (although
> most sources noted this was an issue when configuring Postfix to
> *DELETE* the first header, which I don’t want to do).
> 
> With that in mind, I had two questions:
> 
> ** Is there any anti spam software that checks for the date and time
> at the end of the Received: string ?  My very basic search string does
> not capture the date and time after the semicolon and therefore does
> not show up.
> 
> ** If there is anti spam software that looks for the date and time,
> could you help me construct a “regexp:” compatible search string ?  I
> experimented with captures but again, my regular expression skills are
> bad at the moment.
> 
I am using this:

/^(Received:) from.*]\).*(.{2}by mail\.nimitz\.pl.*Postfix.*) (with
[E]{0,1}SMTP[S]{0,1}[A]{0,1}) (.*)/ REPLACE $1 from mail.nimitz.pl
(localhost [127.0.0.1])$2 with SMTP $4

Just change 'mail.nimitz.pl' with FQDN of your server. This expression
works for me and also removes information about the connection, which in
my case can tell if the mail was sent from webmail (unencrypted
connection from webmail host to postfix host) or client's MUA
(encrypted).

It can probably fail on some systems due to .* matching, which is
greedy, but I wrote it many years ago and it works, so I am not fixing
it.

k.



-- 
Karol Augustin
ka...@augustin.pl
http://karolaugustin.pl/
+353 85 775 5312

Re: Removing trace records on submission MSA

[J Doe]

Hi Philip,

>> Thank you for your reply.
>> 
>> I currently use DKIM and as per the RFC for DKIM, I don’t include trace 
>> headers in the message hash that makes up the DKIM signature.  I am under 
>> the impression that my DKIM signatures should be correct in this case if I 
>> use your solution and it re-writes the first trace header - is that true or 
>> are there any other DKIM issues I might run into ?
> 
> Unless you have specifically configured your DKIM setup to include trace 
> headers in the hash (which you should not do according to the RFC), your DKIM 
> signatures will continue to be correct if you anonymise the first trace 
> header like I do.

Thank you for your reply.

I configured master.cf and created the regular expression lookup table, but my 
installation of Postfix (3.1.0), does not appear to support PCRE as placing 
“pcre:” as the dictionary type in master.cf generated an error that "this 
dictionary type is unsupported".

Some Googling revealed that I may be able to install support for that, but 
rather than install something else I switched to “regexp:”.  Unfortunately, 
regexp stated there was an error in the regular expression string (the error 
indicated the line but not the character in the regexp that it did not like).

My regular expression skills are rusty, so I went with an unoptimized search 
string:

/etc/postifx/submission_header_rules

/Received: from/ REPLACE Received: from [127.0.0.1] (localhost 
[127.0.0.1]) by server.com 

… where server.com  is the FQDN for my mail server.

As I have this configured for submission, I then tested sending e-mail to Gmail 
and can confirm that my DKIM is still valid (as expected - I don’t include 
Received: headers in the DKIM hash, as the DKIM RFC recommends), and this is 
not doing any unwanted edits on mail over port 25.

I figured this was sufficient but further reading indicates that some anti spam 
software pays attention to the Received: headers (although most sources noted 
this was an issue when configuring Postfix to *DELETE* the first header, which 
I don’t want to do).

With that in mind, I had two questions:

** Is there any anti spam software that checks for the date and time at the end 
of the Received: string ?  My very basic search string does not capture the 
date and time after the semicolon and therefore does not show up.

** If there is anti spam software that looks for the date and time, could you 
help me construct a “regexp:” compatible search string ?  I experimented with 
captures but again, my regular expression skills are bad at the moment.

Thanks,

- J

Re: Removing trace records on submission MSA

[Philip Paeps]

On 2018-04-05 08:54:45 (+0800), J Doe wrote:


Hi Phillip,

I have a question in regards to removing some trace records when 
providing submission on Postfix 3.1.x and later.


While reading RFC 6409 (“Message Submission for Mail”), I note 
that the RFC observes that:


  "Even when submitted messages are complete, local site policy may  
  dictate that the message text be examined or modified in some way, 
   e.g., to conceal local name or address spaces.”


By this I take it that I could remove perhaps the initial trace 
message that returns information about internal addresses and 
network names.  It seems to me that both Hotmail/Outlook and Gmail 
do this.


Is this acceptable ?  The only bad side to it would appear to be 
possibly some increased difficulty in troubleshooting.


If it is an acceptable process, how would I configure Postfix to do 
this only on submission ?


I anonymise the initial Received: header with a header_checks on the 
submission service.


In master.cf, I add `-o cleanup_service_name=subcleanup` to the 
submission service.  That service is defined as:


  subcleanup  unix n   -   n   -   0   cleanup
-o syslog_name=postfix/subcleanup
-o 
header_checks=pcre:$config_directory/submission_header_checks.pcre


The submission_header_checks.pcre file contains:

  /^\s*(Received: from .+?(?=\s\())[^\n]*(.*for <.*)/ REPLACE $1 
(localhost [127.0.0.1])$2


I'm sure there are better ways to do this, but this works for me.

It doesn't interfere with debugging much because the logs will 
mentain the replacement and it's easy to grep for.


Thank you for your reply.

I currently use DKIM and as per the RFC for DKIM, I don’t include 
trace headers in the message hash that makes up the DKIM signature.  I 
am under the impression that my DKIM signatures should be correct in 
this case if I use your solution and it re-writes the first trace 
header - is that true or are there any other DKIM issues I might run 
into ?


Unless you have specifically configured your DKIM setup to include trace 
headers in the hash (which you should not do according to the RFC), your 
DKIM signatures will continue to be correct if you anonymise the first 
trace header like I do.


Philip

--
Philip Paeps
Senior Reality Engineer
Ministry of Information

Re: Removing trace records on submission MSA

[J Doe]

Hi Phillip,

>> I have a question in regards to removing some trace records when providing 
>> submission on Postfix 3.1.x and later.
>> 
>> While reading RFC 6409 (“Message Submission for Mail”), I note that the RFC 
>> observes that:
>> 
>>   "Even when submitted messages are complete, local site policy may
>> dictate that the message text be examined or modified in some way,e.g., 
>> to conceal local name or address spaces.”
>> 
>> By this I take it that I could remove perhaps the initial trace message that 
>> returns information about internal addresses and network names.  It seems to 
>> me that both Hotmail/Outlook and Gmail do this.
>> 
>> Is this acceptable ?  The only bad side to it would appear to be possibly 
>> some increased difficulty in troubleshooting.
>> 
>> If it is an acceptable process, how would I configure Postfix to do this 
>> only on submission ?
> 
> I anonymise the initial Received: header with a header_checks on the 
> submission service.
> 
> In master.cf, I add `-o cleanup_service_name=subcleanup` to the submission 
> service.  That service is defined as:
> 
>   subcleanup  unix n   -   n   -   0   cleanup
> -o syslog_name=postfix/subcleanup
> -o header_checks=pcre:$config_directory/submission_header_checks.pcre
> 
> The submission_header_checks.pcre file contains:
> 
>   /^\s*(Received: from .+?(?=\s\())[^\n]*(.*for <.*)/ REPLACE $1 
> (localhost [127.0.0.1])$2
> 
> I'm sure there are better ways to do this, but this works for me.
> 
> It doesn't interfere with debugging much because the logs will mentain the 
> replacement and it's easy to grep for.

Thank you for your reply.

I currently use DKIM and as per the RFC for DKIM, I don’t include trace headers 
in the message hash that makes up the DKIM signature.  I am under the 
impression that my DKIM signatures should be correct in this case if I use your 
solution and it re-writes the first trace header - is that true or are there 
any other DKIM issues I might run into ?

Thanks,

- J

Re: Removing trace records on submission MSA

[Philip Paeps]

On 2018-03-10 16:01:01 (-0500), J Doe wrote:
I have a question in regards to removing some trace records when 
providing submission on Postfix 3.1.x and later.


While reading RFC 6409 (“Message Submission for Mail”), I note that the 
RFC observes that:


   "Even when submitted messages are complete, local site policy may 
   dictate that the message text be examined or modified in some way, 
   e.g., to conceal local name or address spaces.”


By this I take it that I could remove perhaps the initial trace message 
that returns information about internal addresses and network names.  
It seems to me that both Hotmail/Outlook and Gmail do this.


Is this acceptable ?  The only bad side to it would appear to be 
possibly some increased difficulty in troubleshooting.


If it is an acceptable process, how would I configure Postfix to do 
this only on submission ?


I anonymise the initial Received: header with a header_checks on the 
submission service.


In master.cf, I add `-o cleanup_service_name=subcleanup` to the 
submission service.  That service is defined as:


   subcleanup  unix n   -   n   -   0   cleanup
 -o syslog_name=postfix/subcleanup
 -o header_checks=pcre:$config_directory/submission_header_checks.pcre

The submission_header_checks.pcre file contains:

   /^\s*(Received: from .+?(?=\s\())[^\n]*(.*for <.*)/ REPLACE $1 
(localhost [127.0.0.1])$2

I'm sure there are better ways to do this, but this works for me.

It doesn't interfere with debugging much because the logs will mentain 
the replacement and it's easy to grep for.


Philip

--
Philip Paeps
Senior Reality Engineer
Ministry of Information

Removing trace records on submission MSA

[J Doe]

Hi,

I have a question in regards to removing some trace records when providing 
submission on Postfix 3.1.x and later.

While reading RFC 6409 (“Message Submission for Mail”), I note that the RFC 
observes that:

"Even when submitted messages are complete, local site policy may dictate 
that the message text be
   examined or modified in some way, e.g., to conceal local name or address 
spaces.”

By this I take it that I could remove perhaps the initial trace message that 
returns information about internal addresses
and network names.  It seems to me that both Hotmail/Outlook and Gmail do this.

Is this acceptable ?  The only bad side to it would appear to be possibly some 
increased difficulty in troubleshooting.

If it is an acceptable process, how would I configure Postfix to do this only 
on submission ?

Thanks,

- J