Re: Toss load-balancer health checks, but BCC everything else (always_bcc, check_sender_access and 'smtpd_delay_reject = yes')

[deoren]

On 5/17/2018 3:20 PM, Wietse Venema wrote:

Wietse Venema:

deoren:

   /etc/postfix/sender-access.cf:
# First, a rule that matches health-check mail.
smtp-health-che...@example.com DUNNO
# Add a BCC recipient to other email.
* BCC f...@example.com

For the last entry that BCC's "everything else", is the * character a
valid source?


The access map syntax is documented. http://www.postfix.org/access.5.html
There is no '*' in there.


If you must filter on sender address, you can use a regexp: or pcre:
table.

smtpd_xxx_restrictions =
 ...
 check_sender_access pcre:/etc/postfix/sender-access.pcre
 ...

/etc/postfix/sender-access.pcre:
 # First, a rule that matches health-check mail.
 /^smtp-health-checks@example\.com$/ DUNNO
 # Add a BCC recipient to other email.
 . BCC f...@example.com

The ^, $, \, and . are special only when you use regexp: or pcre:.
They are not part of the access map syntax.

Instead of the above pair of rules you could use if..endif, but
again, those are not part of the access map syntax, but of the
specific lookup table.

Wietse



Thank you for your help. Re the '*' syntax, I wasn't sure how to 
accomplish the goal of matching everything, so thanks for clearing that 
up. I ended up going with this setup based off of your feedback and the 
original example I found from Victor Duchovni:


/etc/postfix/main.cf
# BCC mail flowing through this relay EXCEPT for
# health check emails for future troubleshooting
pcre:/etc/postfix/auto_bcc.pcre

# Other checks/actions
proxy:mysql:/etc/postfix/mysql-sender_access.cf,


/etc/postfix/auto_bcc.pcre
# though this content is indented to stand out
# there is no indention in the file itself
if !/^haproxy-smtp-check@example\.com$/
/@/ BCC arch...@example.com
endif


So far it appears to be working just fine. Thanks again for your help.

Re: Toss load-balancer health checks, but BCC everything else (always_bcc, check_sender_access and 'smtpd_delay_reject = yes')

[Wietse Venema]

Wietse Venema:
> deoren:
> >   /etc/postfix/sender-access.cf:
> ># First, a rule that matches health-check mail.
> >smtp-health-che...@example.com DUNNO
> ># Add a BCC recipient to other email.
> >* BCC f...@example.com
> > 
> > For the last entry that BCC's "everything else", is the * character a 
> > valid source?
> 
> The access map syntax is documented. http://www.postfix.org/access.5.html
> There is no '*' in there.

If you must filter on sender address, you can use a regexp: or pcre:
table.

smtpd_xxx_restrictions =
...
check_sender_access pcre:/etc/postfix/sender-access.pcre
...

/etc/postfix/sender-access.pcre:
# First, a rule that matches health-check mail.
/^smtp-health-checks@example\.com$/ DUNNO
# Add a BCC recipient to other email.
. BCC f...@example.com

The ^, $, \, and . are special only when you use regexp: or pcre:.
They are not part of the access map syntax.

Instead of the above pair of rules you could use if..endif, but
again, those are not part of the access map syntax, but of the
specific lookup table.

Wietse

Re: Toss load-balancer health checks, but BCC everything else (always_bcc, check_sender_access and 'smtpd_delay_reject = yes')

[Wietse Venema]

deoren:
>   /etc/postfix/sender-access.cf:
># First, a rule that matches health-check mail.
>smtp-health-che...@example.com DUNNO
># Add a BCC recipient to other email.
>* BCC f...@example.com
> 
> For the last entry that BCC's "everything else", is the * character a 
> valid source?

The access map syntax is documented. http://www.postfix.org/access.5.html
There is no '*' in there.

Wietse

Re: Toss load-balancer health checks, but BCC everything else (always_bcc, check_sender_access and 'smtpd_delay_reject = yes')

[deoren]

On 5/13/2018 11:56 AM, deoren wrote:

On 5/11/2018 2:20 PM, Wietse Venema wrote:

deoren:

My apologies if I overlooked an answer somewhere, but I checked the docs
and performed a brief search of the archives before asking and didn't
spot the answer.

Goal:

BCC everything EXCEPT for health check emails generated by our HAProxy
load-balancer


Use a CIDR map:

/etc/postfix/main.cf
 smtpd_client_restritions =
check_client_access cidr:/etc/postfix/client-access.cidr

etc/postfix/client-access.cidr:
 # First, a rule that matches healtcheck mail.
 1.2.3.4/32 DUNNO
 # Add a BCC recipient to other email.
 0.0.0.0/0 BCC f...@example.com

This requires Postfix 3.0 or later.

Wietse



Thank you for the feedback and the solution.

I assume that the performance for this solution is going to be much 
better than the approach I went with? I'm guessing that this approach 
will also be much more stable over the long run as well. To continue 
that thought, this approach also keeps the two specific actions together 
in a single block which should be easier for anyone reviewing the 
settings to mentally parse.


I had a few moments to go back and update the setup to use the proposed 
solution, but stopped part way through. I see that we're matching on the 
client IP here with an action of DUNNO.


Instead of matching on the IP with check_client_access, could we match 
on the sender address instead with check_sender_access so that 
non-health-check mail from the load-balancer will still be subject to 
the BCC action?


 /etc/postfix/main.cf
  smtpd_client_restrictions =
check_sender_access hash:/etc/postfix/sender-access.cf

 /etc/postfix/sender-access.cf:
  # First, a rule that matches health-check mail.
  smtp-health-che...@example.com DUNNO
  # Add a BCC recipient to other email.
  * BCC f...@example.com

For the last entry that BCC's "everything else", is the * character a 
valid source?


Thanks for your help.

Re: Toss load-balancer health checks, but BCC everything else (always_bcc, check_sender_access and 'smtpd_delay_reject = yes')

[deoren]

On 5/12/2018 1:51 AM, @lbutlr wrote:

On 11 May 2018, at 09:55, deoren  wrote:

BCC everything EXCEPT for health check emails generated by our HAProxy 
load-balancer


Seems it would be much simpler to BCC everything and then discard the few 
messages you don’t want.



Thanks for your feedback.

I thought of that, but with our load-balancer sending in an email to 
each node every X seconds (which, admittedly might be overkill) that 
would result in stream of emails going to the archive account each time 
the health check runs. I was thinking that if there was an issue with 
the relay nodes handing off their mail then those health checks would 
clutter the queue and cause delays for legitimate mail.


I was hoping with some logic in place to block the health check emails 
from going through the standard intake "flow" (I know I'm using the 
right terms here) that other mail wouldn't be held up if there was a 
problem delivering those emails.

Re: Toss load-balancer health checks, but BCC everything else (always_bcc, check_sender_access and 'smtpd_delay_reject = yes')

[deoren]

On 5/11/2018 2:20 PM, Wietse Venema wrote:

deoren:

My apologies if I overlooked an answer somewhere, but I checked the docs
and performed a brief search of the archives before asking and didn't
spot the answer.

Goal:

BCC everything EXCEPT for health check emails generated by our HAProxy
load-balancer


Use a CIDR map:

/etc/postfix/main.cf
 smtpd_client_restritions =
check_client_access cidr:/etc/postfix/client-access.cidr

etc/postfix/client-access.cidr:
 # First, a rule that matches healtcheck mail.
 1.2.3.4/32 DUNNO
 # Add a BCC recipient to other email.
 0.0.0.0/0 BCC f...@example.com

This requires Postfix 3.0 or later.

Wietse



Thank you for the feedback and the solution.

I assume that the performance for this solution is going to be much 
better than the approach I went with? I'm guessing that this approach 
will also be much more stable over the long run as well. To continue 
that thought, this approach also keeps the two specific actions together 
in a single block which should be easier for anyone reviewing the 
settings to mentally parse.

Re: Toss load-balancer health checks, but BCC everything else (always_bcc, check_sender_access and 'smtpd_delay_reject = yes')

[@lbutlr]

On 11 May 2018, at 09:55, deoren  wrote:
> BCC everything EXCEPT for health check emails generated by our HAProxy 
> load-balancer

Seems it would be much simpler to BCC everything and then discard the few 
messages you don’t want.

-- 
I WILL NOT INSTIGATE REVOLUTION Bart chalkboard Ep. 7G06

Re: Toss load-balancer health checks, but BCC everything else (always_bcc, check_sender_access and 'smtpd_delay_reject = yes')

[Wietse Venema]

deoren:
> My apologies if I overlooked an answer somewhere, but I checked the docs 
> and performed a brief search of the archives before asking and didn't 
> spot the answer.
> 
> Goal:
> 
> BCC everything EXCEPT for health check emails generated by our HAProxy 
> load-balancer

Use a CIDR map:

/etc/postfix/main.cf
smtpd_client_restritions = 
check_client_access cidr:/etc/postfix/client-access.cidr

etc/postfix/client-access.cidr:
# First, a rule that matches healtcheck mail.
1.2.3.4/32 DUNNO
# Add a BCC recipient to other email.
0.0.0.0/0 BCC f...@example.com

This requires Postfix 3.0 or later.

Wietse

Toss load-balancer health checks, but BCC everything else (always_bcc, check_sender_access and 'smtpd_delay_reject = yes')

[deoren]

My apologies if I overlooked an answer somewhere, but I checked the docs 
and performed a brief search of the archives before asking and didn't 
spot the answer.


Goal:

BCC everything EXCEPT for health check emails generated by our HAProxy 
load-balancer


I originally tried following the directions provided by Victor Duchovni 
(see link below), but I evidently didn't follow along well enough to 
replicate the results.


I seem to have found a combination of settings which accomplishes my 
goal and have listed them below. Are there any issues that you can spot 
with my approach?


always_bcc = bcc-...@example.com
smtpd_delay_reject = yes
smtpd_recipient_restrictions =
  ...
  check_sender_access proxy:mysql:/etc/postfix/mysql-sender_access.cf,
  ...


Within the access table I'm mapping the health check sender address to 
the DISCARD action. This appears to accomplish the original goal of 
exercising the entire intake workflow (port is open, Postfix is running, 
sender IP is allowed, sender email address is allowed, address 
resolution is working) AND also prevents those health check emails from 
going to the always_bcc email address while still sending everything 
else over.


Thanks in advance for reading this and offering suggestions.

Refs:

http://www.postfix.org/postconf.5.html
http://www.postfix.org/access.5.html
http://www.postfix.org/pcre_table.5.html
http://postfix.1071664.n5.nabble.com/exclude-email-account-from-always-bcc-td13695.html