Re: originating flag not working - critical bug - RelayedOpenRelay / DKIM signing not working

[Damian]

> After I applied the attached patch the issue was resolved. There is
> literally one line that needs to be added:
> "$msginfo->originating(c('originating'));".
>
>
> It would be much appreciated if the following patch could be applied to
> Amavis directly without relying on various distribution maintainers to
> fix the problem.
>
>
> --- amavisd.orig  Tue Apr 26 21:24:33 2016
> +++ amavisd   Fri Jul  1 01:03:15 2016
> @@ -34338,6 +34329,7 @@ sub collect_some_dkim_info($) {
>  $sig_ind++;
>}
>Amavis::load_policy_bank($_,$msginfo) for @bank_names;
> +  $msginfo->originating(c('originating'));
>$msginfo->dkim_signatures_valid(\@signatures_valid)  if
> @signatures_valid;
>  # if (ll(5) && $sig_ind > 0) {
>  #   # show which header fields are covered by which signature
>

An alternative to a patch might be this configuration snippet:

> $policy_bank{'EMPTY'} = {};
> package Amavis::Custom {
>   sub new {
>     my ($class, undef ,$msginfo) = @_;
>     Amavis::load_policy_bank('EMPTY', $msginfo);
>     bless {}, $class;
>   }
> }

Regards
 Damian

Re: originating flag not working - critical bug - RelayedOpenRelay / DKIM signing not working

[Alexander Hoogerhuis]

On 27/05/2018 08:01, Alexander Hoogerhuis wrote:


I just wanted to add feedback to this one. I've had this issue for some 
time, but haven't had time to track it down untill now. I started 
digging and ended with the same conclusions as the two above contributors.


I have had debug turned on, and I am positive that the following holds:
- my client IP is in @mynetworks.
- the recipient is not local.
- the sending domain is in @local_domains_maps.
- Postfix uses xforward and amavis sees the proper client IP.
- the log from amavisd shows the policy bank has originating set.

And yet my log from amavis says RelayedOpenRelay.

My setup is simple; all the possible clients are on private IPs, the 
world is on public


Adding the posted one line patch to 2.11.0 seems to make it all work 
well for me, my logs now show RelayedOutbound for email going out via 
the correct policy bank.




Another datapoint on this.

Another case where I get RelayedOpenRelay is when Exchange generates out 
of office replies for users. Then it gets triggered because the sender 
is blank:



Return-Path: <>
X-Envelope-From: <>
X-Envelope-To: 


Since the sending user is not recognised as a local user.

mvh,
A
--
Alexander Hoogerhuis | http://no.linkedin.com/in/alexh
Boxed Solutions AS   | +47 908 21 485 - al...@boxed.no
"Given enough eyeballs, all bugs are shallow." -Eric S. Raymond

Re: originating flag not working - critical bug - RelayedOpenRelay / DKIM signing not working

[Alexander Hoogerhuis]

On 12/02/2018 10:42, Giovanni wrote:

Karol Augustin  wrote:

Hi,

I am explicitly copying original authors of threads I am referring to in
this email, as I don't know if they are still monitoring the list for
solution to the problem.
I would like to thank Giovanni for supplying the patch, which has now
spread across internet.

There is evidence of a critical bug in quite a few threads on this list
that manifests itself in various ways. Some users have problems with
DKIM signatures of outgoing mail, others with mail marked as
RelayedOpenRelay in the logs.

The issue is caused by Amavis not honoring originating flag, which
causes all sender addresses to be treated as "foreign", which obviously
has a huge potential of breaking mail flow especially in environments
where there are multiple e-mail paths and policy banks configured.


I hit the same problem when I upgraded to 2.11.0 few days ago and asked
similar question in a reply to existing thread.
https://lists.amavis.org/pipermail/amavis-users/2018-February/005284.html

The same issue was described earlier in following thread:
https://lists.amavis.org/pipermail/amavis-users/2017-November/005116.html

Original mention of this problem was made by Giovanni, who kindly
provided a one line fix to the problem:
https://lists.amavis.org/pipermail/amavis-users/2016-July/004428.html


for the records, the patch I sumbitted 2 years ago fixes the bug with postfix;
there are some corner cases (spotted by an Opensmtpd instance, maybe by some 
other mta as well) that needs an additianal one line fix.
Full patch follows.
  Giovanni

--- amavisd.origTue Apr 26 21:24:33 2016
+++ amavisd Fri Aug  5 12:32:39 2016
@@ -22806,6 +22806,7 @@ sub process_smtp_request() {
  }
  # load policy banks from the 'client_ipaddr_policy' lookup
  Amavis::load_policy_bank($_,$msginfo) for @bank_names_cl;
+$msginfo->originating(c('originating'));
  
  $msginfo->client_addr($cl_ip);  # ADDR

  $msginfo->client_port($cl_port);# PORT
@@ -34338,6 +34330,7 @@ sub collect_some_dkim_info($) {
  $sig_ind++;
}
Amavis::load_policy_bank($_,$msginfo) for @bank_names;
+  $msginfo->originating(c('originating'));
$msginfo->dkim_signatures_valid(\@signatures_valid)  if @signatures_valid;
  # if (ll(5) && $sig_ind > 0) {
  #   # show which header fields are covered by which signature



I just wanted to add feedback to this one. I've had this issue for some 
time, but haven't had time to track it down untill now. I started 
digging and ended with the same conclusions as the two above contributors.


I have had debug turned on, and I am positive that the following holds:
- my client IP is in @mynetworks.
- the recipient is not local.
- the sending domain is in @local_domains_maps.
- Postfix uses xforward and amavis sees the proper client IP.
- the log from amavisd shows the policy bank has originating set.

And yet my log from amavis says RelayedOpenRelay.

My setup is simple; all the possible clients are on private IPs, the 
world is on public


Adding the posted one line patch to 2.11.0 seems to make it all work 
well for me, my logs now show RelayedOutbound for email going out via 
the correct policy bank.


mvh,
A
--
Alexander Hoogerhuis | http://no.linkedin.com/in/alexh
Boxed Solutions AS   | +47 908 21 485 - al...@boxed.no
"Given enough eyeballs, all bugs are shallow." -Eric S. Raymond

Re: originating flag not working - critical bug - RelayedOpenRelay / DKIM signing not working

[Giovanni Bechis]

Karol Augustin  wrote:
> On 2018-02-12 9:42, Giovanni wrote:
> 
>> for the records, the patch I sumbitted 2 years ago fixes the bug with 
>> postfix;
>> there are some corner cases (spotted by an Opensmtpd instance, maybe
>> by some other mta as well) that needs an additianal one line fix.
>> Full patch follows.
>>  Giovanni
> 
> 
> Hi Giovanni,
> Thanks for providing the patches. I was suspecting that the original
> patch might be effective only if either or both DKIM
> signing/verification are active, but as that is the case on my system
> (dkim verification) so I haven't tested it with DKIM processing
> disabled. Maybe your Opensmtpd instance had DKIM processing in Amavis
> disabled?
> 
> Your second patch looks like it is solving problem in all possible
> scenarios where originating flag was not honored. I will submit your
> patch to Debian developers for inclusion, also someone using Fedora
> might do this as well for their bug.
> 
this could be possible but untested, my Opensmtpd instance is a send-only 
smtp server.

 Cheers
  Giovanni

> https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=882324
> https://bugzilla.redhat.com/show_bug.cgi?id=1364730
> 
> Thanks again,
> Karol
> 
> 

Re: originating flag not working - critical bug - RelayedOpenRelay / DKIM signing not working

[Karol Augustin]

On 2018-02-12 9:42, Giovanni wrote:

> for the records, the patch I sumbitted 2 years ago fixes the bug with postfix;
> there are some corner cases (spotted by an Opensmtpd instance, maybe
> by some other mta as well) that needs an additianal one line fix.
> Full patch follows.
>  Giovanni


Hi Giovanni,
Thanks for providing the patches. I was suspecting that the original
patch might be effective only if either or both DKIM
signing/verification are active, but as that is the case on my system
(dkim verification) so I haven't tested it with DKIM processing
disabled. Maybe your Opensmtpd instance had DKIM processing in Amavis
disabled?

Your second patch looks like it is solving problem in all possible
scenarios where originating flag was not honored. I will submit your
patch to Debian developers for inclusion, also someone using Fedora
might do this as well for their bug.

https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=882324
https://bugzilla.redhat.com/show_bug.cgi?id=1364730

Thanks again,
Karol


-- 
Karol Augustin
ka...@augustin.pl
http://karolaugustin.pl/
+353 85 775 5312

Re: originating flag not working - critical bug - RelayedOpenRelay / DKIM signing not working

[Karol Augustin]

On 2018-02-12 10:19, Dominic Raferd wrote:
> On 12 February 2018 at 10:06, Dino Edwards
>  wrote:
>> Wouldn't this be avoided by simply using opendkim for DKIM signing instead 
>> of relying on amavis for that?
> 
> ... and use a program to do one thing and do it well -
> https://en.wikipedia.org/wiki/Unix_philosophy

The part "do it well" is affected by this bug. Also if you don't need
additional functionality provided by Amavis you could just stick to bare
spamassassin via spamc/spamd as described here:
https://wiki.apache.org/spamassassin/IntegratedSpamdInPostfix

Amavis provides much more functionality than just spamassassin
integration: there is also virus scanning, DKIM processing, and much
more. It enables you to easily implement very complicated e-mail flows
within your system if you need/desire. This bug impacts many of this
functionalities, which has nothing to do with Unix Philosophy. It's just
a bug with simple two line fix.


Karol


-- 
Karol Augustin
ka...@augustin.pl
http://karolaugustin.pl/
+353 85 775 5312

Re: originating flag not working - critical bug - RelayedOpenRelay / DKIM signing not working

[Karol Augustin]

On 2018-02-12 10:06, Dino Edwards wrote:
> Wouldn't this be avoided by simply using opendkim for DKIM signing
> instead of relying on amavis for that? Or are there other use
> scenarios for the originating flag where this would come into play?
> 
> 
> 

As I mentioned in my original email any use case that involves
distinguishing if email was sent by local user or is originating from
outside (incoming) is affected. That of course includes DKIM signing as
obvious scenario but there are many other configurations that make use
of this functionality as well.

Other example is different SPAM scoring for local users vs incoming
mail. Also adding X-Spam headers is impacted so your outgoing mail has
your spamassassin results attached to it.

I am using opendkim to sign outgoing mail, but as I said, the core
functionality is affected by this bug. Also some people do not like
OpenRelay log entries that are happening with affected version.

In my opinion this bug makes Amavis unusable in many if not all
configurations as distinguishing between local and incoming e-mail is
vital to properly do e-mail spam/virus scanning, header injection, and
DKIM signig/verification (if you are using it).


Karol

-- 
Karol Augustin
ka...@augustin.pl
http://karolaugustin.pl/
+353 85 775 5312

Re: originating flag not working - critical bug - RelayedOpenRelay / DKIM signing not working

[Dominic Raferd]

On 12 February 2018 at 10:06, Dino Edwards
 wrote:
> Wouldn't this be avoided by simply using opendkim for DKIM signing instead of 
> relying on amavis for that?

... and use a program to do one thing and do it well -
https://en.wikipedia.org/wiki/Unix_philosophy

RE: originating flag not working - critical bug - RelayedOpenRelay / DKIM signing not working

[Dino Edwards]

Wouldn't this be avoided by simply using opendkim for DKIM signing instead of 
relying on amavis for that? Or are there other use scenarios for the 
originating flag where this would come into play?



-Original Message-
From: amavis-users 
[mailto:amavis-users-bounces+dino.edwards=mydirectmail@amavis.org] On 
Behalf Of Giovanni
Sent: Monday, February 12, 2018 4:43 AM
To: amavis-users@amavis.org
Subject: Re: originating flag not working - critical bug - RelayedOpenRelay / 
DKIM signing not working

Karol Augustin <ka...@augustin.pl> wrote:
> Hi,
> 
> I am explicitly copying original authors of threads I am referring to 
> in this email, as I don't know if they are still monitoring the list 
> for solution to the problem.
> I would like to thank Giovanni for supplying the patch, which has now 
> spread across internet.
> 
> There is evidence of a critical bug in quite a few threads on this 
> list that manifests itself in various ways. Some users have problems 
> with DKIM signatures of outgoing mail, others with mail marked as 
> RelayedOpenRelay in the logs.
> 
> The issue is caused by Amavis not honoring originating flag, which 
> causes all sender addresses to be treated as "foreign", which 
> obviously has a huge potential of breaking mail flow especially in 
> environments where there are multiple e-mail paths and policy banks 
> configured.
> 
> 
> I hit the same problem when I upgraded to 2.11.0 few days ago and 
> asked similar question in a reply to existing thread.
> https://lists.amavis.org/pipermail/amavis-users/2018-February/005284.h
> tml
> 
> The same issue was described earlier in following thread:
> https://lists.amavis.org/pipermail/amavis-users/2017-November/005116.h
> tml
> 
> Original mention of this problem was made by Giovanni, who kindly 
> provided a one line fix to the problem:
> https://lists.amavis.org/pipermail/amavis-users/2016-July/004428.html
> 
for the records, the patch I sumbitted 2 years ago fixes the bug with postfix; 
there are some corner cases (spotted by an Opensmtpd instance, maybe by some 
other mta as well) that needs an additianal one line fix.
Full patch follows.
 Giovanni

--- amavisd.origTue Apr 26 21:24:33 2016
+++ amavisd Fri Aug  5 12:32:39 2016
@@ -22806,6 +22806,7 @@ sub process_smtp_request() {
 }
 # load policy banks from the 'client_ipaddr_policy' lookup
 Amavis::load_policy_bank($_,$msginfo) for @bank_names_cl;
+$msginfo->originating(c('originating'));
 
 $msginfo->client_addr($cl_ip);  # ADDR
 $msginfo->client_port($cl_port);# PORT
@@ -34338,6 +34330,7 @@ sub collect_some_dkim_info($) {
 $sig_ind++;
   }
   Amavis::load_policy_bank($_,$msginfo) for @bank_names;
+  $msginfo->originating(c('originating'));
   $msginfo->dkim_signatures_valid(\@signatures_valid)  if @signatures_valid;  
# if (ll(5) && $sig_ind > 0) {
 #   # show which header fields are covered by which signature

Re: originating flag not working - critical bug - RelayedOpenRelay / DKIM signing not working

[Giovanni]

Karol Augustin  wrote:
> Hi,
> 
> I am explicitly copying original authors of threads I am referring to in
> this email, as I don't know if they are still monitoring the list for
> solution to the problem.
> I would like to thank Giovanni for supplying the patch, which has now
> spread across internet.
> 
> There is evidence of a critical bug in quite a few threads on this list
> that manifests itself in various ways. Some users have problems with
> DKIM signatures of outgoing mail, others with mail marked as
> RelayedOpenRelay in the logs.
> 
> The issue is caused by Amavis not honoring originating flag, which
> causes all sender addresses to be treated as "foreign", which obviously
> has a huge potential of breaking mail flow especially in environments
> where there are multiple e-mail paths and policy banks configured.
> 
> 
> I hit the same problem when I upgraded to 2.11.0 few days ago and asked
> similar question in a reply to existing thread.
> https://lists.amavis.org/pipermail/amavis-users/2018-February/005284.html
> 
> The same issue was described earlier in following thread:
> https://lists.amavis.org/pipermail/amavis-users/2017-November/005116.html
> 
> Original mention of this problem was made by Giovanni, who kindly
> provided a one line fix to the problem:
> https://lists.amavis.org/pipermail/amavis-users/2016-July/004428.html
> 
for the records, the patch I sumbitted 2 years ago fixes the bug with postfix;
there are some corner cases (spotted by an Opensmtpd instance, maybe by some 
other mta as well) that needs an additianal one line fix.
Full patch follows.
 Giovanni

--- amavisd.origTue Apr 26 21:24:33 2016
+++ amavisd Fri Aug  5 12:32:39 2016
@@ -22806,6 +22806,7 @@ sub process_smtp_request() {
 }
 # load policy banks from the 'client_ipaddr_policy' lookup
 Amavis::load_policy_bank($_,$msginfo) for @bank_names_cl;
+$msginfo->originating(c('originating'));
 
 $msginfo->client_addr($cl_ip);  # ADDR
 $msginfo->client_port($cl_port);# PORT
@@ -34338,6 +34330,7 @@ sub collect_some_dkim_info($) {
 $sig_ind++;
   }
   Amavis::load_policy_bank($_,$msginfo) for @bank_names;
+  $msginfo->originating(c('originating'));
   $msginfo->dkim_signatures_valid(\@signatures_valid)  if @signatures_valid;
 # if (ll(5) && $sig_ind > 0) {
 #   # show which header fields are covered by which signature