On 1 Apr 2018, at 21:09 (-0400), Michael Brunnbauer wrote:
[...]
Figuring out what spamd is using is less simple (and system-specific)
but
since you've been maintaining a system by hand for a long time I
expect
you'll be able to figure out how to do so safely.
This does not sound very helpful of you
Well, it's rather difficult to guess what mechanism you are using to
start spamd and when I wrote that I had not seen your message narrowing
down the range of possibilities to those one might use on Linux.
so I did some debugging on my own and
have more information:
So I guess I was right?
The problem only occurs only when spamd is started in the homedir of
root. If
I start it in any other directory (including subdirs of /root),
Net:DNS
behaves like it should: $answer->rdatastr in dnsbl_uri in Dns.pm
contains IP
addresses in dotted quad notation, like 127.0.0.3. If I start spamd in
/root,
$answer->rdatastr contains strings like "\# 4 7f000003" instead. This
occurs
regardless of any -x or -u flags to spamd.
OK, so that is quite odd.
Is there a tree of Perl modules under /root? Perl before 5.26 includes
'.' in the @INC array that defines where modules might live, which could
cause this if you have a private install tree (as is the default in some
distros.) Also, if you run spamd as root it drops to 'nobody' but if
you're in /root and /root/.spamassassin is world-readable, it will still
get used as the user prefs directory.
A normal startup of spamd (by sysvinit, Upstart, systemd, etc.) is what
you need to diagnose, not a manual startup from a login shell. None of
those normally should put the daemon in /root as a working directory.
Run as a proper system daemon by the normal startup subsystem, spamd
gets a substantially different environment than if you run it by hand
from an interactive shell.
So being in /root when started changes the behavior of spamd. Is it
possible
that this is a timing issue? Could "\# 4 7f000003" be some unprocessed
response that would be converted to 127.0.0.3 a moment later? Or is
there
some other explanation for this?
No, it's not a timing issue. The root cause is that
Net::DNS::RR->rdatastr() should never have been relied upon by SA to
have any particular format because it was always poorly documented and
quietly vanished from the documentation (but not the code) for
Net::DNS::RR.pm in 0.69. What it actually contains is a function of the
specific DNS record and what server generated the response, making an
explanation for any specific oddity something of a guessing game.
More recently, there have been multiple other changes in various
components of the Net-DNS distribution that have caused other problems
in SA, and they may interact with the rdatastr issue. These issues have
all been addressed in the current SA code, both in the 'trunk' and in
the 3.4 branch which will (hopefully soon) become the 3.4.2 release.
Many (most? all?) packagers of SA maintaining it for major platforms
have incorporated some or all of the necessary DNS-related fixes. I've
attached a patch that aggregates all of the fixes to this message. You
could also install SA from the current 3.4 branch or the last 3.4.2
release candidate package, or if you're adventurous, from the SVN
'trunk' that will eventually yield v4.0.
--
Bill Cole
b...@scconsult.com or billc...@apache.org
(AKA @grumpybozo and many *@billmail.scconsult.com addresses)
Currently Seeking Steady Work: https://linkedin.com/in/billcole
Index: lib/Mail/SpamAssassin/Plugin/AskDNS.pm
===================================================================
--- lib/Mail/SpamAssassin/Plugin/AskDNS.pm
(.../tags/spamassassin_release_3_4_1/lib/Mail/SpamAssassin) (revision
1676603)
+++ lib/Mail/SpamAssassin/Plugin/AskDNS.pm
(.../branches/3.4/lib/Mail/SpamAssassin) (working copy)
@@ -140,7 +140,7 @@
multiple character-strings (as defined in Section 3.3 of [RFC1035]), these
strings are concatenated with no delimiters before comparing the result
to the filtering string. This follows requirements of several documents,
-such as RFC 5518, RFC 4408, RFC 4871, RFC 5617. Examples of a plain text
+such as RFC 5518, RFC 7208, RFC 4871, RFC 5617. Examples of a plain text
filtering parameter: "127.0.0.1", "transaction", 'list' .
A regular expression follows a familiar perl syntax like /.../ or m{...}
@@ -192,10 +192,9 @@
use Mail::SpamAssassin::Util qw(decode_dns_question_entry);
use Mail::SpamAssassin::Logger;
-use vars qw(@ISA %rcode_value $txtdata_can_provide_a_list);
-@ISA = qw(Mail::SpamAssassin::Plugin);
+our @ISA = qw(Mail::SpamAssassin::Plugin);
-%rcode_value = ( # http://www.iana.org/assignments/dns-parameters, RFC 6195
+our %rcode_value = ( # http://www.iana.org/assignments/dns-parameters, RFC
6195
NOERROR => 0, FORMERR => 1, SERVFAIL => 2, NXDOMAIN => 3, NOTIMP => 4,
REFUSED => 5, YXDOMAIN => 6, YXRRSET => 7, NXRRSET => 8, NOTAUTH => 9,
NOTZONE => 10, BADVERS => 16, BADSIG => 16, BADKEY => 17, BADTIME => 18,
@@ -202,6 +201,8 @@
BADMODE => 19, BADNAME => 20, BADALG => 21, BADTRUNC => 22,
);
+our $txtdata_can_provide_a_list;
+
sub new {
my($class,$sa_main) = @_;
@@ -539,7 +540,7 @@
@answer = ( undef );
}
- # NOTE: $rr->rdatastr returns the result encoded in a DNS zone file
+ # NOTE: $rr->rdstring returns the result encoded in a DNS zone file
# format, i.e. enclosed in double quotes if a result contains whitespace
# (or other funny characters), and may use \DDD encoding or \X quoting as
# per RFC 1035. Using $rr->txtdata instead avoids this unnecessary encoding
@@ -566,8 +567,16 @@
# special case, no answer records, only rcode can be tested
} else {
$rr_type = uc $rr->type;
- if ($rr->UNIVERSAL::can('txtdata')) { # TXT, SPF
- # join with no intervening spaces, as per RFC 5518
+ if ($rr_type eq 'A') {
+ # Net::DNS::RR::A::address() is available since Net::DNS 0.69
+ $rr_rdatastr = $rr->UNIVERSAL::can('address') ? $rr->address
+ : $rr->rdatastr;
+ if ($rr_rdatastr =~ m/^\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}\z/) {
+ $rdatanum = Mail::SpamAssassin::Util::my_inet_aton($rr_rdatastr);
+ }
+
+ } elsif ($rr->UNIVERSAL::can('txtdata')) {
+ # TXT, SPF: join with no intervening spaces, as per RFC 5518
if ($txtdata_can_provide_a_list || $rr_type ne 'TXT') {
$rr_rdatastr = join('', $rr->txtdata); # txtdata() in list context!
} else { # char_str_list() is only available for TXT records
@@ -574,11 +583,10 @@
$rr_rdatastr = join('', $rr->char_str_list); # historical
}
} else {
- $rr_rdatastr = $rr->rdatastr;
- if ($rr_type eq 'A' &&
- $rr_rdatastr =~ m/^\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}\z/) {
- $rdatanum = Mail::SpamAssassin::Util::my_inet_aton($rr_rdatastr);
- }
+ # rdatastr() is historical, use rdstring() since Net::DNS 0.69
+ $rr_rdatastr = $rr->UNIVERSAL::can('rdstring') ? $rr->rdstring
+ : $rr->rdatastr;
+ utf8::encode($rr_rdatastr) if utf8::is_utf8($rr_rdatastr);
}
# dbg("askdns: received rr type %s, data: %s", $rr_type, $rr_rdatastr);
}
Index: lib/Mail/SpamAssassin/Dns.pm
===================================================================
--- lib/Mail/SpamAssassin/Dns.pm
(.../tags/spamassassin_release_3_4_1/lib/Mail/SpamAssassin) (revision
1676603)
+++ lib/Mail/SpamAssassin/Dns.pm
(.../branches/3.4/lib/Mail/SpamAssassin) (working copy)
@@ -22,7 +22,7 @@
use strict;
use warnings;
-use bytes;
+# use bytes;
use re 'taint';
use Mail::SpamAssassin::Conf;
@@ -35,13 +35,13 @@
use IO::Socket;
use POSIX ":sys_wait_h";
-use vars qw{
- $KNOWN_BAD_DIALUP_RANGES @EXISTING_DOMAINS $IS_DNS_AVAILABLE $LAST_DNS_CHECK
-};
+our $KNOWN_BAD_DIALUP_RANGES; # Nothing uses this var???
+our $LAST_DNS_CHECK;
+
# use very well-connected domains (fast DNS response, many DNS servers,
# geographical distribution is a plus, TTL of at least 3600s)
-@EXISTING_DOMAINS = qw{
+our @EXISTING_DOMAINS = qw{
adelphia.net
akamai.com
apache.org
@@ -64,7 +64,7 @@
yahoo.com
};
-$IS_DNS_AVAILABLE = undef;
+our $IS_DNS_AVAILABLE = undef;
#Removed $VERSION per BUG 6422
#$VERSION = 'bogus'; # avoid CPAN.pm picking up razor ver
@@ -171,7 +171,7 @@
if (substr($rule, 0, 2) eq "__") {
# don't bother with meta rules
} elsif ($answer->type eq 'TXT') {
- # txtdata returns a non- zone-file-format encoded result, unlike rdatastr;
+ # txtdata returns a non- zone-file-format encoded result, unlike rdstring;
# avoid space-separated RDATA <character-string> fields if possible,
# txtdata provides a list of strings in a list context since Net::DNS 0.69
$log = join('',$answer->txtdata);
@@ -215,11 +215,13 @@
my $qname = $question->qname;
- # txtdata returns a non- zone-file-format encoded result, unlike rdatastr;
+ # txtdata returns a non- zone-file-format encoded result, unlike rdstring;
# avoid space-separated RDATA <character-string> fields if possible,
# txtdata provides a list of strings in a list context since Net::DNS 0.69
#
+ # rdatastr() is historical/undocumented, use rdstring() since Net::DNS 0.69
my $rdatastr = $answer->UNIVERSAL::can('txtdata') ? join('',$answer->txtdata)
+ : $answer->UNIVERSAL::can('rdstring') ? $answer->rdstring
: $answer->rdatastr;
if (defined $qname && defined $rdatastr) {
my $qclass = $question->qclass;
@@ -267,8 +269,13 @@
my $answ_type = $answer->type;
# TODO: there are some CNAME returns that might be useful
next if ($answ_type ne 'A' && $answ_type ne 'TXT');
- # skip any A record that isn't on 127/8
- next if ($answ_type eq 'A' && $answer->rdatastr !~ /^127\./);
+ if ($answ_type eq 'A') {
+ # Net::DNS::RR::A::address() is available since Net::DNS 0.69
+ my $ip_address = $answer->UNIVERSAL::can('address') ? $answer->address
+ : $answer->rdatastr;
+ # skip any A record that isn't on 127.0.0.0/8
+ next if $ip_address !~ /^127\./;
+ }
for my $rule (@{$rules}) {
$self->dnsbl_hit($rule, $question, $answer);
}
@@ -284,11 +291,13 @@
sub process_dnsbl_set {
my ($self, $set, $question, $answer) = @_;
- # txtdata returns a non- zone-file-format encoded result, unlike rdatastr;
+ # txtdata returns a non- zone-file-format encoded result, unlike rdstring;
# avoid space-separated RDATA <character-string> fields if possible,
# txtdata provides a list of strings in a list context since Net::DNS 0.69
#
- my $rdatastr = $answer->UNIVERSAL::can('txtdata') ? join('',$answer->txtdata)
+ # rdatastr() is historical/undocumented, use rdstring() since Net::DNS 0.69
+ my $rdatastr = $answer->UNIVERSAL::can('txtdata') ?
join('',$answer->txtdata)
+ : $answer->UNIVERSAL::can('rdstring') ? $answer->rdstring
: $answer->rdatastr;
while (my ($subtest, $rule) = each %{ $self->{dnspost}->{$set} }) {
Index: lib/Mail/SpamAssassin/DnsResolver.pm
===================================================================
--- lib/Mail/SpamAssassin/DnsResolver.pm
(.../tags/spamassassin_release_3_4_1/lib/Mail/SpamAssassin) (revision
1676603)
+++ lib/Mail/SpamAssassin/DnsResolver.pm
(.../branches/3.4/lib/Mail/SpamAssassin) (working copy)
@@ -37,7 +37,7 @@
use strict;
use warnings;
-use bytes;
+# use bytes;
use re 'taint';
require 5.008001; # needs utf8::is_utf8()
@@ -53,7 +53,7 @@
our @ISA = qw();
-use vars qw($io_socket_module_name);
+our $io_socket_module_name;
BEGIN {
if (eval { require IO::Socket::IP }) {
$io_socket_module_name = 'IO::Socket::IP';
@@ -581,7 +581,7 @@
# time, $domain, $type, $packet->id);
1;
} or do {
- # this can if a domain name in a query is invalid, or if a timeout signal
+ # get here if a domain name in a query is invalid, or if a timeout signal
# happened to be trapped by this eval, or if Net::DNS signalled an error
my $eval_stat = $@ ne '' ? $@ : "errno=$!"; chomp $eval_stat;
# resignal if alarm went off
@@ -592,6 +592,9 @@
};
if ($packet) {
+ # RD flag needs to be set explicitly since Net::DNS 1.01, Bug 7223
+ $packet->header->rd(1);
+
# my $udp_payload_size = $self->{res}->udppacketsize;
my $udp_payload_size = $self->{conf}->{dns_options}->{edns};
if ($udp_payload_size && $udp_payload_size > 512) {
@@ -722,6 +725,37 @@
###########################################################################
+=item $id = $res->bgread()
+
+Similar to C<Net::DNS::Resolver::bgread>. Reads a DNS packet from
+a supplied socket, decodes it, and returns a Net::DNS::Packet object
+if successful. Dies on error.
+
+=cut
+
+sub bgread() {
+ my ($self) = @_;
+ my $sock = $self->{sock};
+ my $packetsize = $self->{res}->udppacketsize;
+ $packetsize = 512 if $packetsize < 512; # just in case
+ my $data = '';
+ my $peeraddr = $sock->recv($data, $packetsize+256); # with some size margin
for troubleshooting
+ defined $peeraddr or die "bgread: recv() failed: $!";
+ my $peerhost = $sock->peerhost;
+ $data ne '' or die "bgread: received empty packet from $peerhost";
+ dbg("dns: bgread: received %d bytes from %s", length($data), $peerhost);
+ my($answerpkt, $decoded_length) = Net::DNS::Packet->new(\$data);
+ $answerpkt or die "bgread: decoding DNS packet failed: $@";
+ $answerpkt->answerfrom($peerhost);
+ if (defined $decoded_length && $decoded_length ne "" && $decoded_length !=
length($data)) {
+ warn sprintf("bgread: received a %d bytes packet from %s, decoded %d
bytes\n",
+ length($data), $peerhost, $decoded_length);
+ }
+ return $answerpkt;
+}
+
+###########################################################################
+
=item $nfound = $res->poll_responses()
See if there are any C<bgsend> reply packets ready, and return
@@ -769,13 +803,24 @@
$timeout = 0; # next time around collect whatever is available, then exit
last if $nfound == 0;
- my $packet = $self->{res}->bgread($self->{sock});
+ my $packet;
+ # Bug 7265, use our own bgread() below
+ # $packet = $self->{res}->bgread($self->{sock});
+ eval {
+ $packet = $self->bgread(); # Bug 7265, use our own bgread()
+ } or do {
+ undef $packet;
+ my $eval_stat = $@ ne '' ? $@ : "errno=$!"; chomp $eval_stat;
+ # resignal if alarm went off
+ die $eval_stat if $eval_stat =~ /__alarm__ignore__\(.*\)/s;
+ info("dns: bad dns reply: %s", $eval_stat);
+ };
if (!$packet) {
- my $dns_err = $self->{res}->errorstring;
- # resignal if alarm went off
- die "dns (3) $dns_err\n" if $dns_err =~ /__alarm__ignore__\(.*\)/s;
- info("dns: bad dns reply: $dns_err");
+ # error already reported above
+# my $dns_err = $self->{res}->errorstring;
+# die "dns (3) $dns_err\n" if $dns_err =~ /__alarm__ignore__\(.*\)/s;
+# info("dns: bad dns reply: $dns_err");
} else {
my $header = $packet->header;
if (!$header) {
@@ -861,7 +906,8 @@
This subroutine is a simple synchronous leftover from SpamAssassin version
3.3 and does not participate in packet query caching and callback grouping
as implemented by AsyncLoop::bgsend_and_start_lookup(). As such it should
-be avoided for mainstream usage.
+be avoided for mainstream usage. Currently used through Mail::SPF::Server
+by the SPF plugin.
=cut
Index: lib/Mail/SpamAssassin/Plugin/URIDNSBL.pm
===================================================================
--- lib/Mail/SpamAssassin/Plugin/URIDNSBL.pm
(.../tags/spamassassin_release_3_4_1/lib/Mail/SpamAssassin) (revision
1676603)
+++ lib/Mail/SpamAssassin/Plugin/URIDNSBL.pm
(.../branches/3.4/lib/Mail/SpamAssassin) (working copy)
@@ -298,11 +298,10 @@
use Mail::SpamAssassin::Logger;
use strict;
use warnings;
-use bytes;
+# use bytes;
use re 'taint';
-use vars qw(@ISA);
-@ISA = qw(Mail::SpamAssassin::Plugin);
+our @ISA = qw(Mail::SpamAssassin::Plugin);
use constant LOG_COMPLETION_TIMES => 0;
@@ -942,9 +941,8 @@
next unless (defined($str) && defined($dom));
dbg("uridnsbl: got($j) NS for $dom: $str");
- if ($str =~ /IN\s+NS\s+(\S+)/) {
- my $nsmatch = lc $1;
- $nsmatch =~ s/\.$//;
+ if ($rr->type eq 'NS') {
+ my $nsmatch = lc $rr->nsdname; # available since at least Net::DNS 0.14
my $nsrhblstr = $nsmatch;
my $fullnsrhblstr = $nsmatch;
@@ -1010,10 +1008,9 @@
dbg("uridnsbl: complete_a_lookup aborted %s", $ent->{key});
return;
}
-
dbg("uridnsbl: complete_a_lookup %s", $ent->{key});
+ my $j = 0;
my @answer = $pkt->answer;
- my $j = 0;
foreach my $rr (@answer) {
$j++;
my $str = $rr->string;
@@ -1025,9 +1022,9 @@
}
dbg("uridnsbl: complete_a_lookup got(%d) A for %s: %s", $j,$hname,$str);
- local $1;
- if ($str =~ /IN\s+A\s+(\S+)/) {
- $self->lookup_dnsbl_for_ip($pms, $ent->{obj}, $1);
+ if ($rr->type eq 'A') {
+ my $ip_address = $rr->rdatastr;
+ $self->lookup_dnsbl_for_ip($pms, $ent->{obj}, $ip_address);
}
}
}
@@ -1100,7 +1097,9 @@
my $rr_type = $rr->type;
if ($rr_type eq 'A') {
- $rdatastr = $rr->rdatastr;
+ # Net::DNS::RR::A::address() is available since Net::DNS 0.69
+ $rdatastr = $rr->UNIVERSAL::can('address') ? $rr->address
+ : $rr->rdatastr;
if ($rdatastr =~ m/^\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}$/) {
$rdatanum = Mail::SpamAssassin::Util::my_inet_aton($rdatastr);
}
