> Am 06.04.2016 um 22:11 schrieb Michiel van Es <[email protected]>:
>
>
>> On 06 Apr 2016, at 20:42, Michiel van Es <[email protected]> wrote:
>>
>>
>>> On 06 Apr 2016, at 16:58, Michiel van Es <[email protected]> wrote:
>>>
>>>
>>>> On 06 Apr 2016, at 13:52, Michiel van Es <[email protected]> wrote:
>>>>
>>>>
>>>>> On 06 Apr 2016, at 13:38, Joerg Jung <[email protected]> wrote:
>>>>>
>>>>>
>>>>>
>>>>>> Am 06.04.2016 um 13:08 schrieb Michiel van Es <[email protected]>:
>>>>>>
>>>>>> Hello,
>>>>>>
>>>>>> I also posted this as an issue to the OpenSMTPD github repo but somebody
>>>>>> told me that the mailinglist would be more accurate to post this
>>>>>> question to (I will remove the github issue if preferred).
>>>>>>
>>>>>> it seems whenever I use filter-dnbl with several hostnames, the lookups
>>>>>> always fail.
>>>>>> I tried using ipv6 lookups (although this is something opensmtpd does
>>>>>> right?) and have the latest version of the master branch of
>>>>>> OpenSMTPD-Extras (where this ipv4/ipv6 problem was solved with an
>>>>>> earlier similar issue?).
>>>>>
>>>>> All v6 addresses are just accepted by filter-dnsbl.
>>>>> There is no lookup happening for v6 addresses
>>>>> (just not implemented).
>>>>
>>>> ok, then I won’t use IPv6 for now :)
>>>>
>>>>>
>>>>>> The error I get is:
>>>>>> smtp-in: New session 81cf3e1a4d9ef916 from host
>>>>>> pro-mail-smtp-001.bol.com [185.14.168.222] filter-pause[1337]: debug:
>>>>>> on_connect: sleeping 5 filter-dnsbl-spamhaus[1336]: debug: on_connect:
>>>>>> checking 222.168.14.185.zen.spamhaus.org. filter-dnsbl-spamhaus[1336]:
>>>>>> warn: session 81cf3e1a4d9ef916: event_dispatch: REJECT address smtp-in:
>>>>>> Failed command on session 81cf3e1a4d9ef916: "" => 554 5.7.1 Address in
>>>>>> DNSBL smtp-in: Closing session 81cf3e1a4d9ef916 debug: smtp: 0x24460a0:
>>>>>> deleting session: done
>>>>>
>>>>> This looks legit.
>>>>> Have you tried to lookup/verify the IP manually
>>>>> at the time this happened, was it listed?
>>>>
>>>> No and I know this ip (its the MX for my company and they/we do checks via
>>>> Nagios on most DNSBL’s)
>>>> A lookup shows:
>>>>
>>>> 185.14.168.222 is not listed in the SBL
>>>> 185.14.168.222 is not listed in the PBL
>>>> 185.14.168.222 is not listed in the XBL
>>>>
>>>> or PTR
>>>>
>>>> 222.168.14.185.zen.spamhaus.org is not listed in the DBL
>>>>
>>>> I tried this config with the Google DNS servers and using a caching
>>>> localhost dns (with forwarders to OpenNIC servers) - no difference
>>>
>>> Also tried with some more debugging and now tried the default SORBS dnsbl,
>>> the result (with strace and smtpctl trace all):
>>>
>>> r_session@0x1fc28a0[datalen=0,eom=(nil),ofile=(nil)]]
>>> mproc: pony -> filter-proc : 100 IMSG_CTL_FAIL
>>> filter: waiting for running query
>>> 511e5d1fe40dcd9c[QUERY_CONNECT=178.21.114.197 <->
>>> 185.14.169.222(pro-mail-smtp-002.bol.com),filter_session@0x1fc28a0[datalen=0,eom=(nil),ofile=(nil)]]
>>> filter-pause[9835]: debug: on_connect: sleeping 5
>>> filter: imsg IMSG_FILTER_RESPONSE from procfilter
>>> filter-pause[hooks=0xffffffff,flags=0x0000]
>>> filter: filter_drain_query 511e5d1fe40dcd9c[QUERY_CONNECT=178.21.114.197
>>> <->
>>> 185.14.169.222(pro-mail-smtp-002.bol.com),filter_session@0x1fc28a0[datalen=0,eom=(nil),ofile=(nil)]]
>>> filter: running filter filter:filter-regex[hooks=0xffffffff,flags=0x0000]
>>> for query 511e5d1fe40dcd9c[QUERY_CONNECT=178.21.114.197 <->
>>> 185.14.169.222(pro-mail-smtp-002.bol.com),filter_session@0x1fc28a0[datalen=0,eom=(nil),ofile=(nil)]]
>>> mproc: pony -> filter-proc : 100 IMSG_CTL_FAIL
>>> filter: waiting for running query
>>> 511e5d1fe40dcd9c[QUERY_CONNECT=178.21.114.197 <->
>>> 185.14.169.222(pro-mail-smtp-002.bol.com),filter_session@0x1fc28a0[datalen=0,eom=(nil),ofile=(nil)]]
>>> filter: imsg IMSG_FILTER_RESPONSE from procfilter
>>> filter-regex[hooks=0xffffffff,flags=0x0000]
>>> filter: filter_drain_query 511e5d1fe40dcd9c[QUERY_CONNECT=178.21.114.197
>>> <->
>>> 185.14.169.222(pro-mail-smtp-002.bol.com),filter_session@0x1fc28a0[datalen=0,eom=(nil),ofile=(nil)]]
>>> filter: running filter
>>> filter:filter-dnsbl-sorbs[hooks=0xffffffff,flags=0x0000] for query
>>> 511e5d1fe40dcd9c[QUERY_CONNECT=178.21.114.197 <->
>>> 185.14.169.222(pro-mail-smtp-002.bol.com),filter_session@0x1fc28a0[datalen=0,eom=(nil),ofile=(nil)]]
>>> mproc: pony -> filter-proc : 100 IMSG_CTL_FAIL
>>> filter: waiting for running query
>>> 511e5d1fe40dcd9c[QUERY_CONNECT=178.21.114.197 <->
>>> 185.14.169.222(pro-mail-smtp-002.bol.com),filter_session@0x1fc28a0[datalen=0,eom=(nil),ofile=(nil)]]
>>> filter-dnsbl-sorbs[9834]: debug: on_connect: checking
>>> 222.169.14.185.dnsbl.sorbs.net.
>>> filter-dnsbl-sorbs[9834]: warn: session 511e5d1ea5ee10d1: event_dispatch:
>>> REJECT address
>>> filter: imsg IMSG_FILTER_RESPONSE from procfilter
>>> filter-dnsbl-sorbs[hooks=0xffffffff,flags=0x0000]
>>> filter: filter_drain_query 511e5d1fe40dcd9c[QUERY_CONNECT=178.21.114.197
>>> <->
>>> 185.14.169.222(pro-mail-smtp-002.bol.com),filter_session@0x1fc28a0[datalen=0,eom=(nil),ofile=(nil)]]
>>> filter: filter_end_query 511e5d1fe40dcd9c[QUERY_CONNECT=178.21.114.197 <->
>>> 185.14.169.222(pro-mail-smtp-002.bol.com),filter_session@0x1fc28a0[datalen=0,eom=(nil),ofile=(nil)]]
>>> filter: query 511e5d1fe40dcd9c done: status=FILTER_CLOSE code=554
>>> response="5.7.1 Address in DNSBL"
>>> smtp: 0x20ae090: >>> 554 5.7.1 Address in DNSBL
>>> smtp-in: Failed command on session 511e5d1ea5ee10d1: "" => 554 5.7.1
>>> Address in DNSBL
>>> smtp: 0x20ae090: STATE_CONNECTED -> STATE_QUIT
>>> smtp: 0x20ae090: IO_LOWAT <io:0x20ae0d8 fd=4 to=300000 fl=W ib=0 ob=0>
>>> smtp-in: Closing session 511e5d1ea5ee10d1
>>> debug: smtp: 0x20ae090: deleting session: done
>>>
>>> The hostname is interesting, it seems to do a lookup of
>>> 222.169.14.185.dnsbl.sorbs.net. => on_connect: checking
>>> 222.169.14.185.dnsbl.sorbs.net.
>>>
>>> Off course the . at the end makes it an invalid hostname and a check on
>>> SORBS tells me the same: Bad host/domain 222.169.14.185.dnsbl.sorbs.net.
>>>
>>> Using without the . at then end I get: [222.169.14.185.dnsbl.sorbs.net] Not
>>> found in the database
>>> Trying the ip and the hostname of the MX: [185.14.169.222/32] Not found in
>>> the database & [pro-mail-smtp-002.bol.com] Not found in the database
>>>
>>> So I am a little bit lost here what is going wrong with the lookups..
>>> Can I have more debugging of the filter-dnsbl option?
>>
>> The interesting thing is that also on Ubuntu LTS 16.04 packaged
>> opensmtp-extras the filter-dnsbl fails but with a different error message:
>>
>> lstat("/var/spool/smtpd/offline", {st_mode=S_IFDIR|S_ISVTX|0777,
>> st_size=4096, ...}) = 0
>> open("/var/spool/smtpd/offline", O_RDONLY|O_NONBLOCK|O_DIRECTORY|O_CLOEXEC)
>> = 5
>> fstat(5, {st_mode=S_IFDIR|S_ISVTX|0777, st_size=4096, ...}) = 0
>> getdents(5, /* 2 entries */, 32768) = 48
>> getdents(5, /* 0 entries */, 32768) = 0
>> close(5) = 0
>> write(2, "debug: smtpd: offline scanning d"..., 36debug: smtpd: offline
>> scanning done
>> ) = 36
>> epoll_wait(3,
>> debug: smtp: new client on listener: 0xf68180
>> smtp-in: session fdbb37cd0253c15c: connection from host
>> pro-mail-smtp-001.bol.com [185.14.168.222] established
>> debug: filter-pause: sleeping 5
>> debug: filter-dnsbl: checking 222.168.14.185.zen.spamhaus.org.
>> smtp-in: session fdbb37cd0253c15c: received invalid command: ""
>> smtp-in: session fdbb37cd0253c15c: connection from host
>> pro-mail-smtp-001.bol.com [185.14.168.222] closed (client sent QUIT)
>> debug: smtp: 0xf89ba0: deleting session: done
>>
>> Is filter-dnsbl even working for some of you? It looks like the filter-dnsbl
>> can not get a correct answer (received invalid command: “”)
>
> And with some more debugging:
>
> filter: new query QK_QUERY QUERY_CONNECT
> filter: filter_drain_query
> 8c1aad2fb677222c[QK_QUERY,QUERY_CONNECT=178.21.114.197 <->
> 74.125.82.54(mail-wm0-f54.google.com),filter_session@0x1e3d870[datalen=0,eom=(nil),ofile=(nil)]]
> filter: running filter filter:dnsbl[hooks=0xffffffff,flags=0x0000] for query
> 8c1aad2fb677222c[QK_QUERY,QUERY_CONNECT=178.21.114.197 <->
> 74.125.82.54(mail-wm0-f54.google.com),filter_session@0x1e3d870[datalen=0,eom=(nil),ofile=(nil)]]
> filter: waiting for running query
> 8c1aad2fb677222c[QK_QUERY,QUERY_CONNECT=178.21.114.197 <->
> 74.125.82.54(mail-wm0-f54.google.com),filter_session@0x1e3d870[datalen=0,eom=(nil),ofile=(nil)]]
> debug: filter-dnsbl: checking 54.82.125.74.zen.spamhaus.org.
> filter: imsg IMSG_FILTER_RESPONSE from procfilter
> dnsbl[hooks=0xffffffff,flags=0x0000]
> filter: filter_drain_query
> 8c1aad2fb677222c[QK_QUERY,QUERY_CONNECT=178.21.114.197 <->
> 74.125.82.54(mail-wm0-f54.google.com),filter_session@0x1e3d870[datalen=0,eom=(nil),ofile=(nil)]]
> filter: filter_end_query
> 8c1aad2fb677222c[QK_QUERY,QUERY_CONNECT=178.21.114.197 <->
> 74.125.82.54(mail-wm0-f54.google.com),filter_session@0x1e3d870[datalen=0,eom=(nil),ofile=(nil)]]
> filter: query 8c1aad2fb677222c done: status=FILTER_CLOSE code=0
> response="(null)"
> smtp-in: session 8c1aad2dcce6e1c6: received invalid command: ""
> smtp-in: session 8c1aad2dcce6e1c6: connection from host
> mail-wm0-f54.google.com [74.125.82.54] closed (client sent QUIT)
> debug: smtp: 0x1e353c0: deleting session: done
> filter: new query QK_EVENT EVENT_DISCONNECT
>
> I do see the PTR query in my dnsmasq logfile but I think there is something
> going wrong with the DNSBL lookup after the PTR lookup of the host and the
> response is “(null)” which does not sound right..
I think this is the chroot bug fixed in latest snapshot.
> Also a tcpdump -i ens3 | grep zen does not show any traffic
> Also with psbl.surriel.org the same thing, so it seems there is no lookup at
> all..
>
>
>>
>>>
>>>>
>>>>>
>>>>>> My (sniplet of relevant) config is:
>>>>>>
>>>>>> # filters
>>>>>> filter filter-pause pause
>>>>>> filter filter-regex regex
>>>>>> #filter filter-dnsbl-sorbs dnsbl
>>>>>> #filter filter-dnsbl-surriel dnsbl "-dv" "-h psbl.surriel.com"
>>>>>> #filter filter-dnsbl-spamhaus dnsbl "-h" "zen.spamhaus.org"
>>>>>> filter filter-spamassassin spamassassin "-s accept"
>>>>>> filter filter-clamav clamav
>>>>>> #filter all chain filter-pause filter-regex filter-dnsbl-surriel
>>>>>> filter-dnsbl-spamhaus filter-spamassassin filter-clamav
>>>>>> filter all chain filter-pause filter-regex filter-spamassassin
>>>>>> filter-clamav
>>>>>> #filter all chain filter-pause filter-regex filter-dnsbl-spamhaus
>>>>>> filter-spamassassin filter-clamav
>>>>>> filter sub chain filter-pause filter-spamassassin filter-clamav
>>>>>> # pki/ssl/certs
>>>>>> pki server.pragmasec.nl key
>>>>>> "/etc/letsencrypt/archive/server.pragmasec.nl/privkey1.pem"
>>>>>> pki server.pragmasec.nl certificate
>>>>>> "/etc/letsencrypt/archive/server.pragmasec.nl/fullchain1.pem"
>>>>>> # listen
>>>>>> listen on lo
>>>>>> listen on eth0 port 25 filter all hostname server.pragmasec.nl tls pki
>>>>>> server.pragmasec.nl
>>>>>> listen on eth0 port 587 filter sub hostname server.pragmasec.nl
>>>>>> tls-require pki server.pragmasec.nl auth mask-source
>>>>>> # queue expiry
>>>>>> expire 7d
>>>>>> # virtual domains and users
>>>>>> table vdomains "/usr/local/etc/vdomains"
>>>>>> table vusers "/usr/local/etc/vusers"
>>>>>> # our accepted relays
>>>>>> accept from any for domain <vdomains> virtual <vusers> deliver to mda
>>>>>> "/usr/lib/dovecot/dovecot-lda -f %{sender} -a %{rcpt}"
>>>>>> accept from local for any relay
>>>>>>
>>>>>> Using Ubuntu 14.04.3 LTS with git branch of opensmtpd (OpenSMTPD 5.9.1p1)
>>>>>>
>>>>>> What can I do to troubleshoot or further investigate this?
>>>>>
>>>>> Validate manually with a listed and non-listed IP.
>>>>> Try to rule-out local resolving problems.
>>>>
>>>> It seems everything is listed through the filter rule..even using Gmail or
>>>> other big mail servers.
>>>>
>>>>>
>>>>>> Are there any other spam filters that I can use or might be handy to
>>>>>> follow RFC’s? for example I do use some HELO checks but I think there
>>>>>> might be more then the ones I have:
>>>>>>
>>>>>> # reject helo with leading or trailing dot, and without dots (non-FQDN)
>>>>>> # skipping address literals
>>>>>> helo ! ^\[
>>>>>> helo ^\.
>>>>>> helo \.$
>>>>>> helo ^[^\.]*$
>>>>>
>>>>> In general OpenSMTPD is RFC conform.
>>>>> This helo check is just an additional hard restriction.
>>>>>
>>>>> There are other restrictions possible like enforcing
>>>>> line lengths or forcing valid reverse lookups, but
>>>>> these might not help but likely break legit mails.
>>>>
>>>> Hmm then I keep it as is, thanks for the feedback!
>>>>
>>>>>
>>>>>> Michiel
>>>>>> --
>>>>>> You received this mail because you are subscribed to [email protected]
>>>>>> To unsubscribe, send a mail to: [email protected]
>>>>
>>>>
>>>> --
>>>> You received this mail because you are subscribed to [email protected]
>>>> To unsubscribe, send a mail to: [email protected]
>>>
>>>
>>> --
>>> You received this mail because you are subscribed to [email protected]
>>> To unsubscribe, send a mail to: [email protected]
>>
>>
>> --
>> You received this mail because you are subscribed to [email protected]
>> To unsubscribe, send a mail to: [email protected]
>
>
> --
> You received this mail because you are subscribed to [email protected]
> To unsubscribe, send a mail to: [email protected]
>
--
You received this mail because you are subscribed to [email protected]
To unsubscribe, send a mail to: [email protected]
