Re: Problem with spamlogd
After many hundreds of tests, I have uncovered something that I never found in
any of the pf documents or man pages. If an incoming packet is processed by an
rdr rule, it will always be logged as rdr with the rdr rule number. The pass
action is never logged, even if the rdr rule does not include log and the pass
rule does.
As a result, since spamlogd is specifically looking for a pass action, it will
never see those log entries. Hence, you must ensure that the packets which do
get sent to the mailserver (real one) never are processed by a rdr rule. I
ended up having to use no rdr rules for those to get the logging done such that
spamlogd would find them:
no rdr inet proto tcp from to any port smtp
no rdr inet proto tcp from to any port smtp
rdr pass on $ext_if inet proto tcp to $MAILHOSTS port smtp -> 127.0.0.1 port spa
md
pass in log on $ext_if inet proto tcp to $MAILHOSTS port smtp
This setup works on FreeBSD 7.2 and 9.0. I couldn't find any other that
actually worked including those in the various pf books, man pages, and other
writings on pf.
On 17 June 2012, at 09:40, Doug Hardie wrote:
>
> On 17 June 2012, at 06:29, Matthew Seaman wrote:
>
>> On 17/06/2012 11:45, Doug Hardie wrote:
>>> I am using spamd on several systems and started encountering a problem
>>> awhile ago with FreeBSD 7.2 servers, but let it go since I am in the
>>> process of upgrading the servers. However, I now am encountering the same
>>> issue on FreeBSD 9.0 with spamlogd. It never reads pflog0. pflogd reads
>>> the entries just fine. I set up syslog to log all the spamlogd messages
>>> and when spamlogd is started it gives:
>>>
>>> spamlogd: Listening on pflog0 for all interfaces.
>>>
>>> lsof shows that it is connected to bpf0 as is pflogd. However, pflogd
>>> shows an offset into the file that appears to be the end of the file.
>>> spamlogd shows an offset of 0. It is periodically reading the file as
>>> shown by ktrace but always getting back a 0 size return. spamd itself is
>>> working just fine. However, the expiration times are not being updated so
>>> white entries are timed out way too often. spamlogd used to update them.
>>> The rc.conf entries are:
>>>
>>> obspamd_enable="YES"
>>> obspamd_flags="-G 2:1:1728"
>>> obspamd_setup_flags=""
>>> obspamd_grey=YES
>>> obspamlogd_enable="YES"
>>> obspamlogd_flags="-W 1728"
>>>
>>>
>>> These were established a few years ago and worked up till short while ago.
>>> I don't recall any changes I made to anything, but…
>>>
>>> Looking through the spamlogd source it appears to be building a filter for
>>> the pcap routines with:
>>>
>>> "ip and port 25 and action pass and tcp[13]&0x12=0x2"
>>>
>>> Using that filter on pflog yields no output. I believe the pass item
>>> requires there to be some logging of the pass actions and those are not
>>> appearing in the pflog or in the pfctl counts for those rules. I suspect
>>> that is the problem. The pf.conf is: (mail server is on this machine)
>>>
>>> ext_if="em0"
>>>
>>> table persist file "/etc/blackhole"
>>> table persist
>>> table persist
>>> table persist file "/etc/mail/whitelist"
>>>
>>>
>>> no rdr on { lo0, lo1 } from any to any
>>>
>>> no rdr on { lo0, lo1 } from any to any
>>> MAILHOSTS = "{zool.lafn.org 10.0.1.10}"
>>>
>>> rdr pass log on $ext_if inet proto tcp from to port
>>> smtp -> 127.0.0.1 port smtp
>>> rdr pass log on $ext_if inet proto tcp from to port smtp ->
>>> 127.0.0.1 port smtp
>>> rdr pass log on $ext_if inet proto tcp to $MAILHOSTS port smtp -> 127.0.0.1
>>> port spamd
>>>
>>>
>>> pass in on lo0
>>>
>>> pass in log on $ext_if inet proto tcp to 127.0.0.1 port smtp
>>> pass out log on $ext_if inet proto tcp from 127.0.0.1 to any port smtp
>>>
>>> block in quick log on $ext_if from to any
>>
>> You seem to be logging all the SMTP traffic that passes through pf in
>> any direction. Which doesn't make a lot of sense to me -- obspamlogd
>> will see the logged SMTP packets, assume that's valid traffic and add
>> the hosts to the whitelist. Even if that's the incoming SYN packet from
>> some dubious mailer trying to inject you full of spam.
>
> Right now, I would like spamlogd to be a bit confused ;-) However, its not
> seeing any of the logging. It never receives any input from pflog0. From
> the filter, the pass action indicates it won't look at any of the rdr logging
> (which is in the log) but is waiting for the pass rules to log something.
> The tcp[13]&0x12=0x2 item is the TCP SYN flag so it should be able to
> separate out what it wants from the log. However, the pass rules are never
> being used and hence they never generate any log entries. pfctl -vvsr shows
> all zeros for both of those rules.
>
> I understand that the pass rules are applied after the rdr rules but
> apparently I am getting the matching criteria wrong. At this point switching
> them to a separate log stream won't help since it would never g
Re: Problem with spamlogd
On 17 June 2012, at 06:29, Matthew Seaman wrote:
> On 17/06/2012 11:45, Doug Hardie wrote:
>> I am using spamd on several systems and started encountering a problem
>> awhile ago with FreeBSD 7.2 servers, but let it go since I am in the process
>> of upgrading the servers. However, I now am encountering the same issue on
>> FreeBSD 9.0 with spamlogd. It never reads pflog0. pflogd reads the entries
>> just fine. I set up syslog to log all the spamlogd messages and when
>> spamlogd is started it gives:
>>
>> spamlogd: Listening on pflog0 for all interfaces.
>>
>> lsof shows that it is connected to bpf0 as is pflogd. However, pflogd shows
>> an offset into the file that appears to be the end of the file. spamlogd
>> shows an offset of 0. It is periodically reading the file as shown by
>> ktrace but always getting back a 0 size return. spamd itself is working
>> just fine. However, the expiration times are not being updated so white
>> entries are timed out way too often. spamlogd used to update them. The
>> rc.conf entries are:
>>
>> obspamd_enable="YES"
>> obspamd_flags="-G 2:1:1728"
>> obspamd_setup_flags=""
>> obspamd_grey=YES
>> obspamlogd_enable="YES"
>> obspamlogd_flags="-W 1728"
>>
>>
>> These were established a few years ago and worked up till short while ago.
>> I don't recall any changes I made to anything, but…
>>
>> Looking through the spamlogd source it appears to be building a filter for
>> the pcap routines with:
>>
>> "ip and port 25 and action pass and tcp[13]&0x12=0x2"
>>
>> Using that filter on pflog yields no output. I believe the pass item
>> requires there to be some logging of the pass actions and those are not
>> appearing in the pflog or in the pfctl counts for those rules. I suspect
>> that is the problem. The pf.conf is: (mail server is on this machine)
>>
>> ext_if="em0"
>>
>> table persist file "/etc/blackhole"
>> table persist
>> table persist
>> table persist file "/etc/mail/whitelist"
>>
>>
>> no rdr on { lo0, lo1 } from any to any
>>
>> no rdr on { lo0, lo1 } from any to any
>> MAILHOSTS = "{zool.lafn.org 10.0.1.10}"
>>
>> rdr pass log on $ext_if inet proto tcp from to port smtp
>> -> 127.0.0.1 port smtp
>> rdr pass log on $ext_if inet proto tcp from to port smtp ->
>> 127.0.0.1 port smtp
>> rdr pass log on $ext_if inet proto tcp to $MAILHOSTS port smtp -> 127.0.0.1
>> port spamd
>>
>>
>> pass in on lo0
>>
>> pass in log on $ext_if inet proto tcp to 127.0.0.1 port smtp
>> pass out log on $ext_if inet proto tcp from 127.0.0.1 to any port smtp
>>
>> block in quick log on $ext_if from to any
>
> You seem to be logging all the SMTP traffic that passes through pf in
> any direction. Which doesn't make a lot of sense to me -- obspamlogd
> will see the logged SMTP packets, assume that's valid traffic and add
> the hosts to the whitelist. Even if that's the incoming SYN packet from
> some dubious mailer trying to inject you full of spam.
Right now, I would like spamlogd to be a bit confused ;-) However, its not
seeing any of the logging. It never receives any input from pflog0. From the
filter, the pass action indicates it won't look at any of the rdr logging
(which is in the log) but is waiting for the pass rules to log something. The
tcp[13]&0x12=0x2 item is the TCP SYN flag so it should be able to separate out
what it wants from the log. However, the pass rules are never being used and
hence they never generate any log entries. pfctl -vvsr shows all zeros for
both of those rules.
I understand that the pass rules are applied after the rdr rules but apparently
I am getting the matching criteria wrong. At this point switching them to a
separate log stream won't help since it would never get anything logged to it.
>
> You should only log the SYN packets going out of your upstream (egress)
> interface for obspamlogd -- that way it immediately whitelists anyone
> you send email to, so they can reply without delay due to greylisting.
>
> A good way of doing that is to log SMTP traffic to a separate log
> device. eg:
>
> pass log (to pflog1) on $ext_if proto tcp \
> from any to any port smtp\
> flags S/SA keep state
>
> then in /etc/rc.conf, tell obspamlogd to use pflog1:
>
> obspamlogd_enable="YES"
> obspamlogd_flags="-i em0"
> obspamlogd_pflog_if="pflog1"
>
> That way you can keep pflog0 for doing the normal packet logging that is
> usual with pf -- typically, logging anything that gets dropped by the
> firewall -- without getting obspamlogd confused.
>
> Cheers,
>
> Matthew
>
> --
> Dr Matthew J Seaman MA, D.Phil. 7 Priory Courtyard
> Flat 3
> PGP: http://www.infracaninophile.co.uk/pgpkey Ramsgate
> JID: matt...@infracaninophile.co.uk Kent, CT11 9PW
>
>
>
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mai
Re: Problem with spamlogd
On 17/06/2012 11:45, Doug Hardie wrote:
> I am using spamd on several systems and started encountering a problem awhile
> ago with FreeBSD 7.2 servers, but let it go since I am in the process of
> upgrading the servers. However, I now am encountering the same issue on
> FreeBSD 9.0 with spamlogd. It never reads pflog0. pflogd reads the entries
> just fine. I set up syslog to log all the spamlogd messages and when
> spamlogd is started it gives:
>
> spamlogd: Listening on pflog0 for all interfaces.
>
> lsof shows that it is connected to bpf0 as is pflogd. However, pflogd shows
> an offset into the file that appears to be the end of the file. spamlogd
> shows an offset of 0. It is periodically reading the file as shown by ktrace
> but always getting back a 0 size return. spamd itself is working just fine.
> However, the expiration times are not being updated so white entries are
> timed out way too often. spamlogd used to update them. The rc.conf entries
> are:
>
> obspamd_enable="YES"
> obspamd_flags="-G 2:1:1728"
> obspamd_setup_flags=""
> obspamd_grey=YES
> obspamlogd_enable="YES"
> obspamlogd_flags="-W 1728"
>
>
> These were established a few years ago and worked up till short while ago. I
> don't recall any changes I made to anything, but…
>
> Looking through the spamlogd source it appears to be building a filter for
> the pcap routines with:
>
> "ip and port 25 and action pass and tcp[13]&0x12=0x2"
>
> Using that filter on pflog yields no output. I believe the pass item
> requires there to be some logging of the pass actions and those are not
> appearing in the pflog or in the pfctl counts for those rules. I suspect
> that is the problem. The pf.conf is: (mail server is on this machine)
>
> ext_if="em0"
>
> table persist file "/etc/blackhole"
> table persist
> table persist
> table persist file "/etc/mail/whitelist"
>
>
> no rdr on { lo0, lo1 } from any to any
>
> no rdr on { lo0, lo1 } from any to any
> MAILHOSTS = "{zool.lafn.org 10.0.1.10}"
>
> rdr pass log on $ext_if inet proto tcp from to port smtp
> -> 127.0.0.1 port smtp
> rdr pass log on $ext_if inet proto tcp from to port smtp ->
> 127.0.0.1 port smtp
> rdr pass log on $ext_if inet proto tcp to $MAILHOSTS port smtp -> 127.0.0.1
> port spamd
>
>
> pass in on lo0
>
> pass in log on $ext_if inet proto tcp to 127.0.0.1 port smtp
> pass out log on $ext_if inet proto tcp from 127.0.0.1 to any port smtp
>
> block in quick log on $ext_if from to any
You seem to be logging all the SMTP traffic that passes through pf in
any direction. Which doesn't make a lot of sense to me -- obspamlogd
will see the logged SMTP packets, assume that's valid traffic and add
the hosts to the whitelist. Even if that's the incoming SYN packet from
some dubious mailer trying to inject you full of spam.
You should only log the SYN packets going out of your upstream (egress)
interface for obspamlogd -- that way it immediately whitelists anyone
you send email to, so they can reply without delay due to greylisting.
A good way of doing that is to log SMTP traffic to a separate log
device. eg:
pass log (to pflog1) on $ext_if proto tcp \
from any to any port smtp\
flags S/SA keep state
then in /etc/rc.conf, tell obspamlogd to use pflog1:
obspamlogd_enable="YES"
obspamlogd_flags="-i em0"
obspamlogd_pflog_if="pflog1"
That way you can keep pflog0 for doing the normal packet logging that is
usual with pf -- typically, logging anything that gets dropped by the
firewall -- without getting obspamlogd confused.
Cheers,
Matthew
--
Dr Matthew J Seaman MA, D.Phil. 7 Priory Courtyard
Flat 3
PGP: http://www.infracaninophile.co.uk/pgpkey Ramsgate
JID: matt...@infracaninophile.co.uk Kent, CT11 9PW
signature.asc
Description: OpenPGP digital signature
