> On 20 Apr 2017, at 01:18, Sjöholm Per-Olov <p...@incedo.org> wrote:
>
>
>> On 20 Apr 2017, at 00:39, Fred <open...@crowsons.com> wrote:
>>
>> On 04/19/17 23:30, Sjöholm Per-Olov wrote:
>>> Anyone with a clue would be _very_ much appreciated….
>>> I upgraded from 6.0 to 6.1 two days ago and **did not change anything to
>>> the network** stuff at all. After that clients have random problems
>>> reaching my dmz web server (centos + nginx). I have checked the release
>>> notes, but could not see any clue there. Se logs below
>>> # Relevant rules from PF
>>> LAN_INT="vlan2"
>>> DMZ1_INT="vlan3"
>>> DMZ2_INT="vlan4"
>>> GUEST_INT="vlan1003"
>>> INTERNET_INT="em3
>>> ALL_INTERFACES="{" $LAN_INT $GUEST_INT $DMZ1_INT $DMZ2_INT $INTERNET_INT "}"
>>> pass out on $ALL_INTERFACES inet proto {tcp gre esp udp icmp ipv6} all keep
>>> state
>>> pass out on $ALL_INTERFACES inet6 proto {tcp gre esp udp icmp6} all keep
>>> state
>>> pass out on $IPV6_TUNNEL_INT inet6 all keep state
>>> pass in log quick on $INTERNET_INT inet proto tcp from any to
>>> $DMZ1_DAEDALUS port { 80 443 } label "webstats:$dstport" flags S/SAFR keep
>>> state (max-src-nodes 90, max-src-states 150, max-src-conn 150,
>>> max-src-conn-rate 250/30, overload <bad_hosts> flush global)
>>> # Log that after upgrade shows problems in the logs related to this
>>> directly after the upgrade
>>> root@xanadu:/var/log#tcpdump -e -n -ttt -r /var/log/pflog.6|grep block|grep
>>> 155.4|grep out |grep ': R'
>>> tcpdump: WARNING: snaplen raised from 116 to 160
>>> root@xanadu:/var/log#tcpdump -e -n -ttt -r /var/log/pflog.5|grep block|grep
>>> 155.4|grep out |grep ': R'
>>> tcpdump: WARNING: snaplen raised from 116 to 160
>>> root@xanadu:/var/log#tcpdump -e -n -ttt -r /var/log/pflog.4|grep block|grep
>>> 155.4|grep out |grep ': R'
>>> tcpdump: WARNING: snaplen raised from 116 to 160
>>> root@xanadu:/var/log#tcpdump -e -n -ttt -r /var/log/pflog.3|grep block|grep
>>> 155.4|grep out |grep ': R'
>>> tcpdump: WARNING: snaplen raised from 116 to 160
>>> root@xanadu:/var/log#tcpdump -e -n -ttt -r /var/log/pflog.2|grep block|grep
>>> 155.4|grep out |grep ': R'
>>> tcpdump: WARNING: snaplen raised from 116 to 160
>>> Apr 17 05:43:36.359067 rule 63/(match) block out on em3: 155.4.8.28.80 >
>>> 164.132.161.92.46942: R 0:0(0) ack 2697518940 win 0 (DF)
>>> Apr 17 05:43:37.358688 rule 63/(match) block out on em3: 155.4.8.28.80 >
>>> 164.132.161.92.46942: R 0:0(0) ack 1 win 0 (DF)
>>> Apr 17 05:43:39.362671 rule 63/(match) block out on em3: 155.4.8.28.80 >
>>> 164.132.161.92.46942: R 0:0(0) ack 1 win 0 (DF)
>>> Apr 17 06:10:24.490412 rule 63/(match) block out on em3: 155.4.8.28.80 >
>>> 139.162.111.147.33930: R 0:0(0) ack 1409896759 win 0 (DF)
>>> Apr 17 06:32:45.198754 rule 63/(match) block out on em3: 155.4.8.28.80 >
>>> 180.76.15.26.42835: R 0:0(0) ack 3718886589 win 0 (DF)
>>> Apr 17 06:32:46.198338 rule 63/(match) block out on em3: 155.4.8.28.80 >
>>> 180.76.15.26.42835: R 0:0(0) ack 1 win 0 (DF)
>>> Apr 17 06:41:29.366359 rule 63/(match) block out on em3: 155.4.8.28.80 >
>>> 51.255.65.91.42819: R 0:0(0) ack 4294673273 win 0 (DF)
>>> Apr 17 06:41:30.365396 rule 63/(match) block out on em3: 155.4.8.28.80 >
>>> 51.255.65.91.42819: R 0:0(0) ack 1 win 0 (DF)
>>> Apr 17 06:41:32.369399 rule 63/(match) block out on em3: 155.4.8.28.80 >
>>> 51.255.65.91.42819: R 0:0(0) ack 1 win 0 (DF)
>>> — cut the rest —
>>> What have I missed?
>>> Tnx in advance
>>> Peo
>>> Thanks
>>> Peo
>>
>> You might get some clues from:
>>
>> pfctl -sr -R 63
>>
>> Cheers
>>
>> Fred
>>
>
> I know that that rule is my default block…
>
> root@xanadu:/etc#pfctl -g -sr|grep "@63"
> @63 block drop log all
> root@xanadu:/etc#
>
> But why is this happening after upgrade. I have netiher touched pf.conf,
> sysctl.conf or /etc/hostname* nor found any changes in release notes related
> to this. So I see no reason for the packet to get stuck on that rule. But I
> am probably missing something obvious :)
>
>
> Peo
>
This is a tricky one…. I would very much appreciate “pro” help :)
It seems a reload with pfctl -f /etc/pf.conf make these blocked packages go
away for two hours or so (and everything is working). After that the problem
comes back and I again see frequent intermittent block out on the internet
interface.
root@xanadu:/etc#tcpdump -e -n -ttt -r /var/log/pflog|grep block|grep
155.4|grep out |grep ': R'|tail -20
tcpdump: WARNING: snaplen raised from 116 to 160
Apr 20 12:34:18.052265 rule 63/(match) block out on em3: 155.4.8.28.25 >
194.71.64.14.5910: R 0:0(0) ack 1 win 0 (DF)
Apr 20 12:34:21.249292 rule 63/(match) block out on em3: 155.4.8.28.25 >
194.71.64.14.5910: R 0:0(0) ack 1 win 0 (DF)
Apr 20 12:34:24.449235 rule 63/(match) block out on em3: 155.4.8.28.25 >
194.71.64.14.5910: R 0:0(0) ack 1 win 0 (DF)
Apr 20 12:34:27.649446 rule 63/(match) block out on em3: 155.4.8.28.25 >
194.71.64.14.5910: R 0:0(0) ack 1 win 0 (DF)
Apr 20 12:34:30.849380 rule 63/(match) block out on em3: 155.4.8.28.25 >
194.71.64.14.5910: R 0:0(0) ack 1 win 0 (DF)
Apr 20 12:34:37.049703 rule 63/(match) block out on em3: 155.4.8.28.25 >
194.71.64.14.5910: R 0:0(0) ack 1 win 0 (DF)
Apr 20 12:34:49.253624 rule 63/(match) block out on em3: 155.4.8.28.25 >
194.71.64.14.5910: R 0:0(0) ack 1 win 0 (DF)
Apr 20 12:35:13.449340 rule 63/(match) block out on em3: 155.4.8.28.25 >
194.71.64.14.5910: R 0:0(0) ack 1 win 0 (DF)
Apr 20 12:35:19.053693 rule 63/(match) block out on em3: 155.4.8.28.25 >
41.78.84.231.50310: R 0:0(0) ack 3874760854 win 0 (DF)
Apr 20 12:35:22.048103 rule 63/(match) block out on em3: 155.4.8.28.25 >
41.78.84.231.50310: R 0:0(0) ack 1 win 0 (DF)
Apr 20 12:35:28.045309 rule 63/(match) block out on em3: 155.4.8.28.25 >
41.78.84.231.50310: R 0:0(0) ack 1 win 0 (DF)
Apr 20 12:37:15.284115 rule 63/(match) block out on em3: 155.4.8.28.25 >
112.215.201.32.20948: R 0:0(0) ack 533408652 win 0 (DF)
Apr 20 12:37:18.302353 rule 63/(match) block out on em3: 155.4.8.28.25 >
112.215.201.32.20948: R 0:0(0) ack 1 win 0 (DF)
Apr 20 12:37:24.294055 rule 63/(match) block out on em3: 155.4.8.28.25 >
112.215.201.32.20948: R 0:0(0) ack 1 win 0 (DF)
Apr 20 12:38:24.998973 rule 63/(match) block out on em3: 155.4.8.28.25 >
209.85.218.51.33714: R 0:0(0) ack 3754971547 win 0 (DF)
Apr 20 12:38:25.999057 rule 63/(match) block out on em3: 155.4.8.28.25 >
209.85.218.51.33714: R 0:0(0) ack 1 win 0 (DF)
Apr 20 12:38:27.999024 rule 63/(match) block out on em3: 155.4.8.28.25 >
209.85.218.51.33714: R 0:0(0) ack 1 win 0 (DF)
Apr 20 12:38:31.998695 rule 63/(match) block out on em3: 155.4.8.28.25 >
209.85.218.51.33714: R 0:0(0) ack 1 win 0 (DF)
Apr 20 12:38:39.998591 rule 63/(match) block out on em3: 155.4.8.28.25 >
209.85.218.51.33714: R 0:0(0) ack 1 win 0 (DF)
Apr 20 12:38:55.998538 rule 63/(match) block out on em3: 155.4.8.28.25 >
209.85.218.51.33714: R 0:0(0) ack 1 win 0 (DF)
root@xanadu:/etc#
Could it be any buffers that is causing this in 6.1 but not in 6.0 ?
Peo
