On 28.7.2012. 21:56, Florian Obser wrote:
> Benno and me were looking into why pflow is sending flows with
> starttime after endtime. We believe this was introduced with this
> commit:
>
> ------------------------------------------------------------------------
> CVSROOT: /cvs
> Module name: src
> Changes by: d...@cvs.openbsd.org 2011/11/25 05:52:10
>
> Modified files:
> sys/net : if_pflow.c if_pfsync.c pf.c pf_ioctl.c
> pf_norm.c
>
> Log message:
> use time_uptime to set state creation values as time_second can be
> skewed at runtime by things like date(1) and ntpd. time_uptime is
> monotonic and therefore more useful to compare against.
>
> ok deraadt@ mikeb@
> ------------------------------------------------------------------------
>
> So the startime comes from time_uptime while the expiration time
> depends on time_second:
> in pf_purge_expired_states(u_int32_t maxcheck):
> } else if (pf_state_expires(cur) <= time_second) {
>
> The following patch changes the state timeouts to time_uptime. There
> are two additional patches for fragmentation and src track timeouts.
>
> I believe the remaining usages of time_second are for display purpose
> (in pf_ioctl.c, pf_table.c, pf_if.c and one in pf.c)
Hello,
i have tested your patches with torrent box behind 2 firewalls (nat,
carp,pfsync). One firewall is patched and second box isn't. Both of them
quite often export flows with duration of 429496*
log from nfdump collector:
2012-07-29 17:18:24.005 4294967275.000 TCP 193.198.155.12:56448 ->
211.28.145.239:6789 11 1597 1
2012-07-29 17:18:24.005 4294967291.000 TCP 193.198.155.12:55502 ->
76.98.245.243:61697 1 60 1
2012-07-29 17:18:24.005 4294967293.000 TCP 212.85.66.20:54282 ->
193.198.155.12:32459 4 284 1
2012-07-29 17:18:55.005 4294967290.000 UDP 173.52.237.220:38155 ->
10.161.53.99:32459 3 459 1
2012-07-29 17:18:56.005 4294967294.000 UDP 118.11.12.59:19640 ->
193.198.155.12:32459 1 95 1
2012-07-29 17:19:28.005 4294967278.000 TCP 193.198.155.12:63341 ->
90.190.164.171:17780 5 300 1
pf.conf
set skip on { lo em1 em3 pfsync0 }
set limit { states 25000, src-nodes 25000, table-entries 300000 }
set state-defaults pflow
match out on em0 from vlan600:network nat-to carp0
block in log on em0
pass out
pass in on em0 from { X.X.X.X/22 Y.Y.Y.Y/22 X.X.X.X/28 }
pass on { em0 vlan600 } proto carp keep state (no-sync)
pass in on em0 proto tcp to 193.198.155.12 port 32459 rdr-to 10.161.53.99
pass in on em0 proto udp to 193.198.155.12 port 32459 rdr-to 10.161.53.99
pass in on em0 proto tcp from X.X.X.X/22 to 193.198.155.12 port 5900
rdr-to 10.161.53.99
pass in on em0 proto tcp from Y.Y.Y.Y/22 to 193.198.155.12 port 5800
rdr-to 10.161.53.99
ifconfig
# ifconfig
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> mtu 33196
priority: 0
groups: lo
inet6 ::1 prefixlen 128
inet6 fe80::1%lo0 prefixlen 64 scopeid 0x6
inet 127.0.0.1 netmask 0xff000000
em0:
flags=28b43<UP,BROADCAST,RUNNING,PROMISC,ALLMULTI,SIMPLEX,MULTICAST,NOINET6>
mtu 1500
lladdr 00:1b:21:30:99:ba
description: vanjski_interface
priority: 0
groups: egress
media: Ethernet autoselect (1000baseT full-duplex,rxpause)
status: active
inet 193.198.155.10 netmask 0xfffffff0 broadcast 193.198.155.15
em1:
flags=28b43<UP,BROADCAST,RUNNING,PROMISC,ALLMULTI,SIMPLEX,MULTICAST,NOINET6>
mtu 1500
lladdr 00:1b:21:30:99:bb
priority: 0
media: Ethernet autoselect (1000baseT full-duplex,rxpause)
status: active
em2: flags=28843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,NOINET6> mtu 1500
lladdr 00:11:43:d1:ea:8c
description: corssover
priority: 0
media: Ethernet autoselect (1000baseT full-duplex,rxpause,txpause)
status: active
inet 10.1.1.1 netmask 0xfffffffc broadcast 10.1.1.3
em3: flags=28843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,NOINET6> mtu 1500
lladdr 00:11:43:d1:ea:8d
priority: 0
media: Ethernet autoselect (1000baseT full-duplex,rxpause,txpause)
status: active
inet 10.244.244.1 netmask 0xfffffffc broadcast 10.244.244.3
enc0: flags=0<>
priority: 0
groups: enc
status: active
pflow0: flags=41<UP,RUNNING> mtu 1492
priority: 0
pflow: sender: 193.198.155.10 receiver: 161.53.253.252:9994
version: 5
groups: pflow
vlan600:
flags=28943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST,NOINET6> mtu 1500
lladdr 00:1b:21:30:99:bb
description: unutranji_interface
priority: 0
vlan: 600 parent interface: em1
groups: vlan
status: active
inet 10.161.53.2 netmask 0xffffff00 broadcast 10.161.53.255
pfsync0: flags=20041<UP,RUNNING,NOINET6> mtu 1500
priority: 0
pfsync: syncdev: em3 syncpeer: 10.244.244.2 maxupd: 128 defer: off
groups: carp pfsync
pflog0: flags=141<UP,RUNNING,PROMISC> mtu 33196
priority: 0
groups: pflog
carp0: flags=28843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,NOINET6> mtu 1500
lladdr 00:00:5e:00:01:01
priority: 0
carp: MASTER carpdev em0 vhid 1 advbase 1 advskew 10 carppeer
193.198.155.11
groups: carp
status: master
inet 193.198.155.12 netmask 0xffffff00 broadcast 193.198.155.255
carp1: flags=28843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,NOINET6> mtu 1500
lladdr 00:00:5e:00:01:02
priority: 0
carp: MASTER carpdev vlan600 vhid 2 advbase 1 advskew 10
carppeer 10.161.53.3
groups: carp
status: master
inet 10.161.53.1 netmask 0xff000000 broadcast 10.255.255.255
dmesg
# dmesg
OpenBSD 5.2 (GENERIC.MP) #1: Sun Jul 29 01:05:34 CEST 2012
r...@bcbnfw01.bcbn:/usr/src/sys/arch/i386/compile/GENERIC.MP
cpu0: Intel(R) Xeon(TM) CPU 3.40GHz ("GenuineIntel" 686-class) 3.40 GHz
cpu0:
FPU,V86,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,SBF,NXE,LONG,SSE3,MWAIT,DS-CPL,EST,CNXT-ID,CX16,xTPR
real mem = 2146762752 (2047MB)
avail mem = 2100813824 (2003MB)
mainbus0 at root
bios0 at mainbus0: AT/286+ BIOS, date 04/25/08, BIOS32 rev. 0 @ 0xffe90,
SMBIOS rev. 2.3 @ 0xf9920 (87 entries)
bios0: vendor Dell Computer Corporation version "A07" date 04/25/2008
bios0: Dell Computer Corporation PowerEdge 1850
acpi0 at bios0: rev 0
acpi0: sleep states S0 S4 S5
acpi0: tables DSDT FACP APIC SPCR HPET MCFG
acpi0: wakeup devices PCI0(S5) PALO(S5) PBLO(S5) VPR0(S5) PBHI(S5)
VPR1(S5) PICH(S5)
acpitimer0 at acpi0: 3579545 Hz, 24 bits
acpimadt0 at acpi0 addr 0xfee00000: PC-AT compat
cpu0 at mainbus0: apid 0 (boot processor)
cpu0: apic clock running at 199MHz
cpu1 at mainbus0: apid 6 (application processor)
cpu1: Intel(R) Xeon(TM) CPU 3.40GHz ("GenuineIntel" 686-class) 3.40 GHz
cpu1:
FPU,V86,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,SBF,NXE,LONG,SSE3,MWAIT,DS-CPL,EST,CNXT-ID,CX16,xTPR
ioapic0 at mainbus0: apid 7 pa 0xfec00000, version 20, 24 pins
ioapic0: misconfigured as apic 0, remapped to apid 7
ioapic1 at mainbus0: apid 8 pa 0xfec80000, version 20, 24 pins
ioapic1: misconfigured as apic 0, remapped to apid 8
ioapic2 at mainbus0: apid 9 pa 0xfec83000, version 20, 24 pins
ioapic2: misconfigured as apic 0, remapped to apid 9
acpihpet0 at acpi0: 14318179 Hz
acpimcfg0 at acpi0 addr 0xe0000000, bus 0-255
acpiprt0 at acpi0: bus 0 (PCI0)
acpiprt1 at acpi0: bus 1 (PALO)
acpiprt2 at acpi0: bus 3 (DOBA)
acpiprt3 at acpi0: bus 2 (DOBB)
acpiprt4 at acpi0: bus 4 (PBLO)
acpiprt5 at acpi0: bus 8 (VPR0)
acpiprt6 at acpi0: bus 5 (PBHI)
acpiprt7 at acpi0: bus 6 (PXB1)
acpiprt8 at acpi0: bus 7 (PXB2)
acpiprt9 at acpi0: bus 9 (PICH)
acpicpu0 at acpi0
acpicpu1 at acpi0
bios0: ROM list: 0xc0000/0xb000! 0xcb000/0x4000 0xec000/0x4000!
ipmi at mainbus0 not configured
cpu0: Enhanced SpeedStep disabled by BIOS
pci0 at mainbus0 bus 0: configuration mode 1 (bios)
pchb0 at pci0 dev 0 function 0 "Intel E7520 Host" rev 0x09
ppb0 at pci0 dev 2 function 0 "Intel E7520 PCIE" rev 0x09
pci1 at ppb0 bus 1
ppb1 at pci1 dev 0 function 0 "Intel 6700PXH PCIE-PCIX" rev 0x09
pci2 at ppb1 bus 2
em0 at pci2 dev 11 function 0 "Intel PRO/1000MT (82546GB)" rev 0x03:
apic 7 int 17, address 00:1b:21:30:99:ba
em1 at pci2 dev 11 function 1 "Intel PRO/1000MT (82546GB)" rev 0x03:
apic 7 int 18, address 00:1b:21:30:99:bb
ppb2 at pci1 dev 0 function 2 "Intel 6700PXH PCIE-PCIX" rev 0x09
pci3 at ppb2 bus 3
mpi0 at pci3 dev 5 function 0 "Symbios Logic 53c1030" rev 0x08: msi
scsibus0 at mpi0: 16 targets, initiator 7
sd0 at scsibus0 targ 0 lun 0: <SEAGATE, ST373307LC, DS09> SCSI3 0/direct
fixed serial.SEAGATE_ST373307LC_3HZ9WFYH
sd0: 70007MB, 512 bytes/sector, 143374650 sectors
safte0 at scsibus0 targ 6 lun 0: <PE/PV, 1x2 SCSI BP, 1.0> SCSI2
3/processor fixed
mpi0: target 0 Sync at 160MHz width 16bit offset 63 QAS 0 DT 1 IU 1
ppb3 at pci0 dev 4 function 0 "Intel E7520 PCIE" rev 0x09
pci4 at ppb3 bus 4
ppb4 at pci0 dev 5 function 0 "Intel E7520 PCIE" rev 0x09
pci5 at ppb4 bus 5
ppb5 at pci5 dev 0 function 0 "Intel 6700PXH PCIE-PCIX" rev 0x09
pci6 at ppb5 bus 6
em2 at pci6 dev 7 function 0 "Intel PRO/1000MT (82541GI)" rev 0x05: apic
9 int 0, address 00:11:43:d1:ea:8c
ppb6 at pci5 dev 0 function 2 "Intel 6700PXH PCIE-PCIX" rev 0x09
pci7 at ppb6 bus 7
em3 at pci7 dev 8 function 0 "Intel PRO/1000MT (82541GI)" rev 0x05: apic
9 int 1, address 00:11:43:d1:ea:8d
ppb7 at pci0 dev 6 function 0 "Intel E7520 PCIE" rev 0x09
pci8 at ppb7 bus 8
uhci0 at pci0 dev 29 function 0 "Intel 82801EB/ER USB" rev 0x02: apic 7
int 16
uhci1 at pci0 dev 29 function 1 "Intel 82801EB/ER USB" rev 0x02: apic 7
int 19
uhci2 at pci0 dev 29 function 2 "Intel 82801EB/ER USB" rev 0x02: apic 7
int 18
ehci0 at pci0 dev 29 function 7 "Intel 82801EB/ER USB2" rev 0x02: apic 7
int 23
usb0 at ehci0: USB revision 2.0
uhub0 at usb0 "Intel EHCI root hub" rev 2.00/1.00 addr 1
ppb8 at pci0 dev 30 function 0 "Intel 82801BA Hub-to-PCI" rev 0xc2
pci9 at ppb8 bus 9
vga1 at pci9 dev 13 function 0 "ATI Radeon VE" rev 0x00
wsdisplay0 at vga1 mux 1: console (80x25, vt100 emulation)
wsdisplay0: screen 1-5 added (80x25, vt100 emulation)
radeondrm0 at vga1: apic 7 int 18
drm0 at radeondrm0
ichpcib0 at pci0 dev 31 function 0 "Intel 82801EB/ER LPC" rev 0x02
pciide0 at pci0 dev 31 function 1 "Intel 82801EB/ER IDE" rev 0x02: DMA,
channel 0 configured to compatibility, channel 1 configured to compatibility
atapiscsi0 at pciide0 channel 0 drive 0
scsibus1 at atapiscsi0: 2 targets
cd0 at scsibus1 targ 0 lun 0: <TEAC, CD-224E, K.9A> ATAPI 5/cdrom removable
cd0(pciide0:0:0): using PIO mode 4, Ultra-DMA mode 2
pciide0: channel 1 disabled (no drives)
usb1 at uhci0: USB revision 1.0
uhub1 at usb1 "Intel UHCI root hub" rev 1.00/1.00 addr 1
usb2 at uhci1: USB revision 1.0
uhub2 at usb2 "Intel UHCI root hub" rev 1.00/1.00 addr 1
usb3 at uhci2: USB revision 1.0
uhub3 at usb3 "Intel UHCI root hub" rev 1.00/1.00 addr 1
isa0 at ichpcib0
isadma0 at isa0
com0 at isa0 port 0x3f8/8 irq 4: ns16550a, 16 byte fifo
pckbc0 at isa0 port 0x60/5
pckbd0 at pckbc0 (kbd slot)
pckbc0: using irq 1 for kbd slot
wskbd0 at pckbd0: console keyboard, using wsdisplay0
pcppi0 at isa0 port 0x61
spkr0 at pcppi0
npx0 at isa0 port 0xf0/16: reported by CPUID; using exception 16
fdc0 at isa0 port 0x3f0/6 irq 6 drq 2
fd0 at fdc0 drive 0: 1.44MB 80 cyl, 2 head, 18 sec
mtrr: Pentium Pro MTRR support
uhub4 at uhub0 port 3 "Dell product 0xa001" rev 2.00/0.00 addr 2
vscsi0 at root
scsibus2 at vscsi0: 256 targets
softraid0 at root
scsibus3 at softraid0: 256 targets
root on sd0a (1efcb2ee5700d4be.a) swap on sd0b dump on sd0b
carp1: state transition: BACKUP -> MASTER
carp0: state transition: BACKUP -> MASTER
carp0: state transition: MASTER -> BACKUP
carp1: state transition: MASTER -> BACKUP
carp0: state transition: BACKUP -> MASTER
carp0: state transition: MASTER -> BACKUP
carp1: state transition: BACKUP -> MASTER
carp1: state transition: MASTER -> BACKUP
carp0: state transition: BACKUP -> MASTER
carp0: state transition: MASTER -> BACKUP
carp1: state transition: BACKUP -> MASTER
carp1: state transition: MASTER -> BACKUP
carp0: state transition: BACKUP -> MASTER
carp1: state transition: BACKUP -> MASTER
pf: state key linking mismatch! dir=OUT, if=vlan600, stored af=2, a0:
69.131.171.10:51413, a1: 10.161.53.99:32459, proto=17, found af=2, a0:
10.161.53.99:32459, a1: 187.170.255.239:51413, proto=17
pf: state key linking mismatch! dir=OUT, if=vlan600, stored af=2, a0:
77.96.154.112:51413, a1: 10.161.53.99:32459, proto=17, found af=2, a0:
10.161.53.99:32459, a1: 187.170.255.239:51413, proto=17
pf: state key linking mismatch! dir=OUT, if=vlan600, stored af=2, a0:
59.167.119.136:51413, a1: 10.161.53.99:32459, proto=17, found af=2, a0:
10.161.53.99:32459, a1: 187.170.255.239:51413, proto=17
pf: state key linking mismatch! dir=OUT, if=vlan600, stored af=2, a0:
112.201.225.141:51413, a1: 10.161.53.99:32459, proto=17, found af=2, a0:
10.161.53.99:32459, a1: 187.170.255.239:51413, proto=17
pf: state key linking mismatch! dir=OUT, if=vlan600, stored af=2, a0:
115.69.42.179:51413, a1: 10.161.53.99:32459, proto=17, found af=2, a0:
10.161.53.99:32459, a1: 187.170.255.239:51413, proto=17
pf: state key linking mismatch! dir=OUT, if=vlan600, stored af=2, a0:
75.0.203.247:51413, a1: 10.161.53.99:32459, proto=17, found af=2, a0:
10.161.53.99:32459, a1: 187.170.255.239:51413, proto=17
pf: state key linking mismatch! dir=OUT, if=vlan600, stored af=2, a0:
77.96.154.112:51413, a1: 10.161.53.99:32459, proto=17, found af=2, a0:
10.161.53.99:32459, a1: 187.170.255.239:51413, proto=17
pf: state key linking mismatch! dir=OUT, if=vlan600, stored af=2, a0:
220.233.13.131:51413, a1: 10.161.53.99:32459, proto=17, found af=2, a0:
10.161.53.99:32459, a1: 187.170.255.239:51413, proto=17
pf: state key linking mismatch! dir=OUT, if=vlan600, stored af=2, a0:
86.207.13.162:51413, a1: 10.161.53.99:32459, proto=17, found af=2, a0:
10.161.53.99:32459, a1: 187.170.255.239:51413, proto=17
pf: state key linking mismatch! dir=OUT, if=vlan600, stored af=2, a0:
75.0.203.247:51413, a1: 10.161.53.99:32459, proto=17, found af=2, a0:
10.161.53.99:32459, a1: 187.170.255.239:51413, proto=17
pf: state key linking mismatch! dir=OUT, if=vlan600, stored af=2, a0:
121.131.103.229:51413, a1: 10.161.53.99:32459, proto=17, found af=2, a0:
10.161.53.99:32459, a1: 187.170.255.239:51413, proto=17
pf: state key linking mismatch! dir=OUT, if=vlan600, stored af=2, a0:
74.140.130.82:51413, a1: 10.161.53.99:32459, proto=17, found af=2, a0:
10.161.53.99:32459, a1: 187.170.255.239:51413, proto=17
pf: state key linking mismatch! dir=OUT, if=vlan600, stored af=2, a0:
14.203.161.197:51413, a1: 10.161.53.99:32459, proto=17, found af=2, a0:
10.161.53.99:32459, a1: 187.170.255.239:51413, proto=17
pf: state key linking mismatch! dir=OUT, if=vlan600, stored af=2, a0:
41.249.84.245:51413, a1: 10.161.53.99:32459, proto=17, found af=2, a0:
10.161.53.99:32459, a1: 187.170.255.239:51413, proto=17
pf: state key linking mismatch! dir=OUT, if=vlan600, stored af=2, a0:
58.11.11.11:26085, a1: 10.161.53.99:32459, proto=17, found af=2, a0:
10.161.53.99:32459, a1: 187.170.255.239:26085, proto=17
pf: state key linking mismatch! dir=OUT, if=vlan600, stored af=2, a0:
220.233.13.131:51413, a1: 10.161.53.99:32459, proto=17, found af=2, a0:
10.161.53.99:32459, a1: 187.170.255.239:51413, proto=17
pf: state key linking mismatch! dir=OUT, if=vlan600, stored af=2, a0:
178.2.49.159:51413, a1: 10.161.53.99:32459, proto=17, found af=2, a0:
10.161.53.99:32459, a1: 187.170.255.239:51413, proto=17
