Hi, I have two hosts bounded by a wireguard: superpod(7.4/arm64) and stern (snapshot of today/riscv64).
I have utilized a program that I rewrote yesterday and this morning that I call sipdiv, because it reads SIP signalling off a divert socket. The code is publically available since today: https://github.com/pbug44/misc/tree/main/sipdiv I'm running into problems with the 7.4 host (superpod). It doesn't read off the divert socket for some reason and I want to show the pf rules to start for this. Perhaps you can find the problem immediately. superpod# ps auxww|grep sipdiv root 14841 0.0 0.0 248 516 ?? Ip 10:38AM 0:00.00 sipdiv -c root 76341 0.0 0.0 204 384 p4 R+/1 7:36PM 0:00.00 grep sipdiv superpod# fstat -p 14841 USER CMD PID FD MOUNT INUM MODE R/W SZ|DV root sipdiv 14841 text /usr/local 77788 -r-xr-xr-x r 17944 root sipdiv 14841 wd / 2 drwxr-xr-x r 512 root sipdiv 14841 tr /home 942651 -rw------- rw 64 root sipdiv 14841 0 / 52857 crw-rw-rw- rw null root sipdiv 14841 1 / 52857 crw-rw-rw- rw null root sipdiv 14841 2 / 52857 crw-rw-rw- rw null root sipdiv 14841 3* internet raw divert 0xffffff800b0d1818 So you see descriptor "tr" which has a ktrace.out file of 64 bytes and it's not growing. And there is no compacting being done by this proxy, it boggles me. Now the pf rules are very simple in their structure. I'm not going to list the anchors because it's a quick rule at the beginning that should match. superpod# pfctl -srules block return log all pass all flags S/SA block return in on ! lo0 proto tcp from any to any port 6000:6010 block return out log proto tcp all user = 55 block return out log proto udp all user = 55 pass in log quick on wg1 inet proto udp from 192.168.178.1 to any port = 5060 sc rub (reassemble tcp) divert-packet port 22222 anchor "esp" all anchor "nat6" all ... ... and so on. Since this is a quick rule I'd think it would be caught the very first time, but it doesn't. It gets skipped. I have cleared the states with this logic: superpod# history 1|grep awk 381 pfctl -ss -vv|grep -A2 192\.168\.178\.1 | grep id | awk '{print $2}' 382 pfctl -ss -vv|grep -A2 192\.168\.178\.1 | grep id | awk '{print $2}' | while read i ; do pfctl -k id -k $i; done I'm at the end of wits here. Any help? dmesg follows: The other host (stern) has a similar rule and it works no complaints. Best Regards, -peter OpenBSD 7.4 (GENERIC.MP) #2: Fri Dec 8 15:42:08 MST 2023 [email protected]:/usr/src/sys/arch/arm64/compile/GENERIC.MP real mem = 4185800704 (3991MB) avail mem = 3976454144 (3792MB) random: good seed from bootblocks mainbus0 at root: ACPI psci0 at mainbus0: PSCI 1.0, SMCCC 1.1 efi0 at mainbus0: UEFI 2.7 efi0: EDK II rev 0x10000 smbios0 at efi0: SMBIOS 3.0.0 smbios0: vendor Hetzner version "20171111" date 11/11/2017 smbios0: Hetzner vServer cpu0 at mainbus0 mpidr 0: ARM Neoverse N1 r3p1 cpu0: 64KB 64b/line 4-way L1 PIPT I-cache, 64KB 64b/line 4-way L1 D-cache cpu0: 1024KB 64b/line 8-way L2 cache cpu0: DP,RDM,Atomic,CRC32,SHA2,SHA1,AES+PMULL,LRCPC,DPB,ASID16,PAN+ATS1E1,LO,HPDS,VH,HAFDBS,CSV3,CSV2,SBSS+MSR cpu1 at mainbus0 mpidr 1: ARM Neoverse N1 r3p1 cpu1: 64KB 64b/line 4-way L1 PIPT I-cache, 64KB 64b/line 4-way L1 D-cache cpu1: 1024KB 64b/line 8-way L2 cache cpu1: DP,RDM,Atomic,CRC32,SHA2,SHA1,AES+PMULL,LRCPC,DPB,ASID16,PAN+ATS1E1,LO,HPDS,VH,HAFDBS,CSV3,CSV2,SBSS+MSR apm0 at mainbus0 agintc0 at mainbus0 shift 4:4 nirq 288 nredist 2 ipi: 0, 1, 2: "interrupt-controller" agintcmsi0 at agintc0 agtimer0 at mainbus0: 25000 kHz acpi0 at mainbus0: ACPI 5.1 acpi0: sleep states acpi0: tables DSDT FACP APIC GTDT MCFG SPCR DBG2 IORT BGRT acpi0: wakeup devices acpimcfg0 at acpi0 acpimcfg0: addr 0x4010000000, bus 0-255 acpiiort0 at acpi0 "ACPI0007" at acpi0 not configured "ACPI0007" at acpi0 not configured pluart0 at acpi0 COM0 addr 0x9000000/0x1000 irq 33 pluart0: console "LNRO0015" at acpi0 not configured "LNRO0015" at acpi0 not configured "QEMU0002" at acpi0 not configured "LNRO0005" at acpi0 not configured "LNRO0005" at acpi0 not configured "LNRO0005" at acpi0 not configured "LNRO0005" at acpi0 not configured "LNRO0005" at acpi0 not configured "LNRO0005" at acpi0 not configured "LNRO0005" at acpi0 not configured "LNRO0005" at acpi0 not configured "LNRO0005" at acpi0 not configured "LNRO0005" at acpi0 not configured "LNRO0005" at acpi0 not configured "LNRO0005" at acpi0 not configured "LNRO0005" at acpi0 not configured "LNRO0005" at acpi0 not configured "LNRO0005" at acpi0 not configured "LNRO0005" at acpi0 not configured "LNRO0005" at acpi0 not configured "LNRO0005" at acpi0 not configured "LNRO0005" at acpi0 not configured "LNRO0005" at acpi0 not configured "LNRO0005" at acpi0 not configured "LNRO0005" at acpi0 not configured "LNRO0005" at acpi0 not configured "LNRO0005" at acpi0 not configured "LNRO0005" at acpi0 not configured "LNRO0005" at acpi0 not configured "LNRO0005" at acpi0 not configured "LNRO0005" at acpi0 not configured "LNRO0005" at acpi0 not configured "LNRO0005" at acpi0 not configured "LNRO0005" at acpi0 not configured "LNRO0005" at acpi0 not configured acpipci0 at acpi0 PCI0 pci0 at acpipci0 0:4:0: io address conflict 0x8200/0x8 "Red Hat Host" rev 0x00 at pci0 dev 0 function 0 not configured virtio0 at pci0 dev 1 function 0 "Qumranet Virtio 1.x GPU" rev 0x01 viogpu0 at virtio0: 1024x768, 32bpp wsdisplay0 at viogpu0 mux 1: console (std, vt100 emulation) wsdisplay0: screen 1-5 added (std, vt100 emulation) virtio0: msix per-VQ ppb0 at pci0 dev 2 function 0 vendor "Red Hat", unknown product 0x000c rev 0x00: irq 37 pci1 at ppb0 bus 1 1:0:0: rom address conflict 0xfff80000/0x80000 virtio1 at pci1 dev 0 function 0 "Qumranet Virtio 1.x Network" rev 0x01 vio0 at virtio1: address 96:00:02:1f:61:38 virtio1: msix shared ppb1 at pci0 dev 2 function 1 vendor "Red Hat", unknown product 0x000c rev 0x00: irq 37 pci2 at ppb1 bus 2 xhci0 at pci2 dev 0 function 0 vendor "Red Hat", unknown product 0x000d rev 0x01: msix, xHCI 0.0 usb0 at xhci0: USB revision 3.0 uhub0 at usb0 configuration 1 interface 0 "Red Hat xHCI root hub" rev 3.00/1.00 addr 1 ppb2 at pci0 dev 2 function 2 vendor "Red Hat", unknown product 0x000c rev 0x00: irq 37 pci3 at ppb2 bus 3 virtio2 at pci3 dev 0 function 0 "Qumranet Virtio 1.x Console" rev 0x01 virtio2: no matching child driver; not configured ppb3 at pci0 dev 2 function 3 vendor "Red Hat", unknown product 0x000c rev 0x00: irq 37 pci4 at ppb3 bus 4 virtio3 at pci4 dev 0 function 0 vendor "Qumranet", unknown product 0x1045 rev 0x01 viomb0 at virtio3 virtio3: irq 37 ppb4 at pci0 dev 2 function 4 vendor "Red Hat", unknown product 0x000c rev 0x00: irq 37 pci5 at ppb4 bus 5 virtio4 at pci5 dev 0 function 0 "Qumranet Virtio 1.x RNG" rev 0x01 viornd0 at virtio4 virtio4: irq 37 ppb5 at pci0 dev 2 function 5 vendor "Red Hat", unknown product 0x000c rev 0x00: irq 37 pci6 at ppb5 bus 6 virtio5 at pci6 dev 0 function 0 "Qumranet Virtio 1.x SCSI" rev 0x01 vioscsi0 at virtio5: qsize 128 scsibus0 at vioscsi0: 255 targets cd0 at scsibus0 targ 0 lun 0: <QEMU, QEMU CD-ROM, 2.5+> removable sd0 at scsibus0 targ 0 lun 1: <QEMU, QEMU HARDDISK, 2.5+> sd0: 39064MB, 512 bytes/sector, 80003072 sectors, thin virtio5: msix per-VQ ppb6 at pci0 dev 2 function 6 vendor "Red Hat", unknown product 0x000c rev 0x00: irq 37 pci7 at ppb6 bus 7 7:0:0: rom address conflict 0xfff80000/0x80000 virtio6 at pci7 dev 0 function 0 "Qumranet Virtio 1.x Network" rev 0x01 vio1 at virtio6: address 86:00:00:52:9a:54 virtio6: msix shared ppb7 at pci0 dev 2 function 7 vendor "Red Hat", unknown product 0x000c rev 0x00: irq 37 pci8 at ppb7 bus 8 ppb8 at pci0 dev 3 function 0 vendor "Red Hat", unknown product 0x000c rev 0x00: irq 38 pci9 at ppb8 bus 9 "Red Hat Qemu Serial" rev 0x01 at pci0 dev 4 function 0 not configured acpige0 at acpi0 irq 41 acpibtn0 at acpi0: PWRB uhidev0 at uhub0 port 5 configuration 1 interface 0 "QEMU QEMU USB Tablet" rev 2.00/0.00 addr 2 uhidev0: iclass 3/0 ums0 at uhidev0: 3 buttons, Z dir wsmouse0 at ums0 mux 0 uhidev1 at uhub0 port 6 configuration 1 interface 0 "QEMU QEMU USB Keyboard" rev 2.00/0.00 addr 3 uhidev1: iclass 3/1 ukbd0 at uhidev1: 8 variable keys, 6 key codes wskbd0 at ukbd0 mux 1 wskbd0: connecting to wsdisplay0 vscsi0 at root scsibus1 at vscsi0: 256 targets softraid0 at root scsibus2 at softraid0: 256 targets root on sd0a (3f9b49880fb7b60e.a) swap on sd0b dump on sd0b
