I am helping some folks with some OpenBSD stuff, including at some point being more proactive about updating, but they currently have an OpenBSD 5.9 machine that has been routing traffic happily for a while. Unfortunately, they can't currently update because the proxy arp configuration that they are using on 5.9 doesn't work on anything newer.
Here is the excellent description from someone there, and I was able to reproduce the issue on my laptop, so I don't believe a dmesg will be particularly helpful, but here's one for my laptop that's not terribly old. https://dmesgd.nycbug.org/index.cgi?do=view&id=5279 As I haven't ever actually used proxy arp on OpenBSD, so don't actually know whether this was a glitch that was fixed or added. I do see that 60.html says the routing table is now based on ART, so that seems a possible culprit. --- We've found that a proxy-arp configuration we've relied on (which works perfectly on OpenBSD 5.9 and earlier) no longer works on OpenBSD 6.0 thru 6.7. On OpenBSD 5.9, we can add an "arp -s" entry for a public IP on our public-facing gateway router's external interface, followed by a host-route for that same public IP (pointing to an internal subnet/interface). This allows us to route public IPs to clients through our internal private subnets, so a client can have a public IP assigned on their router, despite actually being several hops deep inside our private network. With this proxy-arp configuration, from both the client router's perspective and from an external perspective, it looks as if the client router is in the public IP subnet. Unfortunately this method doesn't work in OpenBSD 6.0 and later. Here's how it looks when you configure this on OpenBSD 5.9 (working as expected with no errors): root@openbsd-59-pub-gw-rtr~ # arp -s 1.2.3.4 [PUB-GW_em0_EXT-IF_MAC-ADDR] pub root@openbsd-59-pub-gw-rtr~ # route add 1.2.3.4/32 10.0.2.47 add host 1.2.3.4/32: gateway 10.0.2.47 root@openbsd-59-pub-gw-rtr~ # netstat -rn -finet|grep 1.2.3.4 1.2.3.4 10.0.2.47 UGHS 0 0 - 8 em1 1.2.3.4/32 [PUB-GW_em0_EXT-IF_MAC-ADDR] ULS2 0 0 - 8 em0 ##### ^ Success: Routing table shows both the host-route and the "arp -s" entry. That same setup fails on OpenBSD 6.0 and later (tested on OpenBSD 6.0, 6.2, 6.6, and 6.7). If you add an arp -s entry before adding a host route for the same IP, the "route add" fails with "File exists" error: root@openbsd-67-pub-gw-rtr~ # arp -s 1.2.3.4 [PUB-GW_em0_EXT-IF_MAC-ADDR] pub root@openbsd-67-pub-gw-rtr~ # route add 1.2.3.4/32 10.0.2.47 !-> add host 1.2.3.4/32: gateway 10.0.2.47: File exists root@openbsd-67-pub-gw-rtr~ # netstat -rn -finet|grep 1.2.3.4 1.2.3.4/32 [PUB-GW_em0_EXT-IF_MAC-ADDR] ULS2 0 0 - 8 em0 ##### ^ Failure: Routing table only shows the "arp -s" entry (failed to add host-route) If you add the route before the arp -s entry, "arp -s" fails with error "set: proxy entry exists for non 802 device": root@openbsd-67-pub-gw-rtr~ # route add 1.2.3.4/32 10.0.2.47 add host 1.2.3.4/32: gateway 10.0.2.47 root@openbsd-67-pub-gw-rtr~ # arp -s 1.2.3.4 [PUB-GW_em0_EXT-IF_MAC-ADDR] pub !-> set: proxy entry exists for non 802 device root@openbsd-67-pub-gw-rtr~ # netstat -rn -finet|grep 1.2.3.4 1.2.3.4 10.0.2.47 UGHS 0 0 - 8 em1 ##### ^ Failure: Routing table only shows the host-route (failed to add arp -s entry) For most of our clients who request a dedicated public IP, we're able to assign their router a private IP, which we then bi-NAT to a dedicated public IP on our gateway router. Unfortunately the bi-NAT method doesn't work for certain routers' built-in VPNs, so we're looking for a way to re-create the functionality we had with proxy-arp on OpenBSD v5.9. I imagine there's some workaround or alternate method to achieve the same thing on OpenBSD 6.x, but I haven't found it yet.
