Not clear from your message so I was wondering if you have all the following on the same switch
ISP interface External interface of your firewall Internal interface of your firewall Interfaces of your other systems I noticed behaviour similar to what you described when I did something like the above. The arp rewrite attempts stopped when I separated the Internet connection and the external interface of the firewall on one switch and all the internal systems on another switch. Vijay Sent from my iPhone > On Jan 16, 2016, at 12:40, Doug Moss <[email protected]> wrote: > > (my apologies for last message - unfamiliar with Yahoo and forcing plain text email) > > Why is a manually entered permanent arp entry being overwritten? > > > At my home, I have an ISP from which I have 5 static IPv4 addresses. > I use these for my home network, a home email server, jabber server for family/friends, > website related to my academic work, etc, with different domains. > > > The ISP service comes into my home via an ethernet cable which I connect to a switch > (Cisco gigabit) > > Connected to the switch are: > (A) router to my home network (behind which are desktops, a wireless access point, kids laptops, etc) > a low-power, dual NIC OpenBSD amd64 running NAT and unbound (caching) > with IP address 70.20.25.26 > (B) the academic website > a low-power, OpenBSD 5.7 amd64 > with IP address 70.20.25.30 > (plus other servers) > > The ISP gateway/router is IP address 70.20.25.1 > > On the academic website, I noticed that the arp table > showed 70.20.25.26 with the MAC of the ISP gateway > > I thought - why should my private traffic from my personal webserver be routed > through the ISP gateway - why not go directly to my home network on the same switch? > > So on my webserver, I did this: > # sudo arp -s 70.20.25.26 00:25:90:0A:69:B6 permanent > > Then I checked: > # arp -an > Host Ethernet Address Netif Expire Flags > 70.20.25.1 fa:c0:01:75:98:cd em0 19m59s > 70.20.25.26 00:25:90:0a:69:b6 em0 permanent > 70.20.25.30 00:25:90:ea:52:9c em0 permanent l > > The next day, I found this is the logs: > Jan 12 08:17:54 www /bsd: arp info overwritten for 70.20.25.26 by 00:25:90:0a:69:b6 on em0 > Jan 12 08:17:54 www /bsd: arp info overwritten for 70.20.25.26 by fa:c0:01:75:98:cd on em0 > Jan 12 08:37:54 www /bsd: arp info overwritten for 70.20.25.26 by 00:25:90:0a:69:b6 on em0 > Jan 12 08:37:54 www /bsd: arp info overwritten for 70.20.25.26 by fa:c0:01:75:98:cd on em0 > Jan 12 08:57:54 www /bsd: arp info overwritten for 70.20.25.26 by 00:25:90:0a:69:b6 on em0 > Jan 12 08:57:54 www /bsd: arp info overwritten for 70.20.25.26 by fa:c0:01:75:98:cd on em0 > (repeated a couple hundred times) > > $ arp -an > Host Ethernet Address Netif Expire Flags > 70.20.25.1 fa:c0:01:75:98:cd em0 19m54s > 70.20.25.26 fa:c0:01:75:98:cd em0 17m15s > 70.20.25.30 00:25:90:ea:52:9c em0 permanent l > > and > $ traceroute 70.20.25.26 > traceroute to 70.20.25.26 (70.20.25.26), 64 hops max, 40 byte packets > 1 lo0-100.BSTNMA-VFTTP-308.verizon-gni.net (70.20.25.1) 2.841 ms 0.594 ms 3.724 ms > 2 static-70-20-25-26.bstnma.fios.verizon.net (70.20.25.26) 3.544 ms 1.255 ms 3.593 ms > > Am I understanding this correctly? > Is the ISP gateway continuing to try to re-direct the arp table on my home router > to route traffic out to its gateway before coming back to my home network, instead of > directly from my router to the other server connected to ports on the same switch? > > > Have I done something wrong in my configuration? > > Is this (a) expected (b) strange but innocent (c) nefarious, or (d) something else?

