> for i in /proc/sys/net/ipv4/conf/*/arp_announce; do echo 2 > $i;done
+1 setting arp_announce in Linux is essential if being used as a router with more than one subnet. I would also recommend setting arp_ignore. For Linux-based routers, I've found the following settings to be optimal: echo 1 > /proc/sys/net/ipv4/conf/all/arp_announce echo 2 > /proc/sys/net/ipv4/conf/all/arp_ignore On a side note, this underscores what a lot of people on-list are saying: If you don't understand the internals of a Linux system, for example, "rolling your own" will bite you. It's also pretty rare to find a network engineer who is also a Linux system-level developer, so finding and maintaining that talent can often be a challenge. Many make a leap and go on to assert that because of this software-based systems can never be viable, which I disagree with. After all, the latest OS offerings from Cisco run a Linux kernel. Nearly all the Ciena DWDM and ME gear I run is built on Linux. These companies aren't doing quite as much with hardware acceleration as they would lead you to believe. I think Intel DPDK will be a disruptive technology for networking. At the end of the day, I'm pretty anxious to see the days of over-priced routers driving up network service costs go away. On Sun, Dec 29, 2013 at 4:10 AM, Laurent GUERBY <laur...@guerby.net> wrote: > > On Sun, 2013-12-29 at 03:31 +0100, Baldur Norddahl wrote: > > (...) > > The users each have a unique VLAN (Q-in-Q). The question is, what do I put > > on those VLANs, if I do not want to put a full IPv4 subnet on each? > > > > My own answer to that is to have the users share a larger subnet, for > > example I could have a full class C sized subnet shared between 253 > > users/VLANs. > > > > To allow these users to communicate with each other, and so they can > > communicate with the default gateway IP, I will need proxy arp. And in a > > non-OpenFlow solution, also the associated security functions such as > > DHCP-snooping to prevent hijacking of IP addresses. > > > > Which devices can solve this task? > > Hi Baldur, > > Assuming you manage 1.1.1.0/24 and 2001:db8:0::/48 and > have a Linux box on both ends you can get rid of > IPv4 and v6 interco subnets and arp proxy the following way: > > 1/ on the gateway > ip addr add 1.1.1.0/32 dev lo > > for all client VLAN "NN" on eth0 : > ip -6 addr add fe80::1/64 dev eth0.NN > ip -6 route add 2001:db8:0:NN00::/56 via fe80::1:NN dev eth0.NN > > 2/ on user CPE number "NN" CPE WAN interface being eth0 : > ip addr add 1.1.1.NN/32 dev eth0 > ip route add 1.1.1.0/32 dev eth0 > ip route add default via 1.1.1.0 > ip -6 addr add fe80::1:NN/64 dev eth0 > ip -6 route add default via fe80::1 dev eth0 > # ip -6 addr add 2001:db8:0:NN00::1/56 dev eth0 # optional > > Note: NN in hex for IPv6 > > The trick in IPv4 is that linux by default will answer to ARP requests > for "1.1.1.0" on all interfaces even if the adress is on the loopback. > And in IPv6 use static link local on both ends. You can replace > "1.1.1.0" by any IPv4, but since ".0" are rarely assigned to end users > it doesn't waste anything and keep traceroute with public IPv4. > > The nice thing of this setup is that it "virtualizes" the routing from > the client point of view: you can split/balance your clients on multiple > physical gateways and not change a line to the client configuration > while it's being moved, you just have to configure your IGP between > gateways to properly distribute internal routes. > > We (AS197422 / tetaneutral.net) use this for virtual machines too (with > "tapNN" interfaces from KVM instead of "eth0.NN"): it allows us to move > virtual machines around physical machines without user reconfiguration, > not waste any IPv4 and avoid all issues with shared L2 (rogue RA/ARP > spoofing/whatever) since there's no shared L2 anymore between user VM. > It also allows us to not pre split our IPv4 space in a fixed scheme, > we manage only /32 so no waste at all. > > Of course you still have work to do on PPS tuning. > > Sincerely, > > Laurent GUERBY > AS197422 http://tetaneutral.net peering http://as197422.net > > PS: minimum settings on a Linux router > echo 1 > /proc/sys/net/ipv4/ip_forward > for i in /proc/sys/net/ipv6/conf/*; do for j in autoconf accept_ra; do echo 0 > $i/$j; done;done > echo 1 > /proc/sys/net/ipv6/conf/all/forwarding > echo 65536 > /proc/sys/net/ipv6/route/max_size > for i in /proc/sys/net/ipv4/conf/*/arp_announce; do echo 2 > $i;done > > PPS: we also like to give /56 to our users in IPv6, it makes a nice /24 > IPv4 <=> /48 IPv6 correspondance (256 users). > > > -- Ray Patrick Soucy Network Engineer University of Maine System T: 207-561-3526 F: 207-561-3531 MaineREN, Maine's Research and Education Network www.maineren.net