Re: Home stretch on new network - if_bridge looking better
On 02/25/11 01:33, Matthew Dillon wrote: Most informative cheers!
Re: Home stretch on new network - if_bridge looking better
Great news! Is there any chance to support more features in the bridge code? RSTP, span port , filtering based on mac address …. Godot 2011/2/24 Matthew Dillon dil...@apollo.backplane.com: I'm in the home stretch of finishing up the new DragonFly network! It's been pretty unstable the last week or so as I struggled first with the (now failed) attempt at using an att static block with U-Verse and then gave up on that and started working on running a VPN over a dynamic-IP based att U-Verse + comcast internet. I wanted bonding with failover. Most of my struggles with U-Verse were in dealing with the stateful firewall att has that cannot be turned off, even for the static IP block. It had serious issues dealing with many concurrent connections and would drop connections randomly (it would send a RST!). The VPN bypasses the whole mess. The last few days have been spent essentially rewriting half of if_bridge so it would work properly, and testing it while I am still tripple-homed (DSL, U-Verse, and ComCast). Well, it caused a lot of havoc on my network while I was beating it into shape and that's putting it mildly! But I think I now have if_bridge and openvpn and my ipfw and PF rules smacked into shape. I am going to implement line bonding in if_bridge today (on top of the spanning tree and failover which now works) and track down one or two remaining ARP issues and then I'll call it done. The basic setup is as shown below: http://apollo-vc.backplane.com/DFlyMisc/bridge1.txt http://apollo-vc.backplane.com/DFlyMisc/bridge2.txt + There are PF rules and ALTQs on each TAP interface to manage its outgoing bandwidth and keep network latencies down (on both sides of the VC). + IPFW forwarding (fwd) rules to manage multiple default routes based on the source IP. The spanning tree appears to be working properly with the 2x2 and the 3x3 'real' configuration I'm testing it with. Once I get line bonding working I expect my downlink to achieve ~30MBits+ and my uplink will be 4.8MBits. I'm seriously considering keeping both U-Verse and ComCast and just paring the service levels down a little (top tier isn't needed). The poor old DSL with its 600KBit uplink is going to hit the trash heap. It might have been slow, but that ISP served my old /26 static block fairly well for many years. -Matt Matthew Dillon dil...@backplane.com
Re: Home stretch on new network - if_bridge looking better
On 02/24/11 11:50, Matthew Dillon wrote: http://apollo-vc.backplane.com/DFlyMisc/bridge1.txt http://apollo-vc.backplane.com/DFlyMisc/bridge2.txt So - reading over this - is it correct that the setup is roughly like: - assign a local interface (lan0) to a network - add this network to the bridge - create openvpn 'bridged' mode tunnels - add these to the bridge so the L2 bridge / STP will 'map' according to the state of the ethernet bridging, which in turn relates to the openvpn tunnel state? Without diverging any security sensitive whatnot, Is the VPN tunnel created to the ISP or to say, the colo space? (I'd assume the latter) Have been working on my own openvpn (routing mode) fun to a pair of VPS's as well over the last few days so this is of interest :D also - I note in the bridge2.txt file you 'cd /usr/pkg/etc/openvpn' before running - is this so openvpn can find the config files? if so - to note, you can add a 'cd /path/to/configdir' within the config files.. also - assuming you have statics on both end of the tunnels - why did you choose openvpn ethernet bridging over say IP layer + ipsec? (or even openvpn 'routing' mode) with something like OSPF or similar and - do you have hw crypto cards on either endpoint? (my soekris 486 gets a little bogged down by the crypto, which is why I ask) ok enough questions ;) its definitely fun trying to convert consumer internet into a 'real connection' :D - Chris (from a gigabit LAN piggybacked on a sometimes 56k wifi link)
Re: Home stretch on new network - if_bridge looking better
: :Great news! : :Is there any chance to support more features in the bridge code? RSTP, :span port , filtering based on mac address . : :Godot RSTP would be doable as a GSOC project, I think it would be very easy to implement. Perhaps almost too easy but if someone were to do it I would require significant testing to make sure the protocol operates properly. I have to move onto other things myself. (RSTP is STP with a faster recovery time in case of link failure. STP takes about 30 seconds to transition to a new topology while RSTP takes about 10 seconds). The span port is theoretically operational but it has NOT been tested in any way, so something might blow if you try to use it. This would be more of a bug-fix type of thing, not worthy of a GSOC project. MAC based filtering would be worthy of a GSOC project. We don't have it now but IPFW at least already has hooks for ethernet-level firewalling. Doing it w/PF would be a lot more difficult as PF is designed as a routed packet filter (routing vs switching). -Matt
Re: Home stretch on new network - if_bridge looking better
:On 02/24/11 11:50, Matthew Dillon wrote: : : http://apollo-vc.backplane.com/DFlyMisc/bridge1.txt : http://apollo-vc.backplane.com/DFlyMisc/bridge2.txt : :So - reading over this - is it correct that the setup is roughly like: : :- assign a local interface (lan0) to a network :- add this network to the bridge :- create openvpn 'bridged' mode tunnels :- add these to the bridge In the case of my current setup, lan0, uverse0, comcast0, and aerio0 are all physical ethernet ports. lan0 is the LAN, and the other three connect to the three different WAN services I have. Only lan0 and the tunnels (tap0, tap1, tap2) are associated with the bridge. The other physical ethernet ports (uverse0, comcast0, and aerio0) each have a different IP and a different default route and I use IPFW to associate packets sourced from the IP to the default route for each port. Currently uverse0 and comcast0 are both dynamic while aerio0 is a static IP (the old DragonFly net /26). The OpenVPN tunnels are built using these IPs and back the tap devices. The tap devices are then associated with the bridge and the main LAN. The tap devices themselves, and the bridge, have *NO* IPs associated with them. All the local IP spaces are on lan0, including some local NATted spaces (10.x.x.x). The bridge code and the ARP code deal with the inconsistencies and provide a consistent ARP for the bridge members. Also, not shown here, is that I have a massive set of PF rules and ALTQs on each of the TAP interfaces (tap0, tap1, and tap2). In particular I'm running the ALTQs on the TAP devices with fair-share scheduling and tuned to the bandwidth of each WAN so ping times will be low no matter what topology the bridge is using. (Of course I can't do fair-share scheduling on the WAN ports, uverse0, comcast0, and aerio0, because the only thing running over them is the OpenVPN UDP packets and it can't dig into them to see what they represent). :so the L2 bridge / STP will 'map' according to the state of :the ethernet bridging, which in turn relates to the openvpn tunnel :state? Exactly. The if_bridge module does its own 'pinging' using STP config packets so it can detect when a link goes down. OpenVPN itself also has a ping/restart feature. I use both. OpenVPNs internal keepalive auto-restarts openvpn on failure, and the if_bridge's pinging is used to detect actual good flow across the link and controls the failover. :Without diverging any security sensitive whatnot, :Is the VPN tunnel created to the ISP or to say, the colo space? :(I'd assume the latter) Yes, a colo space that the DragonFly project controls, provided by Peter Avalos. OpenVPN itself is running encrypted UDP packets. Very easy to set up. The colo has around 10 MBytes/sec of bandwidth which is plenty for our project. :Have been working on my own openvpn (routing mode) fun to a pair :of VPS's as well over the last few days so this is of interest :D : :also - I note in the bridge2.txt file you 'cd /usr/pkg/etc/openvpn' :before running - is this so openvpn can find the config files? Yes, that's actually a bit broken. I've since changed it to put a 'cd' directive in the config file itself and then just run openvpn with the full path to the config file. Openvpn has problems restarting itself if you don't do this (it winds up getting confused and not being able to find the key files if it restarts). :if so - to note, you can add a 'cd /path/to/configdir' within the :config files.. Yah, found that :-) :also - assuming you have statics on both end of the tunnels - :why did you choose openvpn ethernet bridging over say IP layer + ipsec? :(or even openvpn 'routing' mode) with something like OSPF or similar : :and - do you have hw crypto cards on either endpoint? I originally attempted to route a subnet but the problem is we have a full class C at the colo, but DragonFly isn't really designed to operate with two different subnets where one subnet overlaps the other. Ethernet switching turned out to be the better solution. The colocated box itself is ON the class C, it doesn't have a separate IP outside the class C space. So there was no easy way to swing a routed network. I wouldn't even consider something as complex as OSPF for a simple setup like this, even with a routed solution. :(my soekris 486 gets a little bogged down by the crypto, which is why I ask) : :ok enough questions ;) : :its definitely fun trying to convert consumer internet into a 'real :connection' :D : :- Chris : :(from a gigabit LAN piggybacked on a sometimes 56k wifi link) OpenVPN has options to run in the clear after authentication is complete, I think, but I highly recommend using the crypto TLS support. The instructions on setting up all the files are pretty clear (you can find