I advocate keeping everything routed, and using MPLS/VPLS to move L2 where they need to go, when required.
From: Af [mailto:af-boun...@afmug.com] On Behalf Of Glen Waldrop Sent: Friday, August 7, 2015 3:09 PM To: af@afmug.com Subject: Re: [AFMUG] Routed vs bridge with a twist You guys have given me some light reading to do based on my question above. Sounds like the consensus is a few mid sized L2 rather than one large L2 for backhauls? Or stick with a subnet per link as I have now? ----- Original Message ----- From: Shayne Lebrun <mailto:sleb...@muskoka.com> To: af@afmug.com Sent: Friday, August 07, 2015 1:52 PM Subject: Re: [AFMUG] Routed vs bridge with a twist That’s the basics. On a Mikrotik: Create a bridge with no ports. Call it ‘Loopback’. Assign a /32 to it, and advertise via OSPF. Set your OSPF instance router ID to this IP. MPLS->MPLS, under LDP Settings, select Enabled, use the Loopback IP as the LSR ID and Transport Address. Add the interface under LDP Interface and MPLS Interface. Now, MTU is the big sticking point. On MPLS Interface, I use 1586, which gives plenty of room for full 1500 byte packets plus VLANs, MPLS labels, VPLS labels, and so on. But all equipment needs to support that MTU; backhauls, routers, everything. So no 493 family Mikrotiks. No Canopy FSK or 430 backhauls. Ubiquiti, depends. And so on. Once you have an MPLS network, you can create VPLS tunnels just like EoIP tunnels, only there’ll be no fragmenting and way WAY less encapsulation overhead. RSVP, I think is what Mikrotik calls ‘Traffic Engineering.’ Tell it how much bandwidth you have on each interface, and you can avoid the situation where you have router a->b->c->d and router a->d means the first path is idle as long as the second path is up. I’ll reiterate, though, MTU will be the sticking point. Mikrotik’s wiki has some great write-ups on all this. From: Af [mailto:af-boun...@afmug.com] On Behalf Of Paul Stewart Sent: Friday, August 7, 2015 2:40 PM To: af@afmug.com Subject: Re: [AFMUG] Routed vs bridge with a twist If you have even a couple of routers (ideally with switches off of each), you can simulate some pretty cool stuff… then add a third router into the mix and it’s even more fun. MPLS isn’t something you just learn right away – it’s something that takes time to learn and run through in a lab setting ideally … there’s a lot of complexities that you can use if you want to … or there are simpler approaches…. Very very high level…. (not Microtik specific – I don’t know Microtik very well) Enable loopback interfaces on all routers (which often is already setup) Enable OSPF between the routers (pretty typical) Enable RSVP on the interfaces facing one another (this will be new) Enable MPLS “protocol” on the interfaces facing one another (this will be new) Configure iBGP between the routers (full mesh, peering with loopbacks – not interface IP’s) Configure LSP’s between all routers (remember, LSP’s are unidirectional so need all routers configured to all routers). This is assuming you want an RSVP based MPLS network and not LDP based – RSVP has advantages over LDP but is more complex to setup. You may also prefer using ISIS instead of OSPF in some networks. Once the LSP’s are established then you can look to create l2vpn, l3vpn, vpls, or multicast-vpn instances (there are many things you can do here). Easiest is an l2vpn where you transport a VLAN from one switchport to another switchport via the routers “in the middle”. Once you have some test traffic going, then you can investigate protection options such as fast re-route, node link protection etc… this is where MPLS really starts to “shine” when there is more than one path available to carry the traffic … how you influence how the traffic flows and how fast traffic will failover during an outage etc etc… This is incredibly high level overview and I may be missing something depending on your network hardware and topology …. But again, the basics from a high level. Paul From: Af [mailto:af-boun...@afmug.com] On Behalf Of Mike Hammett Sent: Friday, August 7, 2015 9:04 AM To: af@afmug.com Subject: Re: [AFMUG] Routed vs bridge with a twist I haven't grasped how this would work, but I haven't tried it in a lab yet either. ----- Mike Hammett Intelligent Computing Solutions http://www.ics-il.com _____ From: "Paul Stewart" <p...@paulstewart.org> To: af@afmug.com Sent: Friday, August 7, 2015 7:55:51 AM Subject: Re: [AFMUG] Routed vs bridge with a twist I’d suggest BGP at all locations when building an MPLS network – MPBGP to be specific…. Sometimes folks who are just starting into MPLS presume that by having a full BGP mesh everywhere means that you need to carry the full Internet routing table … not the case and different routing table often (depending on the hardware/os being used). Also, a lot of networks will put the Internet BGP tables into a separate routing instance and leave just their IGP routes in the primary table – provides for a nice level of separation between your routes From: Af [mailto:af-boun...@afmug.com] On Behalf Of Sterling Jacobson Sent: Thursday, August 6, 2015 11:13 PM To: af@afmug.com Subject: Re: [AFMUG] Routed vs bridge with a twist MPLS is where we are heading. In the planning phases right now for MPLS ring network. Seems like it works well if the network has multiple paths, but heads in essentially one location. I think it may break a bit if it’s necessary to involve BGP at multiple locations though. That’s what I’m debating right now. From: Af [mailto:af-boun...@afmug.com] On Behalf Of Mike Hammett Sent: Thursday, August 6, 2015 7:43 PM To: af@afmug.com Subject: Re: [AFMUG] Routed vs bridge with a twist MPLS-enabling a network also reduces your latency on Mikrotik. ----- Mike Hammett Intelligent Computing Solutions http://www.ics-il.com _____ From: "George Skorup" <geo...@cbcast.com> To: af@afmug.com Sent: Thursday, August 6, 2015 7:52:23 PM Subject: Re: [AFMUG] Routed vs bridge with a twist If you already have a routed core network, especially if you have OSPF rings (like we do), I figure it'd make more sense to put MPLS on top. I haven't done it yet because we haven't needed to do anything like customer tunnels for multi-site interconnects, but we're getting there. On 8/6/2015 4:32 PM, Glen Waldrop wrote: I'm running Mikrotik, all routed, got a different subnet for each tower, got a different subnet between each tower, public IP's routed to the customers, all the fun stuff. I'm thinking of restructuring my network so the entire backbone is one big L2 network. If I plug into the switch at the tower at tower 5 it will be no different than tower 1 or 7. Each AP would still have it's own subnet, but the backside of each AP would be on the same L2 as the rest. I'm planning on looping it all the way around and building redundancy into the network, haven't quite decided how I'm going to do that yet, might use STP, that is a little ways down the road. I'll have another fiber feed in case the main goes down and I'd like to have a level of redundancy should a tower go out, I'll only lose the one rather than the ones behind it as well. I've fried my brain today, so if I'm sounding half crazy, just tell me to take the rest of the day off... I'm thinking it might be best to have a few large L2 segments to the backbone, maybe three or four, rather than one big L2 and much simpler than 12+ subnets from tower to tower. Input is appreciated.
