Good questions … as everyone knows and was mentioned, every network operator
has different ideas and requirements so my answer may help or may not :)
Our network is comprised of Juniper MX240,MX480, and MX2010 routers in most
locations. These routers fill various roles depending on the location such as
aggregation, LNS functions (PPPOE), and core/border/edge routing. All of the
MX routers are dual stacked and participate in an iBGP mesh. OSPF is used for
IPv4 p2p/loopback networks only, OSPFv3 for IPv6 etc etc… In addition we run
MPLS across most of the MX routers using RSVP primarily (in some places LDP is
used inside of RSVP).
Our routing tables look like this:
inet.0: 629132 destinations, 3533518 routes (629118 active, 24 holddown, 8
hidden)
Direct: 30 routes, 29 active
Local: 28 routes, 28 active
OSPF: 52 routes, 51 active
BGP: 3533329 routes, 628949 active
Static: 62 routes, 58 active
IGMP: 1 routes, 1 active
PIM: 2 routes, 2 active
RSVP: 14 routes, 0 active
inet.1: 3 destinations, 3 routes (3 active, 0 holddown, 0 hidden)
Multicast: 3 routes, 3 active
inet.3: 12 destinations, 12 routes (12 active, 0 holddown, 0 hidden)
RSVP: 12 routes, 12 active
mpls.0: 140 destinations, 140 routes (140 active, 0 holddown, 0 hidden)
MPLS: 4 routes, 4 active
RSVP: 134 routes, 134 active
VPN: 2 routes, 2 active
bgp.l3vpn.0: 48 destinations, 87 routes (48 active, 0 holddown, 0 hidden)
Direct: 4 routes, 4 active
Local: 2 routes, 2 active
BGP: 80 routes, 41 active
Static: 1 routes, 1 active
inet6.0: 33157 destinations, 218290 routes (33084 active, 13 holddown, 1594
hidden)
Direct: 45 routes, 25 active
Local: 42 routes, 42 active
OSPF3: 35 routes, 33 active
BGP: 218164 routes, 32980 active
Static: 1 routes, 1 active
PIM: 2 routes, 2 active
MLD: 1 routes, 1 active
inet6.1: 1 destinations, 1 routes (1 active, 0 holddown, 0 hidden)
Multicast: 1 routes, 1 active
inet6.3: 12 destinations, 12 routes (12 active, 0 holddown, 0 hidden)
RSVP: 12 routes, 12 active
bgp.l2vpn.0: 14 destinations, 25 routes (14 active, 0 holddown, 0 hidden)
BGP: 25 routes, 14 active
bgp.mvpn.0: 287 destinations, 574 routes (287 active, 1 holddown, 0 hidden)
BGP: 407 routes, 120 active
PIM: 165 routes, 165 active
MVPN: 2 routes, 2 active
bgp.mvpn-inet6.0: 2 destinations, 2 routes (2 active, 0 holddown, 0 hidden)
MVPN: 2 routes, 2 active
<I trimmed out customer specific routing tables>
So, with iBGP (and eBGP) we use extensive communities. These are used for
identification, traffic engineering, and sharing information with other
networks as deemed appropriate.
Our DDOS mitigation uses these communities as well to learn what a certain IP
block is responsible for, which is turn tells the system what type of
mitigation to perform (clean the traffic through surgical mitigation vs
blackhole the route).
Our DPI systems (a vendor that is very popular here on the mailing list) learn
prefix information based on those BGP communities as well, which through some
automation, we can apply rules dynamically on how to deal with certain types of
traffic during any events with congestion.
Our Google caches identify what type of service is deployed in that IP block
(ie. Cable vs fiber vs DSL) and geographic information (again from the
communities).
Our Netflix caches identify geographic information based on communities and
also allows for us to use intra-city redundancy between POP’s should we lose
very many caches for whatever reason.
Our Akamai caches use community information but I can’t recall how/why – pretty
sure it’s purely statistical in nature
For our network topology we find summarization of routes to be more efficient
using iBGP vs OSPF (we have over 15k customers on DSL for example who have
static subnets)
By default, Juniper only resolves BGP routes via inet.3 table – if we kept
routes in IGP such as OSPF then traffic would be routed via IP and not MPLS
(yes there are ways around that but for our needs it was less than ideal)
This is just some of the reasons I could come up with … this “evolution” for us
took a long time especially as MPLS deployments picked up pace which made it
even more compelling for us to deploy using this methodology. For MPLS it came
down to traffic engineering and protection options to keep it short….
I would never say this is the correct way to someone … I would always suggest
people look at what they want to accomplish and what they think they might want
to do in future and then plan for the “layers of the network” (no pun intended).
Cheers,
Paul
From: Af [mailto:[email protected]] On Behalf Of Mike Hammett
Sent: August 27, 2016 12:03 PM
To: [email protected]
Subject: Re: [AFMUG] (OSPF + ibgp) / formerly Mikrotik OSPF weirdness
I keep asking for more because this is a topic I'm extremely interested in.
Tell me more. Tell me more. :-)
-----
Mike Hammett
<http://www.ics-il.com/> Intelligent Computing Solutions
<https://www.facebook.com/ICSIL>
<https://plus.google.com/+IntelligentComputingSolutionsDeKalb>
<https://www.linkedin.com/company/intelligent-computing-solutions>
<https://twitter.com/ICSIL>
<http://www.midwest-ix.com/> Midwest Internet Exchange
<https://www.facebook.com/mdwestix>
<https://www.linkedin.com/company/midwest-internet-exchange>
<https://twitter.com/mdwestix>
<http://www.thebrotherswisp.com/> The Brothers WISP
<https://www.facebook.com/thebrotherswisp>
<https://www.youtube.com/channel/UCXSdfxQv7SpoRQYNyLwntZg>
_____
From: "Paul Stewart" <[email protected] <mailto:[email protected]> >
To: [email protected] <mailto:[email protected]>
Sent: Saturday, August 27, 2016 11:00:51 AM
Subject: Re: [AFMUG] (OSPF + ibgp) / formerly Mikrotik OSPF weirdness
Yes exactly per my earlier post … everyone wants to jump off the OSPF ship for
a couple of reasons:
-Someone told them it’s very bad to scale it up but failed to define what
“scale” is referring to
-misconfiguration or misunderstanding of OSPF (common)
-OS issues (ie. Microtik that’s being talked about a lot)
Of course it’s not just about scale … for me, the benefits that BGP brings to
the table far outweigh the benefits of OSPF .. ie. OSPF tags vs BGP communities
From: Af [mailto:[email protected]] On Behalf Of Faisal Imtiaz
Sent: August 26, 2016 6:02 PM
To: [email protected] <mailto:[email protected]>
Subject: Re: [AFMUG] (OSPF + ibgp) / formerly Mikrotik OSPF weirdness
>>As you grow, you'll find it won't scale well.
Care to elaborate more on this ?
By definition it is pointed out that putting hundreds of routers or hundreds of
routes are a weak point of OSPF, however there are many different techniques
available to manage that.
Regards.
Faisal Imtiaz
Snappy Internet & Telecom
7266 SW 48 Street
Miami, FL 33155
Tel: 305 663 5518 x 232
Help-desk: (305)663-5518 Option 2 or Email: [email protected]
<mailto:[email protected]>
_____
From: "Bruce Robertson" <[email protected] <mailto:[email protected]> >
To: [email protected] <mailto:[email protected]>
Sent: Friday, August 26, 2016 5:23:14 PM
Subject: Re: [AFMUG] (OSPF + ibgp) / formerly Mikrotik OSPF weirdness
As you grow, you'll find it won't scale well.
On 08/26/2016 02:21 PM, George Skorup wrote:
I do redist with OSPF. It works fine if you know what you're doing. MT OSPF
used to act really stupid until ROS v6.27 or thereabouts.
On 8/26/2016 2:16 PM, Faisal Imtiaz wrote:
So just for the sake of a technical discussion...
In your opinion, what is the merit of such a config (osfp + ibgp) ?
It can be argued that such a config,
a) Still depends on OSPF functioning.
b) Layer an additional dynamic protocol on top of it (ibgp)
c) Requires additional Routers (route reflectors).
If the merit of such an approach is to manage manage OSFP behavior in a more
granular fashion, Why not use the those features as they are available in
OSPF / Best Practices...
(OSFP best practices, suggest that, don't advertise connected or static
routes, setup all interfaces as passive, and control prefix advertisements via
the network section of OSPF).
OSPF also tends to be the most common denominator (protocol) across different
mfg. Bgp being the 2nd.
Regards
Faisal Imtiaz
Snappy Internet & Telecom
7266 SW 48 Street
Miami, FL 33155
Tel: 305 663 5518 <callto:305%20663%205518> x 232
Help-desk: (305)663-5518 <callto:%28305%29663-5518> Option 2 or Email:
[email protected] <mailto:[email protected]>
_____
From: "Jesse DuPont" <mailto:[email protected]>
<[email protected]>
To: [email protected] <mailto:[email protected]>
Sent: Friday, August 26, 2016 12:03:58 AM
Subject: Re: [AFMUG] Mikrotik OSPF weirdness
Right, PTP and loopback prefixes are distributed with OSPF (and possibly
management subnets for radios) and "access" network prefixes (customer-facing)
are distributed via iBGP.
I have two of my routers configured as BGP route reflectors and all other
routers peer with only these two; this solves the full mesh and provides
redundancy.
Jesse DuPont
Network Architect
email: [email protected] <mailto:[email protected]>
Celerity Networks LLC
Celerity Broadband LLC
Like us! facebook.com/celeritynetworksllc
Like us! facebook.com/celeritybroadband
<http://127.0.0.1:11862/service/home/%7E/?auth=co&id=1da921bb-b2a8-4368-bc2e-c997a36651f3:95254&part=2>
On 8/25/16 8:40 PM, David Milholen wrote:
He may have meant only have the ptp and loopback addresses listed in networks
On 8/25/2016 9:31 PM, Mike Hammett wrote:
I've heard this concept a few times now. I'm not sure how only using OSPF for
the loopbacks works.
-----
Mike Hammett
<http://www.ics-il.com/> Intelligent Computing Solutions
<https://www.facebook.com/ICSIL>
<https://plus.google.com/+IntelligentComputingSolutionsDeKalb>
<https://www.linkedin.com/company/intelligent-computing-solutions>
<https://twitter.com/ICSIL>
<http://www.midwest-ix.com/> Midwest Internet Exchange
<https://www.facebook.com/mdwestix>
<https://www.linkedin.com/company/midwest-internet-exchange>
<https://twitter.com/mdwestix>
<http://www.thebrotherswisp.com/> The Brothers WISP
<https://www.facebook.com/thebrotherswisp>
<https://www.youtube.com/channel/UCXSdfxQv7SpoRQYNyLwntZg>
_____
From: "Bruce Robertson" <mailto:[email protected]> <[email protected]>
To: [email protected] <mailto:[email protected]>
Sent: Thursday, August 25, 2016 6:28:43 PM
Subject: Re: [AFMUG] Mikrotik OSPF weirdness
I've said it before, and been argued with... this is one of many reasons why
you use iBGP to distribute {customer, dynamic pool, server subnets, anything}
routes, and use OSPF *only* to distribute router loopback addresses.� All
your weird OSPF problems will go away.� My apologies if I'm misunderstanding
the problem, but my point still stands.
On 08/25/2016 10:22 AM, Robert Haas wrote:
Alright, this problem has raised it head again on my network since I started to
renumber some PPPoE pools.
Customer gets a new IP address via PPPoE x.x.x.208/32 (from x.x.x.192/27 pool).
Customer can�t surf and I can�t ping them from my office:
�
[office] � [Bernie Router] � [Braggcity Router] � [Ross Router] �
[Hayti Router] � [customer]
�
A traceroute from my office dies @ the Bernie router but I am not getting any
type of ICMP response from the Bernie router ie no ICMP Host Unreachable/Dest
unreachable etc � just blackholes after my office router.
A traceroute from the Customer to the office again dies at the Bernie router
with no type of response.
�
Checking the routing table on the Bernie router shows a valid route pointing to
the Braggcity router. It is also in the OSPF LSA�s.
--
Another customer gets x.x.x.207/32 and has no issue at all.
�
--
Force the original customer to a new ip address of x.x.x.205/32 and the service
starts working again.
�
--
�
Now � even though there is no valid route to x.x.x.208/32 in the routing
table � traffic destined to the x.x.x.208/32 IP is still getting blackholed..
I should be getting a Destination host unreachable from the Bernie router.
�
This is correct the correct response .206 is not being used and there is no
route to it:
C:\Users\netadmin>ping x.x.x.206
�
Pinging x.x.x.206 with 32 bytes of data:
Reply from y.y.y.1: Destination host unreachable.
Reply from y.y.y.1: Destination host unreachable.
�
Ping statistics for x.x.x.206:
��� Packets: Sent = 2, Received = 2, Lost = 0 (0% loss),
�
C:\Users\netadmin>tracert 74.91.65.206 <callto:74.91.65.206>
�
Tracing route to host-x.x.x.206.bpsnetworks.com [x.x.x.206]
over a maximum of 30 hops:
�
� 1���� 6 ms���� 6 ms���� 7 ms� z.z.z.z
� 2���� 6 ms���� 6 ms���� 6 ms� y.bpsnetworks.com
[y.y.y.1]
� 3� y.bpsnetworks.com [y.y.y.1] �reports: Destination host unreachable.
�
Trace complete.
�
This is what I see to x.x.x.208 even though it is not being used and there is
no route to it.
C:\Users\netadmin>ping x.x.x.208
�
Pinging x.x.x.208 with 32 bytes of data:
Request timed out.
Request timed out.
�
Ping statistics for x.x.x.208:
��� Packets: Sent = 2, Received = 0, Lost = 2 (100% loss),
�
C:\Users\netadmin>tracert x.x.x.208
�
Tracing route to host-x.x.x.208.bpsnetworks.com [x.x.x.208]
over a maximum of 30 hops:
�
� 1���� 6 ms���� 6 ms���� 6 ms� z.z.z.z
� 2���� *������� *������� *����
Request timed out.
� 3���� *������� *���� ^C
�
--
�
I�ve verified there is no firewall that would affect the traffic � I even
put an accept rule in the forward chain for both the source and destination of
x.x.x.208 and neither increment at all. So the traffic is not even making out
of the routing flow and into the firewall..
�
Any pointers are where to start troubleshooting next?
--
<http://127.0.0.1:11862/service/home/%7E/?auth=co&id=1da921bb-b2a8-4368-bc2e-c997a36651f3:95254&part=3>
!DSPAM:2,57c0b2eb92841205749441!
