Re: validating reachability via an ISP

[Andy Litzinger]

Hi Andy,
  The root cause was they regional ISP was failing to advertise my prefix
due to a mistake in their export policy.  While I'm glad we were able to
figure out the issue I'm generally more interested in figuring out a way
that I can programmatically monitor that my ISPs are providing me with good
reachability, even when their route isn't necessarily the best/installed
route to me.

I do have route objects defined in an IRR for this prefix. Perhaps you
wouldn't mind explaining to me how route-server or ISP operators generally
validate route-objects before accepting routes?  Especially in the case
where I'm not peering with your route server but my ISP is.  Do you query
the IRR DB to recurse from the ISP AS to my AS and validate route objects
there?

thanks!
-andy



On Thu, Apr 5, 2018 at 12:49 AM, Andy Davidson  wrote:

>
>
>
>
>
> On 29/03/2018, 00:22, Andy Litzinger 
> wrote:
> >
> >> The root cause is that the our prefix is not being adequately
> >> re-distributed globally by the regional ISP.  This is unexpected and we
> are
> >> working through this with them now.
>
> Hi, Andy —
>
> Are you failing to advertise it, or are they filtering it on ingress, or
> are they failing to send it to their other peers?
>
> One configuration mishap which is starting to come along more and more
> partial or poor reachability caused by route objects which are not
> correctly published in the IRRDB. It is going to be essential to make sure
> that you have properly recorded IRR route objects in, for instance, RADB.
> More BGP speakers properly filter their peers using information that is
> published there.  Avoid future reachability problems by checking this today!
>
> Yours,
> A friendly route-server operator with strict filtering
>
> -a
>
>
>
> --
> Andy DavidsonAsteroid International BV
> https://www.asteroidhq.com@asteroidhq   @andyd
> --
> Local interconnection.  Where you need it.
>
>

Re: validating reachability via an ISP

[Ben Bartsch]

+1 for Route Explorer

On Thu, Apr 5, 2018 at 2:49 AM, Andy Davidson  wrote:

>
>
>
>
>
> On 29/03/2018, 00:22, Andy Litzinger 
> wrote:
> >
> >> The root cause is that the our prefix is not being adequately
> >> re-distributed globally by the regional ISP.  This is unexpected and we
> are
> >> working through this with them now.
>
> Hi, Andy —
>
> Are you failing to advertise it, or are they filtering it on ingress, or
> are they failing to send it to their other peers?
>
> One configuration mishap which is starting to come along more and more
> partial or poor reachability caused by route objects which are not
> correctly published in the IRRDB. It is going to be essential to make sure
> that you have properly recorded IRR route objects in, for instance, RADB.
> More BGP speakers properly filter their peers using information that is
> published there.  Avoid future reachability problems by checking this today!
>
> Yours,
> A friendly route-server operator with strict filtering
>
> -a
>
>
>
> --
> Andy DavidsonAsteroid International BV
> https://www.asteroidhq.com@asteroidhq   @andyd
> --
> Local interconnection.  Where you need it.
>
>

Re: validating reachability via an ISP

[Andy Davidson]

On 29/03/2018, 00:22, Andy Litzinger  wrote:
>
>> The root cause is that the our prefix is not being adequately
>> re-distributed globally by the regional ISP.  This is unexpected and we are
>> working through this with them now.

Hi, Andy —

Are you failing to advertise it, or are they filtering it on ingress, or are 
they failing to send it to their other peers?

One configuration mishap which is starting to come along more and more partial 
or poor reachability caused by route objects which are not correctly published 
in the IRRDB. It is going to be essential to make sure that you have properly 
recorded IRR route objects in, for instance, RADB.  More BGP speakers properly 
filter their peers using information that is published there.  Avoid future 
reachability problems by checking this today!

Yours,
A friendly route-server operator with strict filtering

-a



-- 
Andy DavidsonAsteroid International BV
https://www.asteroidhq.com@asteroidhq   @andyd
--
Local interconnection.  Where you need it.

Re: validating reachability via an ISP

[Andy Litzinger]

Hi Andrew,
  thanks for sharing your repo, it looks very relevant and I should be able
to get it to do what I want with a slight tweak to the regex's.

Hopefully the route-views collectors or some other ones I dig up will be
able to adequately show at least 3 AS path hops through my regional
provider to get to my AS/prefix.  I really want to see AS paths where there
are at least 2 different AS hops before my regional providers AS.  This
will help me feel good that the global partners of my regional ISP are
re-advertising my prefix to their peers.  Otherwise if the AS path only
includes the first and second hop upstream I can only infer that the 2nd
hop AS is accepting routes from the 1st hop AS (my regional ISP), but not
that the 2nd hop AS is actually re-advertising the route to anyone else.

thanks!
 -andy

On Thu, Mar 29, 2018 at 7:51 AM, Andrew Wentzell 
wrote:

> On Wed, Mar 28, 2018 at 7:22 PM, Andy Litzinger
>  wrote:
> > Hi all,
> >   I have an enterprise network and do not provide transit. In one of our
> > datacenters we have our own prefixes and rely on two ISPs as BGP
> neighbors
> > to provide global reachability for our prefixes.  One is a large regional
> > provider and the other is a large global provider.
> >
> > Recently we took our link to the global provider offline to perform
> > maintenance on our router.  Nearly immediately we were hit with alerts
> that
> > our prefix was unreachable and BGPMon alerted that nearly 80 AS's noted
> our
> > route had been withdrawn.  We were not unreachable from every AS, but we
> > certainly were from some of the largest.
> >
> > The root cause is that the our prefix is not being adequately
> > re-distributed globally by the regional ISP.  This is unexpected and we
> are
> > working through this with them now.
> >
> > My question is, how can I monitor global reachability for a prefix via
> this
> > or any specific provider I use over time?  Are there various
> route-servers
> > I can programmatically query for my prefix and get results that include
> AS
> > paths? Then I could verify that an "acceptable" number of paths exist
> that
> > include the AS of the all the ISPs I rely upon.  And what would an
> > "acceptable" number of alternate paths be?
>
> I did something similar a few years ago, by querying routeviews and
> validating AS paths using python and pexpect. You could adapt it for
> your use pretty easily. The code's here:
>
> https://github.com/awentzell/check-as-path
>

Re: validating reachability via an ISP

[Andy Litzinger]

Hi Alex,
  Thanks for the link to the qrator API.  The get-all-paths is interesting
and the output does seem to show exactly the type of data I'm looking for.
Is there some place on the site that lists who Qrator peers with to get
your data?  I searched through the site and couldn't find it.  I'm trying
to get an idea of the breadth of data and whether i can comfortably rely on
it giving me a global view.

Also, how do I authorize programmatic requests to the API?  I created a
user and can use it to run the queries from the web page, but I don't see
how to use it directly with the API.

thanks!
 -andy

On Thu, Mar 29, 2018 at 8:23 AM, Alexander Azimov  wrote:

> Hi Andy,
>
> You can use Qrator.Radar API: https://api.radar.qrator.net/.
> The get-all-paths method will return the set of active paths for selected
> prefix.
>
> PS: I'm sorry if you receive this message two times, something funny is
> happening with my emails on NANOG mailing list.
>
> 2018-03-29 18:09 GMT+03:00 Alexander Azimov :
>
>> Hi Andy,
>>
>> You can use Qrator.Radar API: https://api.radar.qrator.net/.
>> The get-all-paths method will return the set of active paths for selected
>> prefix.
>>
>>
>> 2018-03-29 2:22 GMT+03:00 Andy Litzinger 
>> :
>>
>>> Hi all,
>>>   I have an enterprise network and do not provide transit. In one of our
>>> datacenters we have our own prefixes and rely on two ISPs as BGP
>>> neighbors
>>> to provide global reachability for our prefixes.  One is a large regional
>>> provider and the other is a large global provider.
>>>
>>> Recently we took our link to the global provider offline to perform
>>> maintenance on our router.  Nearly immediately we were hit with alerts
>>> that
>>> our prefix was unreachable and BGPMon alerted that nearly 80 AS's noted
>>> our
>>> route had been withdrawn.  We were not unreachable from every AS, but we
>>> certainly were from some of the largest.
>>>
>>> The root cause is that the our prefix is not being adequately
>>> re-distributed globally by the regional ISP.  This is unexpected and we
>>> are
>>> working through this with them now.
>>>
>>> My question is, how can I monitor global reachability for a prefix via
>>> this
>>> or any specific provider I use over time?  Are there various
>>> route-servers
>>> I can programmatically query for my prefix and get results that include
>>> AS
>>> paths? Then I could verify that an "acceptable" number of paths exist
>>> that
>>> include the AS of the all the ISPs I rely upon.  And what would an
>>> "acceptable" number of alternate paths be?
>>>
>>>
>>> thanks in advance,
>>>   -andy
>>>
>>
>>
>>
>> --
>> | Alexander Azimov  | HLL l QRATOR
>> | tel.: +7 499 241 81 92
>> | mob.: +7 915 360 08 86
>> | skype: mitradir
>> | mailto: a...@qrator.net
>> | visit: www.qrator.net
>>
>
>
>
> --
> | Alexander Azimov  | HLL l QRATOR
> | tel.: +7 499 241 81 92
> | mob.: +7 915 360 08 86
> | skype: mitradir
> | mailto: a...@qrator.net
> | visit: www.qrator.net
>

Re: validating reachability via an ISP

[Don Thomas Jacob]

Hi,

If you are fine with a commercial tool, Packet Design Route Explorer
product supports peering analysis:
BGP-RIB-Visualization-1.png

is
one among its features.

https://www.packetdesign.com/solutions/peering-analysis/

Disclaimer: I work for Packet Design

Thanks,
Don

-
Don Thomas Jacob
LinkedIn  | Twitter



On Thu, Mar 29, 2018 at 8:39 PM, Alexander Azimov  wrote:

> Hi Andy,
>
> You can use Qrator.Radar API: https://api.radar.qrator.net/.
> The get-all-paths method will return the set of active paths for selected
> prefix.
>
>
> 2018-03-29 2:22 GMT+03:00 Andy Litzinger :
>
> > Hi all,
> >   I have an enterprise network and do not provide transit. In one of our
> > datacenters we have our own prefixes and rely on two ISPs as BGP
> neighbors
> > to provide global reachability for our prefixes.  One is a large regional
> > provider and the other is a large global provider.
> >
> > Recently we took our link to the global provider offline to perform
> > maintenance on our router.  Nearly immediately we were hit with alerts
> that
> > our prefix was unreachable and BGPMon alerted that nearly 80 AS's noted
> our
> > route had been withdrawn.  We were not unreachable from every AS, but we
> > certainly were from some of the largest.
> >
> > The root cause is that the our prefix is not being adequately
> > re-distributed globally by the regional ISP.  This is unexpected and we
> are
> > working through this with them now.
> >
> > My question is, how can I monitor global reachability for a prefix via
> this
> > or any specific provider I use over time?  Are there various
> route-servers
> > I can programmatically query for my prefix and get results that include
> AS
> > paths? Then I could verify that an "acceptable" number of paths exist
> that
> > include the AS of the all the ISPs I rely upon.  And what would an
> > "acceptable" number of alternate paths be?
> >
> >
> > thanks in advance,
> >   -andy
> >
>
>
>
> --
> | Alexander Azimov  | HLL l QRATOR
> | tel.: +7 499 241 81 92
> | mob.: +7 915 360 08 86
> | skype: mitradir
> | mailto: a...@qrator.net
> | visit: www.qrator.net
>

Re: validating reachability via an ISP

[Alexander Azimov]

Hi Andy,

You can use Qrator.Radar API: https://api.radar.qrator.net/.
The get-all-paths method will return the set of active paths for selected
prefix.


2018-03-29 2:22 GMT+03:00 Andy Litzinger :

> Hi all,
>   I have an enterprise network and do not provide transit. In one of our
> datacenters we have our own prefixes and rely on two ISPs as BGP neighbors
> to provide global reachability for our prefixes.  One is a large regional
> provider and the other is a large global provider.
>
> Recently we took our link to the global provider offline to perform
> maintenance on our router.  Nearly immediately we were hit with alerts that
> our prefix was unreachable and BGPMon alerted that nearly 80 AS's noted our
> route had been withdrawn.  We were not unreachable from every AS, but we
> certainly were from some of the largest.
>
> The root cause is that the our prefix is not being adequately
> re-distributed globally by the regional ISP.  This is unexpected and we are
> working through this with them now.
>
> My question is, how can I monitor global reachability for a prefix via this
> or any specific provider I use over time?  Are there various route-servers
> I can programmatically query for my prefix and get results that include AS
> paths? Then I could verify that an "acceptable" number of paths exist that
> include the AS of the all the ISPs I rely upon.  And what would an
> "acceptable" number of alternate paths be?
>
>
> thanks in advance,
>   -andy
>



-- 
| Alexander Azimov  | HLL l QRATOR
| tel.: +7 499 241 81 92
| mob.: +7 915 360 08 86
| skype: mitradir
| mailto: a...@qrator.net
| visit: www.qrator.net

Re: validating reachability via an ISP

[Andrew Wentzell]

On Wed, Mar 28, 2018 at 7:22 PM, Andy Litzinger
 wrote:
> Hi all,
>   I have an enterprise network and do not provide transit. In one of our
> datacenters we have our own prefixes and rely on two ISPs as BGP neighbors
> to provide global reachability for our prefixes.  One is a large regional
> provider and the other is a large global provider.
>
> Recently we took our link to the global provider offline to perform
> maintenance on our router.  Nearly immediately we were hit with alerts that
> our prefix was unreachable and BGPMon alerted that nearly 80 AS's noted our
> route had been withdrawn.  We were not unreachable from every AS, but we
> certainly were from some of the largest.
>
> The root cause is that the our prefix is not being adequately
> re-distributed globally by the regional ISP.  This is unexpected and we are
> working through this with them now.
>
> My question is, how can I monitor global reachability for a prefix via this
> or any specific provider I use over time?  Are there various route-servers
> I can programmatically query for my prefix and get results that include AS
> paths? Then I could verify that an "acceptable" number of paths exist that
> include the AS of the all the ISPs I rely upon.  And what would an
> "acceptable" number of alternate paths be?

I did something similar a few years ago, by querying routeviews and
validating AS paths using python and pexpect. You could adapt it for
your use pretty easily. The code's here:

https://github.com/awentzell/check-as-path

Re: validating reachability via an ISP

[Baldur Norddahl]

Also the only traffic you will be receiving on the other provider will be
from parties that did not pick up the more specific prefix. It should
therefore be really obvious. You should not receive any traffic at all, not
even from the transit provider.

Regards,

Baldur


Den tor. 29. mar. 2018 10.41 skrev Baldur Norddahl <
baldur.nordd...@gmail.com>:

> If your prefix is larger than /24 you can test with a more specific prefix
> such as a /24. Announce the test prefix on just one transit provider. Then
> check with BGP services such as the looking glass service provided by the
> NLNOG RING network.
>
> There will be no interruption in the traffic as it will follow the less
> specific prefix everywhere the test prefix was not picked up.
>
> If you only have a /24 you will need to test in a service window.
>
> Regards
>
> Baldur
>
>
> Den tor. 29. mar. 2018 01.24 skrev Andy Litzinger <
> andy.litzinger.li...@gmail.com>:
>
>> Hi all,
>>   I have an enterprise network and do not provide transit. In one of our
>> datacenters we have our own prefixes and rely on two ISPs as BGP neighbors
>> to provide global reachability for our prefixes.  One is a large regional
>> provider and the other is a large global provider.
>>
>> Recently we took our link to the global provider offline to perform
>> maintenance on our router.  Nearly immediately we were hit with alerts
>> that
>> our prefix was unreachable and BGPMon alerted that nearly 80 AS's noted
>> our
>> route had been withdrawn.  We were not unreachable from every AS, but we
>> certainly were from some of the largest.
>>
>> The root cause is that the our prefix is not being adequately
>> re-distributed globally by the regional ISP.  This is unexpected and we
>> are
>> working through this with them now.
>>
>> My question is, how can I monitor global reachability for a prefix via
>> this
>> or any specific provider I use over time?  Are there various route-servers
>> I can programmatically query for my prefix and get results that include AS
>> paths? Then I could verify that an "acceptable" number of paths exist that
>> include the AS of the all the ISPs I rely upon.  And what would an
>> "acceptable" number of alternate paths be?
>>
>>
>> thanks in advance,
>>   -andy
>>
>

Re: validating reachability via an ISP

[Baldur Norddahl]

If your prefix is larger than /24 you can test with a more specific prefix
such as a /24. Announce the test prefix on just one transit provider. Then
check with BGP services such as the looking glass service provided by the
NLNOG RING network.

There will be no interruption in the traffic as it will follow the less
specific prefix everywhere the test prefix was not picked up.

If you only have a /24 you will need to test in a service window.

Regards

Baldur


Den tor. 29. mar. 2018 01.24 skrev Andy Litzinger <
andy.litzinger.li...@gmail.com>:

> Hi all,
>   I have an enterprise network and do not provide transit. In one of our
> datacenters we have our own prefixes and rely on two ISPs as BGP neighbors
> to provide global reachability for our prefixes.  One is a large regional
> provider and the other is a large global provider.
>
> Recently we took our link to the global provider offline to perform
> maintenance on our router.  Nearly immediately we were hit with alerts that
> our prefix was unreachable and BGPMon alerted that nearly 80 AS's noted our
> route had been withdrawn.  We were not unreachable from every AS, but we
> certainly were from some of the largest.
>
> The root cause is that the our prefix is not being adequately
> re-distributed globally by the regional ISP.  This is unexpected and we are
> working through this with them now.
>
> My question is, how can I monitor global reachability for a prefix via this
> or any specific provider I use over time?  Are there various route-servers
> I can programmatically query for my prefix and get results that include AS
> paths? Then I could verify that an "acceptable" number of paths exist that
> include the AS of the all the ISPs I rely upon.  And what would an
> "acceptable" number of alternate paths be?
>
>
> thanks in advance,
>   -andy
>

Re: validating reachability via an ISP

[Frank Habicht]

On 3/29/2018 2:22 AM, Andy Litzinger wrote:
> Hi all,
>   I have an enterprise network and do not provide transit. In one of our
> datacenters we have our own prefixes and rely on two ISPs as BGP neighbors
> to provide global reachability for our prefixes.  One is a large regional
> provider and the other is a large global provider.
> 
> Recently we took our link to the global provider offline to perform
> maintenance on our router.  Nearly immediately we were hit with alerts that
> our prefix was unreachable and BGPMon alerted that nearly 80 AS's noted our
> route had been withdrawn.  We were not unreachable from every AS, but we
> certainly were from some of the largest.
> 
> The root cause is that the our prefix is not being adequately
> re-distributed globally by the regional ISP.  This is unexpected and we are
> working through this with them now.
> 
> My question is, how can I monitor global reachability for a prefix via this
> or any specific provider I use over time?  Are there various route-servers
> I can programmatically query for my prefix and get results that include AS
> paths? Then I could verify that an "acceptable" number of paths exist that
> include the AS of the all the ISPs I rely upon.  And what would an
> "acceptable" number of alternate paths be?

If your global provider supports, you could send your announcements with
a BGP community per RFC1998 telling them to not-prefer-so-much that
advertisement, "use it as a backup".

that would shift a lot of incoming traffic to the other link (regional
provider).
You'll still have the global provider link.
this is a smaller change towards taking global provider offline, keeping
some fallback.

Frank

validating reachability via an ISP

[Andy Litzinger]

Hi all,
  I have an enterprise network and do not provide transit. In one of our
datacenters we have our own prefixes and rely on two ISPs as BGP neighbors
to provide global reachability for our prefixes.  One is a large regional
provider and the other is a large global provider.

Recently we took our link to the global provider offline to perform
maintenance on our router.  Nearly immediately we were hit with alerts that
our prefix was unreachable and BGPMon alerted that nearly 80 AS's noted our
route had been withdrawn.  We were not unreachable from every AS, but we
certainly were from some of the largest.

The root cause is that the our prefix is not being adequately
re-distributed globally by the regional ISP.  This is unexpected and we are
working through this with them now.

My question is, how can I monitor global reachability for a prefix via this
or any specific provider I use over time?  Are there various route-servers
I can programmatically query for my prefix and get results that include AS
paths? Then I could verify that an "acceptable" number of paths exist that
include the AS of the all the ISPs I rely upon.  And what would an
"acceptable" number of alternate paths be?


thanks in advance,
  -andy