Re: Theorical question about cyclic dependency in IRR filtering

[J. Hellenthal via NANOG]

Coin phrase ... IRR (dedup)

-- 
 J. Hellenthal

The fact that there's a highway to Hell but only a stairway to Heaven says a 
lot about anticipated traffic volume.

> On Nov 29, 2021, at 07:17, Job Snijders via NANOG  wrote:
> 
> 
> Hi Anurag,
> 
> Circular dependencies definitely are a thing to keep in mind when designing 
> IRR and RPKI pipelines!
> 
> In the case of IRR: It is quite rare to query the RIR IRR services directly. 
> Instead, the common practise is that utilities such as bgpq3, peval, and 
> bgpq4 query “IRRd” (https://IRRd.net) instances at for example whois.radb.net 
> and rr.ntt.net. You can verify this with tcpdump. These IRRd instances serve 
> as intermediate caches, and will continue to serve old cached data in case 
> the origin is down. This phenomenon in the global IRR deployment avoids a lot 
> of potential for circular dependencies.
> 
> Also, some organisations use threshold checks before deploying new IRR-based 
> filters to reduce risk of “misfiring”.
> 
> 
> The RPKI case is slightly different: the timers are far more aggressive 
> compared to IRR, and until “Publish in Parent” (RFC 8181) becomes common 
> place, there are more publication points, thus more potential for operators 
> to paint themselves into a corner.
> 
> Certainly, in the case of RPKI, all Publication Point (PP) operators need to 
> take special care to not host CAs which have the PP’s INRs listed as 
> subordinate resources inside the PP.
> 
> See RFC 7115 Section 5 for more information: “Operators should be aware that 
> there is a trade-off in placement of an RPKI repository in address space for 
> which the repository’s content is authoritative. On one hand, an operator 
> will wish to maximize control over the repository. On the other hand, if 
> there are reachability problems to the address space, changes in the 
> repository to correct them may not be easily access by others”
> 
> Ryan Sleevi once told me: "yes, it strikes me that you should prevent 
> self-compromise from being able to perpetually own yourself, by limiting an 
> attacker’s ability to persist beyond remediation."
> 
> A possible duct tape approach is outlined at  
> https://bgpfilterguide.nlnog.net/guides/slurm_ta/
> However, I can’t really recommend the SLURM file approach. Instead, RPKI 
> repository operators are probably best off hosting their repository *outside* 
> their own address space.
> 
> Just like with Authoritative DNS servers, make sure you also can serve your 
> records via a competitor! :-)
> 
> For example, if ARIN moved one of their three publication point clusters into 
> address space managed by any of the other four RIRs, some risk would be 
> reduced.
> 
> Kind regards,
> 
> Job
> 
>> On Mon, 29 Nov 2021 at 13:37, Anurag Bhatia  wrote:
>> Hello everyone, 
>> 
>> While discussing IRR on some groups recently, I was thinking if there can be 
>> (and if there is) cycling dependency in filtering where IRR (run by whoever 
>> APNIC, RIPE, RADB etc) uses some upstream and accepts only routes with 
>> existing & valid route object. 
>> 
>> 
>> 
>> So hypothetical case (can apply to any IRR): 
>> 
>> APNIC registry source is whois.apnic.net and points to 202.12.28.136 / 
>> 2001:dc0:1:0:4777::136. The aggregate of both these has a valid route object 
>> at the APNIC registry itself. 
>> 
>> Their upstreams say AS X, Y and Z have tooling in place to generate and push 
>> filters by checking all popular IRRs. All is well till this point. 
>> 
>> Say APNIC has some server/service issue for a few mins and X Y and Z are 
>> updating their filters at the same time. They cannot contact whois.apnic.net 
>> and hence miss generating filters for all APNIC IRR hosted prefixes. 
>> 
>> X, Y and  Z drop APNIC prefixes including those of IRR & the loop goes on 
>> from this point onwards. 
>> 
>> So my question is: Can that actually happen? 
>> If not, do X, Y and Z and possible all upstreams till default-free zone 
>> treat these prefixes in a special manner to avoid such loop in resolution? 
>> 
>> 
>> 
>> 
>> Thanks! 
>> 
>> -- 
>> Anurag Bhatia
>> anuragbhatia.com

Re: Class D addresses? was: Redploying most of 127/8 as unicast public

[Greg Skinner via NANOG]

> On Nov 24, 2021, at 5:08 PM, William Herrin  wrote:
> 
> On Wed, Nov 24, 2021 at 4:36 PM David Conrad  wrote:
>>> I like research but what would the RIRs study? The percentage of the
>> 
>> Lots of people said similar things when 1.0.0.0/8 was allocated to APNIC
>> and they said similar things when 1.1.1.0/24 was stood up as an
>> experiment by Cloudflare and APNIC, yet 1.1.1.1 seems to be pretty popular.
> 
> Hi David,
> 
> I don't recall there being any equipment or software compatibility
> concerns with 1.0.0.0/8. If you do, feel free to refresh my memory. As
> I recall it, there were concerns with prior local use and potential
> trash traffic. It seemed likely those concerns could be addressed with
> experiments, and the experiments in fact addressed them.
> 
> The prior local use worry reared its head again with 240/4 but given
> the prior experience with 1.0.0.0/8 I don't personally believe we need
> to re-run that experiment just because the numbers are part of a
> different block.
> 
> 
>> Seems to me that a number of folks on this list and during this
>> discussion would disagree with a blanket assertion that 240/4
>> is “dysfunctional on the 2021 Internet” - some of them even
>> wrote a draft discussing the possibility.
> 
> Line them up. Show of hands. Who really thinks that if we assign
> 240.0.0.1 to a customer tomorrow without waiting for anyone to clean
> up their software and hardware, you won't get enough complaints about
> things not working that you have to take it back and assign a
> different address instead?
> 
> 
>> 1. Move 240/4 from "reserved" to "unallocated unicast"
>> 
>> OK, but this seems like a quibble.  The status for 240/4 is “
>> RESERVED: designated by the IETF for specific non-global-unicast
>> purposes as noted.”  The “as noted” part is “Future Use”.
> 
> It's not a quibble. Some vendors take the current status to mean
> "treat it like unicast until we're told otherwise." Others take the
> status to mean, "packets with these addresses are bogons without a
> defined routing behavior until we're told otherwise." The result is
> incompatible behavior between vendors. Changing that direction to
> "treat it like unicast" without ambiguity is not a quibble.
> 
> Regards,
> Bill Herrin
> 
> --
> William Herrin
> b...@herrin.us
> https://bill.herrin.us/

For what it’s worth, I’ve been tracking this issue on other netops mailing 
lists.  There is a recent post on the LACNOG list from Leandro Bertholdo 
 
referencing 
https://datatracker.ietf.org/doc/draft-chen-ati-adaptive-ipv4-address-space/ 
, 
a draft proposing another way to make additional IPv4 address space available.  
I haven’t had time to read the draft closely, but I noticed that it involves 
the use of 240/4.  Subsequent googling about the draft turned up a presentation 
 
describing how the techniques described could be deployed.  I noticed that the 
presentation made reference to OpenWRT, so perhaps the authors are aware of the 
work that the authors of the IPv4 Unicast Extensions Project have done in that 
area.

The adaptive-ipv4 draft will expire sometime next month, so anyone interested 
in seeing it move forward should contact the authors.

—gregbo

Re: IPv6 and CDN's

[Owen DeLong via NANOG]

> On Nov 28, 2021, at 23:25 , Mark Tinka  wrote:
> 
> 
> 
> On 11/29/21 03:33, Masataka Ohta wrote:
> 
>> The end result was that our DNS servers became unreachable even though they 
>> were still operational. This made it impossible for the rest of the internet 
>> to find our servers.
> 
> So your suggestion to map machine addresses to human-readable names is... 
> what?

If you can limit the names to the characters a-f, i, o, s, and z then it’s 
possible to do so with IPv6 addresses natively.
(which you can’t do in IPv4).
(o=0, i=1, s=5, and z=2)

I’m not saying this is a good idea, but it is possible. There are a large 
number of english words that can be spelled with just those 9 characters.

Owen

Re: IPv6 and CDN's

[Owen DeLong via NANOG]

> On Nov 28, 2021, at 23:19 , Mark Tinka  wrote:
> 
> 
> 
> On 11/29/21 00:41, scott wrote:
> 
>> Side note: I recently tried to get /48 per customer with ARIN on repeated 
>> emails and they refused.  We were already given an IPv6 block a while back.  
>> I told them I wanted to expand it so I could give out a /48 per customer and 
>> that we had more than 65535 customers, which is the block we got; 65535 
>> /48s.  I didn't even account for our needs.
>> 
>> Without arguing the reasons, we will have to hand out /56s, rather than /48s 
>> because of this.  So, it's not all /48-unicorns, puppies and rainbows.
> 
> We have two types of customers - that that get assigned a /48, and those that 
> get assigned a /56.
> 
> Mark.

So why be stingy to the second class? Why not just assign everyone a /48?

Owen

Re: IPv6 and CDN's

[Owen DeLong via NANOG]

> On Nov 28, 2021, at 15:51 , Mark Andrews  wrote:
> 
> 
> 
>> On 29 Nov 2021, at 09:41, scott  wrote:
>> 
>> 
>> On 11/28/2021 9:47 AM, Owen DeLong via NANOG wrote:
>>> Why not properly assign /48s to customers and /40s to cities?
>>> --
>> 
>> Side note: I recently tried to get /48 per customer with ARIN on repeated 
>> emails and they refused.  We were already given an IPv6 block a while back.  
>> I told them I wanted to expand it so I could give out a /48 per customer and 
>> that we had more than 65535 customers, which is the block we got; 65535 
>> /48s.  I didn't even account for our needs.
>> 
>> Without arguing the reasons, we will have to hand out /56s, rather than /48s 
>> because of this.  So, it's not all /48-unicorns, puppies and rainbows.
>> 
>> scott
> 
> Looks like a policy omission.  You should be able to grow the per customer 
> allocation up to /48 per customer.
> One shouldn’t be stuck with /56 because one made a bad choice of prefix size 
> initially.

There is definitely something wrong here… Policy clearly states that you should 
be able to obtain an allocation large enough to provide /48s to all your 
customers if you so choose.

In fact, it is generally quite generous beyond that point.

Owen

Re: Theorical question about cyclic dependency in IRR filtering

[Christopher Morrow]

On Mon, Nov 29, 2021 at 8:14 AM Job Snijders via NANOG 
wrote:

> Hi Anurag,
>
> Circular dependencies definitely are a thing to keep in mind when
> designing IRR and RPKI pipelines!
>
> In the case of IRR: It is quite rare to query the RIR IRR services
> directly. Instead, the common practise is that utilities such as bgpq3,
> peval, and bgpq4 query “IRRd” (https://IRRd.net) instances at for example
> whois.radb.net and rr.ntt.net. You can verify this with tcpdump. These
> IRRd instances serve as intermediate caches, and will continue to serve old
> cached data in case the origin is down. This phenomenon in the global IRR
> deployment avoids a lot of potential for circular dependencies.
>
> Also, some organisations use threshold checks before deploying new
> IRR-based filters to reduce risk of “misfiring”.
>
>
beyond just 'did the filter deployed change by +/- X%'
you probably don't want to deploy content if you can't actually talk to the
source... which was anurag's proposed problem.

I suppose there are a myriad of actual failure modes though ;) and we'll
always find more as deployments progress... hurray?

Re: Theorical question about cyclic dependency in IRR filtering

[Job Snijders via NANOG]

Hi Anurag,

Circular dependencies definitely are a thing to keep in mind when designing
IRR and RPKI pipelines!

In the case of IRR: It is quite rare to query the RIR IRR services
directly. Instead, the common practise is that utilities such as bgpq3,
peval, and bgpq4 query “IRRd” (https://IRRd.net) instances at for example
whois.radb.net and rr.ntt.net. You can verify this with tcpdump. These IRRd
instances serve as intermediate caches, and will continue to serve old
cached data in case the origin is down. This phenomenon in the global IRR
deployment avoids a lot of potential for circular dependencies.

Also, some organisations use threshold checks before deploying new
IRR-based filters to reduce risk of “misfiring”.


The RPKI case is slightly different: the timers are far more aggressive
compared to IRR, and until “Publish in Parent” (RFC 8181) becomes common
place, there are more publication points, thus more potential for operators
to paint themselves into a corner.

Certainly, in the case of RPKI, all Publication Point (PP) operators need
to take special care to not host CAs which have the PP’s INRs listed as
subordinate resources inside the PP.

See RFC 7115 Section 5 for more information: “Operators should be aware
that there is a trade-off in placement of an RPKI repository in address
space for which the repository’s content is authoritative. On one hand, an
operator will wish to maximize control over the repository. On the other
hand, if there are reachability problems to the address space, changes in
the repository to correct them may not be easily access by others”

Ryan Sleevi once told me: "yes, it strikes me that you should prevent
self-compromise from being able to perpetually own yourself, by limiting an
attacker’s ability to persist beyond remediation."

A possible duct tape approach is outlined at
https://bgpfilterguide.nlnog.net/guides/slurm_ta/
However, I can’t really recommend the SLURM file approach. Instead, RPKI
repository operators are probably best off hosting their repository
*outside* their own address space.

Just like with Authoritative DNS servers, make sure you also can serve your
records via a competitor! :-)

For example, if ARIN moved one of their three publication point clusters
into address space managed by any of the other four RIRs, some risk would
be reduced.

Kind regards,

Job

On Mon, 29 Nov 2021 at 13:37, Anurag Bhatia  wrote:

> Hello everyone,
>
> While discussing IRR on some groups recently, I was thinking if there can
> be (and if there is) cycling dependency in filtering where IRR (run by
> whoever APNIC, RIPE, RADB etc) uses some upstream and accepts only routes
> with existing & valid route object.
>
>
>
> So hypothetical case (can apply to any IRR):
>
>
>1. APNIC registry source is whois.apnic.net and points
>to 202.12.28.136 / 2001:dc0:1:0:4777::136. The aggregate of both these has
>a valid route object at the APNIC registry itself.
>
>2. Their upstreams say AS X, Y and Z have tooling in place to
>generate and push filters by checking all popular IRRs. All is well till
>this point.
>
>3. Say APNIC has some server/service issue for a few mins and X Y and
>Z are updating their filters at the same time. They cannot contact
>whois.apnic.net and hence miss generating filters for all APNIC IRR
>hosted prefixes.
>
>4. X, Y and  Z drop APNIC prefixes including those of IRR & the loop
>goes on from this point onwards.
>
>
> So my question is: Can that actually happen?
> If not, do X, Y and Z and possible all upstreams till default-free zone
> treat these prefixes in a special manner to avoid such loop in resolution?
>
>
>
>
> Thanks!
>
> --
> Anurag Bhatia
> anuragbhatia.com
>

Theorical question about cyclic dependency in IRR filtering

[Anurag Bhatia]

Hello everyone,

While discussing IRR on some groups recently, I was thinking if there can
be (and if there is) cycling dependency in filtering where IRR (run by
whoever APNIC, RIPE, RADB etc) uses some upstream and accepts only routes
with existing & valid route object.



So hypothetical case (can apply to any IRR):


   1. APNIC registry source is whois.apnic.net and points to 202.12.28.136
   / 2001:dc0:1:0:4777::136. The aggregate of both these has a valid route
   object at the APNIC registry itself.

   2. Their upstreams say AS X, Y and Z have tooling in place to
   generate and push filters by checking all popular IRRs. All is well till
   this point.

   3. Say APNIC has some server/service issue for a few mins and X Y and Z
   are updating their filters at the same time. They cannot contact
   whois.apnic.net and hence miss generating filters for all APNIC IRR
   hosted prefixes.

   4. X, Y and  Z drop APNIC prefixes including those of IRR & the loop
   goes on from this point onwards.


So my question is: Can that actually happen?
If not, do X, Y and Z and possible all upstreams till default-free zone
treat these prefixes in a special manner to avoid such loop in resolution?




Thanks!

-- 
Anurag Bhatia
anuragbhatia.com

RE: IPv6 and CDN's

[Jean St-Laurent via NANOG]

I remember when I was a junior in a major NOC, we had this management host with 
a local hosts file for all critical components. 

 

Probably worth reviewing some old school techniques. 

 

If you can automate your gazillion routers business, you probably can also 
automate a couple of hosts file.

 

Jean

 

From: NANOG  On Behalf Of Baldur 
Norddahl
Sent: November 29, 2021 4:22 AM
To: NANOG 
Subject: Re: IPv6 and CDN's

 

 

man. 29. nov. 2021 02.12 skrev Masataka Ohta mailto:mo...@necom830.hpcl.titech.ac.jp> >:



> The only way to truly reduce Opex at scale is automation.

Automation by what? DNS?

Masataka Ohta

 

 

Most of our customers are provisioned by Radius. The remaining are configured 
by scripting using Netconf. 

 

We use DNS to document the network. If our DNS was down and I need to connect 
to a router in some city, do you really expect me to remember the IP address? I 
would have to look it up and our chosen database for that happens to be DNS. It 
has some obvious advantages. 

 

Regards 

 

Baldur 

Re: IPv6 and CDN's

[Baldur Norddahl]

man. 29. nov. 2021 02.12 skrev Masataka Ohta <
mo...@necom830.hpcl.titech.ac.jp>:

>
>
> > The only way to truly reduce Opex at scale is automation.
>
> Automation by what? DNS?
>
> Masataka Ohta
>


Most of our customers are provisioned by Radius. The remaining are
configured by scripting using Netconf.

We use DNS to document the network. If our DNS was down and I need to
connect to a router in some city, do you really expect me to remember the
IP address? I would have to look it up and our chosen database for that
happens to be DNS. It has some obvious advantages.

Regards

Baldur

>