Re: [dns-operations] root? we don't need no stinkin' root!

[Mark Allman]

> Still, I believe that a small resolver instance only needs a few
> DNS queries to root (per TTL), so switching everyone to always
> transferring the whole root should increase the total traffic
> considerably,

An anecdote here ...

I crunched a day's worth of DNS traffic originated at ICSI (which is
pretty much a "small resolver instance") from mid-Oct (which just
happened to be handy).  The entire root zone file would be ~725
full-size TCP packets.  Our two main DNS resolvers together sent
nearly 63K queries to the root nameservers.

I am not arguing either of these is onerous for us.  But, the notion
that snarfing a MB of zone file is somehow a considerable increase
in traffic vs. what we impose on the roots now seems dubious.

allman
___
dns-operations mailing list
dns-operations@lists.dns-oarc.net
https://lists.dns-oarc.net/mailman/listinfo/dns-operations

Re: [dns-operations] root? we don't need no stinkin' root!

[Mark Allman]

>> On 11 Dec 2019, at 12:51, Stephane Bortzmeyer  wrote:
>>
>> IMHO, this is by far the biggest issue with your proposal: TLDs change
>> from one technical operator to another and, when it happens, all name
>> servers change at once.
>
> That’s not correct.
>
> In principle, they could all change at once, In reality, they
> don’t.

I wondered about this.  So, I crunched across our corpus of root
zone files, which spans from Apr 28 2009 to now (I stopped crunching
on Dec 11 2019).  We have one zone file per day (we miss a day here
or there due to glitches, but not many, the corpus is 3,500 days
long).  I found:

  - There are 1,578 TLDs that appear in the root zone file at some
point in the last 10 years.

  - Of those, 1,139 (72.2%) have at least one nameserver (by IP)
that is constant over the entire period the TLD is active.  (I'd
have not guessed it was this high!)

  - For the remaining 439 TLDs, for each day the TLD was active I
calculated how many days into the future would it be until none
of the current set of nameservers (by IP) would no longer be
listed.  For each TLD I took the minimum value.  That shows:

+ 173 TLDs (or 11.0% of all TLDs) at some point have a switch as
  Stephane describes.  I.e., there are no common IP addresses in
  the nameserver set between day X and day X+1.

+ Another 107 TLDs (or 6.8% of all TLDs) had a point where a
  zone file become outdated more in [2,7] days.

+ 75 TLDs (or 4.8% of all TLDs) had a point where a zone file
  become outdated in [8,30] days.

+ 84 TLDs (or 5.4% of all TLDs) only ever became outdated after
  more than 30 days.

FWIW.

allman
___
dns-operations mailing list
dns-operations@lists.dns-oarc.net
https://lists.dns-oarc.net/mailman/listinfo/dns-operations

Re: [dns-operations] root? we don't need no stinkin' root!

[Mark Allman]

Hi Stephane!

Thanks for the note.  I have been thinking about this point a bit.

> IMHO, this is by far the biggest issue with your proposal: TLDs
> change from one technical operator to another and, when it
> happens, all name servers change at once. Should your proposal be
> implemented, we would have to debug problems with root zones
> outdated by 1, 2, 3 months and some TLD working for some resolvers
> but not all. (True, we already have resolver-specific issues, but
> I think it would aggravate the problem.)
>
> This would have anti-competitive consequences, discouraging TLD
> holders to swap technical operators.

I get your point.  But, it's predicated on resolvers using a local
zone file that is "outdated by 1, 2, 3 months".  And, I can't really
quite figure out if that is realistic or, if it is, how much we
should care.  A few comments:

  - To be clear, the scenario is that someone has taken the step to
grab a root zone file and use it in the resolution process, but
then (a) have no effective update process to update the zone file or
(b) have a process that goes bad some point without anyone
noticing.  This would without a doubt happen if we shut down the
root nameservers and forced everyone to use local replicas.
But, I am hard pressed to convince myself it'd happen a lot or
that we should care about (/engineer around) such shoddy
operations.

  - Seems like if we took this approach to run without root
nameservers that we'd first design software to update local
replicas in an automated and robust fashion.  In other words,
this isn't something every operator is going to have to piece
together themselves.

  - To the extent that this is an issue, RFC 7706-style local roots
already have it.  So, this is not a new issue---but, the issue
might be bigger if more local roots existed.

  - Finally, I think there is some incentive to stay up-to-date.  We
do see problems when soft state becomes de-facto hard state
because it doesn't change except for once every eon or so.
E.g., the root hints file.  But, since the root zone file does
change pretty constantly (albeit in small ways), there is an
incentive to keep up, it seems to me.

I guess in sum, after some thought I am not ready to buy that this
situation you describe will constitute a big enough phenomenon to
exert anti-competitive pressure on TLD holders.

allman
___
dns-operations mailing list
dns-operations@lists.dns-oarc.net
https://lists.dns-oarc.net/mailman/listinfo/dns-operations

Re: [dns-operations] root? we don't need no stinkin' root!

[Tony Finch]

Matthew Pounsett  wrote:

> I have yet to witness anyone splitting the NS change up into multiple
> IANA requests.

Amazon did it with their TLDs earlier this year, which is notable because
there were/are so many of them. There have been plenty of other examples
of staged switch-overs.

https://github.com/fanf2/saveroot/commit/9fca29c21fbd1ce61fe56308a86da4bc10f6c18e
(2019-04-23 13:33)

https://github.com/fanf2/saveroot/commit/a35e56c2801df2e3aa7cd4cbc0b3760d484ac5a3
(2019-04-09 18:33)

Sadly github refuses to display root zone diffs, but if you want to clone
a few GB it comes with an ns-only summarizer, amongst others.

Tony.
-- 
f.anthony.n.finchhttp://dotat.at/
Shannon, South Rockall: Westerly or southwesterly 6 to gale 8, perhaps severe
gale 9 later. Very rough or high. Rain or showers. Good, occasionally poor.
___
dns-operations mailing list
dns-operations@lists.dns-oarc.net
https://lists.dns-oarc.net/mailman/listinfo/dns-operations

Re: [dns-operations] root? we don't need no stinkin' root!

[Paul Vixie]

Florian Weimer wrote on 2019-12-11 12:16:

* Jason Livingood:


...


The real question is whether any distribution will be a substantial
improvement over what we have today with NSEC-based NXDOMAIN synthesis
for the root.  I doubt it.


i am +1 to this comment, but in a way that requires explanation.

not only is that assertion doubtful, as you say.

but also, there is absolutely no evidence in favour of it.

so it's not merely doubtful, but also, arbitrary.

--
P Vixie

___
dns-operations mailing list
dns-operations@lists.dns-oarc.net
https://lists.dns-oarc.net/mailman/listinfo/dns-operations

Re: [dns-operations] root? we don't need no stinkin' root!

[Matthew Pounsett]

On Wed, 11 Dec 2019 at 08:24, Jim Reid  wrote:

>
> In principle, they could all change at once, In reality, they don’t.
> 
>

This absolutely does happen.  I've been at the helm of several operator
changes on TLDs that saw all the NS records and glue change in the same
update.   I've seen several others change the same way.  I have yet to
witness anyone splitting the NS change up into multiple IANA requests.
With enough prep and testing it's fairly low risk to do all at once.

But, even if they were split up, the change would still happen on the order
of a few days, maybe two weeks tops if the operators are being extremely
careful and IANA is unusually busy/slow.  That's far too fast a change for
most resolvers to keep their root zone up to date by any current non-XFR
means.
___
dns-operations mailing list
dns-operations@lists.dns-oarc.net
https://lists.dns-oarc.net/mailman/listinfo/dns-operations

Re: [dns-operations] root? we don't need no stinkin' root!

[Florian Weimer]

* Jason Livingood:

> Seems like the answer then is to have the resolver check for updates
> more frequently. The file is tiny and so this is not in the least
> going to be resource-intensive. Just check every XX minutes.

I had hoped that we could use distribution update mechanisms for the
zone, like we do today for the initial set of priming targets for the
root, or for tzdata.  But that's clearly not enough.

The real question is whether any distribution will be a substantial
improvement over what we have today with NSEC-based NXDOMAIN synthesis
for the root.  I doubt it.
___
dns-operations mailing list
dns-operations@lists.dns-oarc.net
https://lists.dns-oarc.net/mailman/listinfo/dns-operations

Re: [dns-operations] root? we don't need no stinkin' root!

[Florian Weimer]

* Stephane Bortzmeyer:

> It doesn't matter. Everything is done in a few days. For a resolver
> updating its copy of the root every month, this is enough to break
> things.

Agreed.  A fancy distribution protocol would be needed instead.

(I see parallels here with compiler changes where people claim
“neglible performance overhead” when it's around 5%, which in reality
can translate into a reversal of many years of work on the
optimizers.)

___
dns-operations mailing list
dns-operations@lists.dns-oarc.net
https://lists.dns-oarc.net/mailman/listinfo/dns-operations

Re: [dns-operations] root? we don't need no stinkin' root!

[Paul Ebersman]

jreid> In principle, they could all change at once, In reality, they
jreid> don't.

dot> But they do. Vanuatu did yesterday, and I mentioned some other
dot> recent examples in this thread a couple of weeks ago:
dot> 
https://lists.dns-oarc.net/pipermail/dns-operations/2019-November/019486.html

Yup. Especially with DNSSEC, where a mix and match of signatures is a
complete nightmare, it's actually cleaner to have dual DS, both sets of
keys in both providers' zones but only one set of NS in the parent, not
mix and match.

See Shumon's draft on multiprovider. It will make your eyes bleed but is
so far the cleanest way we've found of doing provider changes too.

___
dns-operations mailing list
dns-operations@lists.dns-oarc.net
https://lists.dns-oarc.net/mailman/listinfo/dns-operations

Re: [dns-operations] root? we don't need no stinkin' root!

[Stephane Bortzmeyer]

On Wed, Dec 11, 2019 at 03:51:14PM +,
 Livingood, Jason  wrote 
 a message of 7 lines which said:

> Seems like the answer then is to have the resolver check for updates
> more frequently. The file is tiny and so this is not in the least
> going to be resource-intensive. Just check every XX minutes.

This assumes that all resolvers will be well-behaved. If it were the
case, we wouldn't need this proposal at all since proper caching and
RFC 8020 would keep junk local.

My concern is with the bad resolvers: they will have outdated copies
of the zone, that's for sure. The choice is between "damn them,
proceed anyway" (in the spirit of the DNS flag day) or "continue to
accomodate bad resolvers". The first option will be hard to sell to
the "stakeholders". I imagine the announcement "Root name service will
be discontinued on 1 november 2030. Check that your resolver correctly
implements RFC ."
___
dns-operations mailing list
dns-operations@lists.dns-oarc.net
https://lists.dns-oarc.net/mailman/listinfo/dns-operations

Re: [dns-operations] root? we don't need no stinkin' root!

[Livingood, Jason]

> It doesn't matter. Everything is done in a few days. For a resolver
updating its copy of the root every month, this is enough to break
things.

Seems like the answer then is to have the resolver check for updates more 
frequently. The file is tiny and so this is not in the least going to be 
resource-intensive. Just check every XX minutes.



___
dns-operations mailing list
dns-operations@lists.dns-oarc.net
https://lists.dns-oarc.net/mailman/listinfo/dns-operations

Re: [dns-operations] root? we don't need no stinkin' root!

[Tony Finch]

Jim Reid  wrote:
>
> In principle, they could all change at once, In reality, they don’t.

But they do. Vanuatu did yesterday, and I mentioned some other recent
examples in this thread a couple of weeks ago:
https://lists.dns-oarc.net/pipermail/dns-operations/2019-November/019486.html

Tony.
-- 
f.anthony.n.finchhttp://dotat.at/
work to the benefit of all___
dns-operations mailing list
dns-operations@lists.dns-oarc.net
https://lists.dns-oarc.net/mailman/listinfo/dns-operations

Re: [dns-operations] root? we don't need no stinkin' root!

[Rubens Kuhl]

> Em 11 de dez de 2019, à(s) 10:20:000, Jim Reid  escreveu:
> 
> 
> 
>> On 11 Dec 2019, at 12:51, Stephane Bortzmeyer  wrote:
>> 
>> IMHO, this is by far the biggest issue with your proposal: TLDs change
>> from one technical operator to another and, when it happens, all name
>> servers change at once.
> 
> That’s not correct.
> 
> In principle, they could all change at once, In reality, they don’t. When 
> making a change of this nature, established wisdom is to change half of the 
> NS records (or their glue), wait a few days to see that all is well and then 
> change the other half. I think IANA would try to persuade a TLD to do that if 
> they came with a proposal to change all of the TLD's NS records in one 
> transaction. Though if the TLD insisted, IANA would respect their choice.
> 
> Come to think of it, changing all of the NS records at once is generally a 
> bad idea for any zone. That would probably only make sense when all of the 
> existing name servers were dead or no longer serving the zone.
> 


Jim,

That's not of what RSPs (Registry Service Providers), ICANN GDD and ICANN IANA 
have been doing in RSP transitions. What has been working best is to double DS 
the zone with losing KSK and gaining KSK, have both RSPs sign each other ZSKs 
and NSs, and replace all DNS servers at gaining RSP, then losing RSP, then IANA.

One of such transitions in 2019 was .natura and the root zone history can show 
how it was done. I am polishing out a few tidbits in that change process and 
will publish the change process of that case as a template that serves well 
single-registrant TLD transitions.

Rubens




signature.asc
Description: Message signed with OpenPGP
___
dns-operations mailing list
dns-operations@lists.dns-oarc.net
https://lists.dns-oarc.net/mailman/listinfo/dns-operations

Re: [dns-operations] root? we don't need no stinkin' root!

[Stephane Bortzmeyer]

On Wed, Dec 11, 2019 at 01:20:13PM +,
 Jim Reid  wrote 
 a message of 22 lines which said:

> In principle, they could all change at once, In reality, they
> don’t. When making a change of this nature, established wisdom is to
> change half of the NS records (or their glue), wait a few days to
> see that all is well and then change the other half.

It doesn't matter. Everything is done in a few days. For a resolver
updating its copy of the root every month, this is enough to break
things.

___
dns-operations mailing list
dns-operations@lists.dns-oarc.net
https://lists.dns-oarc.net/mailman/listinfo/dns-operations

Re: [dns-operations] root? we don't need no stinkin' root!

[Jim Reid]

> On 11 Dec 2019, at 12:51, Stephane Bortzmeyer  wrote:
> 
> IMHO, this is by far the biggest issue with your proposal: TLDs change
> from one technical operator to another and, when it happens, all name
> servers change at once.

That’s not correct.

In principle, they could all change at once, In reality, they don’t. When 
making a change of this nature, established wisdom is to change half of the NS 
records (or their glue), wait a few days to see that all is well and then 
change the other half. I think IANA would try to persuade a TLD to do that if 
they came with a proposal to change all of the TLD's NS records in one 
transaction. Though if the TLD insisted, IANA would respect their choice.

Come to think of it, changing all of the NS records at once is generally a bad 
idea for any zone. That would probably only make sense when all of the existing 
name servers were dead or no longer serving the zone.


___
dns-operations mailing list
dns-operations@lists.dns-oarc.net
https://lists.dns-oarc.net/mailman/listinfo/dns-operations

Re: [dns-operations] root? we don't need no stinkin' root!

[Stephane Bortzmeyer]

On Mon, Dec 02, 2019 at 10:17:30AM -0500,
 Mark Allman  wrote 
 a message of 36 lines which said:

> Obviously, there could be a more comprehensive analysis, but I think
> that gives some idea about how stable the root zone file is in
> practice.

IMHO, this is by far the biggest issue with your proposal: TLDs change
from one technical operator to another and, when it happens, all name
servers change at once. Should your proposal be implemented, we would
have to debug problems with root zones outdated by 1, 2, 3 months and
some TLD working for some resolvers but not all. (True, we already
have resolver-specific issues, but I think it would aggravate the
problem.)

This would have anti-competitive consequences, discouraging TLD
holders to swap technical operators.

___
dns-operations mailing list
dns-operations@lists.dns-oarc.net
https://lists.dns-oarc.net/mailman/listinfo/dns-operations

Re: [dns-operations] root? we don't need no stinkin' root!

[Stephane Bortzmeyer]

On Wed, Nov 27, 2019 at 10:38:32AM -0500,
 Keith Mitchell  wrote 
 a message of 37 lines which said:

> On garbage-collecting crap traffic, it's worth looking at AS112.

There have been a proposal at IETF to use AS112 as a sinkhole for
"special" TLDs such as .local or .home, which are responsible for some
part of the junk traffic at the root


It got no support, partly because of the privacy issues: leaked
requests would go to a random, unknown, AS112 operator. (Note this is
also an argument against Paul Vixie's "root on AS112-like" proposal.)

___
dns-operations mailing list
dns-operations@lists.dns-oarc.net
https://lists.dns-oarc.net/mailman/listinfo/dns-operations

Re: [dns-operations] root? we don't need no stinkin' root!

[Tony Finch]

Mark Allman  wrote:
>
> Obviously, there could be a more comprehensive analysis

I have a 3.5GB git repository containing 14500 commits with versions of
the root zone going back to March 2014, if anyone wants something to
analyse. I also have a BIND root.jnl file (140MB gzipped) which appears to
start with SOA serial number 2005072701. I'm sure someone else has a less
crappy archive...

Tony.
-- 
f.anthony.n.finchhttp://dotat.at/
Rockall, Malin: Southwest 6 to gale 8, perhaps severe gale 9 later. Very rough
or high. Squally showers, then rain. Moderate or poor, occasionally good.
___
dns-operations mailing list
dns-operations@lists.dns-oarc.net
https://lists.dns-oarc.net/mailman/listinfo/dns-operations

Re: [dns-operations] root? we don't need no stinkin' root!

[Paul Vixie]

David Conrad wrote on 2019-12-04 08:31:
[Sorry for the slow response — US holidays and a resolution not to look 
at my computer over said holidays got in the way]

...
Further, the root servers have to respond to pretty much every DNS query 
that gets thrown at them, both UDP and TCP. A root zone distribution 
service would only need respond to AXFR/IXFR requests over TCP (and this 
could even be gated by whitelisting/blacklisting).


while i agree with this message on all points, i'd like to clarify that 
the ixfr/axfr protocol begins with an SOA query, and there is no current 
requirement that this be done via TCP. a TCP-mostly ze distribution 
service would be unwise to simply ignore TCP -- rather, it would be best 
to answer UDP with TC=1 regardless of the query content. the ixfr/axfr 
protocol also relies on NOTIFY, which is also a UDP-mostly protocol.


of course, a revised protocol could be specified for any given service 
such as a "root zone distribution service" which required that only TCP 
be used, for both the initial SOA query, and NOTIFY if any, and then the 
transfer (ixfr or axfr.) in that event, the above clarification would be 
mooted.


--
P Vixie

___
dns-operations mailing list
dns-operations@lists.dns-oarc.net
https://lists.dns-oarc.net/mailman/listinfo/dns-operations

Re: [dns-operations] root? we don't need no stinkin' root!

[David Conrad]

[Sorry for the slow response — US holidays and a resolution not to look at my 
computer over said holidays got in the way]

> On Nov 28, 2019, at 12:42 AM, Petr Špaček  wrote:
> On 27. 11. 19 21:49, David Conrad wrote:
>> Petr,
>> 
>>> I think there is even more fundamental problem:
>>> Someone has to pay operational costs of "the new system”.
>> 
>> The “new system” is simply the existing network of resolvers, augmented to 
>> have the root zone.  As far as I can tell, the operational cost would be in 
>> (a) ensuring the resolver is upgraded to support obtaining the root zone and 
>> (b) dealing with the fetch of the root zone with some frequency.
> 
> I hypothetise that in the end requirements for "the new system for root zone 
> distribution" will be fairly close to current requirements for current DNS 
> root system... so I do not see where the cost reduction comes from.

Root zone distribution is on different timescales than root query service.  
Even if the root zone distribution service relies only on AXFR, an effective 
DDoS of that service based on SOA timers would need to be maintained for far 
longer than a DDoS against root service based on cache TTLs.  And, of course, 
folks have already been looking at distributing the root zone via stuff other 
than AXFR (e.g., HTTPS).

Further, the root servers have to respond to pretty much every DNS query that 
gets thrown at them, both UDP and TCP. A root zone distribution service would 
only need respond to AXFR/IXFR requests over TCP (and this could even be gated 
by whitelisting/blacklisting).

Regards,
-drc
(Speaking for myself, not any organization I may be affiliated with)





signature.asc
Description: Message signed with OpenPGP
___
dns-operations mailing list
dns-operations@lists.dns-oarc.net
https://lists.dns-oarc.net/mailman/listinfo/dns-operations

Re: [dns-operations] root? we don't need no stinkin' root!

[Thomas, Matthew via dns-operations]

--- Begin Message ---
Verisign has a long track record of working with trusted researchers, 
universities, and an array of partners to publish a significant amount of 
research and peer-reviewed work on the topic of name collisions, sharing 
insights from our unique observation space as a root server operator – see [1] 
for some examples. We’ve also invested extensively to collect and perform 
longitudinal analysis on various aspects of this data and make the findings 
available to the community for consideration, as illustrated in the long list 
of citations available at [1] on name collisions, as well as other more recent 
topics (e.g., to inform KSK roll planning [2]).  If you have technical concerns 
with that work, we welcome your feedback.

As Duane noted, we do NOT monetize root server data.

Conflating those things with the RZM function is not helpful in this context, 
and to the extent you want to access RZM-related data, ICANN / PTI has it and 
is very transparent with it already, IMO.

You’re certainly entitled to your opinion about the results of the studies 
we’ve worked on, but your comments about the motivation behind those studies 
are wrong, unsupported by facts, and frankly out of bounds.  We won’t have 
anything else to say on this matter.

Matt Thomas
Verisign

[1] https://mm.icann.org/pipermail/ncap-discuss/2019-April/08.html
[2] 
http://www.circleid.com/posts/20191126_recognizing_lessons_learned_from_the_first_dnssec_key_rollover/


From: dns-operations  on behalf of Rubens 
Kuhl 
Date: Friday, November 29, 2019 at 8:38 PM
To: "dns-operations@lists.dns-oarc.net" 
Subject: [EXTERNAL] Re: [dns-operations] root? we don't need no stinkin' root!



The data could have monetary value.  Passwords that are otherwise
difficult to come by might be leaking.

Hi Florian,

I can assure you that Verisign does not monetize the root server data.  If
any other operators do, I'm not aware of it.

We do utilize root server data for research purposes from time-to-time.
Recent examples include the KSK rollover and name collisions.  Less recent
examples include understanding TTL/caching behavior and preparing for the
root ZSK size increase.  When DDoS attacks happen, we often analyze the
data to see if we can understand how and why it happened, and to be better
prepared for the next one.


Note that the two paragraphs above contradict each other. The current RZM is 
known to use root server data as anti-competitive measures against new TLD 
operators with the label of name collision studies, including making studies 
that other parties can't reproduce due to being limited to DITL data.


Rubens


--- End Message ---
___
dns-operations mailing list
dns-operations@lists.dns-oarc.net
https://lists.dns-oarc.net/mailman/listinfo/dns-operations

Re: [dns-operations] root? we don't need no stinkin' root!

[Viktor Dukhovni]

On Dec 2, 2019, at 3:09 PM, Mark Allman  wrote:

> > For reachability, it is not enough to consider the nameserver IP
> > addresses, did you also check DS record stability?
>
> I did not.  I was more interested in understanding how much the
> infrastructure churned.  To me the crypto stuff is config that we
> can more readily hack.   And, while I didn't scrutinize your long
> list, I sort of skimmed and it seems the changes may not always keep
> for a year, but are generally more than "a few days".

Yes, the key question is how long before a new DS RR is added is
does it become the only way to authenticate the TLD in question.
And I would not be surprised to find that this is many cases a
matter of days.  So a root zone that's a day or two old is likely
generally sufficient, but much more than that, and some TLDs could
become unreachable.

Of course if the zone distribution model changes, and resolvers
are expected to have somewhat stale root zone copies, then perhaps
TLDs would have to take that into account and roll out changes more
slowly.  For example, the previous DS would have to be good for at
least ~14 days after a new one is added, and outdated TLDs KSK would
only be eligible for retirement after a 14 day overlap...

Can we reasonably expect TLD operators to live within such constraints
and not mess up too often?

-- 
Viktor.
___
dns-operations mailing list
dns-operations@lists.dns-oarc.net
https://lists.dns-oarc.net/mailman/listinfo/dns-operations

Re: [dns-operations] root? we don't need no stinkin' root!

[Mark Allman]

> For reachability, it is not enough to consider the nameserver IP
> addresses, did you also check DS record stability?

I did not.  I was more interested in understanding how much the
infrastructure churned.  To me the crypto stuff is config that we
can more readily hack.   And, while I didn't scrutinize your long
list, I sort of skimmed and it seems the changes may not always keep
for a year, but are generally more than "a few days".

However, that said, ...

> In any case, it seems likely that have a root zone that is a year
> out of date would be problematic for many TLDs.

My point wasn't to argue a zone file that is a year out of date is
somehow OK.  Even without the DNSSEC bits, I think not being able to
reach 50 TLDs is not OK.  However, the infrastructure seems slowly
changing.  And, that has some ramifications.  And, we might be able
to leverage those if we wanted to ...

  - E.g., if we wanted to extend the TTL that doesn't seem like it
would be a big problem.

  - E.g., if *in a pinch* we had to use an expired, but not too old
root zone file to reach a TLD server because we couldn't fetch a
current zone file that would likely be OK, too.

allman
___
dns-operations mailing list
dns-operations@lists.dns-oarc.net
https://lists.dns-oarc.net/mailman/listinfo/dns-operations

Re: [dns-operations] root? we don't need no stinkin' root!

[Viktor Dukhovni]

On Mon, Dec 02, 2019 at 10:17:30AM -0500, Mark Allman wrote:

> Not a direct answer to your question, but a couple empirical bits from
> the paper that started this thread ...
> 
> We analyzed a snapshot of the root zone file from each day in
> April, 2019.  On the first of the month the root zone included
> 1,532~TLDs and one was deleted during the month.  Of the TLDs,
> all but five have at least one nameserver (by IP address) that
> is constant for the entire month.  That is, if a recursive
> resolver used a root zone file that was out of date by one
> month, 99.6\% of the TLDs would remain accessible.  The five
> TLDs that do not have a constant nameserver for the entire month
> are run by NeuStar and use a slowly rotating set of IP addresses
> for the TLD nameservers.  The overlap ensures that a root zone
> file that is no more than 14~days out of date will ensure
> constant TLD reachability.  Further, comparing the root zone
> files on April 1, 2018 and April 1, 2019 we find that all but
> 50~TLDs (3.3\%) would still retain reachability with a root zone
> file that is a year out of date.
> 
> Obviously, there could be a more comprehensive analysis, but I think
> that gives some idea about how stable the root zone file is in
> practice.

For reachability, it is not enough to consider the nameserver IP
addresses, did you also check DS record stability?  If the DS
RRs change, reachability of a bogus zone is not that useful.

Below is a history of TLD DS record updates (hashes truncated to 8
octets) since 01-Jan-2019. [ Start dates of "2017-10-19" are upper
bounds, that's the start date of the DANE/DNSSEC survey dataset. ]

There have been 647 TLD DS RR updates in the root zone this year, 265
additions and 382 deletions.  Sadly, I don't have the data needed to
determine how long one could get by with a stale DS RRset, because I
only track the DNSKEY RRset, but not its RRSIGs, so I don't know which
KSKs were "active" on a given day.  In any case, it seems likely that
have a root zone that is a year out of date would be problematic for
many TLDs.

-- 
Viktor.

TLDalg hash...  startend 
--+---+-+---+--
lancaster   8  eb48f6c2717f808e  2017-10-19  2019-01-07
leclerc 8  cbefca15a28d0fa2  2017-10-19  2019-01-18
ma  8  69c0229c3b381976  2018-01-21  2019-01-18
lu  8  b3d09ea4961045f6  2019-01-18  
mma 8  cc5b8a8fee878f72  2019-01-18  
ma  8  3b9f33c27c469841  2019-01-18  
lu  8  aa8d5290d34665a8  2018-01-09  2019-01-20
aquarelle   8  44b66bce18f1df2d  2019-01-24  
playstation 8  db8186c397cd0ccc  2017-10-25  2019-01-25
playstation 8  bf07dc2b242e3888  2019-01-25  2019-07-13
mma 8  200f10b2987855e5  2017-10-19  2019-01-26
at  8  8f9d0acd78ff2f88  2019-01-30  
berlin  8  b0d792359cb13ab1  2017-10-19  2019-01-30
aquarelle   8  f46778701f505d01  2017-10-19  2019-01-30
berlin  8  b1ddc962a9293cda  2019-01-30  
hamburg 8  8bdc8642f1c02759  2019-01-30  
hamburg 8  4a48334ef87d7fc1  2017-10-19  2019-01-30
versicherung8  991da886cbc5da51  2017-10-19  2019-01-30
versicherung8  925ee924b5bc6876  2019-01-30  
ikano   8  e041cebaf350d90d  2019-02-01  
ikano   8  f41b10d951dc5ee6  2017-10-19  2019-02-01
bridgestone 8  24bb0833fb1f6774  2017-10-19  2019-02-02
hyundai 8  8baa367547efe464  2019-02-02  2019-07-11
bridgestone 8  5d15518740110787  2019-02-02  2019-07-11
hyundai 8  634b1d40dbb1cafc  2017-10-19  2019-02-02
firestone   8  a8c1d22a8eaee432  2017-10-19  2019-02-02
firestone   8  e185d089ab8b3d90  2019-02-02  2019-07-11
monster 7  9dd1cfcf93b8d7f4  2019-02-02  
monster 7  7de5a740d6e45c95  2019-02-02  
kia 8  4926fd32596fb35b  2017-10-19  2019-02-02
kia 8  b6401e5e77ba1d67  2019-02-02  2019-07-11
sharp   8  cfb736a0d3bbb6c0  2017-11-21  2019-02-02
sharp   8  491341d337c8e7a2  2019-02-02  2019-07-13
softbank8  5dfb86188d94f902  2017-12-22  2019-02-02
softbank8  533c77edfb9e60e2  2019-02-02  2019-07-14
au  8  9eda3ec27d09500a  2017-12-07  2019-02-06
au  8  df3d2f347c04eae8  2017-12-07  2019-02-06
si  8  4a69ad581edc6776  2017-12-22  2019-02-06
si  8  8134257f1cf79187  2019-02-06  

Re: [dns-operations] root? we don't need no stinkin' root!

[Mark Allman]

Hi Florian!

> What's the change rate for the root zone?  If there is a full
> transition of the name server addresses for a zone, how long does
> it typically take from the first change to the completion of the
> sequence of changes?

Not a direct answer to your question, but a couple empirical bits
from the paper that started this thread ...

We analyzed a snapshot of the root zone file from each day in
April, 2019.  On the first of the month the root zone included
1,532~TLDs and one was deleted during the month.  Of the TLDs,
all but five have at least one nameserver (by IP address) that
is constant for the entire month.  That is, if a recursive
resolver used a root zone file that was out of date by one
month, 99.6\% of the TLDs would remain accessible.  The five
TLDs that do not have a constant nameserver for the entire month
are run by NeuStar and use a slowly rotating set of IP addresses
for the TLD nameservers.  The overlap ensures that a root zone
file that is no more than 14~days out of date will ensure
constant TLD reachability.  Further, comparing the root zone
files on April 1, 2018 and April 1, 2019 we find that all but
50~TLDs (3.3\%) would still retain reachability with a root zone
file that is a year out of date.

Obviously, there could be a more comprehensive analysis, but I think
that gives some idea about how stable the root zone file is in
practice.

allman
___
dns-operations mailing list
dns-operations@lists.dns-oarc.net
https://lists.dns-oarc.net/mailman/listinfo/dns-operations

Re: [dns-operations] root? we don't need no stinkin' root!

[Rubens Kuhl]

> On 30 Nov 2019, at 14:31, Keith Mitchell  wrote:
> 
> On 11/29/19 8:32 PM, Rubens Kuhl wrote:
> 
>> including making studies that other parties can't reproduce due to
>> being limited to DITL data.
> 
> DITL data is available to any party who signs an OARC Data Sharing
> agreement.


Keith,

DITL data is the gold standard for such analysis. What I mentioned is that the 
RZM publishes studies that can't be reproduced due to use of other data(not 
DITL) only accessible to the RZM, and also try implying that DITL data doesn't 
capture reality due to being short lived, which is kinda like "I know better 
but I can't tell you why". 





Rubens





___
dns-operations mailing list
dns-operations@lists.dns-oarc.net
https://lists.dns-oarc.net/mailman/listinfo/dns-operations

Re: [dns-operations] root? we don't need no stinkin' root!

[Keith Mitchell]

On 11/29/19 8:32 PM, Rubens Kuhl wrote:

> including making studies that other parties can't reproduce due to
> being limited to DITL data.

DITL data is available to any party who signs an OARC Data Sharing
agreement.

Keith
___
dns-operations mailing list
dns-operations@lists.dns-oarc.net
https://lists.dns-oarc.net/mailman/listinfo/dns-operations

Re: [dns-operations] root? we don't need no stinkin' root!

[Rubens Kuhl]

>> 
>> The data could have monetary value.  Passwords that are otherwise
>> difficult to come by might be leaking.
> 
> Hi Florian,
> 
> I can assure you that Verisign does not monetize the root server data.  If
> any other operators do, I'm not aware of it.
> 
> We do utilize root server data for research purposes from time-to-time.
> Recent examples include the KSK rollover and name collisions.  Less recent
> examples include understanding TTL/caching behavior and preparing for the
> root ZSK size increase.  When DDoS attacks happen, we often analyze the
> data to see if we can understand how and why it happened, and to be better
> prepared for the next one.


Note that the two paragraphs above contradict each other. The current RZM is 
known to use root server data as anti-competitive measures against new TLD 
operators with the label of name collision studies, including making studies 
that other parties can't reproduce due to being limited to DITL data. 


Rubens


___
dns-operations mailing list
dns-operations@lists.dns-oarc.net
https://lists.dns-oarc.net/mailman/listinfo/dns-operations

Re: [dns-operations] root? we don't need no stinkin' root!

[Jeremy Harris]

On 29/11/2019 19:34, Tony Finch wrote:
> Attackers can get a small amplification from SYN/ACK retries, and this is
> being used in the wild.
> 
> https://www.darkreading.com/attacks-breaches/new-ddos-attacks-leverage-tcp-amplification-/d/d-id/1336339

This isn't small.  It'd be good to know _what_ is so broken:

"many devices on the Internet can be manipulated to retransmit more than
5,000 SYN-ACK packets in 60 seconds"

-- 
Cheers,
  Jeremy
___
dns-operations mailing list
dns-operations@lists.dns-oarc.net
https://lists.dns-oarc.net/mailman/listinfo/dns-operations

Re: [dns-operations] root? we don't need no stinkin' root!

[Viktor Dukhovni]

On Fri, Nov 29, 2019 at 09:17:32PM +0100, Tom Ivar Helbekkmo wrote:

> > Attackers can get a small amplification from SYN/ACK retries, and this
> > is being used in the wild.
> 
> Can you actually implement a TCP stack without that possibility?

Not in general, but if for a particular service the client always sends the
first application data message, then the server could make this known to the
TCP stack, and in that case the SYN+ACK retransmissions could be avoided,
because the client would retrasmit the SYN if the SYN+ACK was lost, and would
otherwise retransmit its initial message.

When the server sends the first message, it sadly needs to retransmit SYN+ACK,
because when the client's ACK is lost the client is otherwise stuck with an
apparently established connection, and nothing ever sent from the server.

Thus, for example, SMTP servers (over TCP) have no choice but to retransmit
SYN+ACK.

There is not presently a mechanism in any TCP stacks I know of for a server
to indicate that SYN+ACK retransmisison is optional because the client will
send first.

-- 
Viktor.
___
dns-operations mailing list
dns-operations@lists.dns-oarc.net
https://lists.dns-oarc.net/mailman/listinfo/dns-operations

Re: [dns-operations] root? we don't need no stinkin' root!

[Viktor Dukhovni]

On Fri, Nov 29, 2019 at 07:34:56PM +, Tony Finch wrote:

> Viktor Dukhovni  wrote:
> >
> > refection of answers to forged source IPs is not available with TCP
> 
> Attackers can get a small amplification from SYN/ACK retries, and this is
> being used in the wild.
> 
> https://www.darkreading.com/attacks-breaches/new-ddos-attacks-leverage-tcp-amplification-/d/d-id/1336339

Thanks for the link, appreciated.  Perhaps the answer is that a future root
zone retrieval service should be available only via QUIC with always-on address
validation:

https://tools.ietf.org/html/draft-ietf-quic-transport-24#section-8.1.1
https://tools.ietf.org/html/draft-ietf-quic-transport-24#section-8.1

This should also facilitate data integrity.

-- 
Viktor.
___
dns-operations mailing list
dns-operations@lists.dns-oarc.net
https://lists.dns-oarc.net/mailman/listinfo/dns-operations

Re: [dns-operations] root? we don't need no stinkin' root!

[Tony Finch]

Tom Ivar Helbekkmo  wrote:
>
> Can you actually implement a TCP stack without that possibility?

I vaguely speculate that it would be better to rely on SYN retries and
abolish SYN/ACK retries, but I have no idea what it might break.

Tony.
-- 
f.anthony.n.finchhttp://dotat.at/
safeguard the balance of nature and the environment
___
dns-operations mailing list
dns-operations@lists.dns-oarc.net
https://lists.dns-oarc.net/mailman/listinfo/dns-operations

Re: [dns-operations] root? we don't need no stinkin' root!

[Tom Ivar Helbekkmo via dns-operations]

--- Begin Message ---
Tony Finch  writes:

> Attackers can get a small amplification from SYN/ACK retries, and this
> is being used in the wild.

Can you actually implement a TCP stack without that possibility?

-tih
-- 
Most people who graduate with CS degrees don't understand the significance
of Lisp.  Lisp is the most important idea in computer science.  --Alan Kay
--- End Message ---
___
dns-operations mailing list
dns-operations@lists.dns-oarc.net
https://lists.dns-oarc.net/mailman/listinfo/dns-operations

Re: [dns-operations] root? we don't need no stinkin' root!

[Tony Finch]

Florian Weimer  wrote:
>
> But does anyone swap out the name servers for a TLD over the course of
> five days?

Complete replacement of delegation NS RRsets happens fairly frequently. I
don't pay attention to the glue, tho, so I don't know how often these are
just renames as opposed to server platform changes.

One memorable example was .unicom back in June

https://twitter.com/fanf/status/1138372065541677056

The .mm delegation change on 7 Oct was a complete change of names and
addresses.

On 2 Nov, .in and their many IDN TLDs had a rename rather than a change of
addresses.

Tony.
-- 
f.anthony.n.finchhttp://dotat.at/
Whitby to Gibraltar Point: North 3 or 4, becoming variable 3 or less, then
east 3 or 4 later. Slight or moderate. Showers. Good.
___
dns-operations mailing list
dns-operations@lists.dns-oarc.net
https://lists.dns-oarc.net/mailman/listinfo/dns-operations

Re: [dns-operations] root? we don't need no stinkin' root!

[Tony Finch]

Viktor Dukhovni  wrote:
>
> refection of answers to forged source IPs is not available with TCP

Attackers can get a small amplification from SYN/ACK retries, and this is
being used in the wild.

https://www.darkreading.com/attacks-breaches/new-ddos-attacks-leverage-tcp-amplification-/d/d-id/1336339

Tony.
-- 
f.anthony.n.finchhttp://dotat.at/
Trafalgar: Variable 3 or less in southeast, southwesterly 3 to 5 in northwest.
Moderate, occasionally rough in north and slight in southeast. Showers. Good,
occasionally moderate.
___
dns-operations mailing list
dns-operations@lists.dns-oarc.net
https://lists.dns-oarc.net/mailman/listinfo/dns-operations

Re: [dns-operations] root? we don't need no stinkin' root!

[Viktor Dukhovni]

On Thu, Nov 28, 2019 at 09:42:46AM +0100, Petr Špaček wrote:

> Please let me try again:
> 
> Even if "the new system for root zone distribution" is BitTorrent it still:
> - (most likely) needs a set of static IP addresses to solve the bootstrap 
> problem,
> - trackers need to be highly resilient against DDoS,
> - trackers most likely need to be anycasted to limit scope of DDoS.
> 
> I hypothetise that in the end requirements for "the new system for root zone
> distribution" will be fairly close to current requirements for current DNS
> root system... so I do not see where the cost reduction comes from.
> 
> Or in other words:
> If current root system must survive 1 TB/s attack so must the "the new system
> for root zone distribution" system, unless we move to decenralized root.

Yes, but the expected probability of a 1 TB/s attack is likely different for a
file transfer service over TCP than it is for a one-shot query service over
UDP.  The chief reason being that refection of answers to forged source IPs is
not available with TCP, and so attacks on *other* systems via the root servers
are no longer attractive once the root servers just offer bulk data for
download via TCP.

So key question is whether the attacks we're seeing today are aimed at the root
servers themselves, or are DDoS reflection attacks on other systems.  Of course
once we're in the business of waving magic wands, it would be très chic if all
network operators implemented BCP38, and compromised Internet of T!@#$% devices
were no longer able to mask their origin networks.

-- 
Viktor.
___
dns-operations mailing list
dns-operations@lists.dns-oarc.net
https://lists.dns-oarc.net/mailman/listinfo/dns-operations

Re: [dns-operations] root? we don't need no stinkin' root!

[Paul Ebersman]

mallman> I wonder if we're ever allowed to just decide this sort of
mallman> thing is ridiculous old shit and for lots of reasons we can and
mallman> should just garbage collect it away.

ebersman> We aren't allowed as IETF/engineers. The world sort of is. ;)

tale> I see the :) but I'm thinking the first sentence is serious.  

tale> Given the consensus model of the IETF, I absolutely think we're
tale> allowed to decide we no longer have to be backwards compatible with
tale> everything in perpetuity.

Should have been clearer. My point is that we as the IETF can recommend
and specify but we have no control over what users/companies/vendors do
with it. And operationally, we have to be sympathetic to legacy and
obsolescence needs.

Things like DNS flag day are a good start, assuming we do stay flexible
with respect to such user needs.
___
dns-operations mailing list
dns-operations@lists.dns-oarc.net
https://lists.dns-oarc.net/mailman/listinfo/dns-operations

Re: [dns-operations] root? we don't need no stinkin' root!

[Dave Lawrence]

Paul Ebersman writes:
> mallman> I wonder if we're ever allowed to just decide this sort of
> mallman> thing is ridiculous old shit and for lots of reasons we can and
> mallman> should just garbage collect it away.
> 
> We aren't allowed as IETF/engineers. The world sort of is. ;)

I see the :) but I'm thinking the first sentence is serious.  

Given the consensus model of the IETF, I absolutely think we're
allowed to decide we no longer have to be backwards compatible with
everything in perpetuity. We can certainly roll out a spec which
mandates, to the IETF's best ability to mandate anything, a behavior
that is known to be problematic to ancient software.  Of course we'd
take into account just what sore of problematic that is and the
relative risk involved in coming to a decision about whether it's
acceptable, but I've never seen a hard requirement that "ridiculous
old shit" work indefinitely.

The rest of the world's developers and operators can of course decide
whether to go along with the new spec, exactly as they always have
been.
___
dns-operations mailing list
dns-operations@lists.dns-oarc.net
https://lists.dns-oarc.net/mailman/listinfo/dns-operations

Re: [dns-operations] root? we don't need no stinkin' root!

[Paul Vixie]

On Wednesday, 27 November 2019 15:38:32 UTC Keith Mitchell wrote:
> ...
> 
> While AS112 makes a difference, it is far from ubiquitous or optimal.
> Probably there are gains to be made from more aggressive co-ordination
> and advocacy (*), but I suspect these would need stronger resource
> support from a more top-down source. It's far from the whole problem
> space, but makes some difference at the root for sure.

+1 except, there is no top.

-- 
Paul


___
dns-operations mailing list
dns-operations@lists.dns-oarc.net
https://lists.dns-oarc.net/mailman/listinfo/dns-operations

Re: [dns-operations] root? we don't need no stinkin' root!

[Vladimír Čunát]

On 11/26/19 9:58 PM, Tony Finch wrote:
> Mirror zones (validated zone transfers) fall on the wrong side of the
> cost/benefit equation for me. But I might change my mind if there were
> better security for unauthenticated records (NS and glue)

These are why we only implemented the mechanism over HTTPS for now (in
addition to validating signatures).
https://knot-resolver.readthedocs.io/en/stable/modules.html#cache-prefilling

Still, I believe that a small resolver instance only needs a few DNS
queries to root (per TTL), so switching everyone to always transferring
the whole root should increase the total traffic considerably, and HTTPS
and XoT are probably more expensive than DNS-over-UDP given the same
traffic amount.  That's where the aggressive-cache-only approach seems
nice, but (additionally) having full root would also avoid leaking any
of those garbage queries. (Except for those that hit an existing TLD,
but those can't be helped at the root level, and TLDs are generally too
big+dynamic for mirroring.)

--Vladimir

___
dns-operations mailing list
dns-operations@lists.dns-oarc.net
https://lists.dns-oarc.net/mailman/listinfo/dns-operations

Re: [dns-operations] root? we don't need no stinkin' root!

[Petr Špaček]

On 27. 11. 19 21:49, David Conrad wrote:
> Petr,
> 
>> I think there is even more fundamental problem:
>> Someone has to pay operational costs of "the new system”.
> 
> The “new system” is simply the existing network of resolvers, augmented to 
> have the root zone.  As far as I can tell, the operational cost would be in 
> (a) ensuring the resolver is upgraded to support obtaining the root zone and 
> (b) dealing with the fetch of the root zone with some frequency.

Oh, sorry, this is misunderstanding! My reference to "the new system" was meant 
to be "the new system for root zone distribution".
Please let me try again:

Even if "the new system for root zone distribution" is BitTorrent it still:
- (most likely) needs a set of static IP addresses to solve the bootstrap 
problem,
- trackers need to be highly resilient against DDoS,
- trackers most likely need to be anycasted to limit scope of DDoS.

I hypothetise that in the end requirements for "the new system for root zone 
distribution" will be fairly close to current requirements for current DNS root 
system... so I do not see where the cost reduction comes from.

Or in other words:
If current root system must survive 1 TB/s attack so must the "the new system 
for root zone distribution" system, unless we move to decenralized root.

Changing one centralized system to another does not solve the fundamendal 
problem of costly-to-defent-single-point-of-failure.

Hopefully it is clearer this time.
Petr Špaček  @  CZ.NIC


> 
> There would be an additional cost, that of making the root zone available to 
> myriads of resolvers, but I believe this is an easily handled issue.
> 
>> Personally I do not see how transition to another root-zone-distribution 
>> system solves the over-provisioning problem - the new system still has to be 
>> ready to absorm absurdly large DDoS attacks.
> 
> Two ways:
> - greater decentralization: there are a lot more resolvers than the number of 
> instances root server operators are likely to ever deploy. While an 
> individual resolver might melt down, the impact would only be the end users 
> using that resolver (and it is relatively easy for a resolver operator to add 
> more capacity, mitigate the attacking client, etc).
> - the cost of operating and upgrade the service to deal with DDoS is 
> distributed to folks whose job it is to provide that service (namely the ISPs 
> or other network operators that run the resolvers).  Remember that the root 
> server operators have day jobs, some of which are not particularly related to 
> running root service, and they are not (currently) being compensated for the 
> costs of providing root service.
> 
>> Have a look at https://www.knot-dns.cz/benchmark/ . The numbers in charts at 
>> bottom of the page show that a *single server machine* is able to reply 
>> *all* steady state queries for the root today.
>> ...
>> Most of the money is today spent on *massive* over-provisioning. As an 
>> practical example, CZ TLD is over-provisiong factor is in order of 
>> *hunderds* of stead-state query traffic, and the root might have even more.
> 
> Yep. As mentioned before, steady state is largely irrelevant.
> 
> In my view, the fact that root service infrastructure funnels up to a 
> (logical) single point is an architectural flaw that may (assuming DDoS 
> attack capacity continues to grow at the current rate or even grows faster 
> with crappy IoT devices) put the root DNS service at risk.  One of the 
> advantages of putting the root zone in the resolver is that it mitigates that 
> potential risk.
> 
> Regards,
> -drc
> (Speaking for myself, not any organization I may be affiliated with)

___
dns-operations mailing list
dns-operations@lists.dns-oarc.net
https://lists.dns-oarc.net/mailman/listinfo/dns-operations

Re: [dns-operations] root? we don't need no stinkin' root!

[Florian Weimer]

* Ondřej Surý:

>> Raw change rates do not tell us if zones keep at least of some of
>> their servers at constant addresses over really, really long
>> periods of time.
>
> .bank
> - deleted NS {ac1|ac2}.nstld.com. and added NS {a|b|c}.nic.bank. on November 
> 20
> and
> - deleted NS {ac3|ac4}.nstld.com. and added NS {d|e|f}.nic.bank. on November 
> 23
>
> Does that answer your question?

{a,b,c,d,e,f}.nic.bank and ac{1..4}.nstld.com seem to have
non-overlapping addresses.

I think this means that a simple update protocol (such as that
currently used for tzdata, or the root key and initial set of DNS
server address) will not be very reliable (at least until these
practices change).

I'm not sure if a different update protocol would be much of an
improvement over using the current with NSEC synthesis enabled.  (It
would be nice if resolver software allowed configuring that for the
root separately.)

___
dns-operations mailing list
dns-operations@lists.dns-oarc.net
https://lists.dns-oarc.net/mailman/listinfo/dns-operations

Re: [dns-operations] root? we don't need no stinkin' root!

[Ondřej Surý]

> On 28 Nov 2019, at 08:09, Florian Weimer  wrote:
> 
> * Ondřej Surý:
> 
>>> On 27 Nov 2019, at 23:08, Florian Weimer  wrote:
>>> * Mark Allman:
>>> 
 Let me try to get away from what is or is not "big" and ask two
 questions.  (These are legit questions to me.  I have studied the
 DNS a whole bunch, but I do not operate any non-trivial part of the
 DNS and so that viewpoint is valuable to me.)
 
 (1) Setting aside history and how things have been done and why
(which I am happy to stipulate is rational)... At this point,
are there tangible benefits for getting information about the
TLD nameservers to resolvers as needed via a network service?
 
 (2) Are there fundamental problems that would arise in recursive
resolvers if the information about TLD nameservers was no longer
available via a network service, but instead had to come from a
file that was snarfed periodically?
>>> 
>>> What's the change rate for the root zone?
>> 
>> https://twitter.com/diffroot
> 
> Selective quoting does not help to further the discussion.

Including cruft also does not help to further the discussion, but fair enough, 
I reinstated
the rest of your email into the reply.

> Raw change
> rates do not tell us if zones keep at least of some of their servers
> at constant addresses over really, really long periods of time.

.bank
- deleted NS {ac1|ac2}.nstld.com. and added NS {a|b|c}.nic.bank. on November 20
and
- deleted NS {ac3|ac4}.nstld.com. and added NS {d|e|f}.nic.bank. on November 23

Does that answer your question?

That doesn’t include changes to GLUE records which are equally as important, so 
the
only value you can work with and rely on is the TTL. (172800 - 48h).

The TTL on DS is 1 day and is even more important to get the updated DS early
when there’s a breach and DS needs to be swapped as early as possible.

>> If there is a full
>> transition of the name server addresses for a zone, how long does it
>> typically take from the first change to the completion of the sequence
>> of changes?
>> 
>> If the answer, “this has never happened”, then using a fairly static
>> data source should probably be okay (similar to how the browser PKI is
>> maintained).  Due to the way DNSSEC works with its periodic renewal of
>> signatures, validating non-recursive resolvers will automatically
>> verify the freshness of the local root zone copy.  Even if there are
>> few such clients, I expect that for most operators, it will
>> effectively prevent undetected decay due to a stale root zone (where
>> more and more stale delegations accumulate until performance is
>> seriously impacted, and fresh bootstrap using external data is
>> needed).
>> 
>> The other question is whether that data source will make it harder for
>> ICANN or someone else to hand over control over the TLD in a
>> unilateral manner.  And then it's not even clear whether that's a good
>> thing or not.
>> 
>> Other uncertainties relate to the size of the root zone.  It seems
>> that the phase of aggressive growth is more or less over.  But
>> hard-coding an assumption that resolvers can load the root zone into
>> memory is on a different level because it limits policy basically for
>> forever.
>> 
>> I've thought a bit whether the root domain list should be pushed into
>> (non-validating) stub resolvers, but I don't think that's possible
>> because people really like to use local domains.


Ondrej
--
Ondřej Surý
ond...@sury.org


___
dns-operations mailing list
dns-operations@lists.dns-oarc.net
https://lists.dns-oarc.net/mailman/listinfo/dns-operations

Re: [dns-operations] root? we don't need no stinkin' root!

[Florian Weimer]

* Ondřej Surý:

>> On 27 Nov 2019, at 23:08, Florian Weimer  wrote:
>> 
>> What's the change rate for the root zone?
>
> https://twitter.com/diffroot

Selective quoting does not help to further the discussion.  Raw change
rates do not tell us if zones keep at least of some of their servers
at constant addresses over really, really long periods of time.

___
dns-operations mailing list
dns-operations@lists.dns-oarc.net
https://lists.dns-oarc.net/mailman/listinfo/dns-operations

Re: [dns-operations] root? we don't need no stinkin' root!

[Ondřej Surý]

> On 27 Nov 2019, at 23:08, Florian Weimer  wrote:
> 
> What's the change rate for the root zone?

https://twitter.com/diffroot

O.
--
Ondřej Surý
ond...@sury.org


___
dns-operations mailing list
dns-operations@lists.dns-oarc.net
https://lists.dns-oarc.net/mailman/listinfo/dns-operations

Re: [dns-operations] root? we don't need no stinkin' root!

[Florian Weimer]

* Jared Mauch:

>> On Nov 27, 2019, at 5:26 PM, Florian Weimer  wrote:
>> 
>> What's the change rate for the root zone?  If there is a full
>> transition of the name server addresses for a zone, how long does it
>> typically take from the first change to the completion of the sequence
>> of changes?
>
> There are regular changes. 

But does anyone swap out the name servers for a TLD over the course of
five days?
___
dns-operations mailing list
dns-operations@lists.dns-oarc.net
https://lists.dns-oarc.net/mailman/listinfo/dns-operations

Re: [dns-operations] root? we don't need no stinkin' root!

[Jared Mauch]

> On Nov 27, 2019, at 5:26 PM, Florian Weimer  wrote:
> 
> What's the change rate for the root zone?  If there is a full
> transition of the name server addresses for a zone, how long does it
> typically take from the first change to the completion of the sequence
> of changes?

There are regular changes. 

Resolvers regularly have old data. Survey for old root hints to see how bad 
they are. 
___
dns-operations mailing list
dns-operations@lists.dns-oarc.net
https://lists.dns-oarc.net/mailman/listinfo/dns-operations

Re: [dns-operations] root? we don't need no stinkin' root!

[Florian Weimer]

* Mark Allman:

> Let me try to get away from what is or is not "big" and ask two
> questions.  (These are legit questions to me.  I have studied the
> DNS a whole bunch, but I do not operate any non-trivial part of the
> DNS and so that viewpoint is valuable to me.)
>
> (1) Setting aside history and how things have been done and why
> (which I am happy to stipulate is rational)... At this point,
> are there tangible benefits for getting information about the
> TLD nameservers to resolvers as needed via a network service?
>
> (2) Are there fundamental problems that would arise in recursive
> resolvers if the information about TLD nameservers was no longer
> available via a network service, but instead had to come from a
> file that was snarfed periodically?

What's the change rate for the root zone?  If there is a full
transition of the name server addresses for a zone, how long does it
typically take from the first change to the completion of the sequence
of changes?

If the answer, “this has never happened”, then using a fairly static
data source should probably be okay (similar to how the browser PKI is
maintained).  Due to the way DNSSEC works with its periodic renewal of
signatures, validating non-recursive resolvers will automatically
verify the freshness of the local root zone copy.  Even if there are
few such clients, I expect that for most operators, it will
effectively prevent undetected decay due to a stale root zone (where
more and more stale delegations accumulate until performance is
seriously impacted, and fresh bootstrap using external data is
needed).

The other question is whether that data source will make it harder for
ICANN or someone else to hand over control over the TLD in a
unilateral manner.  And then it's not even clear whether that's a good
thing or not.

Other uncertainties relate to the size of the root zone.  It seems
that the phase of aggressive growth is more or less over.  But
hard-coding an assumption that resolvers can load the root zone into
memory is on a different level because it limits policy basically for
forever.

I've thought a bit whether the root domain list should be pushed into
(non-validating) stub resolvers, but I don't think that's possible
because people really like to use local domains.

___
dns-operations mailing list
dns-operations@lists.dns-oarc.net
https://lists.dns-oarc.net/mailman/listinfo/dns-operations

Re: [dns-operations] root? we don't need no stinkin' root!

[David Conrad]

Petr,

> I think there is even more fundamental problem:
> Someone has to pay operational costs of "the new system”.

The “new system” is simply the existing network of resolvers, augmented to have 
the root zone.  As far as I can tell, the operational cost would be in (a) 
ensuring the resolver is upgraded to support obtaining the root zone and (b) 
dealing with the fetch of the root zone with some frequency.

There would be an additional cost, that of making the root zone available to 
myriads of resolvers, but I believe this is an easily handled issue.

> Personally I do not see how transition to another root-zone-distribution 
> system solves the over-provisioning problem - the new system still has to be 
> ready to absorm absurdly large DDoS attacks.

Two ways:
- greater decentralization: there are a lot more resolvers than the number of 
instances root server operators are likely to ever deploy. While an individual 
resolver might melt down, the impact would only be the end users using that 
resolver (and it is relatively easy for a resolver operator to add more 
capacity, mitigate the attacking client, etc).
- the cost of operating and upgrade the service to deal with DDoS is 
distributed to folks whose job it is to provide that service (namely the ISPs 
or other network operators that run the resolvers).  Remember that the root 
server operators have day jobs, some of which are not particularly related to 
running root service, and they are not (currently) being compensated for the 
costs of providing root service.

> Have a look at https://www.knot-dns.cz/benchmark/ 
>  . The numbers in charts at bottom of the 
> page show that a *single server machine* is able to reply *all* steady state 
> queries for the root today.
> ...
> Most of the money is today spent on *massive* over-provisioning. As an 
> practical example, CZ TLD is over-provisiong factor is in order of *hunderds* 
> of stead-state query traffic, and the root might have even more.


Yep. As mentioned before, steady state is largely irrelevant.

In my view, the fact that root service infrastructure funnels up to a (logical) 
single point is an architectural flaw that may (assuming DDoS attack capacity 
continues to grow at the current rate or even grows faster with crappy IoT 
devices) put the root DNS service at risk.  One of the advantages of putting 
the root zone in the resolver is that it mitigates that potential risk.

Regards,
-drc
(Speaking for myself, not any organization I may be affiliated with)


signature.asc
Description: Message signed with OpenPGP
___
dns-operations mailing list
dns-operations@lists.dns-oarc.net
https://lists.dns-oarc.net/mailman/listinfo/dns-operations

Re: [dns-operations] root? we don't need no stinkin' root!

[Fred Morris]

I've been following this thread, and I'm well aware of the massive amounts 
of NXDOMAIN stuff. I don't know enough about this specific issue.


But there are things which happen in Browser Land which would lead me to 
naively conclude the people making browsers don't understand DNS. Two 
recent (actually ongoing) examples:


1) Firefox still (years now, although I haven't filed a bug) doesn't
   understand that it's perfectly ok to have a trailing dot on an FQDN;
   and it serves a purpose. (It's not supposed to be included in the TLS
   Host: header though.)

2) In spite of implementing their own DNS resolvers, browsers seem unable
   to block domains cloaked by CNAMEs (the third party trackers accessing
   first party cookies trope, RPZ works just fine for some odd reason).

On Wed, 27 Nov 2019, Petr Špaček wrote:

[...]
“Coincidence? I think NOT!” 


https://youtu.be/MDpuTqBI0RM?t=53


FYI there is also an issue about this in their tracker:
https://bugs.chromium.org/p/chromium/issues/detail?id=946450#c1


As I understand it these are unadorned labels, unit of one. Two 
parts to this.



What's Chrome's point with this? They've trained monkeys that URLs are for 
Boomers, just type a search string in there. Wild guess here, it goes 
something like this:


User types an undotted hostname on their network. Chrome searches and 
returns a bunch of shopping and social media sites. Ok, that makes monkeys 
upset. Well, we'll go off the reservation (we really hate doing this) and 
see if our operating environment resolves it before searching. Oh drat! 
The operating environment is doing a search! Some monkeys are upset either 
way. Prod Mgr: They're interfering with our search! They don't understand 
the one true way! Dev: I think we should agressively probe the operating 
environment with garbage, the best defense is a good offense. Prod Mgr: 
Let's call it a "friendly" probe and I'm good with it.


Fine, you may not like my personalizations, but is that it? I don't 
believe these people are idiots with no knowledge of DNS operation.


To riff off of an old South Park episode, there seems to be a lot of smug 
in the air. It's not just one thing. It's a pattern of engineering around 
the DNS. Poorly.


Since I don't use Chrome, could somebody please type a local hostname (one 
label) with a trailing dot into the thing and see what happens? Nothing 
good, I'm sure. Those who know the purpose of the trailing dot will know 
that this should outright fail to resolve (probably sends a request to the 
root for the label as a TLD).


Since they're already engineering around the DNS and the trailing dot has 
been a casualty for some time, would it be unthinkable for them to 
repurpose it as a declarative: this label needs to be sent to the 
operating environment resolver (without the dot). Search lists... 
everybody hates search lists.



Let me put it to you this way: which do you hate more, search lists or 
unary labels hitting the roots?


Shouldn't what happens be that they spew their probe at the operating 
environment resolver, it appends things from the search list and tries 
those?


If there's a shared engineering problem here, isn't it that when these 
fail, the resolver tries the naked label? Or tries it first, but in any 
case, tries it.


Isn't the proliferation of "valid" TLDs contributing to this embarrassment 
of riches by making approaches such as selectively whitelisting TLDs so 
increasingly impractical as to obviate consideration?


Should local resolvers reject attempts to resolve single labels as TLDs 
unless RD=0?



I apologize, none of this is fully baked, but the debate doesn't seem to 
be encompassing the entirety of the system.


--

Fred Morris
___
dns-operations mailing list
dns-operations@lists.dns-oarc.net
https://lists.dns-oarc.net/mailman/listinfo/dns-operations

Re: [dns-operations] root? we don't need no stinkin' root!

[Wessels, Duane via dns-operations]

--- Begin Message ---


> On Nov 25, 2019, at 1:23 PM, Bill Woodcock  wrote:
> 
>> On Nov 25, 2019, at 9:54 PM, Florian Weimer  wrote:
>> The query numbers are surprisingly low.  To me at last.
> 
> Duane Wessels did a good study some time ago of queries to the root.  I 
> believe over 99% were bogus, not real queries for resolvable things.

For the record, the number from that study (2003!) was 98%.  I believe it has 
since been reconfirmed to be about the same level in a number of subsequent 
studies by others.

DW




smime.p7s
Description: S/MIME cryptographic signature
--- End Message ---
___
dns-operations mailing list
dns-operations@lists.dns-oarc.net
https://lists.dns-oarc.net/mailman/listinfo/dns-operations

Re: [dns-operations] root? we don't need no stinkin' root!

[Wessels, Duane via dns-operations]

--- Begin Message ---


> On Nov 25, 2019, at 2:19 PM, Florian Weimer  wrote:
> 
> * Jim Reid:
> 
>>> On 25 Nov 2019, at 20:54, Florian Weimer  wrote:
>>> Is it because of the incoming data is interesting?
>> 
>> Define interesting.
> 
> The data could have monetary value.  Passwords that are otherwise
> difficult to come by might be leaking.

Hi Florian,

I can assure you that Verisign does not monetize the root server data.  If
any other operators do, I'm not aware of it.

We do utilize root server data for research purposes from time-to-time.
Recent examples include the KSK rollover and name collisions.  Less recent
examples include understanding TTL/caching behavior and preparing for the
root ZSK size increase.  When DDoS attacks happen, we often analyze the
data to see if we can understand how and why it happened, and to be better
prepared for the next one.

DW




smime.p7s
Description: S/MIME cryptographic signature
--- End Message ---
___
dns-operations mailing list
dns-operations@lists.dns-oarc.net
https://lists.dns-oarc.net/mailman/listinfo/dns-operations

Re: [dns-operations] root? we don't need no stinkin' root!

[Petr Špaček]

On 26. 11. 19 12:46, David Conrad wrote:
> On Nov 26, 2019, at 11:33 AM, Jim Reid  > wrote:
>>> On 26 Nov 2019, at 09:16, Florian Weimer >> > wrote:
>>>
>>> Up until recently, well-behaved recursive resolvers had to forward
>>> queries to the root if they were not already covered by a delegation.
>>> RFC 7816 and in particular RFC 8198 changed that, but before that, it
>>> was just how the protocol was expected to work.
>>
>> So what? These RFCs make very little difference to the volume of queries a 
>> resolving server will send to the root. QNAME minimisation has no impact at 
>> all: the root just sees a query for .com instead of foobar.com 
>> . A recursive resolver should already be supporting 
>> negative caching and will have a reasonably complete picture of what's in 
>> (or not in) the root. RFC8198 will of course help that but not by much IMO.
> 
> It would appear a rather large percentage of queries to the root (like 50% in 
> some samples) are random strings, between 7 to 15 characters long, sometimes 
> longer.  I believe this is Chrome-style probing to determine if there is 
> NXDOMAIN redirection. A good example of the tragedy of the commons, like 
> water pollution and climate change.
> 
> If resolvers would enable DNSSEC validation, there would, in theory, be a 
> reduction in these queries due to aggressive NSEC caching.  Of course, 
> practice may not match theory 
> (https://indico.dns-oarc.net/event/32/contributions/717/attachments/713/1206/2019-10-31-oarc-nsec-caching.pdf).
>  

The discussion after the talk (including hallway :-) was also interesting, and 
with all respect for Geoff's work, these slides should be read with some 
sceptism.

Main points:
1) Load-balancer with N resolver nodes behind it decrease effectivity of 
aggressive cache by factor N, it *does not* invalidate the concept.

In other words, a random subdomain attack which flows through resolver farm 
with N nodes has to fill N caches with NSEC records, and that will simply take 
N times longer when compared with non-load-balanced scenario.

The aggressive cache still provides upper bound for size of NXDOMAIN RRs in 
cache, which is super useful under attack because it prevents individual 
resolvers from dropping all the useful content from cache during the attack.


2) Two out of five major DNS resolver implementations used by large ISPs did 
not implement aggressive caching (yet?), so it needs to be expected that 
deployment is not great. Also the feature is pretty new and large ISPs are 
super conservative and might not deployed new versions yet ...

I forgot the rest so I will conclude with: Watch the video recording and think 
yourself! :-)
Petr Špaček  @  CZ.NIC


> 
> Of course, steady state query load is largely irrelevant since root service 
> has to be provisioned with massive DDoS in mind. In my personal view, the 
> deployment of additional anycast instances by the root server operators is a 
> useful stopgap, but ultimately, given the rate of growth of DoS attack 
> capacity (and assuming that growth will continue due to the stunning security 
> practices of IoT device manufacturers), stuff like what is discussed in that 
> paper is the right long term strategy.

___
dns-operations mailing list
dns-operations@lists.dns-oarc.net
https://lists.dns-oarc.net/mailman/listinfo/dns-operations

Re: [dns-operations] root? we don't need no stinkin' root!

[Petr Špaček]

On 27. 11. 19 9:53, Ondřej Surý wrote:
> Mark,
> 
> I believe that any distributed system that won’t have a fallback to the RZ
> is inevitably doomed and will get out of sync.
> 
> The RFC7706 works because there’s always a safe guard and if the resolver
> is unable to use mirrored zone, it will go to the origin.
> 
> Call me a pessimist, but I’ve yet to see a loosely often neglected 
> distributed system
> that won’t get out of sync.
> 
> So, while the idea of distributing the full RZ to every resolver out there, 
> there are two
> fundamental problems:
> 
> 1. resilience - both against DoS and just plain breakage
> 2. the old clients - while the situation out there is getting better, we will 
> still be stuck with
> old codebase for foreseeable future
> 
> What we can do is to make the load on RZ servers lighter, but we can’t make 
> them just go.

I think there is even more fundamental problem:
Someone has to pay operational costs of "the new system".

Personally I do not see how transition to another root-zone-distribution system 
solves the over-provisioning problem - the new system still has to be ready to 
absorm absurdly large DDoS attacks.

Example:
Have a look at https://www.knot-dns.cz/benchmark/ . The numbers in charts at 
bottom of the page show that a *single server machine* is able to reply *all* 
steady state queries for the root today.

Sure, we have speed-of-light limits, so let's say we need couple hunderd 
servers in well connected places to keep reasonable latency. That's not a huge 
cost overall (keep in mind that these local nodes could be pretty small *if we 
were ignoring the over-provisioning problem*).

Most of the money is today spent on *massive* over-provisioning. As an 
practical example, CZ TLD is over-provisiong factor is in order of *hunderds* 
of stead-state query traffic, and the root might have even more.

Once we have similarly resilient HTTP system it is matter of simple 
configuration :-D
https://knot-resolver.readthedocs.io/en/stable/modules.html#cache-prefilling

-- 
Petr Špaček  @  CZ.NIC
___
dns-operations mailing list
dns-operations@lists.dns-oarc.net
https://lists.dns-oarc.net/mailman/listinfo/dns-operations

Re: [dns-operations] root? we don't need no stinkin' root!

[Keith Mitchell]

On 11/26/19 7:40 PM, Mark Allman wrote:
> I wonder if we're ever allowed to just decide this sort of thing is 
> ridiculous old shit and for lots of reasons we can and should just 
> garbage collect it away.

To some extent, "get rid of ridiculous old sh*t" is kind of what the DNS
Flag Days are working on, though with rather more baby steps than I
suspect you are conceiving. Even these small, rational proposals have
met with push-back in some sectors. It's quite a lot of work to
deprecate stuff in a way that minimizes operational fall-out.

> To me, this whole notion is that we can in fact get rid of this
> giant network service.  If we don't get rid of it then what is the 
> incentive to move one's own resolver away from using the root 
> nameservers?

On garbage-collecting crap traffic, it's worth looking at AS112. Mostly
this runs on a bottom-up community-driven basis, where the incentive to
run an AS112 node comes from the simple self-interested economic basis
of not wanting this crap taking up capacity on one's own outbound
infrastructure.

While AS112 makes a difference, it is far from ubiquitous or optimal.
Probably there are gains to be made from more aggressive co-ordination
and advocacy (*), but I suspect these would need stronger resource
support from a more top-down source. It's far from the whole problem
space, but makes some difference at the root for sure.

Keith


(*) every so often I get a stark reminder of how low awareness of AS112
is...no, we don't want to buy transit for it, thanks..
___
dns-operations mailing list
dns-operations@lists.dns-oarc.net
https://lists.dns-oarc.net/mailman/listinfo/dns-operations

Re: [dns-operations] root? we don't need no stinkin' root!

[Petr Špaček]

On 26. 11. 19 16:04, Roy Arends wrote:
> 
> 
>> On 26 Nov 2019, at 12:46, David Conrad  wrote:
>>
>> It would appear a rather large percentage of queries to the root (like 50% 
>> in some samples) are random strings, between 7 to 15 characters long, 
>> sometimes longer.  I believe this is Chrome-style probing to determine if 
>> there is NXDOMAIN redirection. A good example of the tragedy of the commons, 
>> like water pollution and climate change.
> 
> Yep.
> 
> https://chromium.googlesource.com/chromium/src/+/32352ad08ee673a4d43e8593ce988b224f6482d3/chrome/browser/intranet_redirect_detector.cc
> Line 79: "// We generate a random hostname with between 7 and 15 characters.”
> 
> https://ithi.research.icann.org/graph-m3.html
> Table "Queries to frequently found name patterns” shows that the frequency 
> distribution for queries between 7 and 15 characters are near flat (around 
> 5.2% per character length) AND an order higher than ANY other queries.
> 
> “Coincidence? I think NOT!”  
> 
> https://youtu.be/MDpuTqBI0RM?t=53

FYI there is also an issue about this in their tracker:
https://bugs.chromium.org/p/chromium/issues/detail?id=946450#c1

-- 
Petr Špaček  @  CZ.NIC
___
dns-operations mailing list
dns-operations@lists.dns-oarc.net
https://lists.dns-oarc.net/mailman/listinfo/dns-operations

Re: [dns-operations] root? we don't need no stinkin' root!

[Ondřej Surý]

Mark,

I believe that any distributed system that won’t have a fallback to the RZ
is inevitably doomed and will get out of sync.

The RFC7706 works because there’s always a safe guard and if the resolver
is unable to use mirrored zone, it will go to the origin.

Call me a pessimist, but I’ve yet to see a loosely often neglected distributed 
system
that won’t get out of sync.

So, while the idea of distributing the full RZ to every resolver out there, 
there are two
fundamental problems:

1. resilience - both against DoS and just plain breakage
2. the old clients - while the situation out there is getting better, we will 
still be stuck with
old codebase for foreseeable future

What we can do is to make the load on RZ servers lighter, but we can’t make 
them just go.

Ondrej
--
Ondřej Surý
ond...@sury.org



> On 26 Nov 2019, at 14:41, Mark Allman  wrote:
> 
> 
> Let me try to get away from what is or is not "big" and ask two
> questions.  (These are legit questions to me.  I have studied the
> DNS a whole bunch, but I do not operate any non-trivial part of the
> DNS and so that viewpoint is valuable to me.)
> 
> (1) Setting aside history and how things have been done and why
>(which I am happy to stipulate is rational)... At this point,
>are there tangible benefits for getting information about the
>TLD nameservers to resolvers as needed via a network service?
> 
> (2) Are there fundamental problems that would arise in recursive
>resolvers if the information about TLD nameservers was no longer
>available via a network service, but instead had to come from a
>file that was snarfed periodically?
> 
> Thanks!
> 
> allman
> 
> 
> --
> https://www.icir.org/mallman/
> @mallman_icsi
> ___
> dns-operations mailing list
> dns-operations@lists.dns-oarc.net
> https://lists.dns-oarc.net/mailman/listinfo/dns-operations


___
dns-operations mailing list
dns-operations@lists.dns-oarc.net
https://lists.dns-oarc.net/mailman/listinfo/dns-operations

Re: [dns-operations] root? we don't need no stinkin' root!

[George Michaelson]

I tend to functional questions in these matters. This is not a
symmetric pair, but they go to different sides of the problem

1) what will happen if we imagine these queries not being answered? A
hypothetical (*and, its not zero cost*) front-end process which drops
them

2) what is the consequence of continuing to answer these queries?

Noting 1) is not trivial, I believe if these queries were not
answered, there would be short-term downsides, but long-term upsides.
The problem would (I believe) go away.

Noting 2) is the "do nothing" option. The only clear consequence is
that we're incurring cost, in root instantiations. It is possible if
we go with run-root-on-local/loopback we smear the cost, but do we
reduce it?

-G

On Wed, Nov 27, 2019 at 1:00 PM Mark Allman  wrote:
>
>
> Hi Paul!
>
> > The biggest problem I see here is the legacy/long-tail problem. As
> > of a few years ago, I bumped into BIND 4 servers still
> > active. Wouldn't be shocked to hear they are still being used.
> >
> > IPv4 reachable traditional DNS servers for some tiny group of
> > antique folks will be needed for years, even if we get 99+% of the
> > world to some new system.
>
> I wonder if we're ever allowed to just decide this sort of thing is
> ridiculous old shit and for lots of reasons we can and should just
> garbage collect it away.
>
> > Doesn't mean we shouldn't be thinking about a better way to do it
> > for that 99% though.
>
> Is it better if we only get to 99%?
>
> To me, this whole notion is that we can in fact get rid of this
> giant network service.  If we don't get rid of it then what is the
> incentive to move one's own resolver away from using the root
> nameservers?  I don't have any heartburn with RFC 7706.  But, it is
> a quite minor optimization in the general case.  It may well be
> important in some corner cases, but in general I don't think running
> a local root nameserver helps all that much.
>
> Maybe 99% lets us draw down the size of the root infrastructure...I
> dunno.  But, if we don't say something like "it's going to go away"
> then I am not sure resolvers will move away from it.
>
> allman
>
>
> --
> https://www.icir.org/mallman/
> @mallman_icsi
> ___
> dns-operations mailing list
> dns-operations@lists.dns-oarc.net
> https://lists.dns-oarc.net/mailman/listinfo/dns-operations
___
dns-operations mailing list
dns-operations@lists.dns-oarc.net
https://lists.dns-oarc.net/mailman/listinfo/dns-operations

Re: [dns-operations] root? we don't need no stinkin' root!

[Paul Ebersman]

ebersman> IPv4 reachable traditional DNS servers for some tiny group of
ebersman> antique folks will be needed for years, even if we get 99+% of
ebersman> the world to some new system.

mallman> I wonder if we're ever allowed to just decide this sort of
mallman> thing is ridiculous old shit and for lots of reasons we can and
mallman> should just garbage collect it away.

We aren't allowed as IETF/engineers. The world sort of is. ;)

Eventually, someone wonders why they're burning money on something they
don't see a need for any more.

Sadly, based on the number of IBM AS400s in service, the COBOL programs
with no source still being used, SNA, X.25 and all sorts of other stuff
that you'd think would have been dead decades ago, I'm not betting on
this happening any time soon.

mallman> To me, this whole notion is that we can in fact get rid of this
mallman> giant network service.  If we don't get rid of it then what is
mallman> the incentive to move one's own resolver away from using the
mallman> root nameservers?

But what would we replace it with? Who would run it? How would we get
uniqueness, data integrity, high availability, decent coherence? How
would we get something the entire world would use?

Part of why DNS is so abused and misused is that it's already here and
it mostly scales/works. We did it before the world knew about the
internet. Now there's way more attention, money, and politics that get
in the way of truly massive changes. If DNS started from scratch today,
it's not clear it would happen.

Not saying it's impossible but it will be a daunting task and will have
to be really really compelling (or be the next user loved
shiny-ball/Pokemon). Look at how much fun and progress there is moving
from IPv4 to IPv6.

mallman> Maybe 99% lets us draw down the size of the root
mallman> infrastructure...I dunno.  But, if we don't say something like
mallman> "it's going to go away" then I am not sure resolvers will move
mallman> away from it.

The problem/load isn't the folks that would upgrade. It's crap broken
code/devices that are in many cases forgotten in closets or under
desks. The magic blue smoke will have to pour out the back before they
stop sending useless crap to the root servers.

A6 records were never officially "blessed". We went with . We were
all pretty sure they would never be used. But last I heard, the root
servers still see A6 queries. Google for Geoff Huston's zombie DNS preso
for more scary/bad stories.

Love to see your proposal for a replacement. Just be prepared to have to
support whatever that is and DNS both for a very long time.
___
dns-operations mailing list
dns-operations@lists.dns-oarc.net
https://lists.dns-oarc.net/mailman/listinfo/dns-operations

Re: [dns-operations] root? we don't need no stinkin' root!

[Mark Allman]

Hi Paul!

> The biggest problem I see here is the legacy/long-tail problem. As
> of a few years ago, I bumped into BIND 4 servers still
> active. Wouldn't be shocked to hear they are still being used.
>
> IPv4 reachable traditional DNS servers for some tiny group of
> antique folks will be needed for years, even if we get 99+% of the
> world to some new system.

I wonder if we're ever allowed to just decide this sort of thing is
ridiculous old shit and for lots of reasons we can and should just
garbage collect it away.

> Doesn't mean we shouldn't be thinking about a better way to do it
> for that 99% though.

Is it better if we only get to 99%?

To me, this whole notion is that we can in fact get rid of this
giant network service.  If we don't get rid of it then what is the
incentive to move one's own resolver away from using the root
nameservers?  I don't have any heartburn with RFC 7706.  But, it is
a quite minor optimization in the general case.  It may well be
important in some corner cases, but in general I don't think running
a local root nameserver helps all that much.

Maybe 99% lets us draw down the size of the root infrastructure...I
dunno.  But, if we don't say something like "it's going to go away"
then I am not sure resolvers will move away from it.

allman


--
https://www.icir.org/mallman/
@mallman_icsi
___
dns-operations mailing list
dns-operations@lists.dns-oarc.net
https://lists.dns-oarc.net/mailman/listinfo/dns-operations

Re: [dns-operations] root? we don't need no stinkin' root!

[Tony Finch]

I generally agree with Geoff Huston's thoughts on this subject
http://www.potaroo.net/ispcol/2019-04/root.html

Mirror zones (validated zone transfers) fall on the wrong side of the
cost/benefit equation for me. But I might change my mind if there were
better security for unauthenticated records (NS and glue), e.g.

* xfer-over-TLS - I'm really looking forward to support for authenticated
  server / anonymous client for zone transfers: nice for local root zones
  and cross-campus zone distribution.

* zone digests - interesting for end-to-end verification but maybe too
  complicated?


Mukund Sivaraman  wrote:
>
> There are some Twitter feeds about what kinds of
> changes occur to the root zone and how frequently, e.g.:
>
> https://twitter.com/diffroot

Note that @diffroot does not tweet about changes to glue addresses which
happen a lot more frequently than NS and DS changes.

Tony.
-- 
f.anthony.n.finchhttp://dotat.at/
Biscay: Southwest, veering west, 6 to gale 8, occasionally severe gale 9 until
later. Rough or very rough becoming very rough or high, becoming very rough
later. Thundery showers. Good, occasionally poor.
___
dns-operations mailing list
dns-operations@lists.dns-oarc.net
https://lists.dns-oarc.net/mailman/listinfo/dns-operations

Re: [dns-operations] root? we don't need no stinkin' root!

[Paul Ebersman]

mallman> Setting aside history and how things have been done and why
mallman> (which I am happy to stipulate is rational)... At this point,
mallman> are there tangible benefits for getting information about the
mallman> TLD nameservers to resolvers as needed via a network service?

The biggest problem I see here is the legacy/long-tail problem. As of a
few years ago, I bumped into BIND 4 servers still active. Wouldn't be
shocked to hear they are still being used.

IPv4 reachable traditional DNS servers for some tiny group of antique
folks will be needed for years, even if we get 99+% of the world to some
new system.

Doesn't mean we shouldn't be thinking about a better way to do it for
that 99% though.

mallman> Are there fundamental problems that would arise in recursive
mallman> resolvers if the information about TLD nameservers was no
mallman> longer available via a network service, but instead had to come
mallman> from a file that was snarfed periodically?

/etc/hosts.txt via bittorrent instead of ftp from sri-nic? :)

The DNS is only billed as loosely coherent, so conceptually this could
work. But I'd have to be convinced it was enough better in terms of data
integrity, coherence and availability than the current DNS/DNSSEC to be
worth the pain of changing that much code on all those devices/servers.
___
dns-operations mailing list
dns-operations@lists.dns-oarc.net
https://lists.dns-oarc.net/mailman/listinfo/dns-operations

Re: [dns-operations] root? we don't need no stinkin' root!

[Paul Ebersman]

ebersman> Actually, it's a great argument for longer TTLs and caching
ebersman> doing what they're supposed to.

jim> It would be if the root only got queries from well behaved
jim> recursive resolvers. But we both know Paul that simply isn't true.

jim> Well over 90% of the query traffic at the root has no reason to be
jim> going there at all. For instance stub resolvers that don't care
jim> about TTLs or do any sort of caching, Chrome's 10-character nonce
jim> strings to detect NXDOMAIN rewriting, CPE querying for .home,
jim> enterprises leaking queries for .corp, etc, etc.

You cut off the last line of my post:

ebersman> But compared to a large corp DNS server farm, the root servers
ebersman> shovel a lot of bits. Some of it even valid DNS queries and
ebersman> responses. ;)

Yes. Most of it is crap and the normal DNS rules don't apply. But TTLs
and caching do help (less with root than TLD due to garbage problem) and
the orders of magnitude differences in size of traffic between root/TLD
and large recursive farms is still valid.

We started this with "what's a lot of traffic" and I think you and I
would agree defining "lots" is very dependent on what DNS role you play.

And we've both been around long enough to agree that even if well
behaved and well designed DNS start shifting to local root and similar,
there's enough just crap and enough legacy/old folks needing traditional
root that we're going to be upgrading the traditional root architecture
for a long long time.

But every bit helps, so local root, saner TTLs, solid caching layer are
all still worth building as well.
___
dns-operations mailing list
dns-operations@lists.dns-oarc.net
https://lists.dns-oarc.net/mailman/listinfo/dns-operations

Re: [dns-operations] root? we don't need no stinkin' root!

[John Kristoff]

On Mon, 25 Nov 2019 20:30:32 +
Mark Allman  wrote:

> Left here to be ripped apart ... :-)

Hello Mark,

Without making any explicit remarks on your paper or on the following,
in case you missed it, I remembered I had seen something like this
proposed before:

  Domain Name System Without Root Servers
  Matthäus Wander, Christopher Boelmann, Torben Weis
  

John

___
dns-operations mailing list
dns-operations@lists.dns-oarc.net
https://lists.dns-oarc.net/mailman/listinfo/dns-operations

Re: [dns-operations] root? we don't need no stinkin' root!

[Mukund Sivaraman]

On Tue, Nov 26, 2019 at 08:41:51AM -0500, Mark Allman wrote:
> 
> Let me try to get away from what is or is not "big" and ask two
> questions.  (These are legit questions to me.  I have studied the
> DNS a whole bunch, but I do not operate any non-trivial part of the
> DNS and so that viewpoint is valuable to me.)
> 
> (1) Setting aside history and how things have been done and why
> (which I am happy to stipulate is rational)... At this point,
> are there tangible benefits for getting information about the
> TLD nameservers to resolvers as needed via a network service?
> 
> (2) Are there fundamental problems that would arise in recursive
> resolvers if the information about TLD nameservers was no longer
> available via a network service, but instead had to come from a
> file that was snarfed periodically?

Probably not many. There are some Twitter feeds about what kinds of
changes occur to the root zone and how frequently, e.g.:

https://twitter.com/diffroot

I like how you've posed your questions, at a high-level view.

In the past few years, I've heard of several proposals for improving
root zone resilience, and how it can be made available to resolvers more
readily and robustly.

* RFC 7706 (root zone on loopback)
* Paul Vixie's idea about making one of the root letters have AS112
  style IP addresses
* Root zone mirroring with zone transfer over HTTPS
* Root zone distribution via bittorrent (IIRC I heard it from NLnet Labs
  staff)
* Root zone distribution as a blockchain
* DNS as a blockchain
* Combining zones into a look-aside megazone that is well provisioned

The root zone is another zone. Due to real-world traffic patterns, it is
perhaps more resilient to authoritative service degradation than
top-level zones such as "com." because once a TLD delegation (e.g.,
com.) is cached in a resolver, it is not going to look in the root zone,
when names in the TLD (e.g., com.) are queried, for a very long time.

DNS will always have an under-provisioning problem until there is some
policy about traffic. It will be so as long as a network-connected IP
camera is able to generate packets that saturate its 1Gbps link, and as
long as manufacturers are not liable for how their devices behave
(quality of firmware, updates, etc.).

Resilience applies to _any_ zone, not just the root zone. If an
Alexa-top-10 example.com company's authoritative DNS nameservers are
attacked by an equivalent of the Mirai botnet, or worse, a government
sponsored packet flood / DNS QUERY flood, overprovisioning to handle
such attacks increases its running costs by orders of magnitude. CDNs
with their global anycast DNS service are popular these days, but their
resiliency would depend on the specific attack circumstances. Not a year
goes by where we don't hear of a significant DNS outage due to a
DDoS. Use of such CDNs have also made DNS more centralized, which not
everybody thinks is wise for the general health of the internet.
Availability of a service doesn't depend just on resilience to DDoS
attacks.

If we use response/query percentage (response rate) as a metric for
service quality of a zone, a popular TLD zone would be more vulnerable
than the root when there is similar degradation of response rate.  The
same can be said for CDN-style global anycast authoritiative
DNS too. What would be more noticed and make the popular news - if
root-servers were to drop to 50% of their reply rate, or if Cloudflare
DNS were to drop to 50% of its reply rate?

It does not appear taking over-provisioning for granted would work
well. Borrowing a sibling reply's nomenclature, "large" will not work
robustly now or in the long term. "Clever" has to be better than root on
loopback - it has to work for other parts of DNS too (what if the root
works, but we can't resolve any new queries in the "com." TLDs?)
"Clever" has to be also about regulating, about limits imposed in
equipment by laws, and responsibility for maintaining connected
devices. The average human who installs a security camera knows nothing
about what happens on the wire and cannot realistically be expected to
be responsible.

Mukund
___
dns-operations mailing list
dns-operations@lists.dns-oarc.net
https://lists.dns-oarc.net/mailman/listinfo/dns-operations

Re: [dns-operations] root? we don't need no stinkin' root!

[Roy Arends]

> On 26 Nov 2019, at 12:46, David Conrad  wrote:
> 
> It would appear a rather large percentage of queries to the root (like 50% in 
> some samples) are random strings, between 7 to 15 characters long, sometimes 
> longer.  I believe this is Chrome-style probing to determine if there is 
> NXDOMAIN redirection. A good example of the tragedy of the commons, like 
> water pollution and climate change.

Yep.

https://chromium.googlesource.com/chromium/src/+/32352ad08ee673a4d43e8593ce988b224f6482d3/chrome/browser/intranet_redirect_detector.cc
Line 79: "// We generate a random hostname with between 7 and 15 characters.”

https://ithi.research.icann.org/graph-m3.html
Table "Queries to frequently found name patterns” shows that the frequency 
distribution for queries between 7 and 15 characters are near flat (around 5.2% 
per character length) AND an order higher than ANY other queries.

“Coincidence? I think NOT!”  

https://youtu.be/MDpuTqBI0RM?t=53

Roy


___
dns-operations mailing list
dns-operations@lists.dns-oarc.net
https://lists.dns-oarc.net/mailman/listinfo/dns-operations

Re: [dns-operations] root? we don't need no stinkin' root!

[Roy Arends]

Mark

> On 26 Nov 2019, at 14:49, Mark Allman  wrote:
> 
> 
>> It would appear a rather large percentage of queries to the root
>> (like 50% in some samples) are random strings, between 7 to 15
>> characters long, sometimes longer.  I believe this is Chrome-style
>> probing to determine if there is NXDOMAIN redirection. A good
>> example of the tragedy of the commons, like water pollution and
>> climate change.
> 
> I will note that there have been quite a number of studies over the
> last 20 years that show > 95% of the queries are junk of one kind or
> another.  Someone mentioned Duane's nice paper.  But, this
> observation started with Brownlee, et.al.'s 2001 paper.  Point
> being, Chrome might cause some of this now, but it has been there
> long before Chrome started this particularly probing.

Chrome might cause some of this? That is quite an understatement. If the number 
is around 50%, it is not "some of this". If this 50% disappears, the rest of 
the crap will still be there, and will probably be still > 90 %.

> What's more... in my rudimentary poking of the DITL data [*] it
> seems that 25-50% of the "resolvers" that query the root *never*
> send a legit query.  I.e., we can't ascribe a lot of this junk to
> resolvers that could just work better somehow.

and what percentage of traffic do these 25-50% resolvers account for? 

Roy

> 
> [*] There may be numbers on this sort of thing in the Brownlee,
>Wessels, etc. papers ... I just can't recall them off the top of
>my head.
> 
> allman
> 
> --
> https://www.icir.org/mallman/
> @mallman_icsi
> ___
> dns-operations mailing list
> dns-operations@lists.dns-oarc.net
> https://lists.dns-oarc.net/mailman/listinfo/dns-operations


___
dns-operations mailing list
dns-operations@lists.dns-oarc.net
https://lists.dns-oarc.net/mailman/listinfo/dns-operations

Re: [dns-operations] root? we don't need no stinkin' root!

[Mark Allman]

> It would appear a rather large percentage of queries to the root
> (like 50% in some samples) are random strings, between 7 to 15
> characters long, sometimes longer.  I believe this is Chrome-style
> probing to determine if there is NXDOMAIN redirection. A good
> example of the tragedy of the commons, like water pollution and
> climate change.

I will note that there have been quite a number of studies over the
last 20 years that show > 95% of the queries are junk of one kind or
another.  Someone mentioned Duane's nice paper.  But, this
observation started with Brownlee, et.al.'s 2001 paper.  Point
being, Chrome might cause some of this now, but it has been there
long before Chrome started this particularly probing.

What's more... in my rudimentary poking of the DITL data [*] it
seems that 25-50% of the "resolvers" that query the root *never*
send a legit query.  I.e., we can't ascribe a lot of this junk to
resolvers that could just work better somehow.

[*] There may be numbers on this sort of thing in the Brownlee,
Wessels, etc. papers ... I just can't recall them off the top of
my head.

allman

--
https://www.icir.org/mallman/
@mallman_icsi
___
dns-operations mailing list
dns-operations@lists.dns-oarc.net
https://lists.dns-oarc.net/mailman/listinfo/dns-operations

Re: [dns-operations] root? we don't need no stinkin' root!

[Bill Woodcock]

Yes, in the long term you can only survive by being both large and clever, not 
just one or the other. 

-Bill


> On Nov 26, 2019, at 13:03, David Conrad  wrote:
> 
> On Nov 26, 2019, at 11:33 AM, Jim Reid  wrote:
>>> On 26 Nov 2019, at 09:16, Florian Weimer  wrote:
>>> 
>>> Up until recently, well-behaved recursive resolvers had to forward
>>> queries to the root if they were not already covered by a delegation.
>>> RFC 7816 and in particular RFC 8198 changed that, but before that, it
>>> was just how the protocol was expected to work.
>> 
>> So what? These RFCs make very little difference to the volume of queries a 
>> resolving server will send to the root. QNAME minimisation has no impact at 
>> all: the root just sees a query for .com instead of foobar.com. A recursive 
>> resolver should already be supporting negative caching and will have a 
>> reasonably complete picture of what's in (or not in) the root. RFC8198 will 
>> of course help that but not by much IMO.
> 
> It would appear a rather large percentage of queries to the root (like 50% in 
> some samples) are random strings, between 7 to 15 characters long, sometimes 
> longer.  I believe this is Chrome-style probing to determine if there is 
> NXDOMAIN redirection. A good example of the tragedy of the commons, like 
> water pollution and climate change.
> 
> If resolvers would enable DNSSEC validation, there would, in theory, be a 
> reduction in these queries due to aggressive NSEC caching.  Of course, 
> practice may not match theory 
> (https://indico.dns-oarc.net/event/32/contributions/717/attachments/713/1206/2019-10-31-oarc-nsec-caching.pdf).
>  
> 
> Of course, steady state query load is largely irrelevant since root service 
> has to be provisioned with massive DDoS in mind. In my personal view, the 
> deployment of additional anycast instances by the root server operators is a 
> useful stopgap, but ultimately, given the rate of growth of DoS attack 
> capacity (and assuming that growth will continue due to the stunning security 
> practices of IoT device manufacturers), stuff like what is discussed in that 
> paper is the right long term strategy.
> 
> Regards,
> -drc
> (Speaking for myself)
> 
> ___
> dns-operations mailing list
> dns-operations@lists.dns-oarc.net
> https://lists.dns-oarc.net/mailman/listinfo/dns-operations
___
dns-operations mailing list
dns-operations@lists.dns-oarc.net
https://lists.dns-oarc.net/mailman/listinfo/dns-operations

Re: [dns-operations] root? we don't need no stinkin' root!

[Mark Allman]

Let me try to get away from what is or is not "big" and ask two
questions.  (These are legit questions to me.  I have studied the
DNS a whole bunch, but I do not operate any non-trivial part of the
DNS and so that viewpoint is valuable to me.)

(1) Setting aside history and how things have been done and why
(which I am happy to stipulate is rational)... At this point,
are there tangible benefits for getting information about the
TLD nameservers to resolvers as needed via a network service?

(2) Are there fundamental problems that would arise in recursive
resolvers if the information about TLD nameservers was no longer
available via a network service, but instead had to come from a
file that was snarfed periodically?

Thanks!

allman


--
https://www.icir.org/mallman/
@mallman_icsi
___
dns-operations mailing list
dns-operations@lists.dns-oarc.net
https://lists.dns-oarc.net/mailman/listinfo/dns-operations

Re: [dns-operations] root? we don't need no stinkin' root!

[David Conrad]

On Nov 26, 2019, at 11:33 AM, Jim Reid  wrote:
>> On 26 Nov 2019, at 09:16, Florian Weimer > > wrote:
>> 
>> Up until recently, well-behaved recursive resolvers had to forward
>> queries to the root if they were not already covered by a delegation.
>> RFC 7816 and in particular RFC 8198 changed that, but before that, it
>> was just how the protocol was expected to work.
> 
> So what? These RFCs make very little difference to the volume of queries a 
> resolving server will send to the root. QNAME minimisation has no impact at 
> all: the root just sees a query for .com instead of foobar.com 
> . A recursive resolver should already be supporting 
> negative caching and will have a reasonably complete picture of what's in (or 
> not in) the root. RFC8198 will of course help that but not by much IMO.

It would appear a rather large percentage of queries to the root (like 50% in 
some samples) are random strings, between 7 to 15 characters long, sometimes 
longer.  I believe this is Chrome-style probing to determine if there is 
NXDOMAIN redirection. A good example of the tragedy of the commons, like water 
pollution and climate change.

If resolvers would enable DNSSEC validation, there would, in theory, be a 
reduction in these queries due to aggressive NSEC caching.  Of course, practice 
may not match theory 
(https://indico.dns-oarc.net/event/32/contributions/717/attachments/713/1206/2019-10-31-oarc-nsec-caching.pdf).

Of course, steady state query load is largely irrelevant since root service has 
to be provisioned with massive DDoS in mind. In my personal view, the 
deployment of additional anycast instances by the root server operators is a 
useful stopgap, but ultimately, given the rate of growth of DoS attack capacity 
(and assuming that growth will continue due to the stunning security practices 
of IoT device manufacturers), stuff like what is discussed in that paper is the 
right long term strategy.

Regards,
-drc
(Speaking for myself)



signature.asc
Description: Message signed with OpenPGP
___
dns-operations mailing list
dns-operations@lists.dns-oarc.net
https://lists.dns-oarc.net/mailman/listinfo/dns-operations

Re: [dns-operations] root? we don't need no stinkin' root!

[Florian Weimer]

* Jim Reid:

>> On 26 Nov 2019, at 09:16, Florian Weimer  wrote:
>> 
>> Up until recently, well-behaved recursive resolvers had to forward
>> queries to the root if they were not already covered by a delegation.
>> RFC 7816 and in particular RFC 8198 changed that, but before that, it
>> was just how the protocol was expected to work.
>
> So what? These RFCs make very little difference to the volume of
> queries a resolving server will send to the root. QNAME minimisation
> has no impact at all: the root just sees a query for .com instead of
> foobar.com.

QNAME minimization allows a resolver to blacklist (say) the CORP
subtree, based on the NXDOMAIN response for CORP.  If the full query
is sent to the root, it is only possible to cache the NXDOMAIN for the
exact QNAME, and not its siblings.  (This assumes that the root deals
with empty non-terminals in the expected way, but that seems to be a
reasonable assumption for the root zone.)
___
dns-operations mailing list
dns-operations@lists.dns-oarc.net
https://lists.dns-oarc.net/mailman/listinfo/dns-operations

Re: [dns-operations] root? we don't need no stinkin' root!

[Jim Reid]

> On 26 Nov 2019, at 09:16, Florian Weimer  wrote:
> 
> Up until recently, well-behaved recursive resolvers had to forward
> queries to the root if they were not already covered by a delegation.
> RFC 7816 and in particular RFC 8198 changed that, but before that, it
> was just how the protocol was expected to work.

So what? These RFCs make very little difference to the volume of queries a 
resolving server will send to the root. QNAME minimisation has no impact at 
all: the root just sees a query for .com instead of foobar.com. A recursive 
resolver should already be supporting negative caching and will have a 
reasonably complete picture of what's in (or not in) the root. RFC8198 will of 
course help that but not by much IMO.



___
dns-operations mailing list
dns-operations@lists.dns-oarc.net
https://lists.dns-oarc.net/mailman/listinfo/dns-operations

Re: [dns-operations] root? we don't need no stinkin' root!

[Florian Weimer]

* Jim Reid:

>> On 25 Nov 2019, at 22:19, Florian Weimer  wrote:
>> 
>>> What do you consider to be a lot of queries? The root server system
>>> collectively handles 500K-1M queries per second. That seems rather a
>>> lot to me. YMMV.
>> 
>> But globally?  For the entire planet?
>
> Yes. If you consider a well-behaved recursive resolver would only
> query the root a few times a day (probably < 10), a query load of
> 0.5-1M/second seems on the high side, even for the whole planet.

Up until recently, well-behaved recursive resolvers had to forward
queries to the root if they were not already covered by a delegation.
RFC 7816 and in particular RFC 8198 changed that, but before that, it
was just how the protocol was expected to work.
___
dns-operations mailing list
dns-operations@lists.dns-oarc.net
https://lists.dns-oarc.net/mailman/listinfo/dns-operations

Re: [dns-operations] root? we don't need no stinkin' root!

[Jim Reid]

> On 25 Nov 2019, at 22:31, Paul Ebersman  
> wrote:
> 
> Actually, it's a great argument for longer TTLs and caching doing what
> they're supposed to.

It would be if the root only got queries from well behaved recursive resolvers. 
But we both know Paul that simply isn't true.

Well over 90% of the query traffic at the root has no reason to be going there 
at all. For instance stub resolvers that don't care about TTLs or do any sort 
of caching, Chrome's 10-character nonce strings to detect NXDOMAIN rewriting, 
CPE querying for .home, enterprises leaking queries for .corp, etc, etc.

The amount and breadth of the crap that hits the root is staggering. I suppose 
that'll also be true for the recursive service offered by the likes of google 
or Comcast.
___
dns-operations mailing list
dns-operations@lists.dns-oarc.net
https://lists.dns-oarc.net/mailman/listinfo/dns-operations

Re: [dns-operations] root? we don't need no stinkin' root!

[Jim Reid]

> On 25 Nov 2019, at 22:19, Florian Weimer  wrote:
> 
>> What do you consider to be a lot of queries? The root server system
>> collectively handles 500K-1M queries per second. That seems rather a
>> lot to me. YMMV.
> 
> But globally?  For the entire planet?

Yes. If you consider a well-behaved recursive resolver would only query the 
root a few times a day (probably < 10), a query load of 0.5-1M/second seems on 
the high side, even for the whole planet.

> The data could have monetary value.

Could. But nobody's selling those data. And that's probably illegal anyway 
thanks to GDPR.

> Passwords that are otherwise difficult to come by might be leaking.

Good luck finding those in amongst the terabytes of dross that hits the root 
every day.


___
dns-operations mailing list
dns-operations@lists.dns-oarc.net
https://lists.dns-oarc.net/mailman/listinfo/dns-operations

Re: [dns-operations] root? we don't need no stinkin' root!

[Fred Morris]

Funny you should mention this. It just occurred to me, although it also 
apparently occurred to one other soul on the dnsrpz mailing list, you can 
use RPZ to audit and to some extent contain leakage.


Assuming you own example.com, I'm speaking about entries akin to the 
following:


*.example.example.com CNAME .
*.com.example.com CNAME .
*.net.example.com CNAME .

Entries like the foregoing will return NXDOMAIN for, for example,
dolphin2.com.example.com. ;-) It's also possible to log or direct the 
querant to a honeypot. Granted, most likely the stub resolver is trying 
dolphin2.com.example.com because it already tried dolphin2 and 
dolphin2.com and both of those failed, but at least you know.


You can also see just how good your passive DNS provider's data is, by 
looking for things which resolved to 127.0.53.53. (This is a really good 
way for the casual reader to understand the scope of this problem, by the 
way.)


Running your own caching resolver and dumping the cache and looking for 
stuff is also occasionally advisable; I suspect most of the people on this 
list would know this.


--

Fred Morris

On Mon, 25 Nov 2019, Florian Weimer wrote:



Is it because of the incoming data is interesting?


Define interesting.


The data could have monetary value.  Passwords that are otherwise
difficult to come by might be leaking.

___
dns-operations mailing list
dns-operations@lists.dns-oarc.net
https://lists.dns-oarc.net/mailman/listinfo/dns-operations

Re: [dns-operations] root? we don't need no stinkin' root!

[Paul Ebersman]

jim> What do you consider to be a lot of queries? The root server system
jim> collectively handles 500K-1M queries per second. That seems rather
jim> a lot to me. YMMV.

fw> But globally?  For the entire planet?

fw> It's certainly beyond what I can run out of my basement using spare
fw> parts, but it's also not a mindbogglingly huge number.  I would have
fw> expected something that's clearly impossible to handle from a single
fw> box.

Actually, it's a great argument for longer TTLs and caching doing what
they're supposed to.

The root zones and most TLDs tend to have longer, non trendy (over 5
minute) TTLs, so root servers, TLDs and other auth servers get orders of
magnitude less queries than large recursive farms, which cache and then
get cache hits.

Comcast & Google get 2-3 orders of magnitude more than large TLD servers
and 4-5 orders of magnitude more than the root servers and these two
probably represent something like 1/3 of public recursive server
traffic. The largest Chinese ISP used to do more traffic then either of
the above.

But compared to a large corp DNS server farm, the root servers shovel a
lot of bits. Some of it even valid DNS queries and responses. ;)
___
dns-operations mailing list
dns-operations@lists.dns-oarc.net
https://lists.dns-oarc.net/mailman/listinfo/dns-operations

Re: [dns-operations] root? we don't need no stinkin' root!

[Florian Weimer]

* Jim Reid:

>> On 25 Nov 2019, at 20:54, Florian Weimer  wrote:
>> 
>> The query numbers are surprisingly low.  To me at last.
>
> What do you consider to be a lot of queries? The root server system
> collectively handles 500K-1M queries per second. That seems rather a
> lot to me. YMMV.

But globally?  For the entire planet?

It's certainly beyond what I can run out of my basement using spare
parts, but it's also not a mindbogglingly huge number.  I would have
expected something that's clearly impossible to handle from a single
box.

>> Is it because of the incoming data is interesting?
>
> Define interesting.

The data could have monetary value.  Passwords that are otherwise
difficult to come by might be leaking.
___
dns-operations mailing list
dns-operations@lists.dns-oarc.net
https://lists.dns-oarc.net/mailman/listinfo/dns-operations

Re: [dns-operations] root? we don't need no stinkin' root!

[Jim Reid]

> On 25 Nov 2019, at 20:54, Florian Weimer  wrote:
> 
> The query numbers are surprisingly low.  To me at last.

What do you consider to be a lot of queries? The root server system 
collectively handles 500K-1M queries per second. That seems rather a lot to me. 
YMMV. I don't know of any other IT platform that reliably handles transactions 
at anything close to that volume. Or orders of magnitude lower. IIUC Mastercard 
and Visa each handle around "only" 30K transactions/second.

Root server query numbers are continually rising. This is why suggestions like 
Mark's and RFC7706 need careful consideration. Ultimately, the root server 
operators won't be able to keep on adding capacity and bandwidth to keep up 
with demand or mitigate DDoS attacks. They'll eventually run out of 
money/bits/hardware before the script kiddies and their botnets do. Even though 
the RSOs are winning that arms race today.

> Do we know why the number of root instances has increased?

Partly it will be each RSO adding more instances to improve resilience, 
capacity and performance. They will also be adding more servers to address 
layer 9+ questions from countries who want to have more root servers inside 
their borders. IXPs/ISPs want that too, just like they want extra copies of 
local cache nodes from CDNs.

Some countries perceive the DNS root to be US-centric. When they're not on 
friendly terms with the USA, that can be a problem. Adding anycast root 
instances in say China or Russia can go some way to alleviate some of those 
concerns.

> Is it because of the incoming data is interesting?

Define interesting. IMO instances are being added for the reasons above. 
Whether the ever-growing volume of queries to the root is interesting or not is 
irrelevant IMO.


___
dns-operations mailing list
dns-operations@lists.dns-oarc.net
https://lists.dns-oarc.net/mailman/listinfo/dns-operations

Re: [dns-operations] root? we don't need no stinkin' root!

[Bill Woodcock]

> On Nov 25, 2019, at 9:54 PM, Florian Weimer  wrote:
> The query numbers are surprisingly low.  To me at last.

Duane Wessels did a good study some time ago of queries to the root.  I believe 
over 99% were bogus, not real queries for resolvable things.

> Do we know why the number of root instances has increased?  Is it
> because of the incoming data is interesting?

In some cases perhaps.  In our case, we typically install eight at each 
location, and we’ve passed two hundred locations now.  So this:

>The Domain Name System (DNS) leverages nearly 1K distributed
>servers

…is not exactly correct…  Perhaps it’s only 1K _locations_.

We provide them to make the root more resilient against DDoS, and to reduce 
query latency.  But we’re a non-profit which exists for that purpose, we don’t 
derive any revenue from it, and our finances are publicly audited.  For-profits 
require revenue, and there’s certainly a market for pcaps taken from in front 
of root servers.

-Bill



signature.asc
Description: Message signed with OpenPGP
___
dns-operations mailing list
dns-operations@lists.dns-oarc.net
https://lists.dns-oarc.net/mailman/listinfo/dns-operations

Re: [dns-operations] root? we don't need no stinkin' root!

[bert hubert]

On Mon, Nov 25, 2019 at 09:54:55PM +0100, Florian Weimer wrote:
> Do we know why the number of root instances has increased?  Is it
> because of the incoming data is interesting?

I would venture the latter. This remains a seriously underdiscussed subject. 

There is of course "logging of all data" which is bad enough but people
appear to be getting creative with doing "analyses on the 24 hours of logs
we are allowed to keep".

Bert
___
dns-operations mailing list
dns-operations@lists.dns-oarc.net
https://lists.dns-oarc.net/mailman/listinfo/dns-operations

Re: [dns-operations] root? we don't need no stinkin' root!

[Florian Weimer]

* Mark Allman:

> Left here to be ripped apart ... :-)

The query numbers are surprisingly low.  To me at last.

Do we know why the number of root instances has increased?  Is it
because of the incoming data is interesting?
___
dns-operations mailing list
dns-operations@lists.dns-oarc.net
https://lists.dns-oarc.net/mailman/listinfo/dns-operations

[dns-operations] root? we don't need no stinkin' root!

[Mark Allman]

Left here to be ripped apart ... :-)

  Mark Allman. On Eliminating Root Nameservers from the DNS, ACM
  SIGCOMM Workshop on Hot Topics in Networks (HotNets), November
  2019.
  https://www.icir.org/mallman/pubs/All19b/

  Abstract:
The Domain Name System (DNS) leverages nearly 1K distributed
servers to provide information about the root of the Internet's
namespace. The large size and broad distribution of the root
nameserver infrastructure has a number of benefits, including
providing robustness, low delays to topologically close root
servers and a way to cope with the immense torrent of queries
destined for the root nameservers. While the root nameserver
service operates well, it represents a large community
investment. Due to this large cost, in this paper we take the
position that DNS' root nameservers should be
eliminated. Instead, recursive resolvers should use a local copy
of the root zone file instead of consulting root
nameservers. This paper considers the pros and cons of this
alternate approach.

allman


--
https://www.icir.org/mallman/
@mallman_icsi
___
dns-operations mailing list
dns-operations@lists.dns-oarc.net
https://lists.dns-oarc.net/mailman/listinfo/dns-operations