Re: Configuring different TTLs in multiple RRs for the same domain name, TYPE, and CLASS

[Barry Margolin]

In article ,
 Dave Warren  wrote:

> On 2016-03-25 07:21, Barry Margolin wrote:
> > In article ,
> >   Dave Warren  wrote:
> >
> >> I'm more interested in the impact from the perspective of an
> >> authoritative server operator and in some respects sites that use short
> >> TTLs will increase the odds of my longer-TTL's records staying in the
> >> cache longer before it gets hit by a cache-size limit, but none of my
> >> zones are really large enough to do A/B testing.
> > IMHO, memory is so cheap these days that any server that has to eject
> > cache entries because of memory limits means the server operator isn't
> > really trying to do their job well.
> 
> If you're running a dedicated public/ISP/massive-corporation resolver, 
> sure, this is true. But if your resolver is some random DNS server on a 
> small corporate Active Directory and one of dozens of services on a 
> $1000 server with 1-50 users, who cares if your DNS cache only carries 5 
> minutes, 30 minutes, or 6 hours of cache?
> 
> In fact, if your resolver just forwards queries to your ISP, and your 
> ISP has dedicated caches, there would be very little measurable 
> difference at all. I'm not a fan of forwarding, but many admins set it 
> up because it's there without considering whether it's needed or not.

If you're running a resolver for a small organization, the cache isn't 
going to get huge in the first place. How many different names will 50 
users access in a day?

-- 
Barry Margolin
Arlington, MA
___
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe 
from this list

bind-users mailing list
bind-users@lists.isc.org
https://lists.isc.org/mailman/listinfo/bind-users

Re: Configuring different TTLs in multiple RRs for the same domain name, TYPE, and CLASS

[Dave Warren]

On 2016-03-25 07:21, Barry Margolin wrote:

In article ,
  Dave Warren  wrote:


I'm more interested in the impact from the perspective of an
authoritative server operator and in some respects sites that use short
TTLs will increase the odds of my longer-TTL's records staying in the
cache longer before it gets hit by a cache-size limit, but none of my
zones are really large enough to do A/B testing.

IMHO, memory is so cheap these days that any server that has to eject
cache entries because of memory limits means the server operator isn't
really trying to do their job well.


If you're running a dedicated public/ISP/massive-corporation resolver, 
sure, this is true. But if your resolver is some random DNS server on a 
small corporate Active Directory and one of dozens of services on a 
$1000 server with 1-50 users, who cares if your DNS cache only carries 5 
minutes, 30 minutes, or 6 hours of cache?


In fact, if your resolver just forwards queries to your ISP, and your 
ISP has dedicated caches, there would be very little measurable 
difference at all. I'm not a fan of forwarding, but many admins set it 
up because it's there without considering whether it's needed or not.


--
Dave Warren
http://www.hireahit.com/
http://ca.linkedin.com/in/davejwarren


___
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe 
from this list

bind-users mailing list
bind-users@lists.isc.org
https://lists.isc.org/mailman/listinfo/bind-users

Re: Configuring different TTLs in multiple RRs for the same domain name, TYPE, and CLASS

[John Wobus]

> IMHO, memory is so cheap these days that any server that has to eject 
> cache entries because of memory limits means the server operator isn't 
> really trying to do their job well.

For handling host names, perhaps yes.

But it implies sanity on the part of all apps that your clients use.
App developers can use DNS as a database for any data, and in insanely large 
amounts.
Examples of things like this have been spam databases, and I vaguely recall
some social network doing something.  A single app could easily
push the DNS much harder, the only capacity limit being the tension between
how much memory DNS operators provide and client demands to support this
otherwise-wonderful app.

John Wobus
Cornell IT

___
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe 
from this list

bind-users mailing list
bind-users@lists.isc.org
https://lists.isc.org/mailman/listinfo/bind-users

Re: Configuring different TTLs in multiple RRs for the same domain name, TYPE, and CLASS

[Dave Warren]

On 2016-03-24 18:28, Barry Margolin wrote:

In article ,
  Dave Warren  wrote:


On 2016-03-24 15:20, Tony Finch wrote:

Dave Warren  wrote:

On 2016-03-24 09:46, Ray Bellis wrote:

On 24/03/2016 16:41, Tony Finch wrote:


When I changed our TTLs from 24h to 1h last year, it didn't have a
visible
effect on authoritative server query load, much to my surprise.

I'm not that surprised - there's definitely not a linear correlation
between the TTL of an RRset and how frequently it's queried.

Unless your TTL is very short, forced expulsion from cache (due to
cache-size limits) would cause many clients to re-query for a record far
more frequently than once-per-TTL.

Has anyone ever done any evaluation on this? For average resolvers, what
is the longest TTL that has any utility?

There was a great paper published 15 years ago describing a study of DNS
cache effectiveness at MIT. http://nms.csail.mit.edu/projects/dns/

It concluded (amongst other things) that NS records (and associated
address records) are really important, but leaf records that users ask for
don't matter so much. (Based on cache hits before TTL expiry, IIRC.)

I don't know of a similar study performed more recently.

The internet was a very different place 15 years ago, in particular,
this was before every Windows client machine had it's own DNS cache
service and largely before today's connected mobile devices were a thing.

But it was also before the widespread use of CDNs (Akamai was founded
only 3 years earlier). These days, the most heavily used web sites use
CDNs, which make heavy use of short TTLs for the leaf CNAME and A
records.



Yeah, that's a factor too.

I'm more interested in the impact from the perspective of an 
authoritative server operator and in some respects sites that use short 
TTLs will increase the odds of my longer-TTL's records staying in the 
cache longer before it gets hit by a cache-size limit, but none of my 
zones are really large enough to do A/B testing.



--
Dave Warren
http://www.hireahit.com/
http://ca.linkedin.com/in/davejwarren


___
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe 
from this list

bind-users mailing list
bind-users@lists.isc.org
https://lists.isc.org/mailman/listinfo/bind-users

Re: Configuring different TTLs in multiple RRs for the same domain name, TYPE, and CLASS

[Barry Margolin]

In article ,
 Dave Warren  wrote:

> On 2016-03-24 15:20, Tony Finch wrote:
> > Dave Warren  wrote:
> >> On 2016-03-24 09:46, Ray Bellis wrote:
> >>> On 24/03/2016 16:41, Tony Finch wrote:
> >>>
> > When I changed our TTLs from 24h to 1h last year, it didn't have a 
> > visible
> > effect on authoritative server query load, much to my surprise.
> >>> I'm not that surprised - there's definitely not a linear correlation
> >>> between the TTL of an RRset and how frequently it's queried.
> >>>
> >>> Unless your TTL is very short, forced expulsion from cache (due to
> >>> cache-size limits) would cause many clients to re-query for a record far
> >>> more frequently than once-per-TTL.
> >> Has anyone ever done any evaluation on this? For average resolvers, what
> >> is the longest TTL that has any utility?
> > There was a great paper published 15 years ago describing a study of DNS
> > cache effectiveness at MIT. http://nms.csail.mit.edu/projects/dns/
> >
> > It concluded (amongst other things) that NS records (and associated
> > address records) are really important, but leaf records that users ask for
> > don't matter so much. (Based on cache hits before TTL expiry, IIRC.)
> >
> > I don't know of a similar study performed more recently.
> 
> The internet was a very different place 15 years ago, in particular, 
> this was before every Windows client machine had it's own DNS cache 
> service and largely before today's connected mobile devices were a thing.

But it was also before the widespread use of CDNs (Akamai was founded 
only 3 years earlier). These days, the most heavily used web sites use 
CDNs, which make heavy use of short TTLs for the leaf CNAME and A 
records.

-- 
Barry Margolin
Arlington, MA
___
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe 
from this list

bind-users mailing list
bind-users@lists.isc.org
https://lists.isc.org/mailman/listinfo/bind-users

Re: Configuring different TTLs in multiple RRs for the same domain name, TYPE, and CLASS

[Tony Finch]

Dave Warren  wrote:
> On 2016-03-24 09:46, Ray Bellis wrote:
> > On 24/03/2016 16:41, Tony Finch wrote:
> >
> > > >When I changed our TTLs from 24h to 1h last year, it didn't have a 
> > > >visible
> > > >effect on authoritative server query load, much to my surprise.
> >
> > I'm not that surprised - there's definitely not a linear correlation
> > between the TTL of an RRset and how frequently it's queried.
> >
> > Unless your TTL is very short, forced expulsion from cache (due to
> > cache-size limits) would cause many clients to re-query for a record far
> > more frequently than once-per-TTL.
>
> Has anyone ever done any evaluation on this? For average resolvers, what
> is the longest TTL that has any utility?

There was a great paper published 15 years ago describing a study of DNS
cache effectiveness at MIT. http://nms.csail.mit.edu/projects/dns/

It concluded (amongst other things) that NS records (and associated
address records) are really important, but leaf records that users ask for
don't matter so much. (Based on cache hits before TTL expiry, IIRC.)

I don't know of a similar study performed more recently.

https://00f.net/2012/05/10/distribution-of-dns-ttls/ is also interesting.

Tony.
-- 
f.anthony.n.finch    http://dotat.at/  -  I xn--zr8h punycode
Southeast Iceland: Westerly, becoming cyclonic later, 6 to gale 8, increasing
severe gale 9 for a time. Moderate or rough, becoming rough or very rough.
Occasional rain, wintry showers for a time. Moderate or good, occasionally
poor.
___
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe 
from this list

bind-users mailing list
bind-users@lists.isc.org
https://lists.isc.org/mailman/listinfo/bind-users

Re: Configuring different TTLs in multiple RRs for the same domain name, TYPE, and CLASS

[Dave Warren]

On 2016-03-24 09:50, Barry Margolin wrote:

The problem with this is that when the Office 365 records expire and are
removed from the cache, but the other records have not, the server will
not know that it should re-query for the O365 records. It still has TXT
records in its cache, and it will return them in response to a query.

It won't go back to the authoritative server until ALL the TXT records
expire. During the period between the short TTL and the longest TTL, it
will be as if the short-TTL records don't exist at all.


Or if a caching resolver did make a "note to self" that there are 
missing records that need to be replaced, what would be the point of 
keeping any records with a longer TTL? A resolver would still be sending 
the same queries to refresh the entry with the shortest TTL anyway, so 
it wouldn't reduce the query volume.


--
Dave Warren
http://www.hireahit.com/
http://ca.linkedin.com/in/davejwarren


___
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe 
from this list

bind-users mailing list
bind-users@lists.isc.org
https://lists.isc.org/mailman/listinfo/bind-users

Re: Configuring different TTLs in multiple RRs for the same domain name, TYPE, and CLASS

[Dave Warren]

On 2016-03-24 09:46, Ray Bellis wrote:

On 24/03/2016 16:41, Tony Finch wrote:


>When I changed our TTLs from 24h to 1h last year, it didn't have a visible
>effect on authoritative server query load, much to my surprise.

I'm not that surprised - there's definitely not a linear correlation
between the TTL of an RRset and how frequently it's queried.

Unless your TTL is very short, forced expulsion from cache (due to
cache-size limits) would cause many clients to re-query for a record far
more frequently than once-per-TTL.


Has anyone ever done any evaluation on this? For average resolvers, what 
is the longest TTL that has any utility?


--
Dave Warren
http://www.hireahit.com/
http://ca.linkedin.com/in/davejwarren


___
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe 
from this list

bind-users mailing list
bind-users@lists.isc.org
https://lists.isc.org/mailman/listinfo/bind-users

Re: Configuring different TTLs in multiple RRs for the same domain name, TYPE, and CLASS

[Mark Andrews]

Some of us tried very hard to have the TXT to SPF migration continue
but the working group decided that the transition wasn't happening
fast enough and there were "interoperability" issues.  Somehow a
experiment about which type of data was needed to authenticate email
morphed into a experiment about can you transition from TXT to SPF.

Every DNS person knew that it would take a long time to do the
transition but that it would be effectively complete in about 10
years.  The timescale required to have upgraded code written and
then deployed through normal server upgrades.

Mark

In message , Ben Bridges writes:
> I tend to agree with you about the overloading of TXT records.
> 
> Thanks,
> Ben
> 
> -Original Message-
> From: bind-users-boun...@lists.isc.org [mailto:bind-users-boun...@lists.isc.o
> rg] On Behalf Of Ray Bellis
> Sent: Thursday, March 24, 2016 11:22 AM
> To: bind-users@lists.isc.org
> Subject: Re: Configuring different TTLs in multiple RRs for the same domain n
> ame, TYPE, and CLASS
> 
> On 24/03/2016 16:18, Ben Bridges wrote:
> > TXT records are multiple-purpose.  They can be used for SPF records, 
> > Office 365 "MS" records, DMARC records, or whatever arbitrary uses 
> > someone dreams up, all for the same domain name.  Microsoft wants a 
> > short TTL for their Office 365 records, but I would prefer to 
> > generally use a longer TTL for most records (including other TXT 
> > records) in order to reduce the query load on our servers.  It would 
> > be nice to be able to set a short TTL for the Office 365 record but a 
> > longer TTL for other TXT records for the same domain name.
> 
> OK, I can see why you'd want that, but it's yet another reason why overloadin
> g TXT records for multiple purposes was (and remains) a bad idea :(
> 
> As explained by RFC 2181, an RRset is supposed to be indivisible, and "bad th
> ings happen" if some parts of an RRset expire before others.
> 
> Ray
> 
> 
> ___
> Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe
>  from this list
> 
> bind-users mailing list
> bind-users@lists.isc.org
> https://lists.isc.org/mailman/listinfo/bind-users
> ___
> Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe
>  from this list
> 
> bind-users mailing list
> bind-users@lists.isc.org
> https://lists.isc.org/mailman/listinfo/bind-users
-- 
Mark Andrews, ISC
1 Seymour St., Dundas Valley, NSW 2117, Australia
PHONE: +61 2 9871 4742 INTERNET: ma...@isc.org
___
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe 
from this list

bind-users mailing list
bind-users@lists.isc.org
https://lists.isc.org/mailman/listinfo/bind-users

Re: Configuring different TTLs in multiple RRs for the same domain name, TYPE, and CLASS

[Ray Bellis]

On 24/03/2016 16:41, Tony Finch wrote:

> When I changed our TTLs from 24h to 1h last year, it didn't have a visible
> effect on authoritative server query load, much to my surprise.

I'm not that surprised - there's definitely not a linear correlation
between the TTL of an RRset and how frequently it's queried.

Unless your TTL is very short, forced expulsion from cache (due to
cache-size limits) would cause many clients to re-query for a record far
more frequently than once-per-TTL.

Ray


___
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe 
from this list

bind-users mailing list
bind-users@lists.isc.org
https://lists.isc.org/mailman/listinfo/bind-users

Re: Configuring different TTLs in multiple RRs for the same domain name, TYPE, and CLASS

[Barry Margolin]

In article ,
 Ben Bridges  wrote:

> TXT records are multiple-purpose.  They can be used for SPF records, Office 
> 365 "MS" records, DMARC records, or whatever arbitrary uses someone dreams 
> up, all for the same domain name.  Microsoft wants a short TTL for their 
> Office 365 records, but I would prefer to generally use a longer TTL for most 
> records (including other TXT records) in order to reduce the query load on 
> our servers.  It would be nice to be able to set a short TTL for the Office 
> 365 record but a longer TTL for other TXT records for the same domain name.

The problem with this is that when the Office 365 records expire and are 
removed from the cache, but the other records have not, the server will 
not know that it should re-query for the O365 records. It still has TXT 
records in its cache, and it will return them in response to a query.

It won't go back to the authoritative server until ALL the TXT records 
expire. During the period between the short TTL and the longest TTL, it 
will be as if the short-TTL records don't exist at all.

-- 
Barry Margolin
Arlington, MA
___
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe 
from this list

bind-users mailing list
bind-users@lists.isc.org
https://lists.isc.org/mailman/listinfo/bind-users

RE: Configuring different TTLs in multiple RRs for the same domain name, TYPE, and CLASS

[Tony Finch]

Ben Bridges  wrote:

> Microsoft wants a short TTL for their Office 365 records, but I would
> prefer to generally use a longer TTL for most records (including other
> TXT records) in order to reduce the query load on our servers.

I gather MS ask for a 1 hour TTL - at least, that is what has been
specified in the requests I have dealt with.

When I changed our TTLs from 24h to 1h last year, it didn't have a visible
effect on authoritative server query load, much to my surprise.

If I were you, I wouldn't worry too much :-)

Tony.
-- 
f.anthony.n.finch    http://dotat.at/  -  I xn--zr8h punycode
Forties, Cromarty, Forth, Tyne, Dogger: Southwest, veering northwest for a
time, 4 or 5, increasing 6 at times. Slight or moderate. Occasional rain.
Good, occasionally poor.
___
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe 
from this list

bind-users mailing list
bind-users@lists.isc.org
https://lists.isc.org/mailman/listinfo/bind-users

RE: Configuring different TTLs in multiple RRs for the same domain name, TYPE, and CLASS

[Ben Bridges]

TXT records are multiple-purpose.  They can be used for SPF records, Office 365 
"MS" records, DMARC records, or whatever arbitrary uses someone dreams up, all 
for the same domain name.  Microsoft wants a short TTL for their Office 365 
records, but I would prefer to generally use a longer TTL for most records 
(including other TXT records) in order to reduce the query load on our servers. 
 It would be nice to be able to set a short TTL for the Office 365 record but a 
longer TTL for other TXT records for the same domain name.

Thanks,
Ben

From: bind-users-boun...@lists.isc.org 
[mailto:bind-users-boun...@lists.isc.org] On Behalf Of Darcy Kevin (FCA)
Sent: Thursday, March 24, 2016 9:55 AM
To: bind-users@lists.isc.org
Subject: RE: Configuring different TTLs in multiple RRs for the same domain 
name, TYPE, and CLASS


This is deliberately forbidden by standard. See RFC 2181, Section 5.2 ("TTLs of 
RRs in an RRSet")

Why would you want to do this?


- Kevin


From: bind-users-boun...@lists.isc.org 
[mailto:bind-users-boun...@lists.isc.org] On Behalf Of Ben Bridges
Sent: Thursday, March 24, 2016 10:48 AM
To: bind-users@lists.isc.org
Subject: Configuring different TTLs in multiple RRs for the same domain name, 
TYPE, and CLASS

Greetings.

Is it possible in BIND to configure multiple resource records for the same 
domain name, TYPE, and CLASS with different TTL values?  For example:

Test-txt.example.com 300IN   TXT"Test 300"
Test-txt.example.com 400IN   TXT"Test 400"
Test-txt.example.com 500IN   TXT"Test 500"
Test-txt.example.com 600IN   TXT"Test 600"
Test-txt.example.com 700IN   TXT"Test 700"

I tried it, and BIND set the TTL for all five records to 300 (or more 
specifically, the TTL of the first one of the RRs in the file).  I looked for a 
BIND directive in the manual to change this behavior but could find no obvious 
candidate.

Thanks,

Ben Bridges
Springfield, MO

___
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe 
from this list

bind-users mailing list
bind-users@lists.isc.org
https://lists.isc.org/mailman/listinfo/bind-users

Re: Configuring different TTLs in multiple RRs for the same domain name, TYPE, and CLASS

[Ray Bellis]

On 24/03/2016 14:47, Ben Bridges wrote:
> Greetings.
> 
>  
> 
> Is it possible in BIND to configure multiple resource records for the
> same domain name, TYPE, and CLASS with different TTL values?  For example:
>
> ...
>
> I tried it, and BIND set the TTL for all five records to 300 (or more
> specifically, the TTL of the first one of the RRs in the file).  I
> looked for a BIND directive in the manual to change this behavior but
> could find no obvious candidate.

Doing so would be contrary to §5.2 of RFC 2181:

   Resource Records also have a time to live (TTL).  It is possible for
   the RRs in an RRSet to have different TTLs.  No uses for this have
   been found that cannot be better accomplished in other ways.  This
   can, however, cause partial replies (not marked "truncated") from a
   caching server, where the TTLs for some but not all the RRs in the
   RRSet have expired.

   Consequently the use of differing TTLs in an RRSet is hereby
   deprecated, the TTLs of all RRs in an RRSet must be the same.

Ray

___
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe 
from this list

bind-users mailing list
bind-users@lists.isc.org
https://lists.isc.org/mailman/listinfo/bind-users

RE: Configuring different TTLs in multiple RRs for the same domain name, TYPE, and CLASS

[Darcy Kevin (FCA)]

This is deliberately forbidden by standard. See RFC 2181, Section 5.2 ("TTLs of 
RRs in an RRSet")

Why would you want to do this?


- Kevin


From: bind-users-boun...@lists.isc.org 
[mailto:bind-users-boun...@lists.isc.org] On Behalf Of Ben Bridges
Sent: Thursday, March 24, 2016 10:48 AM
To: bind-users@lists.isc.org
Subject: Configuring different TTLs in multiple RRs for the same domain name, 
TYPE, and CLASS

Greetings.

Is it possible in BIND to configure multiple resource records for the same 
domain name, TYPE, and CLASS with different TTL values?  For example:

Test-txt.example.com 300IN   TXT"Test 300"
Test-txt.example.com 400IN   TXT"Test 400"
Test-txt.example.com 500IN   TXT"Test 500"
Test-txt.example.com 600IN   TXT"Test 600"
Test-txt.example.com 700IN   TXT"Test 700"

I tried it, and BIND set the TTL for all five records to 300 (or more 
specifically, the TTL of the first one of the RRs in the file).  I looked for a 
BIND directive in the manual to change this behavior but could find no obvious 
candidate.

Thanks,

Ben Bridges
Springfield, MO

___
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe 
from this list

bind-users mailing list
bind-users@lists.isc.org
https://lists.isc.org/mailman/listinfo/bind-users

Configuring different TTLs in multiple RRs for the same domain name, TYPE, and CLASS

[Ben Bridges]

Greetings.

Is it possible in BIND to configure multiple resource records for the same 
domain name, TYPE, and CLASS with different TTL values?  For example:

Test-txt.example.com 300IN   TXT"Test 300"
Test-txt.example.com 400IN   TXT"Test 400"
Test-txt.example.com 500IN   TXT"Test 500"
Test-txt.example.com 600IN   TXT"Test 600"
Test-txt.example.com 700IN   TXT"Test 700"

I tried it, and BIND set the TTL for all five records to 300 (or more 
specifically, the TTL of the first one of the RRs in the file).  I looked for a 
BIND directive in the manual to change this behavior but could find no obvious 
candidate.

Thanks,

Ben Bridges
Springfield, MO

___
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe 
from this list

bind-users mailing list
bind-users@lists.isc.org
https://lists.isc.org/mailman/listinfo/bind-users