Re: Configuring different TTLs in multiple RRs for the same domain name, TYPE, and CLASS

[Barry Margolin]

In article ,
 Dave Warren  wrote:

> On 2016-03-24 15:20, Tony Finch wrote:
> > Dave Warren  wrote:
> >> On 2016-03-24 09:46, Ray Bellis wrote:
> >>> On 24/03/2016 16:41, Tony Finch wrote:
> >>>
> > When I changed our TTLs from 24h to 1h last year, it didn't have a 
> > visible
> > effect on authoritative server query load, much to my surprise.
> >>> I'm not that surprised - there's definitely not a linear correlation
> >>> between the TTL of an RRset and how frequently it's queried.
> >>>
> >>> Unless your TTL is very short, forced expulsion from cache (due to
> >>> cache-size limits) would cause many clients to re-query for a record far
> >>> more frequently than once-per-TTL.
> >> Has anyone ever done any evaluation on this? For average resolvers, what
> >> is the longest TTL that has any utility?
> > There was a great paper published 15 years ago describing a study of DNS
> > cache effectiveness at MIT. http://nms.csail.mit.edu/projects/dns/
> >
> > It concluded (amongst other things) that NS records (and associated
> > address records) are really important, but leaf records that users ask for
> > don't matter so much. (Based on cache hits before TTL expiry, IIRC.)
> >
> > I don't know of a similar study performed more recently.
> 
> The internet was a very different place 15 years ago, in particular, 
> this was before every Windows client machine had it's own DNS cache 
> service and largely before today's connected mobile devices were a thing.

But it was also before the widespread use of CDNs (Akamai was founded 
only 3 years earlier). These days, the most heavily used web sites use 
CDNs, which make heavy use of short TTLs for the leaf CNAME and A 
records.

-- 
Barry Margolin
Arlington, MA
___
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe 
from this list

bind-users mailing list
bind-users@lists.isc.org
https://lists.isc.org/mailman/listinfo/bind-users

Re: Configuring different TTLs in multiple RRs for the same domain name, TYPE, and CLASS

[Tony Finch]

Dave Warren  wrote:
> On 2016-03-24 09:46, Ray Bellis wrote:
> > On 24/03/2016 16:41, Tony Finch wrote:
> >
> > > >When I changed our TTLs from 24h to 1h last year, it didn't have a 
> > > >visible
> > > >effect on authoritative server query load, much to my surprise.
> >
> > I'm not that surprised - there's definitely not a linear correlation
> > between the TTL of an RRset and how frequently it's queried.
> >
> > Unless your TTL is very short, forced expulsion from cache (due to
> > cache-size limits) would cause many clients to re-query for a record far
> > more frequently than once-per-TTL.
>
> Has anyone ever done any evaluation on this? For average resolvers, what
> is the longest TTL that has any utility?

There was a great paper published 15 years ago describing a study of DNS
cache effectiveness at MIT. http://nms.csail.mit.edu/projects/dns/

It concluded (amongst other things) that NS records (and associated
address records) are really important, but leaf records that users ask for
don't matter so much. (Based on cache hits before TTL expiry, IIRC.)

I don't know of a similar study performed more recently.

https://00f.net/2012/05/10/distribution-of-dns-ttls/ is also interesting.

Tony.
-- 
f.anthony.n.finch    http://dotat.at/  -  I xn--zr8h punycode
Southeast Iceland: Westerly, becoming cyclonic later, 6 to gale 8, increasing
severe gale 9 for a time. Moderate or rough, becoming rough or very rough.
Occasional rain, wintry showers for a time. Moderate or good, occasionally
poor.
___
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe 
from this list

bind-users mailing list
bind-users@lists.isc.org
https://lists.isc.org/mailman/listinfo/bind-users

Re: Regarding compiling BIND 9.10.3-p4 on a SystemD Distro

[Sean Son]

Thank you for the replies everyone. Are there any major differences between
the BIND package that Red Hat/CentOS provides vs the BIND package provided
by the ISC website?


Thanks

On Thu, Mar 24, 2016 at 8:19 AM, Tony Finch  wrote:

> Mark Andrews  wrote:
> >
> > That said named does wait for the loading to complete before the
> > parent process exits and its exit status reflects if the load
> > succeeded or not.  See ns_os_daemonize() and ns_os_started().
>
> OK, I'm feeling pretty stupid now, and I'm sorry for saying that named
> doesn't do the right thing when starting. (Maybe I didn't have a good
> reason for that anti-race logic after all!) Oh well, on the bright side
> it means that a systemd unit file named can in fact be quite simple, and
> I get to delete some code ...
>
> Tony.
> --
> f.anthony.n.finch    http://dotat.at/  -  I xn--zr8h
> punycode
> Southeast Iceland: Southeasterly 5 or 6, veering southwesterly 6 to gale 8,
> occasionally severe gale 9 later. Moderate or rough, becoming rough or very
> rough. Rain then wintry showers. Good, occasionally poor.
>
> --- rc.named
> +++ rc.named
> @@ -35,15 +35,8 @@ start() {
> log_daemon_msg "Starting name server" "BIND"
> start-stop-daemon --start --oknodo $SSD --startas $TOP/bin/named \
> -- -t $TOP -u named -c /etc/named.conf -L $LOGFILE
> -   i=$(( $? ? 100 : 0 ))
> -   while   [ $i -lt 100 ] &&
> -   ! rndc status >/dev/null 2>&1
> -   do  sleep 0.1
> -   i=$((i+1))
> -   done
> -   chmod g+r $RUN/session.key
> -   rndc status >/dev/null 2>&1
> log_end_msg $?
> +   chmod g+r $RUN/session.key
>  }
>
>  stop() {
> ___
> Please visit https://lists.isc.org/mailman/listinfo/bind-users to
> unsubscribe from this list
>
> bind-users mailing list
> bind-users@lists.isc.org
> https://lists.isc.org/mailman/listinfo/bind-users
>
___
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe 
from this list

bind-users mailing list
bind-users@lists.isc.org
https://lists.isc.org/mailman/listinfo/bind-users

Re: Configuring different TTLs in multiple RRs for the same domain name, TYPE, and CLASS

[Dave Warren]

On 2016-03-24 09:50, Barry Margolin wrote:

The problem with this is that when the Office 365 records expire and are
removed from the cache, but the other records have not, the server will
not know that it should re-query for the O365 records. It still has TXT
records in its cache, and it will return them in response to a query.

It won't go back to the authoritative server until ALL the TXT records
expire. During the period between the short TTL and the longest TTL, it
will be as if the short-TTL records don't exist at all.


Or if a caching resolver did make a "note to self" that there are 
missing records that need to be replaced, what would be the point of 
keeping any records with a longer TTL? A resolver would still be sending 
the same queries to refresh the entry with the shortest TTL anyway, so 
it wouldn't reduce the query volume.


--
Dave Warren
http://www.hireahit.com/
http://ca.linkedin.com/in/davejwarren


___
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe 
from this list

bind-users mailing list
bind-users@lists.isc.org
https://lists.isc.org/mailman/listinfo/bind-users

Re: Configuring different TTLs in multiple RRs for the same domain name, TYPE, and CLASS

[Dave Warren]

On 2016-03-24 09:46, Ray Bellis wrote:

On 24/03/2016 16:41, Tony Finch wrote:


>When I changed our TTLs from 24h to 1h last year, it didn't have a visible
>effect on authoritative server query load, much to my surprise.

I'm not that surprised - there's definitely not a linear correlation
between the TTL of an RRset and how frequently it's queried.

Unless your TTL is very short, forced expulsion from cache (due to
cache-size limits) would cause many clients to re-query for a record far
more frequently than once-per-TTL.


Has anyone ever done any evaluation on this? For average resolvers, what 
is the longest TTL that has any utility?


--
Dave Warren
http://www.hireahit.com/
http://ca.linkedin.com/in/davejwarren


___
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe 
from this list

bind-users mailing list
bind-users@lists.isc.org
https://lists.isc.org/mailman/listinfo/bind-users

Re: Configuring different TTLs in multiple RRs for the same domain name, TYPE, and CLASS

[Mark Andrews]

Some of us tried very hard to have the TXT to SPF migration continue
but the working group decided that the transition wasn't happening
fast enough and there were "interoperability" issues.  Somehow a
experiment about which type of data was needed to authenticate email
morphed into a experiment about can you transition from TXT to SPF.

Every DNS person knew that it would take a long time to do the
transition but that it would be effectively complete in about 10
years.  The timescale required to have upgraded code written and
then deployed through normal server upgrades.

Mark

In message , Ben Bridges writes:
> I tend to agree with you about the overloading of TXT records.
> 
> Thanks,
> Ben
> 
> -Original Message-
> From: bind-users-boun...@lists.isc.org [mailto:bind-users-boun...@lists.isc.o
> rg] On Behalf Of Ray Bellis
> Sent: Thursday, March 24, 2016 11:22 AM
> To: bind-users@lists.isc.org
> Subject: Re: Configuring different TTLs in multiple RRs for the same domain n
> ame, TYPE, and CLASS
> 
> On 24/03/2016 16:18, Ben Bridges wrote:
> > TXT records are multiple-purpose.  They can be used for SPF records, 
> > Office 365 "MS" records, DMARC records, or whatever arbitrary uses 
> > someone dreams up, all for the same domain name.  Microsoft wants a 
> > short TTL for their Office 365 records, but I would prefer to 
> > generally use a longer TTL for most records (including other TXT 
> > records) in order to reduce the query load on our servers.  It would 
> > be nice to be able to set a short TTL for the Office 365 record but a 
> > longer TTL for other TXT records for the same domain name.
> 
> OK, I can see why you'd want that, but it's yet another reason why overloadin
> g TXT records for multiple purposes was (and remains) a bad idea :(
> 
> As explained by RFC 2181, an RRset is supposed to be indivisible, and "bad th
> ings happen" if some parts of an RRset expire before others.
> 
> Ray
> 
> 
> ___
> Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe
>  from this list
> 
> bind-users mailing list
> bind-users@lists.isc.org
> https://lists.isc.org/mailman/listinfo/bind-users
> ___
> Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe
>  from this list
> 
> bind-users mailing list
> bind-users@lists.isc.org
> https://lists.isc.org/mailman/listinfo/bind-users
-- 
Mark Andrews, ISC
1 Seymour St., Dundas Valley, NSW 2117, Australia
PHONE: +61 2 9871 4742 INTERNET: ma...@isc.org
___
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe 
from this list

bind-users mailing list
bind-users@lists.isc.org
https://lists.isc.org/mailman/listinfo/bind-users

Re: Configuring different TTLs in multiple RRs for the same domain name, TYPE, and CLASS

[Ray Bellis]

On 24/03/2016 16:41, Tony Finch wrote:

> When I changed our TTLs from 24h to 1h last year, it didn't have a visible
> effect on authoritative server query load, much to my surprise.

I'm not that surprised - there's definitely not a linear correlation
between the TTL of an RRset and how frequently it's queried.

Unless your TTL is very short, forced expulsion from cache (due to
cache-size limits) would cause many clients to re-query for a record far
more frequently than once-per-TTL.

Ray


___
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe 
from this list

bind-users mailing list
bind-users@lists.isc.org
https://lists.isc.org/mailman/listinfo/bind-users

Re: Configuring different TTLs in multiple RRs for the same domain name, TYPE, and CLASS

[Barry Margolin]

In article ,
 Ben Bridges  wrote:

> TXT records are multiple-purpose.  They can be used for SPF records, Office 
> 365 "MS" records, DMARC records, or whatever arbitrary uses someone dreams 
> up, all for the same domain name.  Microsoft wants a short TTL for their 
> Office 365 records, but I would prefer to generally use a longer TTL for most 
> records (including other TXT records) in order to reduce the query load on 
> our servers.  It would be nice to be able to set a short TTL for the Office 
> 365 record but a longer TTL for other TXT records for the same domain name.

The problem with this is that when the Office 365 records expire and are 
removed from the cache, but the other records have not, the server will 
not know that it should re-query for the O365 records. It still has TXT 
records in its cache, and it will return them in response to a query.

It won't go back to the authoritative server until ALL the TXT records 
expire. During the period between the short TTL and the longest TTL, it 
will be as if the short-TTL records don't exist at all.

-- 
Barry Margolin
Arlington, MA
___
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe 
from this list

bind-users mailing list
bind-users@lists.isc.org
https://lists.isc.org/mailman/listinfo/bind-users

RE: Configuring different TTLs in multiple RRs for the same domain name, TYPE, and CLASS

[Tony Finch]

Ben Bridges  wrote:

> Microsoft wants a short TTL for their Office 365 records, but I would
> prefer to generally use a longer TTL for most records (including other
> TXT records) in order to reduce the query load on our servers.

I gather MS ask for a 1 hour TTL - at least, that is what has been
specified in the requests I have dealt with.

When I changed our TTLs from 24h to 1h last year, it didn't have a visible
effect on authoritative server query load, much to my surprise.

If I were you, I wouldn't worry too much :-)

Tony.
-- 
f.anthony.n.finch    http://dotat.at/  -  I xn--zr8h punycode
Forties, Cromarty, Forth, Tyne, Dogger: Southwest, veering northwest for a
time, 4 or 5, increasing 6 at times. Slight or moderate. Occasional rain.
Good, occasionally poor.
___
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe 
from this list

bind-users mailing list
bind-users@lists.isc.org
https://lists.isc.org/mailman/listinfo/bind-users

RE: Configuring different TTLs in multiple RRs for the same domain name, TYPE, and CLASS

[Ben Bridges]

TXT records are multiple-purpose.  They can be used for SPF records, Office 365 
"MS" records, DMARC records, or whatever arbitrary uses someone dreams up, all 
for the same domain name.  Microsoft wants a short TTL for their Office 365 
records, but I would prefer to generally use a longer TTL for most records 
(including other TXT records) in order to reduce the query load on our servers. 
 It would be nice to be able to set a short TTL for the Office 365 record but a 
longer TTL for other TXT records for the same domain name.

Thanks,
Ben

From: bind-users-boun...@lists.isc.org 
[mailto:bind-users-boun...@lists.isc.org] On Behalf Of Darcy Kevin (FCA)
Sent: Thursday, March 24, 2016 9:55 AM
To: bind-users@lists.isc.org
Subject: RE: Configuring different TTLs in multiple RRs for the same domain 
name, TYPE, and CLASS


This is deliberately forbidden by standard. See RFC 2181, Section 5.2 ("TTLs of 
RRs in an RRSet")

Why would you want to do this?


- Kevin


From: bind-users-boun...@lists.isc.org 
[mailto:bind-users-boun...@lists.isc.org] On Behalf Of Ben Bridges
Sent: Thursday, March 24, 2016 10:48 AM
To: bind-users@lists.isc.org
Subject: Configuring different TTLs in multiple RRs for the same domain name, 
TYPE, and CLASS

Greetings.

Is it possible in BIND to configure multiple resource records for the same 
domain name, TYPE, and CLASS with different TTL values?  For example:

Test-txt.example.com 300IN   TXT"Test 300"
Test-txt.example.com 400IN   TXT"Test 400"
Test-txt.example.com 500IN   TXT"Test 500"
Test-txt.example.com 600IN   TXT"Test 600"
Test-txt.example.com 700IN   TXT"Test 700"

I tried it, and BIND set the TTL for all five records to 300 (or more 
specifically, the TTL of the first one of the RRs in the file).  I looked for a 
BIND directive in the manual to change this behavior but could find no obvious 
candidate.

Thanks,

Ben Bridges
Springfield, MO

___
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe 
from this list

bind-users mailing list
bind-users@lists.isc.org
https://lists.isc.org/mailman/listinfo/bind-users

Re: Configuring different TTLs in multiple RRs for the same domain name, TYPE, and CLASS

[Ray Bellis]

On 24/03/2016 14:47, Ben Bridges wrote:
> Greetings.
> 
>  
> 
> Is it possible in BIND to configure multiple resource records for the
> same domain name, TYPE, and CLASS with different TTL values?  For example:
>
> ...
>
> I tried it, and BIND set the TTL for all five records to 300 (or more
> specifically, the TTL of the first one of the RRs in the file).  I
> looked for a BIND directive in the manual to change this behavior but
> could find no obvious candidate.

Doing so would be contrary to §5.2 of RFC 2181:

   Resource Records also have a time to live (TTL).  It is possible for
   the RRs in an RRSet to have different TTLs.  No uses for this have
   been found that cannot be better accomplished in other ways.  This
   can, however, cause partial replies (not marked "truncated") from a
   caching server, where the TTLs for some but not all the RRs in the
   RRSet have expired.

   Consequently the use of differing TTLs in an RRSet is hereby
   deprecated, the TTLs of all RRs in an RRSet must be the same.

Ray

___
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe 
from this list

bind-users mailing list
bind-users@lists.isc.org
https://lists.isc.org/mailman/listinfo/bind-users

RE: Configuring different TTLs in multiple RRs for the same domain name, TYPE, and CLASS

[Darcy Kevin (FCA)]

This is deliberately forbidden by standard. See RFC 2181, Section 5.2 ("TTLs of 
RRs in an RRSet")

Why would you want to do this?


- Kevin


From: bind-users-boun...@lists.isc.org 
[mailto:bind-users-boun...@lists.isc.org] On Behalf Of Ben Bridges
Sent: Thursday, March 24, 2016 10:48 AM
To: bind-users@lists.isc.org
Subject: Configuring different TTLs in multiple RRs for the same domain name, 
TYPE, and CLASS

Greetings.

Is it possible in BIND to configure multiple resource records for the same 
domain name, TYPE, and CLASS with different TTL values?  For example:

Test-txt.example.com 300IN   TXT"Test 300"
Test-txt.example.com 400IN   TXT"Test 400"
Test-txt.example.com 500IN   TXT"Test 500"
Test-txt.example.com 600IN   TXT"Test 600"
Test-txt.example.com 700IN   TXT"Test 700"

I tried it, and BIND set the TTL for all five records to 300 (or more 
specifically, the TTL of the first one of the RRs in the file).  I looked for a 
BIND directive in the manual to change this behavior but could find no obvious 
candidate.

Thanks,

Ben Bridges
Springfield, MO

___
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe 
from this list

bind-users mailing list
bind-users@lists.isc.org
https://lists.isc.org/mailman/listinfo/bind-users

Configuring different TTLs in multiple RRs for the same domain name, TYPE, and CLASS

[Ben Bridges]

Greetings.

Is it possible in BIND to configure multiple resource records for the same 
domain name, TYPE, and CLASS with different TTL values?  For example:

Test-txt.example.com 300IN   TXT"Test 300"
Test-txt.example.com 400IN   TXT"Test 400"
Test-txt.example.com 500IN   TXT"Test 500"
Test-txt.example.com 600IN   TXT"Test 600"
Test-txt.example.com 700IN   TXT"Test 700"

I tried it, and BIND set the TTL for all five records to 300 (or more 
specifically, the TTL of the first one of the RRs in the file).  I looked for a 
BIND directive in the manual to change this behavior but could find no obvious 
candidate.

Thanks,

Ben Bridges
Springfield, MO

___
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe 
from this list

bind-users mailing list
bind-users@lists.isc.org
https://lists.isc.org/mailman/listinfo/bind-users

Re: Reducing memory usage by using db storage - performance?

[McDonald, Daniel (Dan)]

> On Mar 24, 2016, at 6:28 AM, MURTARI, JOHN  wrote:
> 
> Folks,
> Recently been looking at servers that host almost 200K ARPA 
> zones and load  about 80 million resource records.  They run on good hardware 
> and take only a few minutes to load the zones on a clean start.   The issue 
> is memory utilization of about 23 Gig in RAM.
> It seems a terrible waste of memory and a good portion of 
> those zones probably rarely see queries.

But RAM is cheap.  a 32GB stick is usually under $200. And 80 million 
customers?  Seems like a very small price to pay for reliable PTR records.

> 
> I’ve got extensive experience with mySQL and postgres, but 
> always assumed you’d really take a latency hit.  Plus, we’d be adding more 
> complexity by running a DB server.  The current DNS servers are located in 
> separate data centers – it seems we’d have to also run a master/slave DB 
> setup, with a slave DB server at each site to avoid network overhead.  This 
> all sounds very slow and more complicated.

And the development of such a system would cost significantly more than 
throwing a couple more sticks of RAM into your server, and have on-going 
training and maintenance costs.  Yes, engineers like to create an elegant 
solution, but sometimes the cheap solution just makes a lot of sense.

> 
> Anyone with experience solving this type of issue?
> Many thanks!
> John
> 
> 
> John Murtari – jm5...@att.com 
> Ciberspring
> office: 315-944-0998
> cell: 315-430-2702
> 
> ___
> Please visit https://lists.isc.org/mailman/listinfo/bind-users 
>  to unsubscribe from this 
> list
> 
> bind-users mailing list
> bind-users@lists.isc.org 
> https://lists.isc.org/mailman/listinfo/bind-users 
> 


signature.asc
Description: Message signed with OpenPGP using GPGMail
___
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe 
from this list

bind-users mailing list
bind-users@lists.isc.org
https://lists.isc.org/mailman/listinfo/bind-users

Reducing memory usage by using db storage - performance?

[MURTARI, JOHN]

Folks,
Recently been looking at servers that host almost 200K ARPA 
zones and load  about 80 million resource records.  They run on good hardware 
and take only a few minutes to load the zones on a clean start.   The issue is 
memory utilization of about 23 Gig in RAM.
It seems a terrible waste of memory and a good portion of those 
zones probably rarely see queries.

I've got extensive experience with mySQL and postgres, but 
always assumed you'd really take a latency hit.  Plus, we'd be adding more 
complexity by running a DB server.  The current DNS servers are located in 
separate data centers - it seems we'd have to also run a master/slave DB setup, 
with a slave DB server at each site to avoid network overhead.  This all sounds 
very slow and more complicated.

Anyone with experience solving this type of issue?
Many thanks!
John


John Murtari - jm5...@att.com
Ciberspring
office: 315-944-0998
cell: 315-430-2702

___
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe 
from this list

bind-users mailing list
bind-users@lists.isc.org
https://lists.isc.org/mailman/listinfo/bind-users