Re: Suggestions for a distributed DNS zone hosting solution I'm designing

[Klaus Darilion]

Hi Latitude!

Short answer: I think 2s delay is not possible in a distributed system 
with many global distributed slaves and limited ressources.


Long answer: It all depends on how much money you have and time in 
setting up such a service - long comments inline.


Am 07.03.2018 um 07:10 schrieb Latitude:

I would like to solicit constructive feedback in regards to a distributed DNS
zone hosting proof of concept I'd like to design and establish.

I must deploy a DNS system with the following requirements:
- single master server, multiple slave servers
- minimal time for name resolving for Americas, Europe and Asia
- up to millions records in a domain zone
- changes propagate in real time (master -> slaves), 2 sec max delay
- automatic slave data re-syncing on master link restore after disconnect
- API for zone records manipulation (insert, update, delete)


There is one important thing you did not mention: how often do you 
update the zone? once a day? once an hour? once a minute? a few times 
per second?




So far I am considering using (free) DC/OS on Amazon Web Services with the
latest version of BIND containerized using docker on a Linux or Unix OS. Dyn
and Infoblox are also on my list of items to research but I have never used
either and I enjoy working with BIND on Linux. After all this is the BIND
Users group, but I would be interested to know if someone can make a case
for using Dyn or Infoblox in this case.


The challenges are somewhere else. First, design the distribution, then 
think about OS, software, cloud provider, 



Considerations/questions I have about this deployment for this Bind-Users
forum are:

1. How can I examine DNS resolution times using this platform (or other
platforms to compare with) in different geographic areas of the world
without first deploying it? I will need to have benchmark data to test
against to verify I am getting the fastest speeds possible on name
resolutions.


You can not measure something you have not built yet. But what you can 
do is, measuring what somebody else built, and then clone there setup 
are build it similar, or buy there service. There are plenty of existing 
DNS providers. You can for example us RIPE Atlas to test them. There is 
also dnsperf.com and their entrprise service perfops.net which would 
give you a rough idea what DNS resolution times are possible.


You did not mention what your use case is. Who will do the DNS queries? 
a) A web browser on some standard PC or b) a dedicated application


For a) you do not have any control about the resolver used by the end 
user. HEnce, it may use its provider resolver, 8.8.8.8 (or similar) or 
its own. This resolver may be good in choosing the best announced 
nameserver (from the zone's NS records) or not. In this case I would 
suggest using Anycast - but be warned. Anycast for performance works 
only if you carefully choose your location, where 'location' means the 
location in the network, e.g. which transit providers, which exchanges, 
AS path length,  If you do the hosting all by yourself and carefully 
choose transit providers you can have great performance (ie Cloudflare), 
but it is very expensive. Choosing bad will give you bad performance.


For b) I would definitely avoid Anycast. Provision the application with 
the namne server known in the region, or make the application smart an 
let it probe which name server answers fastest.



2. How to handle millions of records in a DNS zone, and how common is it to
have millions of records in a DNS zone?


It is probably not that common, but there are several TLDs which have 
millions of records. One fo our customers zone has 25Mio RR and we do 
not see any problems with Bind (or NSD or Knot).



3. What API solutions for DNS zone edits currently exist or should I be
lookin into?


It all depends on your setup. If you go with Bind I would suggest DNS 
UPDATE. If you choose some other replication technique there are other 
tools (see below).



I will research more in the next day but so far I know I can manually
configure named.conf to propagate zone changes to slave servers rapidly
(aiming for 2 seconds or less) using NOTIFY messages and zone transfers, and
also configure slave servers to automatically re-synch zone data with the
master server upon reestablishing a connection. That should satisfy two of
my requirements above.


In fact it is not only NOTIFY+XFR. It is:

- Applying the zone change on the master, ie. DNS UPDATE

- NOTIFY all the slaves. With lots of slaves this may take some time and 
AFAIK Bind may throttle NOTIFYs


- NOTIFYs are UDP, and may get lost on the way, so you have 
retransmission. And if all NOTIFYs are lost, the slave will stay out of 
sync until there is the next zone update or SOA refresh value expires. 
This can be worked around ie by pushing your slaves to query the master 
every second for the current SOA (or even more optimzed: request an IXFR 
every second. for sure this will generate load on 

Re: Suggestions for a distributed DNS zone hosting solution I'm designing

[Volker Janzen]

Hi,


> Am 09.03.2018 um 15:26 schrieb Timothe Litt :
> 
>> On 08-Mar-18 07:52, Tony Finch wrote:
>> Best way to achieve this is with anycast, which can be pretty
>> time-consuming to set up - try searching for Nat Morris's presentation
>> "anycast on a shoestring" which he gave at several NOG meetings.
>> The advantage of anycast (as opposed to having NS records in lots of
>> locations) is that you are depending less on resolvers to work out for
>> themselves which of your servers is fastest.
>> 
> Does anyone know what happened to his project?
> 

The project has been shut down aprox a year ago.

> It looked like an interesting secondary DNS, but it seems to be out of 
> business.

It has been indeed, because it was free.


Regards,
Volker

___
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe 
from this list

bind-users mailing list
bind-users@lists.isc.org
https://lists.isc.org/mailman/listinfo/bind-users

Re: Re: Suggestions for a distributed DNS zone hosting solution I'm designing

[Timothe Litt]

On 08-Mar-18 07:52, Tony Finch wrote:
> Best way to achieve this is with anycast, which can be pretty
> time-consuming to set up - try searching for Nat Morris's presentation
> "anycast on a shoestring" which he gave at several NOG meetings.
> The advantage of anycast (as opposed to having NS records in lots of
> locations) is that you are depending less on resolvers to work out for
> themselves which of your servers is fastest.
>
Does anyone know what happened to his project?

It looked like an interesting secondary DNS, but it seems to be out of
business.

noc.esgob.com has a recently expired certificate, and redirects to one
line text page (his name).

The github repository is empty.

So it appears to be defunct.

Timothe Litt
ACM Distinguished Engineer
--
This communication may not represent the ACM or my employer's views,
if any, on the matters discussed. 



smime.p7s
Description: S/MIME Cryptographic Signature
___
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe 
from this list

bind-users mailing list
bind-users@lists.isc.org
https://lists.isc.org/mailman/listinfo/bind-users

Re: Suggestions for a distributed DNS zone hosting solution I'm designing

[Matus UHLAR - fantomas]

Latitude  wrote:

I must deploy a DNS system with the following requirements:
- single master server, multiple slave servers
- minimal time for name resolving for Americas, Europe and Asia


On 08.03.18 12:52, Tony Finch wrote:

Best way to achieve this is with anycast, which can be pretty
time-consuming to set up - try searching for Nat Morris's presentation
"anycast on a shoestring" which he gave at several NOG meetings.
The advantage of anycast (as opposed to having NS records in lots of
locations) is that you are depending less on resolvers to work out for
themselves which of your servers is fastest.


I consider anycast a good solution when providing recursive DNS as an ISP.
I am not sure if it's good idea for authoritative servers, unless you of
course have your own routed IP range and of course multiple POPs over the
world.

...some registries require nameserver IPs in different IP ranges, so they could
refise register domains to your anycast servers.

--
Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
Spam = (S)tupid (P)eople's (A)dvertising (M)ethod
___
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe 
from this list

bind-users mailing list
bind-users@lists.isc.org
https://lists.isc.org/mailman/listinfo/bind-users

Re: Suggestions for a distributed DNS zone hosting solution I'm designing

[Stephane Bortzmeyer]

On Thu, Mar 08, 2018 at 12:52:57PM +,
 Tony Finch  wrote 
 a message of 49 lines which said:

> Best way to achieve this is with anycast, which can be pretty
> time-consuming to set up - try searching for Nat Morris's
> presentation "anycast on a shoestring" which he gave at several NOG
> meetings.  The advantage of anycast (as opposed to having NS records
> in lots of locations) is that you are depending less on resolvers to
> work out for themselves which of your servers is fastest.

It seems to me a DISadvantage of anycast. Resolvers use actual RTT to
find the fastest server while BGP uses metrics which are far from
optimal. The advantage of anycast is isolation of local dDoS, and the
ability to have much more servers. Because of the above DISadvantage,
I would advocate against a zone with only opne anycasted server (even
if it works in theory).

> IXFR+NOTIFY will achieve this, without much effort,

Remember that NOTIFY packets can be lost (it's ordinary UDP). Can BIND
do NOTIFY over TCP?

> > 1. How can I examine DNS resolution times using this platform (or other
> > platforms to compare with) in different geographic areas of the world
> > without first deploying it?
> 
> There are some distributed measurement platforms such as RIPE ATLAS.
> (I can't think of any others off the top of my head.)

Atlas is great but the OP said "without first deploying it". (And I
believe he asks for the impossible.)
___
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe 
from this list

bind-users mailing list
bind-users@lists.isc.org
https://lists.isc.org/mailman/listinfo/bind-users

Re: Suggestions for a distributed DNS zone hosting solution I'm designing

[Tony Finch]

Latitude  wrote:
>
> I must deploy a DNS system with the following requirements:
> - single master server, multiple slave servers
> - minimal time for name resolving for Americas, Europe and Asia

Best way to achieve this is with anycast, which can be pretty
time-consuming to set up - try searching for Nat Morris's presentation
"anycast on a shoestring" which he gave at several NOG meetings.
The advantage of anycast (as opposed to having NS records in lots of
locations) is that you are depending less on resolvers to work out for
themselves which of your servers is fastest.

> - up to millions records in a domain zone

The biggest zone on my servers is an RPZ blocklist with 7 million records.
It's about half a GB and takes nearly 30s to load. Be warned :-)

> - changes propagate in real time (master -> slaves), 2 sec max delay
> - automatic slave data re-syncing on master link restore after disconnect

IXFR+NOTIFY will achieve this, without much effort, tho you may need to be
careful if your xfer distribution topology is at all complicated.

Recovery from outages depends on either the zone's SOA refresh interval,
or the next NOTIFY if the update rate is faster.

> - API for zone records manipulation (insert, update, delete)

As Mukund said, nsupdate :-)

> 1. How can I examine DNS resolution times using this platform (or other
> platforms to compare with) in different geographic areas of the world
> without first deploying it?

There are some distributed measurement platforms such as RIPE ATLAS.
(I can't think of any others off the top of my head.)

Tony.
-- 
f.anthony.n.finch    http://dotat.at/  -  I xn--zr8h punycode
Dogger, Fisher, German Bight: Cyclonic 5 to 7, decreasing 3 or 4. Slight,
occasionally moderate. Occasional sleet. Good occasionally poor.
___
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe 
from this list

bind-users mailing list
bind-users@lists.isc.org
https://lists.isc.org/mailman/listinfo/bind-users

Re: Suggestions for a distributed DNS zone hosting solution I'm designing

[Chris Buxton]

I work for BlueCat, a competitor to Infoblox. Our solution gives you BIND on 
Linux, with root access to the servers when you need it, as well as a robust 
management and monitoring solution. Our servers can be deployed in cloud and 
hosted environments.

That said, you don’t need a commercial product to do what you’ve described. 
BIND on Linux will do everything you’ve described, if properly set up. You 
could set up some simple scripting to give you secure DDNS so that you can 
update the data from anywhere.

I hope that helps.
Chris Buxton

Sent from my iPhone

> On Mar 6, 2018, at 10:10 PM, Latitude  wrote:
> 
> I would like to solicit constructive feedback in regards to a distributed DNS
> zone hosting proof of concept I'd like to design and establish. 
> 
> I must deploy a DNS system with the following requirements:
> - single master server, multiple slave servers
> - minimal time for name resolving for Americas, Europe and Asia
> - up to millions records in a domain zone
> - changes propagate in real time (master -> slaves), 2 sec max delay
> - automatic slave data re-syncing on master link restore after disconnect
> - API for zone records manipulation (insert, update, delete)
> 
> So far I am considering using (free) DC/OS on Amazon Web Services with the
> latest version of BIND containerized using docker on a Linux or Unix OS. Dyn
> and Infoblox are also on my list of items to research but I have never used
> either and I enjoy working with BIND on Linux. After all this is the BIND
> Users group, but I would be interested to know if someone can make a case
> for using Dyn or Infoblox in this case. 
> 
> Considerations/questions I have about this deployment for this Bind-Users
> forum are:
> 
> 1. How can I examine DNS resolution times using this platform (or other
> platforms to compare with) in different geographic areas of the world
> without first deploying it? I will need to have benchmark data to test
> against to verify I am getting the fastest speeds possible on name
> resolutions. 
> 
> 2. How to handle millions of records in a DNS zone, and how common is it to
> have millions of records in a DNS zone?
> 
> 3. What API solutions for DNS zone edits currently exist or should I be
> lookin into?
> 
> I will research more in the next day but so far I know I can manually
> configure named.conf to propagate zone changes to slave servers rapidly
> (aiming for 2 seconds or less) using NOTIFY messages and zone transfers, and
> also configure slave servers to automatically re-synch zone data with the
> master server upon reestablishing a connection. That should satisfy two of
> my requirements above. 
> 
> Any additional advice, hints, or tips for my proof of concept would be
> greatly appreciated! Thanks in advance. This will be a very fun project to
> design and hopefully implement. 
> 
> 
> 
> 
> --
> Sent from: http://bind-users-forum.2342410.n4.nabble.com/
> ___
> Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe 
> from this list
> 
> bind-users mailing list
> bind-users@lists.isc.org
> https://lists.isc.org/mailman/listinfo/bind-users
> 

___
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe 
from this list

bind-users mailing list
bind-users@lists.isc.org
https://lists.isc.org/mailman/listinfo/bind-users

Re: Suggestions for a distributed DNS zone hosting solution I'm designing

[Mukund Sivaraman]

Hi

On Tue, Mar 06, 2018 at 11:10:35PM -0700, Latitude wrote:
> I would like to solicit constructive feedback in regards to a distributed DNS
> zone hosting proof of concept I'd like to design and establish. 
> 
> I must deploy a DNS system with the following requirements:
> - single master server, multiple slave servers
> - minimal time for name resolving for Americas, Europe and Asia
> - up to millions records in a domain zone
> - changes propagate in real time (master -> slaves), 2 sec max delay
> - automatic slave data re-syncing on master link restore after disconnect
> - API for zone records manipulation (insert, update, delete)
> 
> So far I am considering using (free) DC/OS on Amazon Web Services with the
> latest version of BIND containerized using docker on a Linux or Unix OS. Dyn
> and Infoblox are also on my list of items to research but I have never used
> either and I enjoy working with BIND on Linux. After all this is the BIND
> Users group, but I would be interested to know if someone can make a case
> for using Dyn or Infoblox in this case. 
> 
> Considerations/questions I have about this deployment for this Bind-Users
> forum are:
> 
> 1. How can I examine DNS resolution times using this platform (or other
> platforms to compare with) in different geographic areas of the world
> without first deploying it? I will need to have benchmark data to test
> against to verify I am getting the fastest speeds possible on name
> resolutions. 

Changing conditions on the internet affect nameserver selection and
there are several factors involved in what is 'fastest'. When talking
about 'resolution', it also depends on resolvers' and their clients'
connectivity. Short of empirically measuring response times, I don't
have a better answer.

> 2. How to handle millions of records in a DNS zone, and how common is it to
> have millions of records in a DNS zone?

It is uncommon to have millions of records in a DNS zone, but it is
possible and there are some operators who run such large zones. We
routinely test million+ RR zones with BIND.

> 3. What API solutions for DNS zone edits currently exist or should I be
> lookin into?

DNS UPDATE (RFC 2136) is the protocol for modifying zone data.

You may also be interested in web APIs such as: https://dnsimple.com/api

> I will research more in the next day but so far I know I can manually
> configure named.conf to propagate zone changes to slave servers rapidly
> (aiming for 2 seconds or less) using NOTIFY messages and zone transfers, and
> also configure slave servers to automatically re-synch zone data with the
> master server upon reestablishing a connection. That should satisfy two of
> my requirements above. 

There is no guarantee that any nameserver will synchronize zones updates
from primary within 2 seconds max. If the public internet is involved,
the cumulative roundtrip times involved in notifying a secondary and for
the secondary to start a transfer alone may take more than 2 seconds
depending on network conditions and topology, especially if you're
talking about Americas, Europe and Asia together.

Mukund
___
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe 
from this list

bind-users mailing list
bind-users@lists.isc.org
https://lists.isc.org/mailman/listinfo/bind-users