Re: Perfomance issue between 1.5.8-1ubuntu1.1 (xenial) and 1.6.7-1ubuntu2.1 (bionic)

[Robert Edmonds via Unbound-users]

Ralf Hildebrandt via Unbound-users wrote:
> Before the update (runnung unbound 1.5.8-1ubuntu1.1) we were seeing query
> times around 20ms: After the upgrade (1.6.7-1ubuntu2.1) those rose to
> 40ms.
> 
> See these graphs:
> https://www.arschkrebs.de/bugs/dnssvc30d.png
> https://www.arschkrebs.de/bugs/dnssvc1w.png
> 
> I then tinkered with different package versions -- tried upgrading to
> 1.7.3 (no change) and finally downgraded back to to 1.5.8-1ubuntu1.1 -
> and the query times dropped to pre-update levels.

One significant difference between those versions is that the
Debian/Ubuntu unbound package enabled query minimisation by default in
package version 1.5.9-1. You might try installing 1.6.7-1ubuntu2.1 and
commenting out the 'qname-minimisation: yes' line in
/etc/unbound/unbound.conf.d/qname-minimisation.conf.

-- 
Robert Edmonds
edmo...@debian.org

Re: unbound doesn't remove pidfile

[Robert Edmonds via Unbound-users]

Shawn Zhou via Unbound-users wrote:
> I am running unbound 1.5.8 on ubuntu xenial. unbound doesn't run remove the 
> pid file after it's stopped.

I believe the unbound packaging on Ubuntu xenial is old enough that it
still uses the sysv generator to create the service unit. You will
probably want to add this fix to your unbound init script, which I don't
think was ever backported to xenial (it was originally added in 1.5.9-1):

https://salsa.debian.org/dns-team/unbound/commit/1c139abaa0fe58f8d97b64c96da6c3332b1b9e49

-- 
Robert Edmonds
edmo...@debian.org

Re: NOTIMP for unrecognized qtypes

[Robert Edmonds via Unbound-users]

Petr Špaček via Unbound-users wrote:
> Well, the spec is from 1987. Even the meaning of MUST/SHOULD etc. was
> not standardized yet back then ...

Even worse, this language appears to have been copied verbatim from RFC
883, which is even older (1983) :-)

-- 
Robert Edmonds
edmo...@debian.org

Re: NOTIMP for unrecognized qtypes

[Robert Edmonds via Unbound-users]

Jacob Hoffman-Andrews via Unbound-users wrote:
> I'm trying to write some documentation for users of Let's Encrypt about
> CAA. I believe it's the case that standards-conformant authoritative
> resolvers should return NOERROR for qtypes they don't recognize, rather
> than NOTIMP. Is this correct? If so, what is the relevant standard? I
> haven't been able to find a citation in
> https://tools.ietf.org/html/rfc3597,
> https://tools.ietf.org/html/rfc6895, or https://tools.ietf.org/html/rfc1035.

RFC 1035 seems to be pretty clear that NOTIMP applies to the OPCODE, not
the QTYPE.

Mockapetris[Page 25]

RFC 1035Domain Implementation and SpecificationNovember 1987


4.1.1. Header section format

[…]

OPCODE  A four bit field that specifies kind of query in this
message.  This value is set by the originator of a query
and copied into the response.  The values are:

0   a standard query (QUERY)
[…]

RCODE   Response code - this 4 bit field is set as part of
responses.  The values have the following
interpretation:
[…]
4   Not Implemented - The name server does
not support the requested kind of query.
[…]

That is, OPCODE specifies the "kind of query", and NOTIMP indicates that
the "kind of query" (= OPCODE) isn't supported.

-- 
Robert Edmonds
edmo...@debian.org

Re: Trust rules and DNSSEC signatures

[Robert Edmonds via Unbound-users]

Florian Weimer via Unbound-users wrote:
> Does Unbound use otherwise non-trustworthy data simply because it has
> valid DNSSEC signatures?
> 
> I'm asking because of this recent dnsop thread:
> 
>   

Hi, Florian:

It's been a while since I studied the Unbound architecture, but I
believe the answer to your question is "no", due to Unbound's separation
of iteration and validation into separate modules. (E.g.,
'module-config: "validator iterator"'.) If I understand correctly, the
iterator module is responsible for "scrubbing" response messages, which
includes things like deleting out-of-zone information from the response,
and it doesn't scrub conditionally based on whether the validator module
is also present in the module stack.

-- 
Robert Edmonds
edmo...@debian.org

Re: trust-anchor-file, auto-trust-anchor-file, trust-anchor

[Robert Edmonds via Unbound-users]

Edward Lewis via Unbound-users wrote:
> Is the use of trust-anchor-file for the public root zone KSK popular?  Do 
> folks use it much at all (regardless of zone)?  The same for trust-anchor 
> statements, which appear to be in-line of the configuration file.

Hi, Ed:

We ship the Debian package of unbound with an auto-trust-anchor-file
config for the root zone in the default configuration:

http://sources.debian.net/src/unbound/1.6.0-3/debian/unbound.conf.d/root-auto-trust-anchor-file.conf/

I think we've been shipping the root anchor with an
"auto-trust-anchor-file" directive in the default config for around five
years or so.

Debian is the upstream for Ubuntu, which together are pretty popular. If
you also look at the package defaults for Fedora (which is also used as
the upstream for RHEL) you'd probably be covering 80-90% or so of the
Linux distributions by usage.

-- 
Robert Edmonds
edmo...@debian.org

Re: unbound listening sporadically on 0.0.0.0 high ports when configured for 127.0.0.1 ?

[Robert Edmonds via Unbound-users]

Paul Wouters via Unbound-users wrote:
> On Fri, 3 Jun 2016, Daisuke HIGASHI wrote:
> 
> > Subject: Re: unbound listening sporadically on 0.0.0.0 high ports when
> > configured for 127.0.0.1 ?
> 
> >   My guess is: UDP sockets for outgoing query
> > from Unbound to authoritative servers.
> > 
> >  I also see these "listening" UDP sockets at my laptop running unbound
> > when resolver is under load. And I see no them when no load.
> 
> That was my first thought too, but these entries do not have a
> destination IP or port, so it "appears" that these sockets are
> listening. Still it could be that perhaps these are sockets
> that are starting up for use or something?

Unbound doesn't connect() its UDP query sockets to the destination
IP/port, it uses sendto(), TTBOMK. So I wouldn't think the destination
would show up in netstat (or ss or lsof...) output.

-- 
Robert Edmonds
edmo...@debian.org

Re: L-Root IPv6 address renumbering

[Robert Edmonds via Unbound-users]

Dave Warren via Unbound-users wrote:
> On 2016-03-16 10:46, Robert Edmonds via Unbound-users wrote:
> >Not quite, I want to avoid two things:
> >
> >1) The sysadmin should never have to update the root hints by hand.
> >"apt update && apt upgrade" should upgrade any packages needed to bring
> >the root hints up to date.
> >
> >2) The package maintainers shouldn't have to patch and rebuild each
> >package with compiled in root hints when a root server is renumbered.
> 
> At what point would a binary have a newer internal roots hints than the
> filesystem root.hints file when a user is using #1 to keep updated? Is there
> a subset of users who would update the binary but not apt update/upgrade?

This is a good point, it doesn't really matter for the distro user, I
guess.

> I guess to me, it seems better to directly address whatever failed to update
> the external root.hints than to add complexity of a "will-she won't-she" of
> using defined data files.
> 
> Also, does any of this matter? The root hints just used to find the root
> servers on initialization, and then the resolver retrieves and uses the
> current roots anyway. Resolvers need to update eventually, but it's not a
> networking-breaking level of urgency either, is it?

I agree, the consequences are extremely mild in the first place. We
still go to the trouble of backporting the root hint updates, though.

-- 
Robert Edmonds
edmo...@debian.org

Re: L-Root IPv6 address renumbering

[Robert Edmonds via Unbound-users]

W.C.A. Wijngaards via Unbound-users wrote:
> But I think just setting the configuration option for root-hints in
> unbound.conf is probably just what you want?  Do you still need to be
> able to set a default value for the root-hints file location, or is it
> just as good to set it in unbound.conf (or unbound.conf.d/ drop-file) ?

Would setting a compile-time default value also apply to libunbound?
Just setting it in /etc/unbound.conf will only apply to the daemon,
right?

-- 
Robert Edmonds
edmo...@debian.org

Re: L-Root IPv6 address renumbering

[Robert Edmonds via Unbound-users]

W.C.A. Wijngaards via Unbound-users wrote:
> I have updated the default root hints that ship inside the source code
> of Unbound (in the code repository, for future releases).  Thank you for
> the notification.
> 
> Users can upgrade the root hints right now by editing the named.root (or
> named.cache, or root.hints file) and using the root-hints: "filename"
> directive in unbound.conf.  Or they can wait for a source code upgrade
> if they are using defaults.

Hi, Wouter:

I wonder a few things. Probably there will be future root server
re-numberings. We have to patch and rebuild binary packages for several
recursive DNS servers each time this happens.

On Debian, we ship the root hints file in /usr/share/dns/root.hints
(in package dns-root-data). It would be nice if Unbound and other
recursive DNS servers could use the freshest hints available on the
system, either in the system's root.hints file, or in the compiled in
root hints. This would avoid the need to update packages in released
versions of Debian and the need to backport hint updates from trunk to
the latest release.

That is, the two policies available currently are "use hints from an
external file" and "use compiled in hints". Both of these policies can
easily fail, e.g. the "use external file" policy with an out-of-date
hints file and up-to-date binary, or "use compiled in hints" policy with
an out-of-date binary and up-to-date hints file.

What if there were a "use latest hints from either external file or
compiled in hints" policy? Unbound would need to compile in a timestamp
corresponding to the time that the hints were last updated, and compare
this to the mtime on the root hints file. The distros can then enable
this policy by default and point the parameter to a file in /usr whose
mtime will be no later than the time that the hint file was actually
updated (and not the time the package was built or the file was
downloaded, etc.). Since the file would not be a configuration file it
would never be modified by the sysadmin and the mtime should always
accurately reflect its age.

Maybe this is not that big a deal if one or two hint addresses are out
of date out of 20+ addresses, because the root priming algorithm will
update it at process startup, but at least the distros do feel some
pressure to keep the compiled in hints current, across all the recursive
DNS servers in the distro that build in hints and across all the
supported distro releases. If I wrote a patch implementing this policy
would it be something suitable for merging into the mainline?

The other thing I wonder is, if the root-servers.net zone were ever to
be signed with DNSSEC, could we just let the server securely store a
persistent copy of updated root hints into /var, like
auto-trust-anchor-file does for the root trust anchor? :-)

-- 
Robert Edmonds
edmo...@debian.org

Re: python unbound issues

[Robert Edmonds via Unbound-users]

Spike Morelli (DRBA) via Unbound-users wrote:
> 1) unbound-checkconfig complains that the python module isn't there:
> 
> [1456179172] unbound-checkconf[5330:0] fatal error: module conf 'python
> iterator' is not known to work
> 
> looking at the source code this seems to be due to WITH_PYTHONMODULE not
> being set, but I'm not sure where that's not happening. any clue?

Hi, Spike:

The Ubuntu unbound package is closely based on the Debian version. In
this case, you're affected by bug #777193:

https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=777193

unbound-checkconf is only called in certain cases by the init script.
You might be able to work around the issue by altering some of the
package settings. Maybe try setting RESOLVCONF=false in
/etc/default/unbound, if you don't need the resolvconf integration.

-- 
Robert Edmonds
edmo...@debian.org

Re: [patch] insecure-lan-zones

[Robert Edmonds via Unbound-users]

Dag-Erling Smørgrav via Unbound-users wrote:
> I hope I got the Makefile.in part right - it's pretty gross.  Why don't
> you use automake?

+1 to Automake :-)  Hacking on Unbound's Makefile.in is not fun.

-- 
Robert Edmonds
edmo...@debian.org

Unbound and intermittent network connectivity?

[Robert Edmonds via Unbound-users]

Hi,

I have a few recent bug reports from Debian users that Unbound stops
resolving after brief interruptions in network connectivity.  Especially
from users on laptops, which are typically not as well-connected as
servers or workstations with wired Ethernet connections.

https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=791659

https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=808204

A few questions:

Is my guess that Unbound stores unreachability information for
particular nameservers in the "infra cache" correct?  Does this also
apply to forwarders?  Does that mean if a user is running Unbound in
forwarding mode and has a brief network outage, they have to wait until
an "infra-host-ttl" expiration (default 15 minutes) occurs before
resolution service works again?

Is the format of the "dump_infra" output documented anywhere?  I've
started reading source code to figure it out, but it would be nice to
have some "this is good" and "this is bad" examples.  E.g., at first
glance I misread "lame dnssec 0" to mean "this server is lame, and does
not support DNSSEC", which appears to be the opposite of what it means
:-)

Should distros be doing something on network change events to get
Unbound to purge unreachability information?  I think "flush_infra all"
would do it, but isn't this quite disruptive?  (Maybe unreachability
information could be cached with a different TTL than the other
attributes for entries in the infra cache?)

Should distros lower "infra-host-ttl" in general, or for laptop users in
particular?

How should we deal with brief interruptions in network connectivity past
the first hop (say, outage inside the ISP backbone) that don't trigger
events?

Thanks!

-- 
Robert Edmonds
edmo...@debian.org

Re: unbound-anchor ignores net.ipv6.conf.all.disable_ipv6=1 in /etc/sysctl.conf

[Robert Edmonds via Unbound-users]

Tomas Hozza via Unbound-users wrote:
> On 04.11.2015 17:35, Phil Mayers wrote:
> > The code tries to open an IPv6 socket, the kernel tries to load the module, 
> > SELinux denies and logs this. Each of these items is by design. Which are 
> > you suggesting should change?
> 
> I think it makes sense. I'm just not that familiar with how IPv6 works in 
> kernel,
> therefore I was trying to ask you for more information so I can possibly 
> convince
> the Fedora user that the Unbound's behavior is expected and correct.

Hi, Tomas:

Given that this same bug has been reported in:

https://bugzilla.redhat.com/show_bug.cgi?id=641836
[named,sshd,sendmail,rndc]

and duplicates of it:

https://bugzilla.redhat.com/show_bug.cgi?id=669047
[sshd]

https://bugzilla.redhat.com/show_bug.cgi?id=696256
[rpc.statd]

https://bugzilla.redhat.com/show_bug.cgi?id=696371
[named]

https://bugzilla.redhat.com/show_bug.cgi?id=703733
[dhcpd]

https://bugzilla.redhat.com/show_bug.cgi?id=722751
[sendmail]

https://bugzilla.redhat.com/show_bug.cgi?id=728969
[rpc.nfsd]

and that the bug has been closed with status CLOSED CANTFIX, can't you
just mark this as another duplicate and move on?

> > Is it the audit log that is annoying people? If so, the SELinux policy 
> > should be a dontaudit.
> 
> I think it is the fact that they disabled the IPv6, but some userspace 
> component
> is trying to load into kernel a module they they don't want to be loaded.

Given that IPv6 is compiled into the kernel rather than as a module, the
kernel can't load ipv6.ko.  Seems like the real bug is the kernel trying
to load a module that doesn't exist?  (Or, rather, the kernel trying to
autoload the IPv6 module even though it's compiled in and ipv6.disable=1
is specified on the kernel command line.  But apparently the code that
checks the ipv6.disable command line is inside the IPv6 module itself,
leading to a chicken-and-egg problem.)

If the kernel developers don't consider this interaction a bug or don't
consider ipv6.disable=1 a supported configuration, it seems like the
only other solution is to make auditd not log this particular message.
(Note that the only bad behavior reported in any of these bugs is the
log message generated by auditd, not any bad behavior on the part of the
userspace component that happened to be running at the time.)

The only other option would appear to be to convince users to not
attempt to disable IPv6 :-)

-- 
Robert Edmonds
edmo...@debian.org

Re: unbound-anchor ignores net.ipv6.conf.all.disable_ipv6=1 in /etc/sysctl.conf

[Robert Edmonds via Unbound-users]

Hi,

Phil Mayers via Unbound-users wrote:
> On 04/11/2015 15:49, Tomas Hozza wrote:
> 
> >If you have some strong technical argument for this behavior I would
> >be more than glad to hear it. The reason is that similar people will
> >fight hard against having Unbound as the default DNS resolver in
> >Fedora, which is our ultimate plan. Ability to spare hundreds of
> >emails arguing with them would be great :)
> 
> Which "behaviour"?
> 
> I'm honestly confused. As far as I can tell, everything is working as
> designed here.
> 
> The code tries to open an IPv6 socket, the kernel tries to load the module,
> SELinux denies and logs this. Each of these items is by design. Which are
> you suggesting should change?

If I understand correctly, there is no ipv6.ko module on the kernel in
question that can be autoloaded; at least on my Fedora 20 and CentOS 7
machines, the kernels are built with CONFIG_IPV6=y.

When "ipv6.disable=1" is set on the kernel command line, it looks like
the (built-in) IPv6 code bails out without registering the IPv6 TCP,
UDP, etc. protocols:

https://github.com/torvalds/linux/blob/v4.3/net/ipv6/af_inet6.c#L839-L850

So, IIUC, when a userspace program tries to create an AF_INET6 socket,
the kernel sees that the requested protocol isn't available (due to this
early bail out from the ipv6 module), and tries to load the "net-pf-10"
module, even though this would have been provided by code statically
compiled into the kernel.

Is the problem perhaps that "ipv6.disable=1" on the kernel command line
should be accompanied by "alias net-pf-10 off" in the modprobe
configuration in order to prevent useless autoloading attempts?

> Is it the audit log that is annoying people? If so, the SELinux policy
> should be a dontaudit.

I can't see how this audit message wouldn't be triggered by basically
any program that creates an IPv6 socket, which should be close to any
program that uses the network by now?

> Can we agree that unbound-anchor should not be reading sysctls to change
> it's behaviour?

I do, that's just crazy :-)

The userspace interface for detecting the lack of IPv6 support is: if
the call to socket() fails (probably with EAFNOSUPPORT or
EPROTONOSUPPORT), then it's not supported.  And any sane userspace
program already checks for socket() failures.

-- 
Robert Edmonds
edmo...@debian.org

Re: unbound-anchor ignores net.ipv6.conf.all.disable_ipv6=1 in /etc/sysctl.conf

[Robert Edmonds via Unbound-users]

Phil Mayers via Unbound-users wrote:
> On 04/11/2015 17:21, Robert Edmonds wrote:
> >Is the problem perhaps that "ipv6.disable=1" on the kernel command line
> >should be accompanied by "alias net-pf-10 off" in the modprobe
> >configuration in order to prevent useless autoloading attempts?
> 
> Is that config read by modprobe after the kernel has called it? In which
> case it'll still trigger (and deny) the probe. It's been so long since I
> looked at module loading I can't remember.

I don't know that much about SELinux.  Are these "module_request"
denials triggered at the point where the kernel is about to perform the
upcall into userspace to invoke modprobe, or are they triggered after
the modprobe runs and calls back into the kernel to request the module
load?  (If the former, then "alias net-pf-10 off" won't help, of
course.)

> >I can't see how this audit message wouldn't be triggered by basically
> >any program that creates an IPv6 socket, which should be close to any
> >program that uses the network by now?
> 
> They might be running under SELinux contexts which either permit or
> dontaudit the module load I suppose? But I agree, almost everything is going
> to be v6-aware at this point.

Right, so this looks like a duplicate of
https://bugzilla.redhat.com/show_bug.cgi?id=641836, and wow that bug has
a lot of dupes.  So this isn't an Unbound problem at all.

According to a blog post (http://danwalsh.livejournal.com/47118.html),
it would appear the better option is to disable IPv6 addressing.  Or
figure out how to get the system to deny the request without logging it
if it bothers you so much, I guess.

-- 
Robert Edmonds
edmo...@debian.org

Re: unbound-anchor ignores net.ipv6.conf.all.disable_ipv6=1 in /etc/sysctl.conf

[Robert Edmonds via Unbound-users]

Paul Wouters via Unbound-users wrote:
> FYI:
> 
> rhbz#1231946 - unbound-anchor ignores net.ipv6.conf.all.disable_ipv6=1 in 
> /etc/sysctl.conf
> 
> https://bugzilla.redhat.com/show_bug.cgi?id=1231946
> 
> Paul

Hi, Paul:

I'm a bit confused.  unbound-anchor is an ordinary program that uses the
sockets API, so it should have no reason to read Linux kernel specific
sysctl's or change behavior based on their values, since sysctl's are
parameters for the kernel.

kmod="net-pf-10" is the IPv6 protocol family, and Red Hat kernels have
CONFIG_IPV6=y these days, so it could not autoload the module anyway.
Maybe IPv6 is not really fully disabled, and unbound-anchor attempting
to open an AF_INET6 socket triggers this message?

I came across this page (https://access.redhat.com/solutions/482253)
which seems to match the problem description, but the solution is
paywalled, perhaps you have better access :-)

-- 
Robert Edmonds
edmo...@debian.org

Re: unbound and systemd

[Robert Edmonds via Unbound-users]

Sami Kerola via Unbound-users wrote:
> The stuff I did to avoid pkg-config is not nice. Fixing that would
> require dependency that not all projects agree. What is your view
> to add pkg-config dep?

Hi,

Using pkg-config is the documented way to detect the correct library to
link against for libsystemd, see:

http://www.freedesktop.org/software/systemd/man/sd-daemon.html

It's trivial to only use pkg-config (via the PKG_CHECK_MODULES macro)
when systemd integration is requested via --enable-systemd, and to not
fail in the absence of the pkg-config binary when systemd support is not
requested.  And systemd only runs on Linux, where pkg-config is normal
and shared libraries are expected to ship .pc files.

The only downside is that the source for the PKG_CHECK_MODULES macro
(usually shipped in /usr/share/aclocal/pkg.m4 in the "pkgconfig" (.rpm)
or "pkg-config" (.deb) package) needs to be available on the machine
where autoreconf is run.  But ordinarily users don't run autoreconf
themselves, so this is not too onerous.

So IMHO just use pkg-config, it's nicer to write one line of autoconf
instead of 50 :-)

-- 
Robert Edmonds
edmo...@debian.org

Re: inconsistent forward-zone behavior between config files, unbound-control

[Robert Edmonds via Unbound-users]

A. Schulze via Unbound-users wrote:
> Am 22.09.2015 um 19:02 schrieb Mike Brown via Unbound-users:
> >* by default, queries go to my ISP's resolvers (Comcast: 75.75.75.75 &
> >75.75.76.76)
> why would you do that?

Comcast's 75.75.75.75 and 75.75.76.76 nameservers are anycasted.
75.75.75.75 in particular should (probably) be routed to a nearby
Comcast data center.

For instance, on my home Comcast connection, 75.75.75.75, F-Root, and
L-Root all have anycast instances located in a local Comcast facility,
so the RTT to these three servers are identical (~10 ms) for me.

Comcast's servers perform DNSSEC validation and it should in theory be
possible to have a validating DNS server like Unbound forward to those
servers and re-validate the answers, so for a lightly loaded server it
may actually result in *better* performance (latency wise) to forward to
the nearby Comcast cache rather than directly contacting authoritative
nameservers that may be much farther than ~10 ms away.

> Also I'm not aware any unbound configuration is modified in any way by a DHCP 
> client.
> I use to ignore any resolver announced by a DHCP server:

The stock upstream Unbound distribution does not have any DHCP client
integration, true.  However, some OSes (e.g. Debian) have optional
resolvconf integration, which can be used to automatically configure
nameservers learned via DHCP as forwarders for Unbound.  Other OSes may
have similar functionality.  This only works well if those nameservers
perform DNSSEC validation, however.

-- 
Robert Edmonds
edmo...@debian.org

Re: rfc6761 compliance

[Robert Edmonds via Unbound-users]

W.C.A. Wijngaards via Unbound-users wrote:
> It is not a particularly heavy root server load to mitigate, less code
> is better and easier, the unblock-lan-zones statement is a frequently
> asked question from our users.  That said, we could add new code for
> this (and .onion?).

Hi, Wouter:

I would guess that the .test and .invalid zones are much less used in
private networks than the .in-addr.arpa ones, so much less likely to be
a FAQ.  And most of the code to setup default empty zones has been
written already.

Here are the caching DNS considerations for the zones that Unbound
currently doesn't handle:

[ "test." ]
   Caching DNS servers SHOULD recognize test names as special and
   SHOULD NOT, by default, attempt to look up NS records for them,
   or otherwise query authoritative DNS servers in an attempt to
   resolve test names.  Instead, caching DNS servers SHOULD, by
   default, generate immediate negative responses for all such
   queries.  This is to avoid unnecessary load on the root name
   servers and other name servers.  Caching DNS servers SHOULD offer
   a configuration option (disabled by default) to enable upstream
   resolving of test names, for use in networks where test names are
   known to be handled by an authoritative DNS server in said
   private network.

[ "invalid." ]
   Caching DNS servers SHOULD recognize "invalid" names as special
   and SHOULD NOT attempt to look up NS records for them, or
   otherwise query authoritative DNS servers in an attempt to
   resolve "invalid" names.  Instead, caching DNS servers SHOULD
   generate immediate NXDOMAIN responses for all such queries.  This
   is to avoid unnecessary load on the root name servers and other
   name servers.

[ "onion." ]
   Caching DNS Servers: Caching servers, where not explicitly
   adapted to interoperate with Tor, SHOULD NOT attempt to look up
   records for .onion names.  They MUST generate NXDOMAIN for all
   such queries.

I notice the .onion Special-Use registration has a MUST while the other
two only have SHOULDs.

Probably there will be a few more additions to the Special-Use Domain
Names registry, and even if they only generate a trivial amount of root
server load now, that means it's easy to prevent them from becoming a
problem later :-)

-- 
Robert Edmonds
edmo...@debian.org

Re: Query logging performance

[Robert Edmonds via Unbound-users]

W.C.A. Wijngaards via Unbound-users wrote:
 On 03/08/15 19:50, Darren Spruell via Unbound-users wrote:
  Unbound's documentation mentions that query logging can have very 
  adverse performance on server operation. I was curious if the
  project feels this has been optimized to the degree possible
  already, or if an approach similar to what some other projects take
  may be beneficial; namely something like delegating logging
  responsibilities to a different thread (Suricata IDS engine, I
  think) or even using a separate log output process (Squid [1],
  OpenBSD's PF/pflogd(8) [2]).

Hi, Darren:

I looked into how existing DNS servers (BIND, Unbound) implement query
logging when I was originally implementing dnstap support in Unbound.
There are some brief notes on slides 6 and 7 in this slide deck:

http://dnstap.info/slides/dnstap_nanog60.pdf

Benchmarks are shown on slides 36 and 37.  Query logging just kills the
performance of these two servers.

BIND and Unbound are very similar in how their query logging is
performed, though the actual implementations are very different.
It basically comes down to:

 (1) Formatting the query into a textual log entry, usually using a
 printf() family function.

 (2) Using a stdio function or syslog() to export the log entry to a
 file or socket.

 (3) Doing this from the worker thread responsible for processing the
 query.

(The basic idea dates all the way back to BIND 4:
http://dnstap.info/slides/dnstap.html#(8).)

Using printf() family functions in (1) is bad, since format strings have
to be parsed for each entry.  If your log format is highly customizable
or has a lot of fields, etc. you can end up taking a not insubstantial
performance hit.  This can be optimized a bit with custom formatting
code.

(2) + (3) is where the big performance hit comes from, since the C
library is ultimately hiding mutexes in the internal implementations of
typical functions like fprintf() and syslog() used to write the log
output.  With heavily threaded servers like BIND and Unbound, (3) causes
the contention for those mutexes to impact the performance of the
server; you may have many worker threads running and serving clients,
but they will all be contending for the same FILE* or syslog mutex.
(Though, in the syslog case, there's no particularly good reason that
all the threads in a process have to contend for access to the same
socket to the syslog server, other than because that's how the syslog()
implementation provided by the system works.)

If you're only interested in acquiring raw query packets, query logging
in the DNS server is probably not the best approach; there are a variety
of kernel facilities for acquiring that data without affecting the
performance of the DNS server too badly, like BPF/AF_PACKET (pcap),
ulogd, pflogd, etc., depending on kernel.  However, there is additional
metadata that can be exported via in-server logging that doesn't appear
on the wire, stuff like cache status and bailiwick information (the
latter is useful for passive DNS).

The dnstap implementation in Unbound is actually a combination of three
different things: the dnstap.pb protobuf schema itself [0], the
protobuf-c [1] implementation of Protocol Buffers for binary message
serialization, and the fstrm library [2] which provides a dedicated
logging thread and AF_UNIX transport.

[0] https://github.com/dnstap/dnstap.pb

[1] https://github.com/protobuf-c/protobuf-c

[2] https://github.com/farsightsec/fstrm

So, using binary protobuf messages rather than printf() style formatting
addresses (1) above, though protobuf serialization performance can still
be a concern.  The lock contention in (2) and (3) are addressed by the
use of a dedicated logging thread provided by fstrm that provides each
worker thread a lockless way to enqueue log messages.

However, all of the logging work can't be deferred to the logging
thread, so (3) is still a concern.  Namely, the creation of the log
message itself (and any needed memory allocations, memory copies, etc.)
still has to be done by the worker thread.  Since the logging thread
performs work asynchronously to the worker threads, we can't defer 100%
of the work of creating the log message to the point at which the log
data is actually written out: needed data fields that would go into the
rendered log message may have been free()'d and would no longer be
available.  This is because Unbound is written in C and doesn't use any
sort of refcounting or garbage collection scheme that we could latch
onto to avoid the overhead of those memory copies, so we have to do log
message rendering in the worker thread.  And this applies regardless of
whether the log format is text or binary.

In practice, though, that residual overhead from needing to render the
log message in the worker thread is not that bad, see slide 38 in the
slide deck linked above (compare the 'unbound' or
'unbound-dnstap-disabled' plots against the 'unbound-dnstap-discard'
plot).

  Alternately, 

Re: Using unbound-anchor for non-default trust anchor

[Robert Edmonds via Unbound-users]

Edward Lewis via Unbound-users wrote:
 unbound-anchor, by default, pulls DNSSEC trust anchors from data.iana.org.
 
 I am trying to test RFC 5011 capabilities by following these websites:
 
 http://keyroll.systems
 and
 http://icksk.dnssek.info/fauxroot.html
 
 Goal is to run unbound-anchor as a first step before trying to tune
 unbound to either of those experiments.

Hi, Ed:

IIRC, the HTTPS fetch from data.iana.org in unbound-anchor is a
fallback, if the RFC 5011 stuff fails.  You still ought to be able to
test the RFC 5011 stuff alone, if that's what you're trying to do.

I copied the root.db file at the bottom of
http://keyroll.systems/current into /tmp/root.db (would be nice if this
were downloadable as a separate file), and then tried unbound-anchor
with that root zone against the three most recent key files (at the
time) from the bottom of http://keyroll.systems/historic:

# Most recent key.

edmonds@chase{0}:~$ curl -so /tmp/root.key 
http://keyroll.systems/static/K.+008+55039.key
edmonds@chase{0}:~$ unbound-anchor -v -r /tmp/root.db -a /tmp/root.key  
 
/tmp/root.key has content
[1438110527] libunbound[7108:0] warning: root hints /tmp/root.db:16 
skipping type SOA
[1438110527] libunbound[7108:0] warning: root hints /tmp/root.db:26 
skipping type TXT
success: the anchor is ok

# Second most recent key.

edmonds@chase{0}:~$ curl -so /tmp/root.key 
http://keyroll.systems/static/K.+008+27079.key
edmonds@chase{0}:~$ unbound-anchor -v -r /tmp/root.db -a /tmp/root.key  
 
/tmp/root.key has content
[1438110543] libunbound[7113:0] warning: root hints /tmp/root.db:16 
skipping type SOA
[1438110543] libunbound[7113:0] warning: root hints /tmp/root.db:26 
skipping type TXT
success: the anchor is ok

# Third most recent key.

edmonds@chase{0}:~$ curl -so /tmp/root.key 
http://keyroll.systems/static/K.+008+42496.key
edmonds@chase{0}:~$ unbound-anchor -v -r /tmp/root.db -a /tmp/root.key  
 
/tmp/root.key has content
[1438110556] libunbound[7118:0] warning: root hints /tmp/root.db:16 
skipping type SOA
[1438110556] libunbound[7118:0] warning: root hints /tmp/root.db:26 
skipping type TXT
last successful probe: Tue Jul 28 15:09:16 2015
the last successful probe is recent
fail: the anchor is NOT ok and could not be fixed
edmonds@chase{0}:~$ cat /tmp/root.key
; autotrust trust anchor file
;;REVOKED
; The zone has all keys revoked, and is
; considered as if it has no trust anchors.
; the remainder of the file is the last probe.
; to restart the trust anchor, overwrite this file.
; with one containing valid DNSKEYs or DSes.
;;id: . 1
;;last_queried: 1438110556 ;;Tue Jul 28 15:09:16 2015
;;last_success: 1438110556 ;;Tue Jul 28 15:09:16 2015
;;next_probe_time: 0 ;;Wed Dec 31 19:00:00 1969
;;query_failed: 0
;;query_interval: 0
;;retry_time: 0
.   3600IN  DNSKEY  385 3 8 
AwEAAct/IgeZiHmphBTGCJUxJNd1hy9uuqUJFtIsdJgyMr+LLnTjbqXkAF47BskHvSIrlQlIc/SDTDLtUktpM/IVWAjolSsP1+oNYwTi56WwW9nyc+vuJkPG8sxza1p7c7PoTegb2JPPEsmkLGMEDz0kliWHSZkinr9yB1/LxI3SBAYq17Od3CuIAWyU0F0pVxqJwJn/jWI4z1FdSwU9cGhx+/g8FvrnrOkOMyj08g4LlYf5PBpopB+Cz2JNOFa6DRr2WyUuVvbTa9ZnBCOTHcUsaoqVdvs3fihvcdpfWonHm7aJvyUnB3CiUQz/iIzvYTtx3+OF8+mOjy0qFX+Zk4KUg6U=
 ;{id = 42624 (ksk), size = 2048b} ;;state=4 [ REVOKED ] ;;count=0 
;;lastchange=1438110556 ;;Tue Jul 28 15:09:16 2015
edmonds@chase{0}:~$ 

Hope this helps!

-- 
Robert Edmonds
edmo...@debian.org