Re: ISC BIND 9 breakage?

[Mark Tinka]

On 25/Mar/20 19:20, Nick Hilliard wrote:

> The fix is either to remove "dnssec-lookaside auto;" from the config
> or else set "dnssec-lookaside no;" and then reload named.

We had issues with that feature back in 2018. We disabled it since then
as a matter of course:

    //dnssec-lookaside auto;

Mark.

Re: ISC BIND 9 breakage?

[Ray Bellis]

On 26/03/2020 06:29, Mark Andrews wrote:

> There should be a official report sometime tomorrow.

Our report is at:



Ray Bellis
Director of DNS Operations, ISC.

Re: ISC BIND 9 breakage?

[Mike Lewinski]

Nick Hilliard wrote:

> forgot to re-sign the zone on dlv.isc.org or forgot to remove 
> dnssec-lookaside from the config?
>
> Not kidding here.  People need to take responsibility for their 
> configurations.

Anyone running BIND provided with CentOS 6 has a release from ~2012 (bind 
9.8.2) and it is understandable why their documentation is out-of-date (like 
OP).

To get more recent bugs and fixes from ISC directly, install from ISC's copr:

https://copr.fedorainfracloud.org/coprs/isc/bind-esv/

On CentOS 7 I needed to install dnf and yum-plugin-copr first. I don't see 
these in the usual places for CentOS 6, so getting copr sources enabled is the 
first challenge.

ISC sources for other distros:
https://www.isc.org/blogs/bind-9-packages/

Mike

Re: ISC BIND 9 breakage?

[Nick Hilliard]

Clayton Zekelman wrote on 26/03/2020 09:49:

Was it a "glitch" or someone just plain old forgot to do it?


forgot to re-sign the zone on dlv.isc.org or forgot to remove 
dnssec-lookaside from the config?


Not kidding here.  People need to take responsibility for their 
configurations.


Nick

Re: ISC BIND 9 breakage?

[Clayton Zekelman]

Was it a "glitch" or someone just plain old forgot to do it?



At 02:29 AM 26/03/2020, Mark Andrews wrote:

It was a glitch with the re-signing of the zone. There should be a official
report sometime tomorrow.  That said "dnssec-lookaside auto;" has been a no-op
in BIND since BIND 9.9.12, BIND 9.10.7, BIND 9.11.3 and a fatal configuration
error as of BIND 9.12.0.  We didn’t want the 
DLV lookup traffic and provides no

benefit as the zone has been empty since 2017.

If you have dnssec-lookaside configured in 
named.conf please remove it otherwise
the DLV code in the validator has to 
cryptographically prove that DLV records don’t
exist before returning that the response is 
insecure.  That requires talking to the
servers for dlv.isc.org.  It does this every 
hour for a active validating resolver

that is still running DNSSEC lookaside validation.

Mark

> On 26 Mar 2020, at 04:18, Drew Weaver  wrote:
>
> Did anyone else on CentOS 6 just have some DNS resolvers totally fall over?
>
> I noticed that this command: dnssec-lookaside 
auto; was causing the issue. The issue occurred right at about 1PM EST.

>
> I see this note in the ISC key file..
>
> # ISC DLV: See https://www.isc.org/solutions/dlv for details.
> #
> # NOTE: The ISC DLV zone is being phased out as of February 2017;
> # the key will remain in place but 
the zone will be otherwise empty.

> # Configuring "dnssec-lookaside auto;" to activate this key is
> # harmless, but is no longer useful and is not recommended.
>
> It’s not harmless anymore.

--
Mark Andrews, ISC
1 Seymour St., Dundas Valley, NSW 2117, Australia
PHONE: +61 2 9871 4742  INTERNET: ma...@isc.org


--

Clayton Zekelman
Managed Network Systems Inc. (MNSi)
3363 Tecumseh Rd. E
Windsor, Ontario
N8W 1H4

tel. 519-985-8410
fax. 519-985-8409

Re: ISC BIND 9 breakage?

[Mark Andrews]

It was a glitch with the re-signing of the zone. There should be a official
report sometime tomorrow.  That said "dnssec-lookaside auto;" has been a no-op
in BIND since BIND 9.9.12, BIND 9.10.7, BIND 9.11.3 and a fatal configuration
error as of BIND 9.12.0.  We didn’t want the DLV lookup traffic and provides no
benefit as the zone has been empty since 2017.

If you have dnssec-lookaside configured in named.conf please remove it otherwise
the DLV code in the validator has to cryptographically prove that DLV records 
don’t
exist before returning that the response is insecure.  That requires talking to 
the
servers for dlv.isc.org.  It does this every hour for a active validating 
resolver
that is still running DNSSEC lookaside validation.

Mark

> On 26 Mar 2020, at 04:18, Drew Weaver  wrote:
> 
> Did anyone else on CentOS 6 just have some DNS resolvers totally fall over?
>  
> I noticed that this command: dnssec-lookaside auto; was causing the issue. 
> The issue occurred right at about 1PM EST.
>  
> I see this note in the ISC key file..
>  
> # ISC DLV: See https://www.isc.org/solutions/dlv for details.
> #
> # NOTE: The ISC DLV zone is being phased out as of February 2017;
> # the key will remain in place but the zone will be otherwise empty.
> # Configuring "dnssec-lookaside auto;" to activate this key is
> # harmless, but is no longer useful and is not recommended.
>  
> It’s not harmless anymore.

-- 
Mark Andrews, ISC
1 Seymour St., Dundas Valley, NSW 2117, Australia
PHONE: +61 2 9871 4742  INTERNET: ma...@isc.org

Re: ISC BIND 9 breakage?

[Owen DeLong]

Yeah, looks like that comment should have been updated to “harmless until…”

Owen


> On Mar 25, 2020, at 10:32 , Drew Weaver  wrote:
> 
> We just left the dnssec-lookaside auto; configuration in there. Probably 
> because it specifically says in the documentation from ISC that it won't hurt 
> anything to leave it in there...
> 
> # Configuring "dnssec-lookaside auto;" to activate this key is
># harmless
> 
> Guess not?
> 
> Thanks,
> -Drew
> 
> 
> 
> 
> -Original Message-
> From: Stephane Bortzmeyer  
> Sent: Wednesday, March 25, 2020 1:27 PM
> To: Drew Weaver 
> Cc: 'nanog@nanog.org' 
> Subject: Re: ISC BIND 9 breakage?
> 
> On Wed, Mar 25, 2020 at 05:18:49PM +,  Drew Weaver 
>  wrote  a message of 97 lines which said:
> 
>> Did anyone else on CentOS 6 just have some DNS resolvers totally fall over?
> 
> dlv.isc.org signatures just expired.
> 
>># NOTE: The ISC DLV zone is being phased out as of February
>>2017;
> 
> And yet some people still use it, it seems.

RE: ISC BIND 9 breakage?

[Drew Weaver]

We just left the dnssec-lookaside auto; configuration in there. Probably 
because it specifically says in the documentation from ISC that it won't hurt 
anything to leave it in there...

# Configuring "dnssec-lookaside auto;" to activate this key is
# harmless

Guess not?

Thanks,
-Drew




-Original Message-
From: Stephane Bortzmeyer  
Sent: Wednesday, March 25, 2020 1:27 PM
To: Drew Weaver 
Cc: 'nanog@nanog.org' 
Subject: Re: ISC BIND 9 breakage?

On Wed, Mar 25, 2020 at 05:18:49PM +,  Drew Weaver  
wrote  a message of 97 lines which said:

> Did anyone else on CentOS 6 just have some DNS resolvers totally fall over?

dlv.isc.org signatures just expired.

> # NOTE: The ISC DLV zone is being phased out as of February
> 2017;

And yet some people still use it, it seems.

Re: ISC BIND 9 breakage?

[Stephane Bortzmeyer]

On Wed, Mar 25, 2020 at 05:18:49PM +,
 Drew Weaver  wrote 
 a message of 97 lines which said:

> Did anyone else on CentOS 6 just have some DNS resolvers totally fall over?

dlv.isc.org signatures just expired.

> # NOTE: The ISC DLV zone is being phased out as of February
> 2017;

And yet some people still use it, it seems.

RE: ISC BIND 9 breakage?

[Drew Weaver]

Oh, yes. I am aware.

I am asking if anyone has any info as to why it just randomly stopped running 
perfectly normally at exactly 1PM EST?

Thanks,
-Drew


-Original Message-
From: Nick Hilliard  
Sent: Wednesday, March 25, 2020 1:21 PM
To: Drew Weaver 
Cc: 'nanog@nanog.org' 
Subject: Re: ISC BIND 9 breakage?

The fix is either to remove "dnssec-lookaside auto;" from the config or else 
set "dnssec-lookaside no;" and then reload named.

Nick

Drew Weaver wrote on 25/03/2020 17:18:
> Did anyone else on CentOS 6 just have some DNS resolvers totally fall over?
> 
> I noticed that this command: dnssec-lookaside auto; was causing the 
> issue. The issue occurred right at about 1PM EST.
> 
> I see this note in the ISC key file..
> 
> # ISC DLV: See https://www.isc.org/solutions/dlv for details.
> 
>      #
> 
>      # NOTE: The ISC DLV zone is being phased out as of February 
> 2017;
> 
>      # the key will remain in place but the zone will be otherwise 
> empty.
> 
>      # Configuring "dnssec-lookaside auto;" to activate this key 
> is
> 
>      # harmless, but is no longer useful and is not recommended.
> 
> It's not harmless anymore.
> 

Re: ISC BIND 9 breakage?

[Nick Hilliard]

The fix is either to remove "dnssec-lookaside auto;" from the config or 
else set "dnssec-lookaside no;" and then reload named.


Nick

Drew Weaver wrote on 25/03/2020 17:18:

Did anyone else on CentOS 6 just have some DNS resolvers totally fall over?

I noticed that this command: dnssec-lookaside auto; was causing the 
issue. The issue occurred right at about 1PM EST.


I see this note in the ISC key file..

# ISC DLV: See https://www.isc.org/solutions/dlv for details.

     #

     # NOTE: The ISC DLV zone is being phased out as of February 2017;

     # the key will remain in place but the zone will be otherwise 
empty.


     # Configuring "dnssec-lookaside auto;" to activate this key is

     # harmless, but is no longer useful and is not recommended.

It’s not harmless anymore.