Re: [NOC] ARIN contact needed: something bad happens with legacy IPv4 block's reverse delegations

[William Herrin]

On Mon, Mar 20, 2017 at 3:27 PM, Brett Frankenberger 
wrote:

> If ARIN delegated 32.11.10.in-addr.arpa through 47.11.10.in-addr.arpa
> to a RIPE nameserver, there's no good way for RIPE to then delegate,
> say, 10.11.34.0/24 (34.11.10.in-addr.arpa) to the nameserver of the
> entity to which RIPE has allocated 10.11.34.0.  (Sure, it can be done,
> using the same techniques as are used for allocations of
> longer-than-/24, but recipients of /24 and larger reasonably expect to
> have the X.X.X.in-addr.arpa delegated to their nameservers.)
>

Hi Brett,

The last I tried it, the servers which the glue claims are authoritative
for a zone could assert that they themselves are not authoritative and
offer new glue for completely different servers asserted to be
authoritative. I had to fake a parent zone in Bind. This was before DNSSEC.

Regards,
Bill Herrin




-- 
William Herrin  her...@dirtside.com  b...@herrin.us
Dirtside Systems . Web: 

Re: [NOC] ARIN contact needed: something bad happens with legacy IPv4 block's reverse delegations

[Brett Frankenberger]

On Sat, Mar 18, 2017 at 09:27:11PM -0700, Doug Barton wrote:
> 
> > As to why DNS-native zone operations are not utilized, the challenge
> > is that reverse DNS zones for IPv4 and DNS operations are on octet
> > boundaries, but IPv4 address blocks may be aligned on any bit
> > boundary.
> 
> Yes, deeply familiar with that problem. Are you dealing with any address
> blocks smaller than a /24? If the answer is no (which it almost certainly
> is), what challenges are you facing that you haven't figured out how to
> overcome yet? (Even < /24 blocks can be dealt with, obviously, but I'd be
> interested to learn that there are problems with /24 and up that are too
> difficult to solve.)

Hypotheically:

10.11.0.0/16 (11.10.in-addr.arpa) is managed by ARIN
10.11.16.0/20 is ARIN space
10.11.32.0/20 is RIPE space

If ARIN delegated 32.11.10.in-addr.arpa through 47.11.10.in-addr.arpa
to a RIPE nameserver, there's no good way for RIPE to then delegate,
say, 10.11.34.0/24 (34.11.10.in-addr.arpa) to the nameserver of the
entity to which RIPE has allocated 10.11.34.0.  (Sure, it can be done,
using the same techniques as are used for allocations of
longer-than-/24, but recipients of /24 and larger reasonably expect to
have the X.X.X.in-addr.arpa delegated to their nameservers.)

So, instead, RIPE communicates to ARIN the proper delegations for
32.11.10.in-addr.arpa through 47.11.10.in-addr.arpa, and ARIN merges
those into 11.10.in-addr.arpa.

One way for RIPE to communicate those delegations to ARIN is to put
then into some other zone, which ARIN could then zone-transfer.  But
ARIN would still need a process to merge the data from that other e
with the real 11.10.in-addr.arpa zone.  But that has the same risks as
the current process, which apparently communicates those delegations
via something other than zone-transfer.

 -- Brett

Re: [NOC] ARIN contact needed: something bad happens with legacy IPv4 block's reverse delegations

[Romeo Zwart]

Dear John, Bill and all,

On 17/03/17 19:31 , John Curran wrote:
> On 17 Mar 2017, at 2:17 PM, William Herrin  wrote:
>>
>> On Fri, Mar 17, 2017 at 2:14 PM, John Curran  wrote:
>>
>>> See previous reply.  The data was both correctly formatted and signed,
>>> so the agreed integrity checks passed.
>>>
>> Ah, okay. So it wasn't bad counts as originally reported but no data with
>> counts that confirmed no data. Thanks for the clarification!
> 
> Bill - 
> 
> Glad to help (and apologies for the information coming out in pieces –
> we’ve opted to go with updates as we learn more rather than some for 
> comprehensive but less timely report.) 

We have been slow to clarify this from the RIPE NCC end, for which I
apologize. As was already mentioned by Mark and John in previous
messages in this thread, the initial report from the RIPE NCC wasn't
complete, which has lead to unnecessary confusion.

A follow up message with additional detail was sent to the RIPE NCC DNS
working group list earlier today:
 https://www.ripe.net/ripe/mail/archives/dns-wg/2017-March/003401.html

We hope that this clarifies matters sufficiently.

Kind regards,
Romeo Zwart
RIPE NCC

Re: [NOC] ARIN contact needed: something bad happens with legacy IPv4 block's reverse delegations

[Doug Barton]

On 03/18/2017 10:53 PM, John Curran wrote:

On 19 Mar 2017, at 12:50 AM, Doug Barton mailto:do...@dougbarton.us>> wrote:

...
Meanwhile, my offer to help y'all fix your DNS was a sincere one. Feel
free to hit me up off list.


Doug -

  You’d want to make that offer to the RIPE NCC


My offer was in response to your assertion that normal DNS techniques of 
delegation were not sufficient to the unique problems ARIN has to deal 
with in regards to the address space you manage delegations for.


Subsequent to our conversation however, Shane Kerr was kind enough to 
explain the problem that the "zonelets" are designed to solve:


https://www.ripe.net/ripe/mail/archives/dns-wg/2017-March/003406.html


Short version, ARIN maintains foo/8, but bar/16 within it is managed by 
RIPE, who wants to delegate it directly to the registered party for that 
block. They use a zonelet to tell ARIN how to do that.


As you have indicated that ARIN will not make any changes to its 
existing practices without specific instructions from RIPE, I will offer 
my suggestions to them instead.  :)


best,

Doug

Re: [NOC] ARIN contact needed: something bad happens with legacy IPv4 block's reverse delegations

[John Curran]

On 19 Mar 2017, at 12:50 AM, Doug Barton 
mailto:do...@dougbarton.us>> wrote:
...
Meanwhile, my offer to help y'all fix your DNS was a sincere one. Feel free to 
hit me up off list.

Doug -

  You’d want to make that offer to the RIPE NCC (as they experienced the issues 
with their DNS systems.)

  Let me know if you need contact info.

Thanks,
/John

John Curran
President and CEO
ARIN

Re: [NOC] ARIN contact needed: something bad happens with legacy IPv4 block's reverse delegations

[Doug Barton]

On 03/18/2017 09:40 PM, John Curran wrote:

On 19 Mar 2017, at 12:27 AM, Doug Barton  wrote:

...

Despite the associated risk, we are happy to install such checks if
RIPE requests them, but are this time are processing them as we
agreed to do so – which is whenever we receive correctly formatted
and properly signed requests from them. (You should inquire to RIPE
for more detail regarding their future intentions in this regard.)


Already did, thanks. :)  Meanwhile, one could make a legitimate argument that 
even absent specific guidance from RIPE, ARIN should have a sufficient level of 
concern for the health of the larger Internet to consider unilateral action 
here. At least in the form of delaying implementation until some human 
coordination takes place.


We’ll process RIPE’s requests in whatever manner they direct, as it is their
customers that are affected by whatever decision they make in this regard.


I'll let others chime in on whether they they think that's a reasonable 
response. I've already said my piece.


Meanwhile, my offer to help y'all fix your DNS was a sincere one. Feel 
free to hit me up off list.


Doug

Re: [NOC] ARIN contact needed: something bad happens with legacy IPv4 block's reverse delegations

[John Curran]

On 19 Mar 2017, at 12:27 AM, Doug Barton  wrote:
> ...
>> Despite the associated risk, we are happy to install such checks if
>> RIPE requests them, but are this time are processing them as we
>> agreed to do so – which is whenever we receive correctly formatted
>> and properly signed requests from them. (You should inquire to RIPE
>> for more detail regarding their future intentions in this regard.)
> 
> Already did, thanks. :)  Meanwhile, one could make a legitimate argument that 
> even absent specific guidance from RIPE, ARIN should have a sufficient level 
> of concern for the health of the larger Internet to consider unilateral 
> action here. At least in the form of delaying implementation until some human 
> coordination takes place.

We’ll process RIPE’s requests in whatever manner they direct, as it is their 
customers that are affected by whatever decision they make in this regard.

Thanks!
/John

John Curran
President and CEO
ARIN

Re: [NOC] ARIN contact needed: something bad happens with legacy IPv4 block's reverse delegations

[Doug Barton]

Thanks for the response, John. Some thoughts below.

On 03/18/2017 08:58 PM, John Curran wrote:

On 18 Mar 2017, at 9:58 PM, Doug Barton mailto:do...@dougbarton.us>> wrote:


My eyebrows reacted to this the same way Bill's did. It sounds
like this is at least a semi-automated system. Such things should
have sanity checks on the receiving side when told to remove large
gobs of data, even if the instructions validate correctly.

More fundamentally, according to the RIPE report they are sending
you something called "zonelets" which you then process into actual
DNS data. Can you say something about the relative merit of this
system, vs. simply delegating the right zones to the right parties
and letting the DNS do what it was intended to do?

At minimum the fact that this automated system was allowed to wipe
out great chunks of important data calls it into question. And
sure, you can all 3 fix the bugs you found this time around, but up
until these bugs were triggered you all thought the system was
functioning perfectly, in spite of it ending up doing something
that obviously was not intended.


Doug -

We could indeed decide to ignore correctly formatted and signed
information if it doesn’t match some heuristics that we put in place
(e.g. empty zone, zone with only 1 entry, zone that changes more than
10% in size, etc.)


I have used the latter type of system with good results, for what it's 
worth. And funny you should mention 10%, as that's what I've found to be 
fairly commonly at least a yellow flag, if not a big red one.



Some downsides with this approach is that that: 1) we’d be
establishing heuristics for data that originates with a different
organization and absent knowledge of their business changes,


If you're not already having ongoing discussions about said changes well 
in advance, your system is broken.



and 2)
this would be mean that there could be occasions where proper data
cannot be installed without manual intervention (because the changes
happens to be outside of whatever heuristics have previously been
put in place.)


See above. Also, not putting in place *new* changes on a scale 
sufficient to trip the heuristics is far superior to automatically 
putting in place changes that take huge chunks of address space off the 
network. Or am I missing something?


And if you're having regular conversations with your "customers" in this 
scenario about upcoming major changes, tripping the alarm should only 
happen if someone, somewhere, made a mistake. Thus, human intervention 
is required regardless.


But, see below.


Despite the associated risk, we are happy to install such checks if
RIPE requests them, but are this time are processing them as we
agreed to do so – which is whenever we receive correctly formatted
and properly signed requests from them. (You should inquire to RIPE
for more detail regarding their future intentions in this regard.)


Already did, thanks. :)  Meanwhile, one could make a legitimate argument 
that even absent specific guidance from RIPE, ARIN should have a 
sufficient level of concern for the health of the larger Internet to 
consider unilateral action here. At least in the form of delaying 
implementation until some human coordination takes place.


Personally, I don't buy, "They told us to do it!" as a legitimate 
response here.



As to why DNS-native zone operations are not utilized, the challenge
is that reverse DNS zones for IPv4 and DNS operations are on octet
boundaries, but IPv4 address blocks may be aligned on any bit
boundary.


Yes, deeply familiar with that problem. Are you dealing with any address 
blocks smaller than a /24? If the answer is no (which it almost 
certainly is), what challenges are you facing that you haven't figured 
out how to overcome yet? (Even < /24 blocks can be dealt with, 
obviously, but I'd be interested to learn that there are problems with 
/24 and up that are too difficult to solve.)


Personally, I would be happy to donate my time to help y'all sort this 
out, and I'm sure there are others who would also be willing to help.


Doug

Re: [NOC] ARIN contact needed: something bad happens with legacy IPv4 block's reverse delegations

[John Curran]

On 18 Mar 2017, at 9:58 PM, Doug Barton  wrote:
> 
> My eyebrows reacted to this the same way Bill's did. It sounds like this is 
> at least a semi-automated system. Such things should have sanity checks on 
> the receiving side when told to remove large gobs of data, even if the 
> instructions validate correctly.
> 
> More fundamentally, according to the RIPE report they are sending you 
> something called "zonelets" which you then process into actual DNS data. Can 
> you say something about the relative merit of this system, vs. simply 
> delegating the right zones to the right parties and letting the DNS do what 
> it was intended to do?
> 
> At minimum the fact that this automated system was allowed to wipe out great 
> chunks of important data calls it into question. And sure, you can all 3 fix 
> the bugs you found this time around, but up until these bugs were triggered 
> you all thought the system was functioning perfectly, in spite of it ending 
> up doing something that obviously was not intended.

Doug - 
 
   We could indeed decide to ignore correctly formatted and signed information 
if 
   it doesn’t match some heuristics that we put in place (e.g. empty zone, zone 
with 
   only 1 entry, zone that changes more than 10% in size, etc.)

   Some downsides with this approach is that that: 1) we’d be establishing 
heuristics 
   for data that originates with a different organization and absent knowledge 
of their
   business changes, and 2) this would be mean that there could be occasions 
where 
   proper data cannot be installed without manual intervention (because the 
changes 
   happens to be outside of whatever heuristics have previously been put in 
place.)

   Despite the associated risk, we are happy to install such checks if  RIPE 
requests 
   them, but are this time are processing them as we agreed to do so – which is 
   whenever we receive correctly formatted and properly signed requests from 
them. 
   (You should inquire to RIPE for more detail regarding their future 
intentions in this
   regard.) 

   As to why DNS-native zone operations are not utilized, the challenge is that 
reverse DNS 
   zones for IPv4 and DNS operations are on octet boundaries, but IPv4 address 
blocks may 
   be aligned on any bit boundary.  Thus, a single IPv4 octet range may contain 
IPv4 address 
   blocks that are administered by multiple RIRs, making it is necessary for 
one RIR to be 
   authoritative for the entire zone and other RIRs to send information 
seperately on their IPv4 
  address blocks in that same range so that it gets included in the appropriate 
zone file. 

Excellent questions - thanks!
/John

John Curran
President and CEO
ARIN

Re: [NOC] ARIN contact needed: something bad happens with legacy IPv4 block's reverse delegations

[Doug Barton]

On 03/17/2017 10:42 AM, Mark Kosters wrote:

On 3/17/17, 12:26 PM, "NANOG on behalf of William Herrin"
 wrote:

On Fri, Mar 17, 2017 at 7:52 AM, Romeo Zwart 
wrote:

RIPE NCC have issued a statement about the issue here:

https://www.ripe.net/ripe/mail/archives/dns-wg/2017-March/003394.html







Our apologies for the inconvenience caused.


Hmm. That sounds like an ARIN-side bug too. ARIN's code responded to
corrupted data by zeroing out the data instead of using the last
known good data. That's awfully brittle for such a critical service.

Regards, Bill Herrin


Hi Bill,

The analysis was not yet complete when the notice went out from RIPE.
After doing a post-mortum, there were no bugs in ARIN’s software in
regards to this issue. We followed exactly what RIPE told us to do.
When we noticed an issue with RIPE’s updates yesterday, we notified
them as well.


My eyebrows reacted to this the same way Bill's did. It sounds like this 
is at least a semi-automated system. Such things should have sanity 
checks on the receiving side when told to remove large gobs of data, 
even if the instructions validate correctly.


More fundamentally, according to the RIPE report they are sending you 
something called "zonelets" which you then process into actual DNS data. 
Can you say something about the relative merit of this system, vs. 
simply delegating the right zones to the right parties and letting the 
DNS do what it was intended to do?


At minimum the fact that this automated system was allowed to wipe out 
great chunks of important data calls it into question. And sure, you can 
all 3 fix the bugs you found this time around, but up until these bugs 
were triggered you all thought the system was functioning perfectly, in 
spite of it ending up doing something that obviously was not intended.


Doug

Re: [NOC] ARIN contact needed: something bad happens with legacy IPv4 block's reverse delegations

[William Herrin]

On Fri, Mar 17, 2017 at 2:31 PM, John Curran  wrote:

> Glad to help (and apologies for the information coming out in pieces –
> we’ve opted to go with updates as we learn more rather than some for
> comprehensive but less timely report.)
>

Also very much appreciated.

Regards,
Bill Herrin


-- 
William Herrin  her...@dirtside.com  b...@herrin.us
Dirtside Systems . Web: 

Re: [NOC] ARIN contact needed: something bad happens with legacy IPv4 block's reverse delegations

[John Curran]

On 17 Mar 2017, at 2:17 PM, William Herrin  wrote:
> 
> On Fri, Mar 17, 2017 at 2:14 PM, John Curran  wrote:
> 
>> See previous reply.  The data was both correctly formatted and signed,
>> so the agreed integrity checks passed.
>> 
> 
> Ah, okay. So it wasn't bad counts as originally reported but no data with
> counts that confirmed no data. Thanks for the clarification!

Bill - 

Glad to help (and apologies for the information coming out in pieces –
we’ve opted to go with updates as we learn more rather than some for 
comprehensive but less timely report.) 

Thanks!
/John

John Curran
President and CEO
ARIN

Re: [NOC] ARIN contact needed: something bad happens with legacy IPv4 block's reverse delegations

[William Herrin]

On Fri, Mar 17, 2017 at 2:14 PM, John Curran  wrote:

> See previous reply.  The data was both correctly formatted and signed,
> so the agreed integrity checks passed.
>

Ah, okay. So it wasn't bad counts as originally reported but no data with
counts that confirmed no data. Thanks for the clarification!

Regards,
Bill Herrin



-- 
William Herrin  her...@dirtside.com  b...@herrin.us
Dirtside Systems . Web: 

Re: [NOC] ARIN contact needed: something bad happens with legacy IPv4 block's reverse delegations

[John Curran]

On 17 Mar 2017, at 2:08 PM, William Herrin  wrote:
> 
> On Fri, Mar 17, 2017 at 1:42 PM, Mark Kosters  wrote:
>> On 3/17/17, 12:26 PM, "NANOG on behalf of William Herrin" <
> nanog-boun...@nanog.org on behalf of b...@herrin.us> wrote:
>>>Hmm. That sounds like an ARIN-side bug too. ARIN's code responded to
>>>corrupted data by zeroing out the data instead of using the last
> known good
>> 
>> there were no bugs in ARIN’s software in regards to this issue. We
> followed exactly what RIPE told us to do.
> 
> Hi Mark,
> 
> That shot my eyebrow up. You misspoke here, right? There's no bug -solely
> because- you did what the design said to do? The design calls for some
> self-check information and it's not a critical design bug to zero-out the
> publish if the self-check fails?

Bill - 

See previous reply.  The data was both correctly formatted and signed, 
so the agreed integrity checks passed. 

Thanks,
/John

John Curran
President and CEO
ARIN

Re: [NOC] ARIN contact needed: something bad happens with legacy IPv4 block's reverse delegations

[John Curran]

On 17 Mar 2017, at 12:26 PM, William Herrin 
mailto:b...@herrin.us>> wrote:

On Fri, Mar 17, 2017 at 7:52 AM, Romeo Zwart 
mailto:rz%2b...@zwart.com>> wrote:
> RIPE NCC have issued a statement about the issue here:
>
>  https://www.ripe.net/ripe/mail/archives/dns-wg/2017-March/003394.html
>
> Our apologies for the inconvenience caused.

Hmm. That sounds like an ARIN-side bug too. ARIN's code responded to corrupted 
data by zeroing out the data instead of using the last known good data. That's 
awfully brittle for such a critical service.

Agreed in principle - receiving incorrect data (improperly formatted, 
corrupted, or not properly signed)
should result in appropriate notice and no change to the running system.  This 
is actually the case with
ARIN’s systems.

However, we received a properly formatted and signed zonelet file, albeit one 
which contained zero
records.   APNIC also received similar correctly formatted/signed zonelet files 
as a record of the RIPE
bug, and the three RIRs have been working closely together to get the correct 
RIPE data loaded back
into our authoritative DNS systems.

Thanks!
/John

John Curran
President and CEO
ARIN

Re: [NOC] ARIN contact needed: something bad happens with legacy IPv4 block's reverse delegations

[Mark Kosters]

On 3/17/17, 12:26 PM, "NANOG on behalf of William Herrin" 
 wrote:

On Fri, Mar 17, 2017 at 7:52 AM, Romeo Zwart  wrote:
> RIPE NCC have issued a statement about the issue here:
>
>  https://www.ripe.net/ripe/mail/archives/dns-wg/2017-March/003394.html
>
> Our apologies for the inconvenience caused.

Hmm. That sounds like an ARIN-side bug too. ARIN's code responded to
corrupted data by zeroing out the data instead of using the last known good
data. That's awfully brittle for such a critical service.

Regards,
Bill Herrin
  

Hi Bill,

The analysis was not yet complete when the notice went out from RIPE. After 
doing a post-mortum, there were no bugs in ARIN’s software in regards to this 
issue. We followed exactly what RIPE told us to do. When we noticed an issue 
with RIPE’s updates yesterday, we notified them as well.

Two zones managed by APNIC that received delegations from RIPE were also 
affected by RIPE’s bug – it was more than just a RIPE/ARIN thing.

The three impacted RIR’s have been working closely together to get RIPE’s DNS 
entries correctly added into our respective authoritative DNS systems.

Thanks,
Mark
ARIN CTO
 

Re: [NOC] ARIN contact needed: something bad happens with legacy IPv4 block's reverse delegations

[William Herrin]

On Fri, Mar 17, 2017 at 1:42 PM, Mark Kosters  wrote:
> On 3/17/17, 12:26 PM, "NANOG on behalf of William Herrin" <
nanog-boun...@nanog.org on behalf of b...@herrin.us> wrote:
>> Hmm. That sounds like an ARIN-side bug too. ARIN's code responded to
>> corrupted data by zeroing out the data instead of using the last
known good
>
> there were no bugs in ARIN’s software in regards to this issue. We
followed exactly what RIPE told us to do.

Hi Mark,

That shot my eyebrow up. You misspoke here, right? There's no bug -solely
because- you did what the design said to do? The design calls for some
self-check information and it's not a critical design bug to zero-out the
publish if the self-check fails?

Regards,
Bill Herrin



-- 
William Herrin  her...@dirtside.com  b...@herrin.us
Dirtside Systems . Web: 

Re: [NOC] ARIN contact needed: something bad happens with legacy IPv4 block's reverse delegations

[George William Herbert]

> On Mar 17, 2017, at 10:28 AM, valdis.kletni...@vt.edu wrote:
> 
> On Fri, 17 Mar 2017 17:42:11 +0100, Bjørn Mork said:
>> Well, it was a nice smoke test of the "RDNS required" anti-feature.  All
>> of a sudden we couldn't even send email to ourselves, having smarthosts
>> in one of the affected zones. Nice.
> 
> If you don't have a chaos monkey, the Internet will provide one free of 
> charge.

Or to a service partner or underlying infrastructure you rely on.

Isn't complexity grand...


-george

Sent from my iPhone

Re: [NOC] ARIN contact needed: something bad happens with legacy IPv4 block's reverse delegations

[valdis . kletnieks]

On Fri, 17 Mar 2017 17:42:11 +0100, Bjørn Mork said:
> Well, it was a nice smoke test of the "RDNS required" anti-feature.  All
> of a sudden we couldn't even send email to ourselves, having smarthosts
> in one of the affected zones. Nice.

If you don't have a chaos monkey, the Internet will provide one free of charge.



pgpQlswegClVA.pgp
Description: PGP signature

Re: [NOC] ARIN contact needed: something bad happens with legacy IPv4 block's reverse delegations

[Jared Mauch]

On Fri, Mar 17, 2017 at 05:42:11PM +0100, Bjørn Mork wrote:
> William Herrin  writes:
> > On Fri, Mar 17, 2017 at 7:52 AM, Romeo Zwart  wrote:
> >> RIPE NCC have issued a statement about the issue here:
> >>
> >>  https://www.ripe.net/ripe/mail/archives/dns-wg/2017-March/003394.html
> >>
> >> Our apologies for the inconvenience caused.
> >
> > Hmm. That sounds like an ARIN-side bug too. ARIN's code responded to
> > corrupted data by zeroing out the data instead of using the last known good
> > data. That's awfully brittle for such a critical service.
> 
> Well, it was a nice smoke test of the "RDNS required" anti-feature.  All
> of a sudden we couldn't even send email to ourselves, having smarthosts
> in one of the affected zones. Nice.
> 
> Maybe time to re-evaluate the usefulness of that config...

or proper whitelisting of your own infrastructure :-)

- Jared

-- 
Jared Mauch  | pgp key available via finger from ja...@puck.nether.net
clue++;  | http://puck.nether.net/~jared/  My statements are only mine.

Re: [NOC] ARIN contact needed: something bad happens with legacy IPv4 block's reverse delegations

[Bjørn Mork]

William Herrin  writes:
> On Fri, Mar 17, 2017 at 7:52 AM, Romeo Zwart  wrote:
>> RIPE NCC have issued a statement about the issue here:
>>
>>  https://www.ripe.net/ripe/mail/archives/dns-wg/2017-March/003394.html
>>
>> Our apologies for the inconvenience caused.
>
> Hmm. That sounds like an ARIN-side bug too. ARIN's code responded to
> corrupted data by zeroing out the data instead of using the last known good
> data. That's awfully brittle for such a critical service.

Well, it was a nice smoke test of the "RDNS required" anti-feature.  All
of a sudden we couldn't even send email to ourselves, having smarthosts
in one of the affected zones. Nice.

Maybe time to re-evaluate the usefulness of that config...



Bjørn

Re: [NOC] ARIN contact needed: something bad happens with legacy IPv4 block's reverse delegations

[William Herrin]

On Fri, Mar 17, 2017 at 7:52 AM, Romeo Zwart  wrote:
> RIPE NCC have issued a statement about the issue here:
>
>  https://www.ripe.net/ripe/mail/archives/dns-wg/2017-March/003394.html
>
> Our apologies for the inconvenience caused.

Hmm. That sounds like an ARIN-side bug too. ARIN's code responded to
corrupted data by zeroing out the data instead of using the last known good
data. That's awfully brittle for such a critical service.

Regards,
Bill Herrin



-- 
William Herrin  her...@dirtside.com  b...@herrin.us
Dirtside Systems . Web: 

Re: [NOC] ARIN contact needed: something bad happens with legacy IPv4 block's reverse delegations

[Romeo Zwart]

Dear all,

RIPE NCC have issued a statement about the issue here:

 https://www.ripe.net/ripe/mail/archives/dns-wg/2017-March/003394.html

Our apologies for the inconvenience caused.

Kind regards,
Romeo Zwart
RIPE NCC

On 17/03/17 12:31 , John Curran wrote:
> Eygene - 
> 
>   We are aware there’s an issue and working on it presently with RIPE. 
>   Expect additional updates shortly.
> 
> /John
> 
> John Curran
> President and CEO
> ARIN
> 
>> On 17 Mar 2017, at 11:04 AM, Eygene Ryabinkin  wrote:
>>
>> Job, good day.
>>
>> Fri, Mar 17, 2017 at 09:55:56AM +, Job Snijders wrote:
>>> 171 also seems affected.
>>
>> Not the whole one, it seems:
>> {{{
>> $ dig +trace -t soa 1.171.in-addr.arpa
>>
>> ; <<>> DiG 9.10.4-P6 <<>> +trace -t soa 1.171.in-addr.arpa
>> ;; global options: +cmd
>> .202634  IN  NS  m.root-servers.net.
>> .202634  IN  NS  i.root-servers.net.
>> .202634  IN  NS  k.root-servers.net.
>> .202634  IN  NS  c.root-servers.net.
>> .202634  IN  NS  a.root-servers.net.
>> .202634  IN  NS  d.root-servers.net.
>> .202634  IN  NS  l.root-servers.net.
>> .202634  IN  NS  e.root-servers.net.
>> .202634  IN  NS  g.root-servers.net.
>> .202634  IN  NS  b.root-servers.net.
>> .202634  IN  NS  j.root-servers.net.
>> .202634  IN  NS  h.root-servers.net.
>> .202634  IN  NS  f.root-servers.net.
>> .511016  IN  RRSIG   NS 8 0 518400 2017032917 
>> 2017031616 61045 . 
>> OjsjoA6WCcThV3BqhAyMaXI3bU1m228Gl8nvaR074qBtem/RjaFJh1Oe 
>> r7LPI6W15jgbRGuCY7/GUNgDex4ZM43yvx2iY+2GpSk9b2/pKGbDaDIp 
>> X1Hd8418206eow1P/SgPqtT+2LzjM+lz/HKUvCyPoN4uX5/7GDExn8ir 
>> 2Q/vw81Za2nJ4Ji6oeGAmE8g6V3RJfTI0El/CP9Vl2/r0jWDZZ+wtYRA 
>> uwqvQYj+7VZc5+UJDJ3/Gne1LBnzXKnZRwnfJtZenhfZ7X20D9PiXvfJ 
>> M2s1ryqZvg8K5RqYt/r8pIiq1Udd1KWQBgdN7Q629+jNSngZQtH19R9H fM7h1w==
>> ;; Received 1097 bytes from 127.0.0.1#53(127.0.0.1) in 0 ms
>>
>> in-addr.arpa.172800  IN  NS  a.in-addr-servers.arpa.
>> in-addr.arpa.172800  IN  NS  b.in-addr-servers.arpa.
>> in-addr.arpa.172800  IN  NS  c.in-addr-servers.arpa.
>> in-addr.arpa.172800  IN  NS  d.in-addr-servers.arpa.
>> in-addr.arpa.172800  IN  NS  e.in-addr-servers.arpa.
>> in-addr.arpa.172800  IN  NS  f.in-addr-servers.arpa.
>> in-addr.arpa.86400   IN  DS  47054 8 2 
>> 5CAFCCEC201D1933B4C9F6A9C8F51E51F3B39979058AC21B8DF1B1F2 81CBC6F2
>> in-addr.arpa.86400   IN  DS  53696 8 2 
>> 13E5501C56B20394DA921B51412D48B7089C5EB6957A7C58553C4D4D 424F04DF
>> in-addr.arpa.86400   IN  DS  63982 8 2 
>> AAF4FB5D213EF25AE44679032EBE3514C487D7ABD99D7F5FEC3383D0 30733C73
>> in-addr.arpa.86400   IN  RRSIG   DS 8 2 86400 
>> 2017033000 2017031623 33786 arpa. 
>> gqNN6MmLVxFtSG0oxdyoYVSXZlp6vY9yxpnJW89TYCHfTkv+ZmklM76O 
>> IHdTznEhLHXbyr4BaxjFCshsC3WacdH/YZYa/SZynJ9Q1N6/bogrIUTn 
>> qs61Y/YD4Sk5lI7YvUxVnPrF3lk1pY8dpoTkprTLZngoeQEsm552inqG ypU=
>> ;; Received 731 bytes from 193.0.14.129#53(k.root-servers.net) in 41 ms
>>
>> 171.in-addr.arpa.86400   IN  NS  ns1.apnic.net.
>> 171.in-addr.arpa.86400   IN  NS  ns2.lacnic.net.
>> 171.in-addr.arpa.86400   IN  NS  ns3.apnic.net.
>> 171.in-addr.arpa.86400   IN  NS  ns4.apnic.net.
>> 171.in-addr.arpa.86400   IN  NS  apnic.authdns.ripe.net.
>> 171.in-addr.arpa.86400   IN  NS  apnic1.dnsnode.net.
>> 171.in-addr.arpa.86400   IN  NS  tinnie.arin.net.
>> 171.in-addr.arpa.86400   IN  DS  49699 5 1 
>> 0240801C96480900CC6D93130AF45CFE86EDE940
>> 171.in-addr.arpa.86400   IN  DS  49699 5 2 
>> 6E0BA96F05B4D39C1668979731E040213BB6130DE33E86E063B5F7F7 5C465C88
>> 171.in-addr.arpa.86400   IN  RRSIG   DS 8 3 86400 20170330042932 
>> 20170309125911 4341 in-addr.arpa. 
>> RKcPZJdG1MBrwpfa1mIK7ikIU585i1Jv+UkEBHxuM3BxxAbD+ht0W04C 
>> ljboyXJfYK8ly/YTDqNUkWKZLwDHlkq/rgNwTG63sPT8iK8qya+2qUVB 
>> iTXVsD2HaV1V/KwgJjlWHRNnlwkK0YZ5Q7tevXaq4yyLT2Q2mu5dkhFC evNQOyI=
>> ;; Received 482 bytes from 199.253.183.183#53(b.in-addr-servers.arpa) in 69 
>> ms
>>
>> 171.in-addr.arpa.0   IN  SOA ns1.apnic.net. 
>> read-txt-record-of-zone-first-dns-admin.apnic.net. 5276 7200 1800 604800 
>> 172800
>> 171.in-addr.arpa.0   IN  RRSIG   SOA 5 3 172800 20170416100102 
>> 20170317090102 7858 171.in-addr.arpa. 
>> dH+uuGq9pgMc9TEMkTwrf9NGMj5zFnbZt3kjlNSF0kPzGk3WUr5g84jv 
>> 0SN88Y26eP/Op4Vv8JewyVu2IT1POnmiNpDYELkJxrTWnPPc6CJ2wugR 
>> TrFLOq6eIsqtrCcWbatQ7XSXXj1

Re: ARIN contact needed: something bad happens with legacy IPv4 block's reverse delegations

[Alberto Delgado]

I just receive an answer from RIPE and now its working again:



Dear Alberto,

Our apologies for this. There was a bug in our DNS provisioning system, 
and it temporarily caused some delegations (where the parent zone is 
operated by ARIN) to be removed. The bug has been fixed, and the correct 
data is now being published on the RIPE NCC FTP server. We expect ARIN's 
DNS provisioning system to pick it up shortly, and the delegations will 
be restored.


Regards,
Anand Buddhdev
RIPE NCC



El 2017-03-17 11:49, Alberto Delgado escribió:

Me too.

 We have a 164.138.208.0/22 and we have issues with dns reverse too

 We sent an email to ripe support, but no answer yet


 El 2017-03-17 10:53, Stephane Bortzmeyer escribió:

On Fri, Mar 17, 2017 at 12:03:58PM +0300,
 Eygene Ryabinkin  wrote
 a message of 71 lines which said:


Seems like the other /16 from 144.in-addr.arpa are affected too
(at least).


Also in 164.in-addr.arpa, it seems?

Re: ARIN contact needed: something bad happens with legacy IPv4 block's reverse delegations

[Alberto Delgado]

 Me too.

 We have a 164.138.208.0/22 and we have issues with dns reverse too

 We sent an email to ripe support, but no answer yet


 El 2017-03-17 10:53, Stephane Bortzmeyer escribió:

On Fri, Mar 17, 2017 at 12:03:58PM +0300,
 Eygene Ryabinkin  wrote
 a message of 71 lines which said:


Seems like the other /16 from 144.in-addr.arpa are affected too
(at least).


Also in 164.in-addr.arpa, it seems?

Re: [NOC] ARIN contact needed: something bad happens with legacy IPv4 block's reverse delegations

[John Curran]

Folks -

RIPE NCC has backed out the change at issue, and we are again processing
zonelet files received from them.   They’ve posted more details here -


FYI,
/John

John Curran
President and CEO
ARIN

On 17 Mar 2017, at 12:31 PM, John Curran 
mailto:jcur...@istaff.org>> wrote:

Eygene -

 We are aware there’s an issue and working on it presently with RIPE.
 Expect additional updates shortly.

/John

John Curran
President and CEO
ARIN

On 17 Mar 2017, at 11:04 AM, Eygene Ryabinkin 
mailto:rea+na...@grid.kiae.ru>> wrote:

Job, good day.

Fri, Mar 17, 2017 at 09:55:56AM +, Job Snijders wrote:
171 also seems affected.

Not the whole one, it seems:
{{{
$ dig +trace -t soa 1.171.in-addr.arpa

; <<>> DiG 9.10.4-P6 <<>> +trace -t soa 1.171.in-addr.arpa
;; global options: +cmd
. 202634 IN NS m.root-servers.net.
. 202634 IN NS i.root-servers.net.
. 202634 IN NS k.root-servers.net.
. 202634 IN NS c.root-servers.net.
. 202634 IN NS a.root-servers.net.
. 202634 IN NS d.root-servers.net.
. 202634 IN NS l.root-servers.net.
. 202634 IN NS e.root-servers.net.
. 202634 IN NS g.root-servers.net.
. 202634 IN NS b.root-servers.net.
. 202634 IN NS j.root-servers.net.
. 202634 IN NS h.root-servers.net.
. 202634 IN NS f.root-servers.net.
. 511016 IN RRSIG NS 8 0 518400 2017032917 2017031616 61045 . 
OjsjoA6WCcThV3BqhAyMaXI3bU1m228Gl8nvaR074qBtem/RjaFJh1Oe 
r7LPI6W15jgbRGuCY7/GUNgDex4ZM43yvx2iY+2GpSk9b2/pKGbDaDIp 
X1Hd8418206eow1P/SgPqtT+2LzjM+lz/HKUvCyPoN4uX5/7GDExn8ir 
2Q/vw81Za2nJ4Ji6oeGAmE8g6V3RJfTI0El/CP9Vl2/r0jWDZZ+wtYRA 
uwqvQYj+7VZc5+UJDJ3/Gne1LBnzXKnZRwnfJtZenhfZ7X20D9PiXvfJ 
M2s1ryqZvg8K5RqYt/r8pIiq1Udd1KWQBgdN7Q629+jNSngZQtH19R9H fM7h1w==
;; Received 1097 bytes from 127.0.0.1#53(127.0.0.1) in 0 ms

in-addr.arpa. 172800 IN NS a.in-addr-servers.arpa.
in-addr.arpa. 172800 IN NS b.in-addr-servers.arpa.
in-addr.arpa. 172800 IN NS c.in-addr-servers.arpa.
in-addr.arpa. 172800 IN NS d.in-addr-servers.arpa.
in-addr.arpa. 172800 IN NS e.in-addr-servers.arpa.
in-addr.arpa. 172800 IN NS f.in-addr-servers.arpa.
in-addr.arpa. 86400 IN DS 47054 8 2 
5CAFCCEC201D1933B4C9F6A9C8F51E51F3B39979058AC21B8DF1B1F2 81CBC6F2
in-addr.arpa. 86400 IN DS 53696 8 2 
13E5501C56B20394DA921B51412D48B7089C5EB6957A7C58553C4D4D 424F04DF
in-addr.arpa. 86400 IN DS 63982 8 2 
AAF4FB5D213EF25AE44679032EBE3514C487D7ABD99D7F5FEC3383D0 30733C73
in-addr.arpa. 86400 IN RRSIG DS 8 2 86400 2017033000 2017031623 33786 
arpa. gqNN6MmLVxFtSG0oxdyoYVSXZlp6vY9yxpnJW89TYCHfTkv+ZmklM76O 
IHdTznEhLHXbyr4BaxjFCshsC3WacdH/YZYa/SZynJ9Q1N6/bogrIUTn 
qs61Y/YD4Sk5lI7YvUxVnPrF3lk1pY8dpoTkprTLZngoeQEsm552inqG ypU=
;; Received 731 bytes from 
193.0.14.129#53(k.root-servers.net) in 41 ms

171.in-addr.arpa. 86400 IN NS ns1.apnic.net.
171.in-addr.arpa. 86400 IN NS ns2.lacnic.net.
171.in-addr.arpa. 86400 IN NS ns3.apnic.net.
171.in-addr.arpa. 86400 IN NS ns4.apnic.net.
171.in-addr.arpa. 86400 IN NS 
apnic.authdns.ripe.net.
171.in-addr.arpa. 86400 IN NS apnic1.dnsnode.net.
171.in-addr.arpa. 86400 IN NS tinnie.arin.net.
171.in-addr.arpa. 86400 IN DS 49699 5 1 0240801C96480900CC6D93130AF45CFE86EDE940
171.in-addr.arpa. 86400 IN DS 49699 5 2 
6E0BA96F05B4D39C1668979731E040213BB6130DE33E86E063B5F7F7 5C465C88
171.in-addr.arpa. 86400 IN RRSIG DS 8 3 86400 20170330042932 20170309125911 
4341 in-addr.arpa. RKcPZJdG1MBrwpfa1mIK7ikIU585i1Jv+UkEBHxuM3BxxAbD+ht0W04C 
ljboyXJfYK8ly/YTDqNUkWKZLwDHlkq/rgNwTG63sPT8iK8qya+2qUVB 
iTXVsD2HaV1V/KwgJjlWHRNnlwkK0YZ5Q7tevXaq4yyLT2Q2mu5dkhFC evNQOyI=
;; Received 482 bytes from 199.253.183.183#53(b.in-addr-servers.arpa) in 69 ms

171.in-addr.arpa. 0 IN SOA ns1.apnic.net. 
read-txt-record-of-zone-first-dns-admin.apnic.net.
 5276 7200 1800 604800 172800
171.in-addr.arpa. 0 IN RRSIG SOA 5 3 172800 20170416100102 20170317090102 7858 
171.in-addr.arpa. dH+uuGq9pgMc9TEMkTwrf9NGMj5zFnbZt3kjlNSF0kPzGk3WUr5g84jv 
0SN88Y26eP/Op4Vv8JewyVu2IT1POnmiNpDYELkJxrTWnPPc6CJ2wugR 
TrFLOq6eIsqtrCcWbatQ7XSXXj1UglqVVydlX1ExoIh9MdP+1eBneD1I 4OU=
171.in-addr.arpa. 172800 IN NSEC 10.171.in-addr.arpa. NS SOA TXT RRSIG NSEC 
DNSKEY
171.in-addr.arpa. 172800 IN RRSIG NSEC 5 3 172800 20170416100102 20170317090102 
7858 171.in-addr.arpa. QieHZ0W7jx3QCgLuvqtKtFoLPYkxn5R9nRO1oMbSSXkdF+S4iQQkVlX7 
DyQvoL69hKA+S9idfernf6DwNTQgeWFfCdjUyOjMCa3Ag/S8ARZs7K4F 
N9DxB/cBCZm7KTzwAzXFe3cJ/d1Wo45Tx9jf

Re: [NOC] ARIN contact needed: something bad happens with legacy IPv4 block's reverse delegations

[Eygene Ryabinkin]

John, Alex, Romeo,

Fri, Mar 17, 2017 at 12:31:06PM +0100, John Curran wrote:
>   We are aware there’s an issue and working on it presently with RIPE. 
>   Expect additional updates shortly.

Fri, Mar 17, 2017 at 12:50:48PM +0100, Alex Band wrote:
> You can find a detailed announcement from the RIPE NCC here:
> https://www.ripe.net/ripe/mail/archives/dns-wg/2017-March/003394.html 
> 

Fri, Mar 17, 2017 at 12:52:10PM +0100, Romeo Zwart wrote:
> RIPE NCC have issued a statement about the issue here:
> 
>  https://www.ripe.net/ripe/mail/archives/dns-wg/2017-March/003394.html
> 
> Our apologies for the inconvenience caused.

Thanks for your work: the issue seem to be fixed.  I am trying to
verify if everything works as expected, there are some neats,
{{{
206.144.in-addr.arpa.   172800  IN  NS  ns1.rrcki.ru.
206.144.in-addr.arpa.   172800  IN  NS  ns.kiae.ru.
206.144.in-addr.arpa.   172800  IN  NS  ns3.rrcki.ru.
206.144.in-addr.arpa.   172800  IN  NS  ns2.grid.kiae.ru.
206.144.in-addr.arpa.   172800  IN  NS  ns.grid.kiae.ru.
206.144.in-addr.arpa.   10800   IN  NSEC207.144.in-addr.arpa. NS RRSIG 
NSEC
206.144.in-addr.arpa.   10800   IN  RRSIG   NSEC 5 4 10800 20170331110430 
20170317100430 33345 144.in-addr.arpa. vbFwaKdRa7Jd70aAbJ
5mC37BsTzMg3nWVI5gqQLLOqSaCZfH0XUez+Uk 
MbTNvepziCRzH+HgSLabuvRSo4nIUP1SjOd2WX0wySSdb/blqhfmjw3l 
n8laqOxy/lj8TDiIuxOdw2JhM1v5x/DH4aDnwdGFfUEOdgzCU5k8LdAT oyA=   

  ;; Received 373 bytes from 199.180.180.63#53(r.arin.net) in 198 ms

;; Question section mismatch: got 206.144.0.0.in-addr.arpa/PTR/IN
;; Question section mismatch: got 206.144.0.0.in-addr.arpa/PTR/IN
206.144.in-addr.arpa.   3600IN  SOA ns.kiae.ru. noc.rrcki.ru. 
2017020803 10800 3600 180 3600
;; Received 105 bytes from 144.206.239.1#53(ns.grid.kiae.ru) in 0 ms
}}}
about question mismatch, though my guts feel that this is the local
issue at rrcki.ru.

Thanks again!  And for a nicely-written summary -- too.
-- 
Eygene Ryabinkin, National Research Centre "Kurchatov Institute"

Always code as if the guy who ends up maintaining your code will be
a violent psychopath who knows where you live.

Re: [NOC] ARIN contact needed: something bad happens with legacy IPv4 block's reverse delegations

[Alex Band]

You can find a detailed announcement from the RIPE NCC here:
https://www.ripe.net/ripe/mail/archives/dns-wg/2017-March/003394.html 


-Alex Band

> On 17 Mar 2017, at 12:31, John Curran  wrote:
> 
> Eygene - 
> 
>  We are aware there’s an issue and working on it presently with RIPE. 
>  Expect additional updates shortly.
> 
> /John
> 
> John Curran
> President and CEO
> ARIN
> 
>> On 17 Mar 2017, at 11:04 AM, Eygene Ryabinkin  wrote:
>> 
>> Job, good day.
>> 
>> Fri, Mar 17, 2017 at 09:55:56AM +, Job Snijders wrote:
>>> 171 also seems affected.
>> 
>> Not the whole one, it seems:
>> {{{
>> $ dig +trace -t soa 1.171.in-addr.arpa
>> 
>> ; <<>> DiG 9.10.4-P6 <<>> +trace -t soa 1.171.in-addr.arpa
>> ;; global options: +cmd
>> .202634  IN  NS  m.root-servers.net.
>> .202634  IN  NS  i.root-servers.net.
>> .202634  IN  NS  k.root-servers.net.
>> .202634  IN  NS  c.root-servers.net.
>> .202634  IN  NS  a.root-servers.net.
>> .202634  IN  NS  d.root-servers.net.
>> .202634  IN  NS  l.root-servers.net.
>> .202634  IN  NS  e.root-servers.net.
>> .202634  IN  NS  g.root-servers.net.
>> .202634  IN  NS  b.root-servers.net.
>> .202634  IN  NS  j.root-servers.net.
>> .202634  IN  NS  h.root-servers.net.
>> .202634  IN  NS  f.root-servers.net.
>> .511016  IN  RRSIG   NS 8 0 518400 2017032917 
>> 2017031616 61045 . 
>> OjsjoA6WCcThV3BqhAyMaXI3bU1m228Gl8nvaR074qBtem/RjaFJh1Oe 
>> r7LPI6W15jgbRGuCY7/GUNgDex4ZM43yvx2iY+2GpSk9b2/pKGbDaDIp 
>> X1Hd8418206eow1P/SgPqtT+2LzjM+lz/HKUvCyPoN4uX5/7GDExn8ir 
>> 2Q/vw81Za2nJ4Ji6oeGAmE8g6V3RJfTI0El/CP9Vl2/r0jWDZZ+wtYRA 
>> uwqvQYj+7VZc5+UJDJ3/Gne1LBnzXKnZRwnfJtZenhfZ7X20D9PiXvfJ 
>> M2s1ryqZvg8K5RqYt/r8pIiq1Udd1KWQBgdN7Q629+jNSngZQtH19R9H fM7h1w==
>> ;; Received 1097 bytes from 127.0.0.1#53(127.0.0.1) in 0 ms
>> 
>> in-addr.arpa.172800  IN  NS  a.in-addr-servers.arpa.
>> in-addr.arpa.172800  IN  NS  b.in-addr-servers.arpa.
>> in-addr.arpa.172800  IN  NS  c.in-addr-servers.arpa.
>> in-addr.arpa.172800  IN  NS  d.in-addr-servers.arpa.
>> in-addr.arpa.172800  IN  NS  e.in-addr-servers.arpa.
>> in-addr.arpa.172800  IN  NS  f.in-addr-servers.arpa.
>> in-addr.arpa.86400   IN  DS  47054 8 2 
>> 5CAFCCEC201D1933B4C9F6A9C8F51E51F3B39979058AC21B8DF1B1F2 81CBC6F2
>> in-addr.arpa.86400   IN  DS  53696 8 2 
>> 13E5501C56B20394DA921B51412D48B7089C5EB6957A7C58553C4D4D 424F04DF
>> in-addr.arpa.86400   IN  DS  63982 8 2 
>> AAF4FB5D213EF25AE44679032EBE3514C487D7ABD99D7F5FEC3383D0 30733C73
>> in-addr.arpa.86400   IN  RRSIG   DS 8 2 86400 
>> 2017033000 2017031623 33786 arpa. 
>> gqNN6MmLVxFtSG0oxdyoYVSXZlp6vY9yxpnJW89TYCHfTkv+ZmklM76O 
>> IHdTznEhLHXbyr4BaxjFCshsC3WacdH/YZYa/SZynJ9Q1N6/bogrIUTn 
>> qs61Y/YD4Sk5lI7YvUxVnPrF3lk1pY8dpoTkprTLZngoeQEsm552inqG ypU=
>> ;; Received 731 bytes from 193.0.14.129#53(k.root-servers.net) in 41 ms
>> 
>> 171.in-addr.arpa.86400   IN  NS  ns1.apnic.net.
>> 171.in-addr.arpa.86400   IN  NS  ns2.lacnic.net.
>> 171.in-addr.arpa.86400   IN  NS  ns3.apnic.net.
>> 171.in-addr.arpa.86400   IN  NS  ns4.apnic.net.
>> 171.in-addr.arpa.86400   IN  NS  apnic.authdns.ripe.net.
>> 171.in-addr.arpa.86400   IN  NS  apnic1.dnsnode.net.
>> 171.in-addr.arpa.86400   IN  NS  tinnie.arin.net.
>> 171.in-addr.arpa.86400   IN  DS  49699 5 1 
>> 0240801C96480900CC6D93130AF45CFE86EDE940
>> 171.in-addr.arpa.86400   IN  DS  49699 5 2 
>> 6E0BA96F05B4D39C1668979731E040213BB6130DE33E86E063B5F7F7 5C465C88
>> 171.in-addr.arpa.86400   IN  RRSIG   DS 8 3 86400 20170330042932 
>> 20170309125911 4341 in-addr.arpa. 
>> RKcPZJdG1MBrwpfa1mIK7ikIU585i1Jv+UkEBHxuM3BxxAbD+ht0W04C 
>> ljboyXJfYK8ly/YTDqNUkWKZLwDHlkq/rgNwTG63sPT8iK8qya+2qUVB 
>> iTXVsD2HaV1V/KwgJjlWHRNnlwkK0YZ5Q7tevXaq4yyLT2Q2mu5dkhFC evNQOyI=
>> ;; Received 482 bytes from 199.253.183.183#53(b.in-addr-servers.arpa) in 69 
>> ms
>> 
>> 171.in-addr.arpa.0   IN  SOA ns1.apnic.net. 
>> read-txt-record-of-zone-first-dns-admin.apnic.net. 5276 7200 1800 604800 
>> 172800
>> 171.in-addr.arpa.0   IN  RRSIG   SOA 5 3 172800 20170416100102 
>> 20170317090102 7858 171.in-addr.arpa. 
>> dH+uuGq9pgMc9TEMkTwrf9NGMj5zFnbZt3kjlNSF0kPzGk3WUr5g84jv 
>> 0SN88Y26eP/Op4Vv8JewyVu2IT1POnmiNpDYELkJxrTWnPPc6CJ2wugR 
>> TrFLOq6eIsqtr

Re: [NOC] ARIN contact needed: something bad happens with legacy IPv4 block's reverse delegations

[John Curran]

Eygene - 

  We are aware there’s an issue and working on it presently with RIPE. 
  Expect additional updates shortly.

/John

John Curran
President and CEO
ARIN

> On 17 Mar 2017, at 11:04 AM, Eygene Ryabinkin  wrote:
> 
> Job, good day.
> 
> Fri, Mar 17, 2017 at 09:55:56AM +, Job Snijders wrote:
>> 171 also seems affected.
> 
> Not the whole one, it seems:
> {{{
> $ dig +trace -t soa 1.171.in-addr.arpa
> 
> ; <<>> DiG 9.10.4-P6 <<>> +trace -t soa 1.171.in-addr.arpa
> ;; global options: +cmd
> . 202634  IN  NS  m.root-servers.net.
> . 202634  IN  NS  i.root-servers.net.
> . 202634  IN  NS  k.root-servers.net.
> . 202634  IN  NS  c.root-servers.net.
> . 202634  IN  NS  a.root-servers.net.
> . 202634  IN  NS  d.root-servers.net.
> . 202634  IN  NS  l.root-servers.net.
> . 202634  IN  NS  e.root-servers.net.
> . 202634  IN  NS  g.root-servers.net.
> . 202634  IN  NS  b.root-servers.net.
> . 202634  IN  NS  j.root-servers.net.
> . 202634  IN  NS  h.root-servers.net.
> . 202634  IN  NS  f.root-servers.net.
> . 511016  IN  RRSIG   NS 8 0 518400 2017032917 
> 2017031616 61045 . 
> OjsjoA6WCcThV3BqhAyMaXI3bU1m228Gl8nvaR074qBtem/RjaFJh1Oe 
> r7LPI6W15jgbRGuCY7/GUNgDex4ZM43yvx2iY+2GpSk9b2/pKGbDaDIp 
> X1Hd8418206eow1P/SgPqtT+2LzjM+lz/HKUvCyPoN4uX5/7GDExn8ir 
> 2Q/vw81Za2nJ4Ji6oeGAmE8g6V3RJfTI0El/CP9Vl2/r0jWDZZ+wtYRA 
> uwqvQYj+7VZc5+UJDJ3/Gne1LBnzXKnZRwnfJtZenhfZ7X20D9PiXvfJ 
> M2s1ryqZvg8K5RqYt/r8pIiq1Udd1KWQBgdN7Q629+jNSngZQtH19R9H fM7h1w==
> ;; Received 1097 bytes from 127.0.0.1#53(127.0.0.1) in 0 ms
> 
> in-addr.arpa. 172800  IN  NS  a.in-addr-servers.arpa.
> in-addr.arpa. 172800  IN  NS  b.in-addr-servers.arpa.
> in-addr.arpa. 172800  IN  NS  c.in-addr-servers.arpa.
> in-addr.arpa. 172800  IN  NS  d.in-addr-servers.arpa.
> in-addr.arpa. 172800  IN  NS  e.in-addr-servers.arpa.
> in-addr.arpa. 172800  IN  NS  f.in-addr-servers.arpa.
> in-addr.arpa. 86400   IN  DS  47054 8 2 
> 5CAFCCEC201D1933B4C9F6A9C8F51E51F3B39979058AC21B8DF1B1F2 81CBC6F2
> in-addr.arpa. 86400   IN  DS  53696 8 2 
> 13E5501C56B20394DA921B51412D48B7089C5EB6957A7C58553C4D4D 424F04DF
> in-addr.arpa. 86400   IN  DS  63982 8 2 
> AAF4FB5D213EF25AE44679032EBE3514C487D7ABD99D7F5FEC3383D0 30733C73
> in-addr.arpa. 86400   IN  RRSIG   DS 8 2 86400 2017033000 
> 2017031623 33786 arpa. 
> gqNN6MmLVxFtSG0oxdyoYVSXZlp6vY9yxpnJW89TYCHfTkv+ZmklM76O 
> IHdTznEhLHXbyr4BaxjFCshsC3WacdH/YZYa/SZynJ9Q1N6/bogrIUTn 
> qs61Y/YD4Sk5lI7YvUxVnPrF3lk1pY8dpoTkprTLZngoeQEsm552inqG ypU=
> ;; Received 731 bytes from 193.0.14.129#53(k.root-servers.net) in 41 ms
> 
> 171.in-addr.arpa. 86400   IN  NS  ns1.apnic.net.
> 171.in-addr.arpa. 86400   IN  NS  ns2.lacnic.net.
> 171.in-addr.arpa. 86400   IN  NS  ns3.apnic.net.
> 171.in-addr.arpa. 86400   IN  NS  ns4.apnic.net.
> 171.in-addr.arpa. 86400   IN  NS  apnic.authdns.ripe.net.
> 171.in-addr.arpa. 86400   IN  NS  apnic1.dnsnode.net.
> 171.in-addr.arpa. 86400   IN  NS  tinnie.arin.net.
> 171.in-addr.arpa. 86400   IN  DS  49699 5 1 
> 0240801C96480900CC6D93130AF45CFE86EDE940
> 171.in-addr.arpa. 86400   IN  DS  49699 5 2 
> 6E0BA96F05B4D39C1668979731E040213BB6130DE33E86E063B5F7F7 5C465C88
> 171.in-addr.arpa. 86400   IN  RRSIG   DS 8 3 86400 20170330042932 
> 20170309125911 4341 in-addr.arpa. 
> RKcPZJdG1MBrwpfa1mIK7ikIU585i1Jv+UkEBHxuM3BxxAbD+ht0W04C 
> ljboyXJfYK8ly/YTDqNUkWKZLwDHlkq/rgNwTG63sPT8iK8qya+2qUVB 
> iTXVsD2HaV1V/KwgJjlWHRNnlwkK0YZ5Q7tevXaq4yyLT2Q2mu5dkhFC evNQOyI=
> ;; Received 482 bytes from 199.253.183.183#53(b.in-addr-servers.arpa) in 69 ms
> 
> 171.in-addr.arpa. 0   IN  SOA ns1.apnic.net. 
> read-txt-record-of-zone-first-dns-admin.apnic.net. 5276 7200 1800 604800 
> 172800
> 171.in-addr.arpa. 0   IN  RRSIG   SOA 5 3 172800 20170416100102 
> 20170317090102 7858 171.in-addr.arpa. 
> dH+uuGq9pgMc9TEMkTwrf9NGMj5zFnbZt3kjlNSF0kPzGk3WUr5g84jv 
> 0SN88Y26eP/Op4Vv8JewyVu2IT1POnmiNpDYELkJxrTWnPPc6CJ2wugR 
> TrFLOq6eIsqtrCcWbatQ7XSXXj1UglqVVydlX1ExoIh9MdP+1eBneD1I 4OU=
> 171.in-addr.arpa. 172800  IN  NSEC10.171.in-addr.arpa. NS SOA TXT 
> RRSIG NSEC DNSKEY
> 171.in-addr.arpa. 172800  IN  RRSIG   NSEC 5 3 172800 20170416100102 
> 20170317090102 7858 171.in-addr.arpa. 
> QieHZ0W7jx3QCgLuvqtKtFoLPYkxn5R9nRO1oMbSSXkdF+S4iQQkVlX7 
> DyQvoL69hKA+S9idfernf6DwNTQgeWFfCdjUyOjMCa3Ag/S8ARZs7K4F 
> N9DxB/cBCZm7KTzwAzXFe3cJ

Re: [NOC] ARIN contact needed: something bad happens with legacy IPv4 block's reverse delegations

[Eygene Ryabinkin]

Job, good day.

Fri, Mar 17, 2017 at 09:55:56AM +, Job Snijders wrote:
> 171 also seems affected.

Not the whole one, it seems:
{{{
$ dig +trace -t soa 1.171.in-addr.arpa

; <<>> DiG 9.10.4-P6 <<>> +trace -t soa 1.171.in-addr.arpa
;; global options: +cmd
.   202634  IN  NS  m.root-servers.net.
.   202634  IN  NS  i.root-servers.net.
.   202634  IN  NS  k.root-servers.net.
.   202634  IN  NS  c.root-servers.net.
.   202634  IN  NS  a.root-servers.net.
.   202634  IN  NS  d.root-servers.net.
.   202634  IN  NS  l.root-servers.net.
.   202634  IN  NS  e.root-servers.net.
.   202634  IN  NS  g.root-servers.net.
.   202634  IN  NS  b.root-servers.net.
.   202634  IN  NS  j.root-servers.net.
.   202634  IN  NS  h.root-servers.net.
.   202634  IN  NS  f.root-servers.net.
.   511016  IN  RRSIG   NS 8 0 518400 2017032917 
2017031616 61045 . OjsjoA6WCcThV3BqhAyMaXI3bU1m228Gl8nvaR074qBtem/RjaFJh1Oe 
r7LPI6W15jgbRGuCY7/GUNgDex4ZM43yvx2iY+2GpSk9b2/pKGbDaDIp 
X1Hd8418206eow1P/SgPqtT+2LzjM+lz/HKUvCyPoN4uX5/7GDExn8ir 
2Q/vw81Za2nJ4Ji6oeGAmE8g6V3RJfTI0El/CP9Vl2/r0jWDZZ+wtYRA 
uwqvQYj+7VZc5+UJDJ3/Gne1LBnzXKnZRwnfJtZenhfZ7X20D9PiXvfJ 
M2s1ryqZvg8K5RqYt/r8pIiq1Udd1KWQBgdN7Q629+jNSngZQtH19R9H fM7h1w==
;; Received 1097 bytes from 127.0.0.1#53(127.0.0.1) in 0 ms

in-addr.arpa.   172800  IN  NS  a.in-addr-servers.arpa.
in-addr.arpa.   172800  IN  NS  b.in-addr-servers.arpa.
in-addr.arpa.   172800  IN  NS  c.in-addr-servers.arpa.
in-addr.arpa.   172800  IN  NS  d.in-addr-servers.arpa.
in-addr.arpa.   172800  IN  NS  e.in-addr-servers.arpa.
in-addr.arpa.   172800  IN  NS  f.in-addr-servers.arpa.
in-addr.arpa.   86400   IN  DS  47054 8 2 
5CAFCCEC201D1933B4C9F6A9C8F51E51F3B39979058AC21B8DF1B1F2 81CBC6F2
in-addr.arpa.   86400   IN  DS  53696 8 2 
13E5501C56B20394DA921B51412D48B7089C5EB6957A7C58553C4D4D 424F04DF
in-addr.arpa.   86400   IN  DS  63982 8 2 
AAF4FB5D213EF25AE44679032EBE3514C487D7ABD99D7F5FEC3383D0 30733C73
in-addr.arpa.   86400   IN  RRSIG   DS 8 2 86400 2017033000 
2017031623 33786 arpa. 
gqNN6MmLVxFtSG0oxdyoYVSXZlp6vY9yxpnJW89TYCHfTkv+ZmklM76O 
IHdTznEhLHXbyr4BaxjFCshsC3WacdH/YZYa/SZynJ9Q1N6/bogrIUTn 
qs61Y/YD4Sk5lI7YvUxVnPrF3lk1pY8dpoTkprTLZngoeQEsm552inqG ypU=
;; Received 731 bytes from 193.0.14.129#53(k.root-servers.net) in 41 ms

171.in-addr.arpa.   86400   IN  NS  ns1.apnic.net.
171.in-addr.arpa.   86400   IN  NS  ns2.lacnic.net.
171.in-addr.arpa.   86400   IN  NS  ns3.apnic.net.
171.in-addr.arpa.   86400   IN  NS  ns4.apnic.net.
171.in-addr.arpa.   86400   IN  NS  apnic.authdns.ripe.net.
171.in-addr.arpa.   86400   IN  NS  apnic1.dnsnode.net.
171.in-addr.arpa.   86400   IN  NS  tinnie.arin.net.
171.in-addr.arpa.   86400   IN  DS  49699 5 1 
0240801C96480900CC6D93130AF45CFE86EDE940
171.in-addr.arpa.   86400   IN  DS  49699 5 2 
6E0BA96F05B4D39C1668979731E040213BB6130DE33E86E063B5F7F7 5C465C88
171.in-addr.arpa.   86400   IN  RRSIG   DS 8 3 86400 20170330042932 
20170309125911 4341 in-addr.arpa. 
RKcPZJdG1MBrwpfa1mIK7ikIU585i1Jv+UkEBHxuM3BxxAbD+ht0W04C 
ljboyXJfYK8ly/YTDqNUkWKZLwDHlkq/rgNwTG63sPT8iK8qya+2qUVB 
iTXVsD2HaV1V/KwgJjlWHRNnlwkK0YZ5Q7tevXaq4yyLT2Q2mu5dkhFC evNQOyI=
;; Received 482 bytes from 199.253.183.183#53(b.in-addr-servers.arpa) in 69 ms

171.in-addr.arpa.   0   IN  SOA ns1.apnic.net. 
read-txt-record-of-zone-first-dns-admin.apnic.net. 5276 7200 1800 604800 172800
171.in-addr.arpa.   0   IN  RRSIG   SOA 5 3 172800 20170416100102 
20170317090102 7858 171.in-addr.arpa. 
dH+uuGq9pgMc9TEMkTwrf9NGMj5zFnbZt3kjlNSF0kPzGk3WUr5g84jv 
0SN88Y26eP/Op4Vv8JewyVu2IT1POnmiNpDYELkJxrTWnPPc6CJ2wugR 
TrFLOq6eIsqtrCcWbatQ7XSXXj1UglqVVydlX1ExoIh9MdP+1eBneD1I 4OU=
171.in-addr.arpa.   172800  IN  NSEC10.171.in-addr.arpa. NS SOA TXT 
RRSIG NSEC DNSKEY
171.in-addr.arpa.   172800  IN  RRSIG   NSEC 5 3 172800 20170416100102 
20170317090102 7858 171.in-addr.arpa. 
QieHZ0W7jx3QCgLuvqtKtFoLPYkxn5R9nRO1oMbSSXkdF+S4iQQkVlX7 
DyQvoL69hKA+S9idfernf6DwNTQgeWFfCdjUyOjMCa3Ag/S8ARZs7K4F 
N9DxB/cBCZm7KTzwAzXFe3cJ/d1Wo45Tx9jf6Drx551oqOw1gmDIL4y6 +k4=
;; Received 530 bytes from 202.12.31.140#53(ns4.apnic.net) in 347 ms
}}}
since it is APNIC who respond to queries about it.  Any more specific
reverse records that are cross ARIN and die there from 171.x.y.z?
-- 
Eygene Ryabinkin, National Research Centre "Kurchatov Institute"

Alw

Re: ARIN contact needed: something bad happens with legacy IPv4 block's reverse delegations

[Stephane Bortzmeyer]

On Fri, Mar 17, 2017 at 12:03:58PM +0300,
 Eygene Ryabinkin  wrote 
 a message of 71 lines which said:

> We (at Kurchatov Insitute) still use 144.206.0.0/16, the legacy
> block, and seeing the breakage rooted at ARIN since this night,
> {{{
> $ dig +trace -t soa 206.144.in-addr.arpa

According to DNSDB, it arrived yesterday, around 1630 UTC.

Re: ARIN contact needed: something bad happens with legacy IPv4 block's reverse delegations

[Job Snijders]

171 also seems affected.

Job

On Fri, 17 Mar 2017 at 10:54, Stephane Bortzmeyer  wrote:

> On Fri, Mar 17, 2017 at 12:03:58PM +0300,
>  Eygene Ryabinkin  wrote
>  a message of 71 lines which said:
>
> > Seems like the other /16 from 144.in-addr.arpa are affected too
> > (at least).
>
> Also in 164.in-addr.arpa, it seems?
>

Re: ARIN contact needed: something bad happens with legacy IPv4 block's reverse delegations

[Stephane Bortzmeyer]

On Fri, Mar 17, 2017 at 12:03:58PM +0300,
 Eygene Ryabinkin  wrote 
 a message of 71 lines which said:

> Seems like the other /16 from 144.in-addr.arpa are affected too
> (at least).

Also in 164.in-addr.arpa, it seems?

ARIN contact needed: something bad happens with legacy IPv4 block's reverse delegations

[Eygene Ryabinkin]

We (at Kurchatov Insitute) still use 144.206.0.0/16, the legacy
block, and seeing the breakage rooted at ARIN since this night,
{{{
$ dig +trace -t soa 206.144.in-addr.arpa

; <<>> DiG 9.10.4-P6 <<>> +trace -t soa 206.144.in-addr.arpa
;; global options: +cmd
.   206351  IN  NS  a.root-servers.net.
.   206351  IN  NS  c.root-servers.net.
.   206351  IN  NS  d.root-servers.net.
.   206351  IN  NS  e.root-servers.net.
.   206351  IN  NS  l.root-servers.net.
.   206351  IN  NS  f.root-servers.net.
.   206351  IN  NS  g.root-servers.net.
.   206351  IN  NS  h.root-servers.net.
.   206351  IN  NS  j.root-servers.net.
.   206351  IN  NS  m.root-servers.net.
.   206351  IN  NS  k.root-servers.net.
.   206351  IN  NS  i.root-servers.net.
.   206351  IN  NS  b.root-servers.net.
.   514733  IN  RRSIG   NS 8 0 518400 2017032917 
2017031616 61045 . OjsjoA6WCcThV3BqhAyMaXI3bU1m228Gl8nvaR074qBtem/RjaFJh1Oe 
r7LPI6W15jgbRGuCY7/GUNgDex4ZM43yvx2iY+2GpSk9b2/pKGbDaDIp 
X1Hd8418206eow1P/SgPqtT+2LzjM+lz/HKUvCyPoN4uX5/7GDExn8ir 
2Q/vw81Za2nJ4Ji6oeGAmE8g6V3RJfTI0El/CP9Vl2/r0jWDZZ+wtYRA 
uwqvQYj+7VZc5+UJDJ3/Gne1LBnzXKnZRwnfJtZenhfZ7X20D9PiXvfJ 
M2s1ryqZvg8K5RqYt/r8pIiq1Udd1KWQBgdN7Q629+jNSngZQtH19R9H fM7h1w==
;; Received 1097 bytes from 127.0.0.1#53(127.0.0.1) in 0 ms

in-addr.arpa.   172800  IN  NS  e.in-addr-servers.arpa.
in-addr.arpa.   172800  IN  NS  f.in-addr-servers.arpa.
in-addr.arpa.   172800  IN  NS  d.in-addr-servers.arpa.
in-addr.arpa.   172800  IN  NS  c.in-addr-servers.arpa.
in-addr.arpa.   172800  IN  NS  b.in-addr-servers.arpa.
in-addr.arpa.   172800  IN  NS  a.in-addr-servers.arpa.
in-addr.arpa.   86400   IN  DS  53696 8 2 
13E5501C56B20394DA921B51412D48B7089C5EB6957A7C58553C4D4D 424F04DF
in-addr.arpa.   86400   IN  DS  63982 8 2 
AAF4FB5D213EF25AE44679032EBE3514C487D7ABD99D7F5FEC3383D0 30733C73
in-addr.arpa.   86400   IN  DS  47054 8 2 
5CAFCCEC201D1933B4C9F6A9C8F51E51F3B39979058AC21B8DF1B1F2 81CBC6F2
in-addr.arpa.   86400   IN  RRSIG   DS 8 2 86400 2017033000 
2017031623 33786 arpa. 
gqNN6MmLVxFtSG0oxdyoYVSXZlp6vY9yxpnJW89TYCHfTkv+ZmklM76O 
IHdTznEhLHXbyr4BaxjFCshsC3WacdH/YZYa/SZynJ9Q1N6/bogrIUTn 
qs61Y/YD4Sk5lI7YvUxVnPrF3lk1pY8dpoTkprTLZngoeQEsm552inqG ypU=
;; Received 733 bytes from 198.41.0.4#53(a.root-servers.net) in 64 ms

144.in-addr.arpa.   86400   IN  NS  r.arin.net.
144.in-addr.arpa.   86400   IN  NS  u.arin.net.
144.in-addr.arpa.   86400   IN  NS  x.arin.net.
144.in-addr.arpa.   86400   IN  NS  y.arin.net.
144.in-addr.arpa.   86400   IN  NS  z.arin.net.
144.in-addr.arpa.   86400   IN  NS  arin.authdns.ripe.net.
144.in-addr.arpa.   86400   IN  DS  62856 5 1 
484154736CF9D4DC4F1C2B807BDC06E5B1645AFF
144.in-addr.arpa.   86400   IN  RRSIG   DS 8 3 86400 20170328085556 
20170307105911 4341 in-addr.arpa. 
X3nPKgjQbRef6BxK3msEXi/IcfpG1rylY2xeWwfNO6fJB5x/QT38ZCA5 
XMs6CBeXb59tpQFbgjwZX+prqSpjGFtZ+phFuzsaGmuPOF0FFypMHj7F 
swykEFvaPY5z4d8mo9FKFJetF2gZfzYthwJWW0x2oo2H2BG0lEedVUCb 1aGs/HE=
;; Received 380 bytes from 193.0.9.1#53(f.in-addr-servers.arpa) in 48 ms

144.in-addr.arpa.   0   IN  SOA z.arin.net. dns-ops.arin.net. 
2017022432 1800 900 691200 10800
144.in-addr.arpa.   0   IN  RRSIG   SOA 5 3 86400 20170331083220 
20170317073220 33345 144.in-addr.arpa. 
k2sZXuQQR3MyyRiB9Wd7fw45yZTbNokjQtFFNY+aCEa/85AnSyuomLlZ 
jZs4A0bO376/Z1v+bTHoeFqsyHr/xuWHX74r0QC/TCeP0amc+w8RqCvw 
Yox1TfoJxwhblGNRCgyLw52WszMYyr49EfWPfpHAKFU+G94gdilf3C6x GOM=
144.in-addr.arpa.   10800   IN  NSEC0.144.in-addr.arpa. NS SOA 
RRSIG NSEC DNSKEY
144.in-addr.arpa.   10800   IN  RRSIG   NSEC 5 3 10800 20170331083220 
20170317073220 33345 144.in-addr.arpa. 
opvG8SC7+UjHSTtBBtekWNkhLFMIxFxOdejJ7sM7t8lyGNzM6sOABeSI 
ADXfo8q3p4YFBHgBU5fRCAhBdiQ/AiZC0dLMnHb4WdKEv5bDsBbq5Pt6 
vabJaIv7Gizw7kBy1JZ/ZrC4Jmp4EX0HiYA+Ak+aQggD7gdI/6tFDNpl N7M=
205.144.in-addr.arpa.   10800   IN  NSEC207.144.in-addr.arpa. NS RRSIG 
NSEC
205.144.in-addr.arpa.   10800   IN  RRSIG   NSEC 5 4 10800 20170331083220 
20170317073220 33345 144.in-addr.arpa. 
F1e5mA7C3Mx10YSaNtshzfzfER8pudDOQkUoe+jeDhHeDZeR8FM1FoqW 
dqrNh5X6JVNlTdWGio6evnSkxnaoJFYyeYJtc+tJ8dTd5APJxJCSOfhH 
VfRNy7VHs2PdElRo1rRwMw+5Zncc0u8uPdmFDv2ZG8Vv3zj5C6l7rMla 3SA=
;; Received 718 bytes from 199.212.0.63#53(z.arin.net) in 128 ms
}}}

Seems like the other /16 from 144.in-addr.arpa are affected too
(at least).

ARIN ticke