Re: [dns-operations] ag.gov not providing NXDOMAIN responses

[Puneet Sood via dns-operations]

--- Begin Message ---
On Fri, Apr 12, 2024 at 3:10 PM David Zych  wrote:
>
> On 4/12/24 05:13, Petr Špaček wrote:
>
> On 11. 04. 24 6:15, Stephane Bortzmeyer wrote:
>
> On Tue, Apr 09, 2024 at 01:09:20PM -0500,
>   David Zych  wrote:
>
> The problem: when queried for a record underneath ag.gov. which does
> not exist, these nameservers do not return a proper NXDOMAIN
> response; instead, they don't answer at all.
>
>
> Funny enough, it depends on the QTYPE.
>
>
> Answering QTYPE=NS queries may be a new development; thanks to Victor's 
> suggestion I was able to reach someone using the SOA RNAME address, and their 
> first reply on Apr 10 was "The reported issue of the domain 
> "www.[.]tucson[.]ars[.]ag[.]gov" (formatted to avoid url protection) not 
> resolving has been corrected."  It might be that this is the thing they 
> changed; I'm not sure.
>
> Our continued back-and-forth discussion about the importance of NXDOMAIN 
> responses in general is ongoing.
>
> It's also funny to me that the exact same nameservers do answer `dig +norec 
> @ns1.usda.gov thissubdomaindoesnotexist.usda.gov a` (for usda.gov instead of 
> ag.gov)
>
> The practical trouble this causes has to do with an increasingly popular DNS 
> privacy feature called QNAME Minimization, which depends upon authoritative 
> DNS servers like yours responding in a standards-compliant way to queries like
>
> _.ag.gov IN A
> _.ars.ag.gov IN A
> _.tucson.ars.ag.gov IN A
>
>
> More fun: the previous version of QNAME minimisation used QTYPE=NS. It
> then changed to QTYPE=A precisely to work around broken
> middleboxes. (And also to avoid sticking out.)
>
>
> This is not only in violation of
> https://datatracker.ietf.org/doc/html/rfc8906 but it is an outright security 
> issue because it allows attackers to mess up load balancing in resolvers. See
> https://indico.dns-oarc.net/event/47/contributions/1018/attachments/959/1802/pre-silence-not-golden-dns-orac.pdf
> I predict you have much better chance getting this fixed if you go through 
> respective CERT team and point them to this presentation.
>
>
> Thanks, that's a great link and I will be sure to include it in my next reply 
> to the authoritative nameserver operator.  (It does appear that someone 
> considers the status quo posture a "security protocol", so hopefully they 
> will see this as a compelling argument.)
>
> Answering before some asks: No, we are not going to workaround this in BIND 
> resolver. It has to be fixed on the auth side. This is not a security bug in 
> BIND. See
> https://bind9.readthedocs.io/en/latest/chapter7.html#dns-resolvers
>
>
> I certainly would never suggest that it was a bug in BIND, but FWIW: I would 
> see some value in being able to configure `qname-minimization` within a 
> `server` block.
>
> Suppose I as the recursive nameserver operator hypothetically found myself 
> under pressure from non-technical people to "just make it work so we can get 
> to the website, this is embarrassing" (fortunately not the case right now, 
> but it's not hard to imagine in a higher profile scenario).  I could 
> accomplish that right now using either `qname-minimization disabled;` or in 
> this case also (ironically) `qname-minimization strict;`.  However, a global 
> setting of strict might cause problems resolving other domains, so 
> realistically my best unilateral recourse would be to disable minimization 
> completely, even though that feels like a step backward.  Given the choice, 
> I'd much rather disable it just for two known misbehaving servers and leave 
> it on for everything else (while in parallel still reaching out to the 
> authoritative server operator to address the root cause).
>
> It seems to me that such a feature would likely also be helpful when strict 
> eventually does become the default, if a few authoritative nameservers in the 
> wild still aren't ready (preferable at that point for recursive server 
> operators to set relaxed or disabled just for problem cases vs globally).

Per nameserver IP settings are convenient and can provide tactical
fixes to a resolution problem.

However they suffer from a big maintenance problem as nameserver IPs
for zones change over time. Also when making a per nameserver config,
did the operator configure it for all the relevant IPs?


>
> Thanks,
> David
>
> --
> David Zych (he/him)
> Lead Network Service Engineer
> University of Illinois Urbana-Champaign
> Office of the Chief Information Officer
> Technology Services
>
> Under the Illinois Freedom of Information Act any written communication to or 
> from university employees regarding university business is a public record 
> and may be subject to public disclosure.
>
> ___
> dns-operations mailing list
> dns-operations@lists.dns-oarc.net
> https://lists.dns-oarc.net/mailman/listinfo/dns-operations

--- End Message ---
___
dns-operations mailing list

Re: [dns-operations] [Ext] dns-operationsMysteries of DNSSEC

[Puneet Sood via dns-operations]

--- Begin Message ---
On Tue, Apr 2, 2024 at 12:20 PM Paul Hoffman  wrote:

> On Apr 2, 2024, at 08:42, Wes Hardaker  wrote:
>
> > Do check/worry about DDoS reflections from UDP requests for DNSKEYs.
>
> Why? Of what value is worrying about this? From what you and John says,
> it's pretty clear that you can't do anything effective to remediate
> whatever it is they are doing. Recent DDoS stats indicate that redirected
> DNS over UDP is no longer a significant source in real-world attacks. Short
> of being fodder for yet another "UDP considered harmful" discussion, why
> even note this?
>

Agree with this sentiment. There are over 1 M unique name server IPs. There
are probably many more resolver IPs seen by auths. The latter probably
includes researchers, random probes and so on.

Limiting UDP responses to < 1500 bytes and truncating otherwise should be
the response these days.


> --Paul Hoffman
>
>
>
> ___
> dns-operations mailing list
> dns-operations@lists.dns-oarc.net
> https://lists.dns-oarc.net/mailman/listinfo/dns-operations
>
--- End Message ---
___
dns-operations mailing list
dns-operations@lists.dns-oarc.net
https://lists.dns-oarc.net/mailman/listinfo/dns-operations

[dns-operations] COM referral responses from root without glue and TC bit

[Puneet Sood via dns-operations]

--- Begin Message ---
Hi dns operators,

While investigating a production issue, we found the following
interesting response from some root name servers.

Note: query does not use EDNS0; response does not set TC bit and has no glue.

$ dig @m.root-servers.net
kcmbrvwjafupdyztdq2ifvi6ye7fcacaaben6jaavmoaaaeqnqaaa2qnh7j.a5erjsqwn7zic34e7psoufcfue6rsznpw34cx57gjhhqqj6edwr6o57wikagcdv.ard6pjajyuo6kmpbm6ohbbjppyhmkivhxxmgqgb5xjpl2cvvlzo34erwypot4fw.lh4aa5rzkni7yihszvyxxw43w4aa3cysaws7jtjg.dns.uas-1.optnl.com
+noedns +ignore

; <<>> DiG 9.10.6 <<>> @m.root-servers.net
kcmbrvwjafupdyztdq2ifvi6ye7fcacaaben6jaavmoaaaeqnqaaa2qnh7j.a5erjsqwn7zic34e7psoufcfue6rsznpw34cx57gjhhqqj6edwr6o57wikagcdv.ard6pjajyuo6kmpbm6ohbbjppyhmkivhxxmgqgb5xjpl2cvvlzo34erwypot4fw.lh4aa5rzkni7yihszvyxxw43w4aa3cysaws7jtjg.dns.uas-1.optnl.com
+noedns +ignore
; (2 servers found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 44314
;; flags: qr rd; QUERY: 1, ANSWER: 0, AUTHORITY: 13, ADDITIONAL: 0
;; WARNING: recursion requested but not available

;; QUESTION SECTION:
;kcmbrvwjafupdyztdq2ifvi6ye7fcacaaben6jaavmoaaaeqnqaaa2qnh7j.a5erjsqwn7zic34e7psoufcfue6rsznpw34cx57gjhhqqj6edwr6o57wikagcdv.ard6pjajyuo6kmpbm6ohbbjppyhmkivhxxmgqgb5xjpl2cvvlzo34erwypot4fw.lh4aa5rzkni7yihszvyxxw43w4aa3cysaws7jtjg.dns.uas-1.optnl.com.
IN A

;; AUTHORITY SECTION:
com. 172800 IN NS d.gtld-servers.net.
com. 172800 IN NS m.gtld-servers.net.
com. 172800 IN NS a.gtld-servers.net.
com. 172800 IN NS g.gtld-servers.net.
com. 172800 IN NS b.gtld-servers.net.
com. 172800 IN NS i.gtld-servers.net.
com. 172800 IN NS j.gtld-servers.net.
com. 172800 IN NS e.gtld-servers.net.
com. 172800 IN NS l.gtld-servers.net.
com. 172800 IN NS h.gtld-servers.net.
com. 172800 IN NS c.gtld-servers.net.
com. 172800 IN NS k.gtld-servers.net.
com. 172800 IN NS f.gtld-servers.net.

;; Query time: 180 msec
;; SERVER: 2001:dc3::35#53(2001:dc3::35)
;; WHEN: Fri Jan 12 13:19:56 EST 2024
;; MSG SIZE  rcvd: 497


We confirmed similar behavior from some of the other root operators.

for s in a b c d e f g h i j k l m; do echo -n ${s} && (dig
@${s}.root-servers.net
kcmbrvwjafupdyztdq2ifvi6ye7fcacaaben6jaavmoaaaeqnqaaa2qnh7j.a5erjsqwn7zic34e7psoufcfue6rsznpw34cx57gjhhqqj6edwr6o57wikagcdv.ard6pjajyuo6kmpbm6ohbbjppyhmkivhxxmgqgb5xjpl2cvvlzo34erwypot4fw.lh4aa5rzkni7yihszvyxxw43w4aa3cysaws7jtjg.dns.uas-1.optnl.com
+noedns +ignore | grep flags); done

a;; flags: qr tc rd; QUERY: 1, ANSWER: 0, AUTHORITY: 13, ADDITIONAL: 1
b;; flags: qr rd; QUERY: 1, ANSWER: 0, AUTHORITY: 13, ADDITIONAL: 1
c;; flags: qr rd; QUERY: 1, ANSWER: 0, AUTHORITY: 13, ADDITIONAL: 0
d;; flags: qr tc rd; QUERY: 1, ANSWER: 0, AUTHORITY: 13, ADDITIONAL: 0
e;; flags: qr tc rd; QUERY: 1, ANSWER: 0, AUTHORITY: 13, ADDITIONAL: 1
f;; flags: qr tc rd; QUERY: 1, ANSWER: 0, AUTHORITY: 13, ADDITIONAL: 1
g;; flags: qr rd; QUERY: 1, ANSWER: 0, AUTHORITY: 13, ADDITIONAL: 0
h;; flags: qr tc rd; QUERY: 1, ANSWER: 0, AUTHORITY: 13, ADDITIONAL: 0
i;; flags: qr rd; QUERY: 1, ANSWER: 0, AUTHORITY: 13, ADDITIONAL: 0
j;; flags: qr tc rd; QUERY: 1, ANSWER: 0, AUTHORITY: 13, ADDITIONAL: 1
k;; flags: qr rd; QUERY: 1, ANSWER: 0, AUTHORITY: 13, ADDITIONAL: 0
l;; flags: qr tc rd; QUERY: 1, ANSWER: 0, AUTHORITY: 13, ADDITIONAL: 0
m;; flags: qr rd; QUERY: 1, ANSWER: 0, AUTHORITY: 13, ADDITIONAL: 0
==

While the above is a somewhat contrived example in 2023 (with larger
UDP bufsize and qname minimization), there can still be other
scenarios where a referral response with in-bailiwick name servers
does not fit in the available bufsize. The problem while rare for root
can be a bigger issue since root does provide glue for most if not all
delegations.

Question for root server operators: can the responses be fixed to set
TC bit when glue is dropped?

In general relevant name server software vendors should also fix this
but will take time to be deployed by operators.

Relevant text from RFC 9471 abstract: If message size constraints
prevent the inclusion of all glue records for in-domain name servers,
the server must set the TC (Truncated) flag to inform the client that
the response is incomplete.

Thanks,
Puneet
--- End Message ---
___
dns-operations mailing list
dns-operations@lists.dns-oarc.net
https://lists.dns-oarc.net/mailman/listinfo/dns-operations

Re: [dns-operations] Call for Participation -- ICANN DNSSEC and Security Workshop for ICANN Community Forum

[Puneet Sood via dns-operations]

--- Begin Message ---
On Wed, Nov 15, 2023 at 4:32 PM Barry Raveendran Greene 
wrote:

>
> The irony - having a DNS workshop in Puerto Rico where DNS Resiliency
> failures aggravated recovery during Hurricane Maria.
>
> My recommendation is to have an entire session on DNS resiliency in the
> face of disaster.
>

Do you have references handy you can share?



>
> On Nov 15, 2023, at 08:49, Kathy Schnitt  wrote:
>
> 
>
> *Call for Participation -- ICANN DNSSEC and Security Workshop for ICANN
> Community Forum*
>
>
>
> In cooperation with the ICANN Security and Stability Advisory Committee
> (SSAC), we are planning a DNSSEC and Security Workshop for the ICANN79
> Community Forum being held as a hybrid meeting from 02-07 March 2024 San
> Juan, Puerto Rico in the Atlantic Standard Time - AST (UTC-4). This
> workshop date will be determined once ICANN creates a block schedule for us
> to follow; then we will be able to request a day and time. The DNSSEC and
> Security Workshop has been a part of ICANN meetings for several years and
> has provided a forum for both experienced and new people to meet, present
> and discuss current and future DNSSEC deployments.  For reference, the most
> recent session was held at the ICANN78 The Annual General Meeting on
> Wednesday, 25 October 2023. The presentations and transcripts are available
> at: https://icann78.sched.com/.
>
>
>
> The DNSSEC Workshop Program Committee is developing a program for the
>
> upcoming meeting.  Proposals will be considered for the following topic
> areas and included if space permits.  In addition, we welcome suggestions
> for additional topics either for inclusion in the ICANN78 workshop, or for
> consideration for future workshops.
>
>
>
> *1.  Global DNSSEC Activities Panel*
>
> For this panel, we are seeking participation from those who have been
> involved in DNSSEC deployment as well as from those who have not deployed
> DNSSEC but who have a keen interest in the challenges and benefits of
> deployment, including Root Key Signing Key (KSK) Rollover activities and
> plans.
>
>
>
> *2.  DNSSEC Best Practice*
>
> Now that DNSSEC has become an operational norm for many registries,
> registrars, and ISPs, what have we learned about how we manage DNSSEC?
>
>
>- Do you still submit/accept DS records with Digest Type 1?
>- What is the best practice around key roll-overs?
>- What about Algorithm roll-overs?
>- Do you use and support DNSKEY Algorithms 13-16?
>- How often do you review your disaster recovery procedures?
>- Is there operational familiarity within your customer support teams?
>- What operational statistics have been gathered about DNSSEC?
>- Are there experiences being documented in the form of best
>practices, or something similar, for transfer of signed zones?
>
>
>
> Activities and issues related to DNSSEC in the DNS Root Zone are also
> desired.
>
>
>
> *3. DNSSEC Deployment Challenges*
>
> The program committee is seeking input from those that are interested in
> implementation of DNSSEC but have general or particular concerns with
> DNSSEC.  In particular, we are seeking input from individuals that would be
> willing to participate in a panel that would discuss questions of the
> following nature:
>
>
>
>
>- Are there any policies directly or indirectly impeding your DNSSEC
>deployment? (RRR model, CDS/CDNSKEY automation)
>- What are your most significant concerns with DNSSEC, e.g.,
>complexity, training, implementation, operation or something else?
>- What do you expect DNSSEC to do for you and what doesn't it do?
>- What do you see as the most important trade-offs with respect to
>doing or not doing DNSSEC?
>
>
>
> *4. Security Panel*
>
> The program committee is looking for presentations on DNS, DNSSEC, routing
> and other topics that could impact the security and/or stability of the
> Internet.
>
>
>
> We are looking for presentations that cover implementation issues,
> challenges, opportunities and best practices for:
>
>
>
>- Emerging threats that could impact the security and/or stability of
>the Internet
>- DoH and DoT
>- RPKI (Resource Public Key Infrastructure)
>- BGP routing & secure implementations
>- MANRS ( Mutually Agreed Norms for Routing Security)
>- Browser security – DNS, DNSSEC, DoH
>- EMAIL & DNS related security – DMARC, DKIM, TLSA, etc…
>
>
>
> If you are interested in participating, please send a brief (1-3 sentence)
> description of your proposed presentation to
> dnssec-security-works...@icann.org *by COB Friday, 19 January 2024.*
>
>
>
> Thank you,
>
> Kim and Kathy
>
> On behalf of the DNSSEC Workshop Program Committee:
>
> Steve Crocker, Edgemoor Research Institute
>
> Mark Elkins, DNS/ZARC
>
> Jacques Latour, .CA
>
> Russ Mundy, Tislabs
>
> Yoshiro Yoneya
>
> Dan York, Internet Society
>
>
>
>
>
>
>
>
> ___
> dns-operations mailing list
> 

Re: [dns-operations] Google Public DNS has enabled case randomization globally

[Puneet Sood via dns-operations]

--- Begin Message ---
On Sat, Jul 29, 2023 at 5:06 PM Evan Hunt  wrote:
>
> (Resending because I accidentally replied privately.)
>
> On Sat, Jul 29, 2023 at 09:07:21AM -0700, Paul Vixie via dns-operations wrote:
> > < > draft “Use of Bit 0x20 in DNS Labels to Improve Transaction Identity”, is
> > highly effective and widely supported.>>
> >
> > would the google dns team be willing to contribute to this draft in the ietf
> > dns wg? we have not pressed the matter since 2008 simply because noone
> > cared. with google now deploying it for quad8, i think we might get a
> > different result today than we got 14 years ago.
>
> Case randomization has been supported in quite a lot of resolvers for
> quite a long while.  I know for sure that unbound and knot resolver both
> have it.  (BIND doesn't, I'm not sure why not; we just never got around to
> it, I suppose.)
>
> If, on top of these other implementations, google is now deploying it, then
> they must have found it non-harmful, which would imply that all or nearly
> all currently-deployed authoritative server software must be repsonding to
> case-randomized queries correctly.

There are still a number of operators that are broken where we have
had to auto-detect and/or config disable case randomization. The worst
are the small number that return NXDOMAIN for the queries or timeout.

>
> As I recall, the 0x20 draft was mostly discussion of the problem space; the
> only normative part was a protocol clarification that the question section
> has to be copied bit-for-bit into replies. That was already implicit in
> other RFCs at the time... and, though I can't remember where at the moment,
> I could just about swear it's been made explicit since then. (I remember
> discussing this with Paul Hoffman at an OARC meeting in 2014; perhaps he
> can call up the chapter and verse?)
>
> If I'm mistaken about that, and it's still only implicit, then I'd support
> clarifying the protocol in that way.  If it's already been clarified,
> though, then I'm not sure why a 0x20 RFC is needed now.

I do not recall this during my attendance of the IETF dnsop sessions
but I could easily have missed this. Will have to wait for Paul
Hoffman to chime in here. Barring such an RFC update, I believe a
requirement update would be helpful for compliance.

-Puneet Sood

>
> --
> Evan Hunt -- e...@isc.org
> Internet Systems Consortium, Inc.
>
> ___
> dns-operations mailing list
> dns-operations@lists.dns-oarc.net
> https://lists.dns-oarc.net/mailman/listinfo/dns-operations

--- End Message ---
___
dns-operations mailing list
dns-operations@lists.dns-oarc.net
https://lists.dns-oarc.net/mailman/listinfo/dns-operations

Re: [dns-operations] Important change for the .ga TLD 6th june 2023

[Puneet Sood via dns-operations]

--- Begin Message ---
Stephane,

Looks like google.ga is returning NXDOMAIN from the nic.fr servers.
Can you please check what could be wrong with the delegation info?

Thanks,
Puneet

$ dig @d.nic.fr google.ga

; <<>> DiG 9.10.6 <<>> @d.nic.fr google.ga
; (2 servers found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NXDOMAIN, id: 55926
;; flags: qr aa rd; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 1
;; WARNING: recursion requested but not available

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1232
;; QUESTION SECTION:
;google.ga. IN A

;; AUTHORITY SECTION:
ga. 600 IN SOA a.nic.fr. dnsmaster.afnic.fr. 222484 3600 1800 360 600

;; Query time: 97 msec
;; SERVER: 2001:678:c::1#53(2001:678:c::1)
;; WHEN: Sun Jun 04 22:22:48 EDT 2023
;; MSG SIZE  rcvd: 100

On Sun, Jun 4, 2023 at 10:46 AM Stephane Bortzmeyer  wrote:
>
> On Fri, Jun 02, 2023 at 09:28:24AM +0200,
>  Stephane Bortzmeyer  wrote
>  a message of 56 lines which said:
>
> > The .ga TLD will change its mode of operation on 6th june 2023. The majority
> > of domain names, registered under disputable conditions, will be removed. Do
> > not be surprised if many domains will yield NXDOMAIN.
>
> Done in advance, this night (gabonese time).
>
> % check-soa -i ga
> d.nic.fr.
> 2001:678:c::1: OK: 222414 (6 ms)
> 194.0.9.1: OK: 222414 (14 ms)
> f.ext.nic.fr.
> 2001:67c:1010:11::53: OK: 222414 (17 ms)
> 194.146.106.46: OK: 222414 (19 ms)
> g.ext.nic.fr.
> 2001:678:4c::1: OK: 222414 (5 ms)
> 194.0.36.1: OK: 222414 (5 ms)
> ___
> dns-operations mailing list
> dns-operations@lists.dns-oarc.net
> https://lists.dns-oarc.net/mailman/listinfo/dns-operations

--- End Message ---
___
dns-operations mailing list
dns-operations@lists.dns-oarc.net
https://lists.dns-oarc.net/mailman/listinfo/dns-operations

Re: [dns-operations] why DNS can't have nice things

[Puneet Sood via dns-operations]

--- Begin Message ---
On Fri, Apr 14, 2023 at 6:31 PM Evan Hunt  wrote:
>
> > once an embedded dns recursive server works well enough, it ships, is
> > widely deployed, and becomes abandonware. the apps which don't work are
> > found (by others) later. there is no complaint path.
>
> Alas, if only the IETF would adopt my simple recommendation that
> non-standards-compliant software be required to literally smell bad.
>
> (For some reason I still haven't gotten my Nobel prize for that. Maybe I
> should check today's mail...)

It's in the spam folder.

>
> --
> Evan Hunt -- e...@isc.org
> Internet Systems Consortium, Inc.
> ___
> dns-operations mailing list
> dns-operations@lists.dns-oarc.net
> https://lists.dns-oarc.net/mailman/listinfo/dns-operations

--- End Message ---
___
dns-operations mailing list
dns-operations@lists.dns-oarc.net
https://lists.dns-oarc.net/mailman/listinfo/dns-operations

Re: [dns-operations] Cloudflare TYPE65283

[Puneet Sood via dns-operations]

--- Begin Message ---
This part of the discussion probably should move to the dnsop wg thread for
the draft. I will send some comments there.

On Mon, Mar 27, 2023 at 5:36 PM Shumon Huque  wrote:

> On Tue, Mar 28, 2023 at 6:19 AM Viktor Dukhovni 
> wrote:
>
>>
>> A possibly inconvenient question, just to make sure we're not ignoring
>> the obvious sceptical position:
>>
>> * How compelling are compact lies?
>>
>> The reason to ask is that both the original and now modified protocols
>> involve non-trivial complexity, and would have resolvers responding
>> differently to queries with the DO bit set (tell them the truth) vs.
>> queries that don't request validated answers (unmask the lie).
>>
>> The savings vs. actual by-the-book NSEC responses appear to be a 2x
>> reduction in the number of signatures to compute (the SOA RRSIG is
>> presumably easily cached) and a 1.5x reduction in the number of
>> signatures to transmit (SOA + 1 NSEC, vs. SOA + 2 NSEC).
>>
>> Do the CPU and packet size reductions justify the additional protocol
>> complexity?
>>
>
> That's a reasonable question, and perhaps best directed to the originators
> of the scheme at Cloudflare. I don't know if there have been any
> measurement studies or analyses of the cost benefits vs by-the-book DNSSEC.
> There are currently 3 large commercial DNS providers that have had it
> deployed for a while now, so I suspect that it is here to stay.
>


>
> Note that one other provider (UltraDNS) does support traditional NSEC
> White "Lies" that give by-the-book DNSSEC proofs for NXDOMAIN, so
> apparently they are bearing the additional costs just fine.
>
> One other point -- without the additional rcode substitution schemes under
> discussion, Compact Answers can cause additional work for authority
> servers, since NODATA responses may lead to follow-on queries by DNS client
> applications (e.g. the common  followed by A pattern). So, the
> per-response crypt & size reductions need to also be weighed against the
> cost of these additional queries.
>

The draft should also specify desired behavior for queries for the new
NXNAME RR type at the auth and at the resolver.



>
> Shumon.
>
> ___
> dns-operations mailing list
> dns-operations@lists.dns-oarc.net
> https://lists.dns-oarc.net/mailman/listinfo/dns-operations
>
--- End Message ---
___
dns-operations mailing list
dns-operations@lists.dns-oarc.net
https://lists.dns-oarc.net/mailman/listinfo/dns-operations

Re: [dns-operations] Name servers returning incorrectly truncated UDP responses

[Puneet Sood via dns-operations]

--- Begin Message ---
On Sat, Jul 30, 2022 at 10:26 AM Dave Lawrence  wrote:

> Greg Choules via dns-operations writes:
> > I am including in this mail the RNAME from the SOA (same for both
> > zones) in the hope that someone who is responsible for DNS at Sony
> > entertainment will see this and take note.
>
> And tell us what in the world DNS software they're running, and why
> they chose it.
>

Jaap up-thread used fpdns to figure out the first question.

fpdns e.ns.email.sonyentertainmentnetwork.com
fingerprint (e.ns.email.sonyentertainmentnetwork.com, 207.251.96.133): DJ
Bernstein TinyDNS 1.05 [Old Rules]

> ___
> dns-operations mailing list
> dns-operations@lists.dns-oarc.net
> https://lists.dns-oarc.net/mailman/listinfo/dns-operations
>
--- End Message ---
___
dns-operations mailing list
dns-operations@lists.dns-oarc.net
https://lists.dns-oarc.net/mailman/listinfo/dns-operations

[dns-operations] Name servers returning incorrectly truncated UDP responses

[Puneet Sood via dns-operations]

--- Begin Message ---
Hello,

While making our DNS response validation stricter, we have noticed that a
number of name servers return badly truncated UDP responses. This sometimes
happens with incorrect Answer section RR count.

$ dig m.email.sonyentertainmentnetwork.com. TXT @
e.ns.email.sonyentertainmentnetwork.com
;; Warning: Message parser reports malformed message packet.
;; Truncated, retrying in TCP mode.

; <<>> DiG 9.18.3-1+build1-Debian <<>> m.email.sonyentertainmentnetwork.com.
TXT @e.ns.email.sonyentertainmentnetwork.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 24446
;; flags: qr aa rd; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0
;; WARNING: recursion requested but not available

;; QUESTION SECTION:
;m.email.sonyentertainmentnetwork.com. IN TXT

;; ANSWER SECTION:
m.email.sonyentertainmentnetwork.com. 3600 IN TXT "v=spf1 a mx ip4:
63.236.31.220/31 ip4:8.30.201.100/31 ip4:63.236.84.160 ip4:8.30.201.16
ip4:4.22.42.19 ip4:4.22.42.20/30 ip4:4.2" "2.42.24/31 ip4:4.22.42.26 ip4:
72.166.182.10/31 ip4:72.166.182.12/31 ip4:72.166.182.18/31 ip4:
72.166.182.20/30 ip4:207.251.96.0/" "24 ip4:65.125.54.0/24 ip4:
63.232.57.0/24 ip4:208.49.63.128/28 ip4:63.211.90.16/29 ip4:8.7.42.16/29
ip4:8.7.43.16/29 ip4:63.232." "236.144/29 ip4:8.7.44.144/29 ip4:
63.236.31.128/26 ip4:63.236.76.0/23 ip4:8.30.201.0/26 ~all"

;; Query time: 4 msec
;; SERVER: 207.251.96.133#53(e.ns.email.sonyentertainmentnetwork.com) (TCP)
;; WHEN: Fri Jul 29 16:57:51 EDT 2022
;; MSG SIZE  rcvd: 542


While the affected operators are spread around the world, the similarity of
the bad response across operators appears to suggest the DNS software may
be from the same or closely related source. These servers do not respond to
a version.bind query.

Have you seen similar bad responses? Do you have an idea of the provenance
of this software?

Thanks,
Puneet
--- End Message ---
___
dns-operations mailing list
dns-operations@lists.dns-oarc.net
https://lists.dns-oarc.net/mailman/listinfo/dns-operations

Re: [dns-operations] Freenom TLDs not working through Google Public DNS

[Puneet Sood via dns-operations]

--- Begin Message ---
We have been investigating the issue. This was a BGP route
announcement issue. It is mostly resolved now.

Tracking bug: https://b.corp.google.com/issues/214911324

On Wed, Jan 19, 2022 at 10:11 AM Stephane Bortzmeyer  wrote:
>
> I did not investigate yet but it may be fun:
>
> https://issuetracker.google.com/issues?q=.ml=1
> ___
> dns-operations mailing list
> dns-operations@lists.dns-oarc.net
> https://lists.dns-oarc.net/mailman/listinfo/dns-operations
--- End Message ---
___
dns-operations mailing list
dns-operations@lists.dns-oarc.net
https://lists.dns-oarc.net/mailman/listinfo/dns-operations

Re: [dns-operations] Lot's of TXT queries from Google

[Puneet Sood via dns-operations]

--- Begin Message ---
On Fri, Oct 8, 2021 at 3:42 AM Moritz Müller via dns-operations
 wrote:
>
>
>
>
> -- Forwarded message --
> From: "Moritz Müller" 
> To: "Blacka, David via dns-operations" 
> Cc:
> Bcc:
> Date: Fri, 8 Oct 2021 09:37:34 +0200
> Subject: Re: [dns-operations] Lot's of TXT queries from Google
> Thank you for trying to help out folks!
>
> > On 7 Oct 2021, at 16:56, Viktor Dukhovni  wrote:
> >
> > I wonder whether this is an attempt to collect the NSEC3 chain for an
> > off-line dictionary attack?  12 character random names are long enough
> > to sample the space very well, though shorter strings would also do.
>
> That sounds possible, but doesn’t explain the _dmarc/default labels, right?
>
> @puneet
> Would that be worth further exploring on your side?
> At some point, we received 16264 qps of those type of queries at one site.

I think this falls in the category of unusual but non-critical traffic
spikes. Unless there is a pattern here to suggest future risk it is
not worthwhile to investigate further.

This is another case where NSEC3 opt-out interferes with effective
NSEC{3} response caching which would reduce queries to the TLD.

-Puneet

>
> —
> Moritz
>
>
>
>
> -- Forwarded message --
> From: "Moritz Müller via dns-operations" 
> To: "Blacka, David via dns-operations" 
> Cc:
> Bcc:
> Date: Fri, 8 Oct 2021 09:37:34 +0200
> Subject: Re: [dns-operations] Lot's of TXT queries from Google
> ___
> dns-operations mailing list
> dns-operations@lists.dns-oarc.net
> https://lists.dns-oarc.net/mailman/listinfo/dns-operations

--- End Message ---
___
dns-operations mailing list
dns-operations@lists.dns-oarc.net
https://lists.dns-oarc.net/mailman/listinfo/dns-operations

Re: [dns-operations] Lot's of TXT queries from Google

[Puneet Sood via dns-operations]

--- Begin Message ---
On Thu, Oct 7, 2021 at 11:22 AM Viktor Dukhovni  wrote:
>
> On Thu, Oct 07, 2021 at 02:53:36PM +, Wessels, Duane via dns-operations 
> wrote:
>
> > I can't explain the TXT queries, but the NS queries seem to be
> > Google's method of doing qname minimization, with an added nonce
> > value.  See https://indico.dns-oarc.net/event/39/contributions/864/
> > and
> > https://developers.google.com/speed/public-dns/docs/security?hl=en#nonce_prefixes
>
> The odd thing is though that queries with Google's nonce labels to .NL
> would be expected to have the appended label after some desired 2LD:
>
> nonce.extant-2ld.nl
>
> I would not expect Google to append 2LD rather than 3LD nonces in
> queries to the .NL auth servers, those elicit NXDOMAIN, rather than the
> desired nonce-salted referrals.

Correct. These are not nonce prefixes appended by GPDNS. Also we are
mostly querying for NS records when nonce prefixes are used. Given the
RR types being queried, this is likely to be what Matt Nordhoff
mentioned above.

On a related note, the queries you mention send more than two labels
to the NL nameservers. This happens in some scenarios with our qname
minimization implementation. We are making some changes which should
reduce the labels in the query to just two (plus an optional nonce) in
almost all cases.

-Puneet

>
> --
> Viktor.
> ___
> dns-operations mailing list
> dns-operations@lists.dns-oarc.net
> https://lists.dns-oarc.net/mailman/listinfo/dns-operations
--- End Message ---
___
dns-operations mailing list
dns-operations@lists.dns-oarc.net
https://lists.dns-oarc.net/mailman/listinfo/dns-operations

Re: [dns-operations] Lot's of TXT queries from Google

[Puneet Sood via dns-operations]

--- Begin Message ---
On Thu, Oct 7, 2021 at 11:22 AM Viktor Dukhovni  wrote:
>
> On Thu, Oct 07, 2021 at 02:53:36PM +, Wessels, Duane via dns-operations 
> wrote:
>
> > I can't explain the TXT queries, but the NS queries seem to be
> > Google's method of doing qname minimization, with an added nonce
> > value.  See https://indico.dns-oarc.net/event/39/contributions/864/
> > and
> > https://developers.google.com/speed/public-dns/docs/security?hl=en#nonce_prefixes
>
> The odd thing is though that queries with Google's nonce labels to .NL
> would be expected to have the appended label after some desired 2LD:
>
> nonce.extant-2ld.nl
>
> I would not expect Google to append 2LD rather than 3LD nonces in
> queries to the .NL auth servers, those elicit NXDOMAIN, rather than the
> desired nonce-salted referrals.

Correct. These are not nonce prefixes appended by GPDNS. Also we are
mostly querying for NS records when nonce prefixes are used. Given the
RR types being queried, this is likely to be what Matt Nordhoff
mentioned above.

On a related note, the queries you mention send more than two labels
to the NL nameservers. This happens in some scenarios with our qname
minimization implementation. We are making some changes which should
reduce the labels in the query to just two (plus an optional nonce) in
almost all cases.

-Puneet

>
> --
> Viktor.
> ___
> dns-operations mailing list
> dns-operations@lists.dns-oarc.net
> https://lists.dns-oarc.net/mailman/listinfo/dns-operations
--- End Message ---
___
dns-operations mailing list
dns-operations@lists.dns-oarc.net
https://lists.dns-oarc.net/mailman/listinfo/dns-operations

Re: [dns-operations] slack.com bogus

[Puneet Sood via dns-operations]

--- Begin Message ---
Missed this in the previous email:

* We cache most records needed for DNS resolution with a 6 hour TTL.

On Fri, Oct 1, 2021 at 12:03 PM Puneet Sood  wrote:
>
> Some information on what happened during this incident with the Google
> Public DNS service.
>
> * GPDNS did not configure an NTA for slack.com
> * We observed a small percentage of SERVFAILs during 10:05-10:47 AM PT
> on 20210930. Which was fixed by
> * a number of user-initiated cache flush requests for slack.com
> records (dname, ds, a, soa) between 10:46 AM to 11:28 AM PT on
> 20210930.
>
> Our general policy on NTAs is to only add them after evaluating the
> specific scenario. We never add them by default.
>
> -Puneet
>
> On Thu, Sep 30, 2021 at 3:20 PM Viktor Dukhovni  
> wrote:
> >
> > On Thu, Sep 30, 2021 at 02:31:32PM -0400, vom513 wrote:
> >
> > > I see it as broken at home on my validating BIND instance (I get 
> > > SERVFAIL).
> > >
> > > Interesting (at least to me) - checking 8.8.8.8 and 1.1.1.1 I get A
> > > records back just fine.  However no ‘ad’ flag (using dig).  Other
> > > DNSSEC signed zones still give ad off quad-8/quad-1 though.
> > >
> > > So perhaps a dumb question - could Google and Cloudflare be hitting
> > > some kind of “manual overrride” to not validate a given zone - i.e.
> > > human intervention / look the other way ?
> > >
> > > Or is there a more technical explanation that I could be missing ?
> >
> > It suffices to cap DS RRSet TTLs well below the .COM TLDs 24 hour
> > default.  If they cap TTLs at say 1 hour, the issue would have been
> > resolved 1 hour after the DS RRSet was deleted from the .COM zone.
> >
> > Of course the DS RRSet might also have been manually flushed from the
> > cache, as special treatment for "slack.com", given their popularity and
> > the verifiable deletion of the DS records.
> >
> > A final possibility is automatic refresh of DS records for domains that
> > go bogus.
> >
> > So there are a number of ways to reduce the impact duration to below the
> > full ~1 day maximum.
> >
> > --
> > Viktor.
> > ___
> > dns-operations mailing list
> > dns-operations@lists.dns-oarc.net
> > https://lists.dns-oarc.net/mailman/listinfo/dns-operations

--- End Message ---
___
dns-operations mailing list
dns-operations@lists.dns-oarc.net
https://lists.dns-oarc.net/mailman/listinfo/dns-operations

Re: [dns-operations] slack.com bogus

[Puneet Sood via dns-operations]

--- Begin Message ---
Some information on what happened during this incident with the Google
Public DNS service.

* GPDNS did not configure an NTA for slack.com
* We observed a small percentage of SERVFAILs during 10:05-10:47 AM PT
on 20210930. Which was fixed by
* a number of user-initiated cache flush requests for slack.com
records (dname, ds, a, soa) between 10:46 AM to 11:28 AM PT on
20210930.

Our general policy on NTAs is to only add them after evaluating the
specific scenario. We never add them by default.

-Puneet

On Thu, Sep 30, 2021 at 3:20 PM Viktor Dukhovni  wrote:
>
> On Thu, Sep 30, 2021 at 02:31:32PM -0400, vom513 wrote:
>
> > I see it as broken at home on my validating BIND instance (I get SERVFAIL).
> >
> > Interesting (at least to me) - checking 8.8.8.8 and 1.1.1.1 I get A
> > records back just fine.  However no ‘ad’ flag (using dig).  Other
> > DNSSEC signed zones still give ad off quad-8/quad-1 though.
> >
> > So perhaps a dumb question - could Google and Cloudflare be hitting
> > some kind of “manual overrride” to not validate a given zone - i.e.
> > human intervention / look the other way ?
> >
> > Or is there a more technical explanation that I could be missing ?
>
> It suffices to cap DS RRSet TTLs well below the .COM TLDs 24 hour
> default.  If they cap TTLs at say 1 hour, the issue would have been
> resolved 1 hour after the DS RRSet was deleted from the .COM zone.
>
> Of course the DS RRSet might also have been manually flushed from the
> cache, as special treatment for "slack.com", given their popularity and
> the verifiable deletion of the DS records.
>
> A final possibility is automatic refresh of DS records for domains that
> go bogus.
>
> So there are a number of ways to reduce the impact duration to below the
> full ~1 day maximum.
>
> --
> Viktor.
> ___
> dns-operations mailing list
> dns-operations@lists.dns-oarc.net
> https://lists.dns-oarc.net/mailman/listinfo/dns-operations

--- End Message ---
___
dns-operations mailing list
dns-operations@lists.dns-oarc.net
https://lists.dns-oarc.net/mailman/listinfo/dns-operations

Re: [dns-operations] Google (formerly also CF) public DNS sometimes forwards incomplete subset of NSEC RRs

[Puneet Sood via dns-operations]

--- Begin Message ---
The problem has been fixed for a couple of months now. Thanks for reporting.

$ dig @8.8.8.8 _4._tcp.mx.runbox.com. IN TLSA +dnssec +nocomment +nocrypto

; <<>> DiG 9.10.6 <<>> @8.8.8.8 _4._tcp.mx.runbox.com. IN TLSA +dnssec
+nocomment +nocrypto
; (1 server found)
;; global options: +cmd
;_4._tcp.mx.runbox.com. IN TLSA
runbox.com. 1204 IN SOA dns61.copyleft.no. hostmaster.copyleft.no.
308995 14400 3600 1296000 3600
runbox.com. 1204 IN RRSIG SOA 13 2 86400 20210809115838 20210726102838
7485 runbox.com. [omitted]
mx.runbox.com. 3003 IN NSEC _25._tcp.mx.runbox.com. A RRSIG NSEC
mx.runbox.com. 3003 IN RRSIG NSEC 13 3 3600 20210809115838
20210726102838 7485 runbox.com. [omitted]
_25._tcp.mx.runbox.com. 3003 IN NSEC ipmi.mysql01.runbox.com. CNAME RRSIG NSEC
_25._tcp.mx.runbox.com. 3003 IN RRSIG NSEC 13 5 3600 20210809115838
20210726102838 7485 runbox.com. [omitted]
;; Query time: 57 msec
;; SERVER: 8.8.8.8#53(8.8.8.8)
;; WHEN: Wed Jul 28 22:03:46 EDT 2021
;; MSG SIZE  rcvd: 525

Cheers,
Puneet

On Wed, Sep 16, 2020 at 4:35 PM Viktor Dukhovni  wrote:
>
> On Wed, Sep 16, 2020 at 11:50:31AM -0700, Marek Vavruša wrote:
> > Hi Viktor, I forgot to update this thread, but this should be fixed.
>
> Thanks!  Looks much better now.  Now it is Google's turn.  I still see
> an incomplete NSEC3 RRset from 8.8.8.8:
>
> $ hsdig -n8.8.8.8 -D -t tlsa _25._tcp.mx.runbox.com
> _25._tcp.mx.runbox.com. IN TLSA ? ; NoError AD=1
> runbox.com. IN SOA dns61.copyleft.no. hostmas...@copyleft.no. 308499 
> 14400 3600 1296000 3600
> runbox.com. IN RRSIG SOA 13 2 86400 20200930104345 20200916091345 18202 
> runbox.com. 
> *.runbox.com. IN NSEC _acme-challenge.runbox.com. A MX RRSIG NSEC
> *.runbox.com. IN RRSIG NSEC 13 2 3600 20200930104345 20200916091345 18202 
> runbox.com. 
>
> but the NSEC establishing the zone apex as the closest encloser (now
> present in the CF responses):
>
> $ hsdig -n1.0.0.1 -D -t tlsa _25._tcp.mx.runbox.com
> _25._tcp.mx.runbox.com. IN TLSA ? ; NoError AD=1
> runbox.com. IN SOA dns61.copyleft.no. hostmas...@copyleft.no. 308499 
> 14400 3600 1296000 3600
> runbox.com. IN RRSIG SOA 13 2 86400 20200930104345 20200916091345 18202 
> runbox.com. 
> munin01.runbox.com. IN NSEC ipmi.mysql01.runbox.com. A RRSIG NSEC
> munin01.runbox.com. IN RRSIG NSEC 13 3 3600 20200930104345 20200916091345 
> 18202 runbox.com. 
> *.runbox.com. IN NSEC _acme-challenge.runbox.com. A MX RRSIG NSEC
> *.runbox.com. IN RRSIG NSEC 13 2 3600 20200930104345 20200916091345 18202 
> runbox.com. 
>
> is missing from the GOOG responses.
>
> --
> Viktor.
> ___
> dns-operations mailing list
> dns-operations@lists.dns-oarc.net
> https://lists.dns-oarc.net/mailman/listinfo/dns-operations

--- End Message ---
___
dns-operations mailing list
dns-operations@lists.dns-oarc.net
https://lists.dns-oarc.net/mailman/listinfo/dns-operations

[dns-operations] DNS contact for .ml TLD

[Puneet Sood via dns-operations]

--- Begin Message ---
Hello,

I need to reach a live contact for the .ml TLD. Emailing
i...@malidili.com has not gotten a response.

If someone has a contact, please let me know.

Thanks,
Puneet
--- End Message ---
___
dns-operations mailing list
dns-operations@lists.dns-oarc.net
https://lists.dns-oarc.net/mailman/listinfo/dns-operations

Re: [dns-operations] Inconsistent NSEC response for unsigned zone from AWS

[Puneet Sood via dns-operations]

--- Begin Message ---
On Tue, Jun 22, 2021, 12:35 AM Matt Nordhoff  wrote:

> On Tue, Jun 22, 2021 at 3:54 AM Viktor Dukhovni 
> wrote:
> > On Tue, Jun 22, 2021 at 03:30:39AM +, Matt Nordhoff wrote:
> >
> > > > Indeed I see the same:
> > > >
> > > > $ dig +noall +dnssec +norecur +nocrypto +ans +auth +add -t NS
> corp.ibexglobal.com @ns-725.awsdns-26.net.
> > > > corp.ibexglobal.com.172800  IN  NS
> ns-1415.awsdns-48.org.
> > > > corp.ibexglobal.com.172800  IN  NS
> ns-1804.awsdns-33.co.uk.
> > > > corp.ibexglobal.com.172800  IN  NS
> ns-29.awsdns-03.com.
> > > > corp.ibexglobal.com.172800  IN  NS
> ns-945.awsdns-54.net.
> > > > corp.ibexglobal.com.86400   IN  NSEC\
> 000.corp.ibexglobal.com. RRSIG NSEC
> > > > corp.ibexglobal.com.86400   IN  RRSIG   NSEC 13 3 86400
> 20210623012420 20210621232420 36517 ibexglobal.com. [omitted]
> > > >
> > > > This violates <
> https://datatracker.ietf.org/doc/html/rfc4035#section-2.3>:
> > >
> > > I haven't spent the time to understand precisely what this thread is
> > > talking about, but that's how NSEC white/black lies work. NS1 does the
> > > same thing (give or take possible bugs) as AWS.
> >
> > No, there's a subtle difference, this qname actually exists, and has an
> > NS RRSet.  The NSEC bitmap needs to reflect this.
>
> Ah, now I understand the issue, thank you.
>
> I'm sorry if my previous email seemed glib. I didn't get enough sleep
> to easily understand NSEC problems, but thought it was probably better
> to reply now than to wait.
>
> My email might still matter, but it's not relevant to this thread.
>
> [FWIW, Cloudflare handles insecure referrals correctly, AFAIK. I have
> no idea about NS1, but there's no reason to suspect anything.]
>

I did not see this issue with cloudflare.

For some reason my post forked into two threads. Gavin from AWS replied on
the other thread.

Puneet

> --
> Matt Nordhoff
> ___
> dns-operations mailing list
> dns-operations@lists.dns-oarc.net
> https://lists.dns-oarc.net/mailman/listinfo/dns-operations
>
--- End Message ---
___
dns-operations mailing list
dns-operations@lists.dns-oarc.net
https://lists.dns-oarc.net/mailman/listinfo/dns-operations

[dns-operations] Inconsistent NSEC response for unsigned zone from AWS

[Puneet Sood via dns-operations]

--- Begin Message ---
Hello dnssec experts,

I am noticing an inconsistent NSEC response in a delegation. Depending
on the RR type specified in the query the response includes NS in the
set of RR types in the NSEC RR proving the absence of the /DS
record. Is this behavior below within what nameservers can return?
Ideally all cases will list the NS RR type in the NSEC record.

I suspect the absence of NS in the NSEC is confusing our NSEC checking
logic. Validation is working correctly but in a suboptimal fashion.

 Example domain: corp.ibexglobal.com

$ dig ns corp.ibexglobal.com +short
ns-1415.awsdns-48.org.
ns-1804.awsdns-33.co.uk.
ns-29.awsdns-03.com.
ns-945.awsdns-54.net.

 With type NS, NS not included in NSEC RR.

$ dig corp.ibexglobal.com -t NS +dnssec +nocrypto +nocomment
@ns-725.awsdns-26.net.

;corp.ibexglobal.com.   IN  NS
corp.ibexglobal.com.172800  IN  NS  ns-1415.awsdns-48.org.
corp.ibexglobal.com.172800  IN  NS  ns-1804.awsdns-33.co.uk.
corp.ibexglobal.com.172800  IN  NS  ns-29.awsdns-03.com.
corp.ibexglobal.com.172800  IN  NS  ns-945.awsdns-54.net.
corp.ibexglobal.com.86400   IN  NSEC
\000.corp.ibexglobal.com. RRSIG NSEC
corp.ibexglobal.com.86400   IN  RRSIG   NSEC 13 3 86400
20210623002754 20210621222754 36517 ibexglobal.com. [omitted]

 With type DS or A, NS included in NSEC RR.

$ dig corp.ibexglobal.com -t A +dnssec +nocrypto +nocomment
@ns-725.awsdns-26.net.
;corp.ibexglobal.com.   IN  A
corp.ibexglobal.com.172800  IN  NS  ns-1415.awsdns-48.org.
corp.ibexglobal.com.172800  IN  NS  ns-1804.awsdns-33.co.uk.
corp.ibexglobal.com.172800  IN  NS  ns-29.awsdns-03.com.
corp.ibexglobal.com.172800  IN  NS  ns-945.awsdns-54.net.
corp.ibexglobal.com.86400   IN  NSEC
\000.corp.ibexglobal.com. NS RRSIG NSEC
corp.ibexglobal.com.86400   IN  RRSIG   NSEC 13 3 86400
20210623002809 20210621222809 36517 ibexglobal.com. [omitted]

$ dig corp.ibexglobal.com -t DS +dnssec +nocrypto +nocomment
@ns-725.awsdns-26.net.
;corp.ibexglobal.com.   IN  DS
ibexglobal.com. 900 IN  SOA ns-380.awsdns-47.com.
awsdns-hostmaster.amazon.com. 1 7200 900 1209600 86400
ibexglobal.com. 900 IN  RRSIG   SOA 13 2 900
20210622004320 20210621222820 36517 ibexglobal.com. [omitted]
corp.ibexglobal.com.86400   IN  NSEC
\000.corp.ibexglobal.com. NS RRSIG NSEC
corp.ibexglobal.com.86400   IN  RRSIG   NSEC 13 3 86400
20210623002820 20210621222820 36517 ibexglobal.com. [omitted]

Thanks,
Puneet
--- End Message ---
___
dns-operations mailing list
dns-operations@lists.dns-oarc.net
https://lists.dns-oarc.net/mailman/listinfo/dns-operations

Re: [dns-operations] Google public DNS sometimes forwards incomplete subset of NSEC RRs

[Puneet Sood via dns-operations]

--- Begin Message ---
Viktor, dnsop community,

If there additional missing NSEC|NSEC3 RR issues with Google Public
DNS you are aware of, please respond here or file a ticket
(https://developers.google.com/speed/public-dns/groups#issue_tracker).

On Sat, Feb 6, 2021 at 1:17 AM Viktor Dukhovni  wrote:
>
> > On Sep 16, 2020, at 6:31 PM, Viktor Dukhovni  wrote:
> >
> > Now it is Google's turn.  I still see an incomplete NSEC3 RRset from 
> > 8.8.8.8:
> >
> >$ hsdig -n8.8.8.8 -D -t tlsa _25._tcp.mx.runbox.com
> >_25._tcp.mx.runbox.com. IN TLSA ? ; NoError AD=1
> >runbox.com. IN SOA dns61.copyleft.no. hostmas...@copyleft.no. 308499 
> > 14400 3600 1296000 3600
> >runbox.com. IN RRSIG SOA 13 2 86400 20200930104345 20200916091345 18202 
> > runbox.com. 
> >*.runbox.com. IN NSEC _acme-challenge.runbox.com. A MX RRSIG NSEC
> >*.runbox.com. IN RRSIG NSEC 13 2 3600 20200930104345 20200916091345 
> > 18202 runbox.com. 
>
> I am seeing this issue again, intermittently from various Google
> DNS servers.  Here's an example from 8.8.4.4:
>
>   _25._tcp.mx.runbox.com. IN TLSA ? ; NoError AD=1
>   runbox.com. IN SOA dns61.copyleft.no. hostmas...@copyleft.no. 308714 
> 14400 3600 1296000 3600
>   runbox.com. IN RRSIG SOA 13 2 86400 20210219161924 20210205144924 12629 
> runbox.com. 
>   *.runbox.com. IN NSEC _acme-challenge.runbox.com. A MX RRSIG NSEC
>   *.runbox.com. IN RRSIG NSEC 13 2 3600 20210219161924 20210205144924 12629 
> runbox.com. 

I can reproduce the issue internally. A fix should be coming in the near future.

>
> Or DNSViz (3 of the four public IPs):
>
>   https://dnsviz.net/d/_25._tcp.mx.runbox.com/e/437682/dnssec/
>
> --
> Viktor.
>
>
> ___
> dns-operations mailing list
> dns-operations@lists.dns-oarc.net
> https://lists.dns-oarc.net/mailman/listinfo/dns-operations
--- End Message ---
___
dns-operations mailing list
dns-operations@lists.dns-oarc.net
https://lists.dns-oarc.net/mailman/listinfo/dns-operations

Re: [dns-operations] IMPORTANT: Please ensure your NSEC3 iteration count is sufficiently low

[Puneet Sood via dns-operations]

--- Begin Message ---
Hi Viktor,

Thanks for bringing this issue to everyone's attention and your ongoing
work on DNSSEC.

Google Public DNS is also planning to cap NSEC3 iterations to a safe value.
Do you have data you can share on the prevalence of high iteration count
NSEC3 zones?

-Puneet
On Thu, Apr 1, 2021 at 9:42 AM Viktor Dukhovni 
wrote:

> RFC 5155 defined NSEC3 iterations to scale up with the RSA/DSA key size
> up to perhaps as high as 2500 iterations for 4096-bit keys.  In
> retrospect such a generous iteration cap is counter-productive.  It is
> neither particularly effective at keeping your zone content "secret",
> nor sufficiently cheap to avoid negative impact on authoritative and
> iterative resolver performance.
>
> In that light, Wes Hardaker and I have authored an Internet-Draft that
> strongly recommends setting the NSEC3 additional iteration count to 0
> (at least one initial SHA1 hash is always performed).
>
> https://tools.ietf.org/html/draft-hardaker-dnsop-nsec3-guidance-02
>
> Today, the Knot resolver became the first one to cap NSEC3 iterations
> for now at 150, but this will likely be reduced further (perhaps
> ultimately as low as ~25):
>
> https://gitlab.nic.cz/knot/knot-resolver/-/tags/v5.3.1
>
> and is expected to be done by more resolvers.
>
> Since iteration counts above the resolver cap make denial-of-existence
> for the entire zone insecure, it is important that all domains with a
> high NSEC3 iteration count proactively lower it ideally to 0, but
> otherwise ~10 or less.
>
> While DNSSEC still precludes forged positive answers, any signed
> NXDomain or NODATA NSEC3 response can be replayed for any query,
> regardless of the qname.
>
> This impacts any protocol in which negative responses have security
> consequences.  Potential exposures include:
>
> - Forged absence of RFC7672 DANE SMTP TLSA records
> - Forged absence of CAA records
> - Forged absence of HTTPS records enabling various downgrades
> - ...
>
> A number of TLDs have already lowred their iteration counts, and it is
> expected that most of the rest will follow soon.
>
> TLD before  after
> --- --  -
> la 150  1
> xn--q7ce6a 150  1
> blue   100 10
> green  100 10
> lat100 10
> mx 100 10
> pink   100 10
> red100 10
> schaeffler 100 10
> by 100  3
> creditunion100  3
> ally   100  1
> autos  100  1
> boats  100  1
> homes  100  1
> motorcycles100  1
> yachts 100  1
>
> If your DNS zone is configured to use NSEC3, please:
>
> - Reduce the iteration count to 10 or less.
>
> - Disable opt-out, you're very unlikely to need it.
>
> - Either rotate the salt each time you sign, or skip
>   it entirely.  But a short fixed salt is harmless if
>   leaving it alone easier than changing it.
>
> Of course, if your zone is small enough (just the zone apex and a
> handful of already public or easy to guess names) or in any case has
> nothing to hide, even better is to use just plain NSEC.  You get smaller
> negative replies (less exposure to DoS) and more effective negative
> caching at resolvers.  So in many cases, it is even simpler to abandon
> NSEC3 entirely.  Please also consider the pros/cons of that option.
>
> --
> Viktor.
> ___
> dns-operations mailing list
> dns-operations@lists.dns-oarc.net
> https://lists.dns-oarc.net/mailman/listinfo/dns-operations
>
--- End Message ---
___
dns-operations mailing list
dns-operations@lists.dns-oarc.net
https://lists.dns-oarc.net/mailman/listinfo/dns-operations

Re: [dns-operations] OpenDNS, Google, Nominet - New delegation update failure mode

[Puneet Sood via dns-operations]

--- Begin Message ---
On Mon, Nov 16, 2020 at 7:09 PM Doug Barton  wrote:
>
> Puneet, or anyone else at Google, are there publicly available
> references to this configuration choice? Web searches have all returned
> pages of generic discussion about parent/child config. I'm having a
> discussion with some colleagues and it would be helpful to reference
> something official.

For Google Public DNS, the public mention of parent centric behavior
is at 
https://developers.google.com/speed/public-dns/docs/troubleshooting/domains#delegation.
It does not provide detailed information about the implementation.
This is not a configuration option - this is the only way our resolver
service works. Other key points:
* we maintain a separate delegation cache; distinct from the response cache
* user queries for NS records result in queries to nameservers for the
zone and return NS RRs from the zone and are cached in response cache

Note that the delegation cache behavior is subject to change as we
improve our implementation and review
https://datatracker.ietf.org/doc/draft-ietf-dnsop-ns-revalidation/.

-Puneet

>
> If anyone from Nominet, or Knot, or other folks who referenced that
> their software is also parent-centric have references, that would be
> helpful as well.
>
> Best regards,
>
> Doug
>
> On 2020-04-02 13:12, Puneet Sood wrote:
> > Hi Doug,
> >
> > Google Public DNS resolution is working now.
> >
> > Google Public DNS is “parent-centric”—meaning that it only uses the
> > name servers that are returned in the referral responses from the
> > parent zone name servers, and does not make NS queries to this child
> > zone. So updating the parent delegation to include both NS sets will
> > help with Google Public DNS resolution.
> >
> > -Puneet

--- End Message ---
___
dns-operations mailing list
dns-operations@lists.dns-oarc.net
https://lists.dns-oarc.net/mailman/listinfo/dns-operations

Re: [dns-operations] DNS Flag Day 2020 will become effective on 2020-10-01

[Puneet Sood via dns-operations]

--- Begin Message ---
Google Public DNS posted a message on github about our plans:
https://github.com/dns-violations/dnsflagday/issues/139#issuecomment-673489183.

Background: Similar to what Ralf Weber said, over the last couple of
years we have seen issues with large domains related to UDP
fragmentation. It is not always obvious from the initial problem
report, which tends to be about a problem at the application level,
that the root cause is DNS UDP packet fragmentation. Setting lower
EDNS0 buffer size values solved the reported problems. Doing this more
widely for our resolver service will reduce operational overhead.

We plan to start setting a smaller EDNS0 buffer size (1400) starting
on October 1. Our experiments have shown that 1400 has minimal
difference from 1232. This will be a gradual rollout over 4-6 weeks.

-Puneet

On Fri, Sep 11, 2020 at 11:04 AM Ralf Weber  wrote:
>
> Moin!
>
> On 11 Sep 2020, at 10:29, Viktor Dukhovni wrote:
>
> > Paul is not arguing against avoiding fragmentation, IIRC his name is on
> > a draft recommending fragmentation avoidance.  So I think the issue is
> > really about which numbers to go with.
> I think nobody is thinking that. We do have an agreement that fragmentation
> does not work and thus we must make sure to not send packets that will
> fragment. The whole discussion is about picking a default number.
>
> > While 1232 is in the ballpark, it may be too conservative, the case for
> > 1232 rather than perhaps say 1400 didn't look that compelling.  For most
> > users the larger number is also fine, and sometimes even avoids (notably
> > rare) problems where a larger value works, but the smaller does not.
> I recall an incident where an auth server was accepting a maximum size
> around 700 bytes, and we could not resolve it as at that time our default
> size was 4000. However some other resolver software could resolve as it
> was advertising an 512 byte EDNS0 buffer in those circumstances. These
> are the workaround we want to get rid of and as a vendor and operator of
> DNS resolver software. As an operator or vendor you will get calls/tickets
> when your software can not resolve something and other software can, and
> the consensus of the major resolver software vendors is to use 1232 as
> the default EDNS0 buffer size.
>
> Most if not all software still will have a switch where you can make it
> bigger or smaller. The only benefit of having a larger size is less
> switches to TCP and the impact of this is way smaller then the impact of
> DoH and DoT also coming to authoritative servers.
>
> So I still think 1232 is a good default number, and rather then fighting
> over a few bytes the DNS community should work on other stuff.
>
> So long
> -Ralf
> —--
> Ralf Weber
> ___
> dns-operations mailing list
> dns-operations@lists.dns-oarc.net
> https://lists.dns-oarc.net/mailman/listinfo/dns-operations

--- End Message ---
___
dns-operations mailing list
dns-operations@lists.dns-oarc.net
https://lists.dns-oarc.net/mailman/listinfo/dns-operations

Re: [dns-operations] DNS Flag Day 2020 will become effective on 2020-10-01

[Puneet Sood via dns-operations]

--- Begin Message ---
Google Public DNS posted a message on github about our plans:
https://github.com/dns-violations/dnsflagday/issues/139#issuecomment-673489183.

Background: Similar to what Ralf Weber said, over the last couple of
years we have seen issues with large domains related to UDP
fragmentation. It is not always obvious from the initial problem
report, which tends to be about a problem at the application level,
that the root cause is DNS UDP packet fragmentation. Setting lower
EDNS0 buffer size values solved the reported problems. Doing this more
widely for our resolver service will reduce operational overhead.

We plan to start setting a smaller EDNS0 buffer size (1400) starting
on October 1. Our experiments have shown that 1400 has minimal
difference from 1232. This will be a gradual rollout over 4-6 weeks.

-Puneet

On Fri, Sep 11, 2020 at 11:04 AM Ralf Weber  wrote:
>
> Moin!
>
> On 11 Sep 2020, at 10:29, Viktor Dukhovni wrote:
>
> > Paul is not arguing against avoiding fragmentation, IIRC his name is on
> > a draft recommending fragmentation avoidance.  So I think the issue is
> > really about which numbers to go with.
> I think nobody is thinking that. We do have an agreement that fragmentation
> does not work and thus we must make sure to not send packets that will
> fragment. The whole discussion is about picking a default number.
>
> > While 1232 is in the ballpark, it may be too conservative, the case for
> > 1232 rather than perhaps say 1400 didn't look that compelling.  For most
> > users the larger number is also fine, and sometimes even avoids (notably
> > rare) problems where a larger value works, but the smaller does not.
> I recall an incident where an auth server was accepting a maximum size
> around 700 bytes, and we could not resolve it as at that time our default
> size was 4000. However some other resolver software could resolve as it
> was advertising an 512 byte EDNS0 buffer in those circumstances. These
> are the workaround we want to get rid of and as a vendor and operator of
> DNS resolver software. As an operator or vendor you will get calls/tickets
> when your software can not resolve something and other software can, and
> the consensus of the major resolver software vendors is to use 1232 as
> the default EDNS0 buffer size.
>
> Most if not all software still will have a switch where you can make it
> bigger or smaller. The only benefit of having a larger size is less
> switches to TCP and the impact of this is way smaller then the impact of
> DoH and DoT also coming to authoritative servers.
>
> So I still think 1232 is a good default number, and rather then fighting
> over a few bytes the DNS community should work on other stuff.
>
> So long
> -Ralf
> —--
> Ralf Weber
> ___
> dns-operations mailing list
> dns-operations@lists.dns-oarc.net
> https://lists.dns-oarc.net/mailman/listinfo/dns-operations

--- End Message ---
___
dns-operations mailing list
dns-operations@lists.dns-oarc.net
https://lists.dns-oarc.net/mailman/listinfo/dns-operations

Re: [dns-operations] [Ext] Nameserver responses from different IP than destination of request

[Puneet Sood via dns-operations]

--- Begin Message ---
On Tue, Sep 8, 2020 at 5:00 PM John Levine  wrote:
>
> In article <20200908181130.gd4...@straasha.imrryr.org> you write:
> >> Seems to me that would be true for any software that uses the usual
> >> BSD or linux socket calls that match the host and port ...
>
> >You're conflating binding the UDP socket which specifies the *local end*
> >of the UDP socket (and behaves as you describe) with the somewhat less
> >common practice of "connecting" the UDP socket (done by DNS resolvers of
> >various stripes) which then also limits the *remote peer* ...
>
> Right, but I'd think that would be the usual way to do it. I suppose
> the alternative is for each request, pick a port, do a send using that
> port, then do a separate recv on the same port, but unless you're
> actively trying to work around the wrong IP bug, why would you do
> that?

A single recursive resolver process can make a large number of
outbound requests to thousands (if not more) of nameservers. Keeping
one socket for each unique combination of (resolver IP, nameserver IP)
becomes expensive in such an environment. Using more than one resolver
IP provides additional entropy for the queries.

-Puneet

>
> ___
> dns-operations mailing list
> dns-operations@lists.dns-oarc.net
> https://lists.dns-oarc.net/mailman/listinfo/dns-operations
--- End Message ---
___
dns-operations mailing list
dns-operations@lists.dns-oarc.net
https://lists.dns-oarc.net/mailman/listinfo/dns-operations

Re: [dns-operations] [Ext] Nameserver responses from different IP than destination of request

[Puneet Sood via dns-operations]

--- Begin Message ---
There are 2 sets of nameservers which account for the bulk of this
behavior. I can share some of the nameserver IPs (that have a high
volume of responses from different IPs) directly with operators who
are interested in reviewing these further. Please contact me directly.

-Puneet


On Sun, Sep 6, 2020 at 12:36 AM Brian Somers  wrote:
>
> FWIW, OpenDNS resolvers have always ignored the response source address.
>
> In light of this conversation and in light of RFC 2181 section 4 and RFC 5452
> section 9.1, I’ll create a ticket to change this….
>
> Unfortunately I have not statistics around how many responses we receive
> from differing IPs.
>
> —
> Brian
>
> > On Sep 2, 2020, at 12:38 AM, Thomas Mieslinger  wrote:
> >
> > On 9/1/20 9:15 PM, Andreas Ott wrote:
> >> On Mon, Aug 31, 2020 at 8:00 PM P Vixie  >> > wrote:
> >>[...] the observation that something
> >>
> >>bad is not happening to somebody doesn't mean it's not happening to
> >>anybody.
> >>
> >> May I please ask an operational question to experts: though I am only
> >> running a small number of authoritative and recursive servers, I am
> >> coming up short looking up what logging I need to turn on in BIND 9.16
> >> and what logged strings I need to parse out to see responses coming from
> >> a different IP? I have various log channels enabled per the BIND logging
> >> "FAQ" but either I am missing config bits or the problem does not occur
> >> (on my servers). This is in a network lab setup and I am able to share 
> >> data.
> >
> > I don't think this is implemented in a way need for this kind of
> > analysis in any recursive dns software.
> >
> > I have chosen to do dnscap on the interface with outgoing traffic and
> > may do correlation of request/reponses based on qname/qtype and look for
> > mismatches in dst ip/src ip afterwards.
> >
> > Another option that comes to my mind is to tweak/reuse the collectd dns
> > plugin which also opens the packetflow on a configurable interface with
> > libpcap and may be able to do some online data correlation.
> >
> > Just my 5¢
> >
> > Thomas
> >
> > ___
> > dns-operations mailing list
> > dns-operations@lists.dns-oarc.net
> > https://lists.dns-oarc.net/mailman/listinfo/dns-operations
>
>
> ___
> dns-operations mailing list
> dns-operations@lists.dns-oarc.net
> https://lists.dns-oarc.net/mailman/listinfo/dns-operations

--- End Message ---
___
dns-operations mailing list
dns-operations@lists.dns-oarc.net
https://lists.dns-oarc.net/mailman/listinfo/dns-operations

Re: [dns-operations] [Ext] Nameserver responses from different IP than destination of request

[Puneet Sood via dns-operations]

--- Begin Message ---
On Sat, Aug 29, 2020 at 11:50 AM Paul Hoffman  wrote:
>
> On Aug 28, 2020, at 3:24 PM, Puneet Sood via dns-operations 
>  wrote:
> > We would be interested in hearing other operator's experience here.
> > Are recursive servers seeing similar behavior from authoritative
> > servers? If yes, are you discarding these responses?
> > Are there authoritative server operators who still need the
> > flexibility afforded by RFC 1035?
>
> Please note that Puneet was asking for other operators' experiences, not the 
> opinions of those of us who believe we should tell Google what to do. (And, 
> yes, I certainly put myself in the latter category.) I, too, would like to 
> hear if other resolver operators see this, and if possible to what extent 
> they are seeing it, and if we're really lucky to hear at least a few names 
> for which this is happening. The latter is not to name-and-shame, but instead 
> to be able to talk to the authoritative operators about what their 
> configuration is so that we can maybe guide others away from this path.

Paul is right. We do plan to close this oversight in our
request/response validation. Before doing that we wanted to see how
prevalent it is and not cause unnecessary breakage.

-Puneet

>
> --Paul Hoffman

--- End Message ---
___
dns-operations mailing list
dns-operations@lists.dns-oarc.net
https://lists.dns-oarc.net/mailman/listinfo/dns-operations

Re: [dns-operations] Nameserver responses from different IP than destination of request

[Puneet Sood via dns-operations]

--- Begin Message ---
On Sat, Aug 29, 2020 at 12:18 AM Robert Edmonds  wrote:
>
> Puneet Sood via dns-operations wrote:
> > RFC 1035 section 7.3 (https://tools.ietf.org/html/rfc1035)
> >  Some name servers send their responses from different
> >  addresses than the one used to receive the query.  That is, a
> >  resolver cannot rely that a response will come from the same
> >  address which it sent the corresponding query to.  This name
> >  server bug is typically encountered in UNIX systems.
> >
> > RFC 2181 (https://tools.ietf.org/html/rfc2181#section-4)
> >Most, if not all, DNS clients, expect the address from which a reply
> >is received to be the same address as that to which the query
> >eliciting the reply was sent.  This is true for servers acting as
> >clients for the purposes of recursive query resolution, as well as
> >simple resolver clients.  The address, along with the identifier (ID)
> >in the reply is used for disambiguating replies, and filtering
> >spurious responses.  This may, or may not, have been intended when
> >the DNS was designed, but is now a fact of life.
> >
> >Some multi-homed hosts running DNS servers generate a reply using a
> >source address that is not the same as the destination address from
> >the client's request packet.  Such replies will be discarded by the
> >client because the source address of the reply does not match that of
> >a host to which the client sent the original request.  That is, it
> >appears to be an unsolicited response.
>
> See also RFC 5452 section 9.1
> (https://tools.ietf.org/html/rfc5452#section-9.1) which puts the
> clarification in RFC 2181 into mandatory RFC 2119 language.
>
> 9.1.  Query Matching Rules
>
>A resolver implementation MUST match responses to all of the
>following attributes of the query:
>
>o  Source address against query destination address
>
>o  Destination address against query source address
>
>o  Destination port against query source port
>
>o  Query ID
>
>o  Query name
>
>o  Query class and type
>
>before applying DNS trustworthiness rules (see Section 5.4.1 of
>[RFC2181]).
>
>A mismatch and the response MUST be considered invalid.

Thanks Rob for that reference. It is quite clear. I didn't follow the
"Updated By" links from 1035 to this section :(

Is there an online tool that does mark up on RFCs to show which other
RFCs are referring to specific sections in it?

>
> --
> Robert Edmonds
--- End Message ---
___
dns-operations mailing list
dns-operations@lists.dns-oarc.net
https://lists.dns-oarc.net/mailman/listinfo/dns-operations

[dns-operations] Nameserver responses from different IP than destination of request

[Puneet Sood via dns-operations]

--- Begin Message ---
Hello,

We (Google Public DNS) have noticed some instances of nameserver
responses for a query coming from a different IP. Our initial plan was
to consider these responses invalid and discard them. However after
reading the text in RFC 1035 and the update in RFC 2181, we wanted to
check what other recursive resolvers are seeing and how they are
handling such responses.

RFC 1035 section 7.3 (https://tools.ietf.org/html/rfc1035)
 Some name servers send their responses from different
 addresses than the one used to receive the query.  That is, a
 resolver cannot rely that a response will come from the same
 address which it sent the corresponding query to.  This name
 server bug is typically encountered in UNIX systems.

RFC 2181 (https://tools.ietf.org/html/rfc2181#section-4)
   Most, if not all, DNS clients, expect the address from which a reply
   is received to be the same address as that to which the query
   eliciting the reply was sent.  This is true for servers acting as
   clients for the purposes of recursive query resolution, as well as
   simple resolver clients.  The address, along with the identifier (ID)
   in the reply is used for disambiguating replies, and filtering
   spurious responses.  This may, or may not, have been intended when
   the DNS was designed, but is now a fact of life.

   Some multi-homed hosts running DNS servers generate a reply using a
   source address that is not the same as the destination address from
   the client's request packet.  Such replies will be discarded by the
   client because the source address of the reply does not match that of
   a host to which the client sent the original request.  That is, it
   appears to be an unsolicited response.

Couple of observations:
1. The difference between original and response nameserver IP happens
for a small percentage of total queries (< 0.1%).
2. Where the original and response nameserver IPs are different,
manual analysis of the top few response IPs showed that both IPs are
on the same operator's network. So these are likely responses from the
operator's DNS servers.

We would be interested in hearing other operator's experience here.
Are recursive servers seeing similar behavior from authoritative
servers? If yes, are you discarding these responses?
Are there authoritative server operators who still need the
flexibility afforded by RFC 1035?

Thanks,
Puneet
--- End Message ---
___
dns-operations mailing list
dns-operations@lists.dns-oarc.net
https://lists.dns-oarc.net/mailman/listinfo/dns-operations

Re: [dns-operations] [Ext] OpenDNS, Google, Nominet - New delegation update failure mode

[Puneet Sood via dns-operations]

--- Begin Message ---
On Fri, Apr 3, 2020 at 1:53 PM Paul Hoffman  wrote:
>
> Shouldn't this part of the thread (proposed changes base on an Internet 
> Draft) be in the DNSOP WG in the IETF? Said another way, if you don't move it 
> there soon, when the topic appears there, you'll have to repeat yourselves.

+1. From Google Public DNS perspective, I also have thoughts on this
topic. Would prefer to discuss on the dnsop WG list.


>
> --Paul Hoffman___
> dns-operations mailing list
> dns-operations@lists.dns-oarc.net
> https://lists.dns-oarc.net/mailman/listinfo/dns-operations
--- End Message ---
___
dns-operations mailing list
dns-operations@lists.dns-oarc.net
https://lists.dns-oarc.net/mailman/listinfo/dns-operations

Re: [dns-operations] OpenDNS, Google, Nominet - New delegation update failure mode

[Puneet Sood via dns-operations]

--- Begin Message ---
Pasted wrong output above.

dig @8.8.4.4 shopdisney.co.uk

; <<>> DiG 9.11.5-P4-5.1+build2-Debian <<>> @8.8.4.4 shopdisney.co.uk
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 15107
;; flags: qr rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 512
;; QUESTION SECTION:
;shopdisney.co.uk.  IN  A

;; ANSWER SECTION:
shopdisney.co.uk.   299 IN  A   13.248.150.189
shopdisney.co.uk.   299 IN  A   76.223.18.1

;; Query time: 17 msec
;; SERVER: 8.8.4.4#53(8.8.4.4)
;; WHEN: Thu Apr 02 16:13:12 EDT 2020
;; MSG SIZE  rcvd: 77

On Thu, Apr 2, 2020 at 4:12 PM Puneet Sood  wrote:
>
> Hi Doug,
>
> Google Public DNS resolution is working now.
>
> Google Public DNS is “parent-centric”—meaning that it only uses the
> name servers that are returned in the referral responses from the
> parent zone name servers, and does not make NS queries to this child
> zone. So updating the parent delegation to include both NS sets will
> help with Google Public DNS resolution.
>
> -Puneet
>
> $ dig @8.8.4.4 shopdisney.co.uk ns
>
> ; <<>> DiG 9.11.5-P4-5.1+build2-Debian <<>> @8.8.4.4 shopdisney.co.uk ns
> ; (1 server found)
> ;; global options: +cmd
> ;; Got answer:
> ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 8397
> ;; flags: qr rd ra; QUERY: 1, ANSWER: 6, AUTHORITY: 0, ADDITIONAL: 1
>
> ;; OPT PSEUDOSECTION:
> ; EDNS: version: 0, flags:; udp: 512
> ;; QUESTION SECTION:
> ;shopdisney.co.uk.  IN  NS
>
> ;; ANSWER SECTION:
> shopdisney.co.uk.   167 IN  NS  a12-66.akam.net.
> shopdisney.co.uk.   167 IN  NS  a18-64.akam.net.
> shopdisney.co.uk.   167 IN  NS  a28-65.akam.net.
> shopdisney.co.uk.   167 IN  NS  a1-127.akam.net.
> shopdisney.co.uk.   167 IN  NS  a9-66.akam.net.
> shopdisney.co.uk.   167 IN  NS  a13-67.akam.net.
>
> ;; Query time: 20 msec
> ;; SERVER: 8.8.4.4#53(8.8.4.4)
> ;; WHEN: Thu Apr 02 16:08:57 EDT 2020
> ;; MSG SIZE  rcvd: 178
>
> On Thu, Apr 2, 2020 at 4:01 PM Doug Barton  wrote:
> >
> > Howdy,
> >
> > I redelegated shopdisney.co.uk this morning. I can see that all of the
> > Nominet authorities are returning the correct new NS set, however I have
> > a number of reports of resolution failures. There are resolvers from
> > OpenDNS, Google, Virgin, O2, and others that are not finding any name
> > servers at all, and refusing to re-query. This is causing address record
> > resolution failures for users behind those resolvers.
> >
> > What is odd to me is that earlier this week we cross-pollinated the old
> > and new zone files with both the old and new sets of name servers. I
> > have seen situations in the past where cutting cleanly from one set of
> > name servers to a completely different set has caused problems, so we
> > take this extra step of updating the zones so that no matter what point
> > in the process we're at the resolving name servers will always have at
> > least one good set to query. It's always worked for me in the past.
> >
> > What's even more strange is that we also did shopdisney.it this morning,
> > having done the same preparation, and it's solid as a rock. It's only
> > the CO.UK name that is failing. When querying OpenDNS or Google directly
> > I get the same result when it fails:
> >
> > dig @8.8.4.4 shopdisney.co.uk ns
> > ; <<>> DiG 9.10.6 <<>> @8.8.4.4 shopdisney.co.uk ns
> > ; (1 server found)
> > ;; global options: +cmd
> > ;; Got answer:
> > ;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 1587
> > ;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1
> >
> > ;; OPT PSEUDOSECTION:
> > ; EDNS: version: 0, flags:; udp: 512
> > ;; QUESTION SECTION:
> > ;shopdisney.co.uk.  IN  NS
> >
> > ;; Query time: 501 msec
> > ;; SERVER: 8.8.4.4#53(8.8.4.4)
> > ;; WHEN: Thu Apr 02 12:28:46 PDT 2020
> > ;; MSG SIZE  rcvd: 45
> >
> > The flags are the same for the OpenDNS servers.
> >
> > Has anyone seen this happen before? I've seen plenty of cases where
> > resolvers have hung onto the old NS set for too long (following the
> > parent TTL instead of the child), which is why I have been adding both
> > sets of name servers to both zones in advance of the redelegation. But I
> > have literally never seen a case where a resolver not only has no NS
> > records, but also will not re-query.
> >
> > My first thought was that Nominet withdrew the delegation for a short
> > period, and the resolvers have a negative cache entry, but when doing
> > the UAT this morning I happened to catch the exact point at which they
> > changed. In serial number 1308977661 they had the old NS set, and in
> > 1308977662 they had the new one. So that doesn't seem to be the problem.
> >
> > If anyone from OpenDNS and/or Google can take a look at a resolver that
> > is failing for 

Re: [dns-operations] OpenDNS, Google, Nominet - New delegation update failure mode

[Puneet Sood via dns-operations]

--- Begin Message ---
Hi Doug,

Google Public DNS resolution is working now.

Google Public DNS is “parent-centric”—meaning that it only uses the
name servers that are returned in the referral responses from the
parent zone name servers, and does not make NS queries to this child
zone. So updating the parent delegation to include both NS sets will
help with Google Public DNS resolution.

-Puneet

$ dig @8.8.4.4 shopdisney.co.uk ns

; <<>> DiG 9.11.5-P4-5.1+build2-Debian <<>> @8.8.4.4 shopdisney.co.uk ns
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 8397
;; flags: qr rd ra; QUERY: 1, ANSWER: 6, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 512
;; QUESTION SECTION:
;shopdisney.co.uk.  IN  NS

;; ANSWER SECTION:
shopdisney.co.uk.   167 IN  NS  a12-66.akam.net.
shopdisney.co.uk.   167 IN  NS  a18-64.akam.net.
shopdisney.co.uk.   167 IN  NS  a28-65.akam.net.
shopdisney.co.uk.   167 IN  NS  a1-127.akam.net.
shopdisney.co.uk.   167 IN  NS  a9-66.akam.net.
shopdisney.co.uk.   167 IN  NS  a13-67.akam.net.

;; Query time: 20 msec
;; SERVER: 8.8.4.4#53(8.8.4.4)
;; WHEN: Thu Apr 02 16:08:57 EDT 2020
;; MSG SIZE  rcvd: 178

On Thu, Apr 2, 2020 at 4:01 PM Doug Barton  wrote:
>
> Howdy,
>
> I redelegated shopdisney.co.uk this morning. I can see that all of the
> Nominet authorities are returning the correct new NS set, however I have
> a number of reports of resolution failures. There are resolvers from
> OpenDNS, Google, Virgin, O2, and others that are not finding any name
> servers at all, and refusing to re-query. This is causing address record
> resolution failures for users behind those resolvers.
>
> What is odd to me is that earlier this week we cross-pollinated the old
> and new zone files with both the old and new sets of name servers. I
> have seen situations in the past where cutting cleanly from one set of
> name servers to a completely different set has caused problems, so we
> take this extra step of updating the zones so that no matter what point
> in the process we're at the resolving name servers will always have at
> least one good set to query. It's always worked for me in the past.
>
> What's even more strange is that we also did shopdisney.it this morning,
> having done the same preparation, and it's solid as a rock. It's only
> the CO.UK name that is failing. When querying OpenDNS or Google directly
> I get the same result when it fails:
>
> dig @8.8.4.4 shopdisney.co.uk ns
> ; <<>> DiG 9.10.6 <<>> @8.8.4.4 shopdisney.co.uk ns
> ; (1 server found)
> ;; global options: +cmd
> ;; Got answer:
> ;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 1587
> ;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1
>
> ;; OPT PSEUDOSECTION:
> ; EDNS: version: 0, flags:; udp: 512
> ;; QUESTION SECTION:
> ;shopdisney.co.uk.  IN  NS
>
> ;; Query time: 501 msec
> ;; SERVER: 8.8.4.4#53(8.8.4.4)
> ;; WHEN: Thu Apr 02 12:28:46 PDT 2020
> ;; MSG SIZE  rcvd: 45
>
> The flags are the same for the OpenDNS servers.
>
> Has anyone seen this happen before? I've seen plenty of cases where
> resolvers have hung onto the old NS set for too long (following the
> parent TTL instead of the child), which is why I have been adding both
> sets of name servers to both zones in advance of the redelegation. But I
> have literally never seen a case where a resolver not only has no NS
> records, but also will not re-query.
>
> My first thought was that Nominet withdrew the delegation for a short
> period, and the resolvers have a negative cache entry, but when doing
> the UAT this morning I happened to catch the exact point at which they
> changed. In serial number 1308977661 they had the old NS set, and in
> 1308977662 they had the new one. So that doesn't seem to be the problem.
>
> If anyone from OpenDNS and/or Google can take a look at a resolver that
> is failing for shopdisney.co.uk and tell me what's in the logs I would
> deeply appreciate it. Since I can't figure out what happened, I'm not
> sure how to mitigate it for the next change.
>
> In the past I've taken the intermediate step of also updating the parent
> delegation to include both NS sets, which I plan to do for the next set
> of updates just to be on the safe side, but given this fun new failure
> mode it's not clear to me that even doing that will insulate us.
>
> Any thoughts/help/advice welcome,
>
> Doug
> ___
> dns-operations mailing list
> dns-operations@lists.dns-oarc.net
> https://lists.dns-oarc.net/mailman/listinfo/dns-operations

--- End Message ---
___
dns-operations mailing list
dns-operations@lists.dns-oarc.net
https://lists.dns-oarc.net/mailman/listinfo/dns-operations

Re: [dns-operations] Google DNS Admin

[Puneet Sood via dns-operations]

--- Begin Message ---
[Google Public DNS engineer here]

As others have mentioned this appears to be a connectivity or load
problem. We are getting successful resolution in about 50% of metros
globally and timeouts in the rest of the metros. The locations are
consistent across 3 attempts.

FWIW we see failures in Belgium but not in Amsterdam.

For Google Public DNS specific problems, you can reach us via mailing
list or issue tracker:
https://developers.google.com/speed/public-dns/groups.

Thanks,
Puneet

On Wed, Jan 8, 2020 at 10:26 AM Viktor Dukhovni  wrote:
>
> On Wed, Jan 08, 2020 at 02:57:17PM +, Tony Finch wrote:
>
> > Daniel Corbe  wrote:
> > >
> > > Every well-known recursor is returning valid results for as57335.net
> > > except for 8.8.8.8 and 8.8.4.4 and I'd like some assistance getting
> > > down to the root of the issue.
> >
> > Maybe connectivity problems? I can't get to any of the nameservers from
> > 131.111.0.0/16 or 2a05:b400::/32. DNSviz can see the domain OK but
> > zonemaster cannot.
> >
> > https://dnsviz.net/d/as57335.net/dnssec/
> >
> > https://zonemaster.net/result/0e70c5e9893a0ce8
>
> Same for me, only I see the IPv4 NS hosts in 185.203.204/22:
>
> @a.ns.as57335.net.[185.203.204.10]
> @b.ns.as57335.net.[185.203.205.10]
> @c.ns.as57335.net.[185.203.206.10]
> @d.ns.as57335.net.[185.203.207.10]
> ;; connection timed out; no servers could be reached
>
> @a.ns.as57335.net.[2a0c:d2c0::53:5:7335]
> @b.ns.as57335.net.[2a0c:d2c2::53:5:7335]
> @c.ns.as57335.net.[2a0c:d2c4::53:5:7335]
> @d.ns.as57335.net.[2a0c:d2c6::53:5:7335]
> ;; connection timed out; no servers could be reached
>
> The delegation and glue RRs from .net were:
>
> as57335.net. IN NS a.ns.as57335.net.
> as57335.net. IN NS b.ns.as57335.net.
> as57335.net. IN NS c.ns.as57335.net.
> as57335.net. IN NS d.ns.as57335.net.
> ;
> a.ns.as57335.net. IN A 185.203.204.10
> a.ns.as57335.net. IN  2a0c:d2c0::53:5:7335
> b.ns.as57335.net. IN A 185.203.205.10
> b.ns.as57335.net. IN  2a0c:d2c2::53:5:7335
> c.ns.as57335.net. IN A 185.203.206.10
> c.ns.as57335.net. IN  2a0c:d2c4::53:5:7335
> d.ns.as57335.net. IN A 185.203.207.10
> d.ns.as57335.net. IN  2a0c:d2c6::53:5:7335
>
> --
> Viktor.
> ___
> dns-operations mailing list
> dns-operations@lists.dns-oarc.net
> https://lists.dns-oarc.net/mailman/listinfo/dns-operations
--- End Message ---
___
dns-operations mailing list
dns-operations@lists.dns-oarc.net
https://lists.dns-oarc.net/mailman/listinfo/dns-operations

Re: [dns-operations] sophosxl.net problem?

[Puneet Sood via dns-operations]

--- Begin Message ---
On Wed, Oct 30, 2019 at 7:31 PM Mark Andrews  wrote:
>
>
>
> > On 31 Oct 2019, at 12:02 am, Bob Harold  wrote:
> >
> >
> > On Tue, Oct 29, 2019 at 9:07 PM Paul Vixie  wrote:
> >
> >
> > Mark Andrews wrote on 2019-10-27 19:24:
> > > ...
> > >
> > > BIND tried to fix named to reject AA=0 from authoritative servers a
> > > few years back but pandora.tv was returning AA=0 from all servers at
> > > the time and we had to back the change out.  We still want to make
> > > that change.
> >
> > please consider making this a config option so that those of us who are
> > willing to endure outages for nonconforming domains can turn it on. it
> > could even become part of some annual so-called dns flag day.
> >
> > --
> > P Vixie
> >
> > I agree.
> >
> > But if someone thinks that is too drastic, would it be reasonable to make a 
> > config option, plus an exception list?   Then someone could make exceptions 
> > for the known cases, but break any new cases, to avoid this problem getting 
> > any worse.
> >
> > --
> > Bob Harold
> >
>
> First thing is to get Google, Cloudflare etc. on board.  “But it works using 
> 8.8.8.8 or 1.1.1.1” etc.
> is the biggest problem with actually being able to deploy fixes.  The second 
> problem is being able
> to contact the server administrators.

Can you file a bug at
https://developers.google.com/speed/public-dns/groups#issues?

> --
> Mark Andrews, ISC
> 1 Seymour St., Dundas Valley, NSW 2117, Australia
> PHONE: +61 2 9871 4742  INTERNET: ma...@isc.org
>
>
> ___
> dns-operations mailing list
> dns-operations@lists.dns-oarc.net
> https://lists.dns-oarc.net/mailman/listinfo/dns-operations

--- End Message ---
___
dns-operations mailing list
dns-operations@lists.dns-oarc.net
https://lists.dns-oarc.net/mailman/listinfo/dns-operations

Re: [dns-operations] s3.amazonaws.com problem?

[Puneet Sood via dns-operations]

--- Begin Message ---
Google Public DNS also observed the increased CNAME queries to
s3.amazonaws.com yesterday. This morning we are seeing most normal
queries getting responses from their nameservers.

On Wed, Oct 23, 2019 at 4:31 AM Greg Choules via dns-operations
 wrote:
>
>
>
>
> -- Forwarded message --
> From: Greg Choules 
> To: dns-operations@lists.dns-oarc.net
> Cc:
> Bcc:
> Date: Wed, 23 Oct 2019 09:25:20 +0100
> Subject: s3.amazonaws.com problem?
> Is anyone else experiencing resolution issues for names in this domain? We 
> are seeing a lot of queries of the form:
>
> CNAME? bvusfvyq.s3.amazonaws.com
>
> (the label before ".s3" looks random) and a lot of SERVFAIL responses.
>
> Any clues would be appreciated.
> thanks, Greg
>
>
>
> -- Forwarded message --
> From: Greg Choules via dns-operations 
> To: dns-operations@lists.dns-oarc.net
> Cc:
> Bcc:
> Date: Wed, 23 Oct 2019 09:25:20 +0100
> Subject: [dns-operations] s3.amazonaws.com problem?
> ___
> dns-operations mailing list
> dns-operations@lists.dns-oarc.net
> https://lists.dns-oarc.net/mailman/listinfo/dns-operations
--- End Message ---
___
dns-operations mailing list
dns-operations@lists.dns-oarc.net
https://lists.dns-oarc.net/mailman/listinfo/dns-operations