[Touch-packages] [Bug 1898590] Re: Verify DNS fingerprints not working
*** This bug is a duplicate of bug 1897744 *** https://bugs.launchpad.net/bugs/1897744 @Dan - now I saw your update - that might have shortened my dnssec trip :-) It indeed is a duplicate of that - marking as such. ** This bug has been marked a duplicate of bug 1897744 VerifyHostKeyDNS not working due to missing trust-ad flag -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to openssh in Ubuntu. https://bugs.launchpad.net/bugs/1898590 Title: Verify DNS fingerprints not working Status in systemd: Unknown Status in glibc package in Ubuntu: Invalid Status in openssh package in Ubuntu: Invalid Status in systemd package in Ubuntu: Fix Released Status in systemd source package in Focal: New Status in openssh package in Debian: Unknown Bug description: When setting in /etc/ssh/ssh_config VerifyHostKeyDNS to yes the fingerprints are fetched, but the result is always: debug1: found n insecure fingerprints in DNS With dig +dnssec -tsshfp hostname the result is ok: ad flg is set. To manage notifications about this bug go to: https://bugs.launchpad.net/systemd/+bug/1898590/+subscriptions -- Mailing list: https://launchpad.net/~touch-packages Post to : touch-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~touch-packages More help : https://help.launchpad.net/ListHelp
[Touch-packages] [Bug 1898590] Re: Verify DNS fingerprints not working
*** This bug is a duplicate of bug 1897744 *** https://bugs.launchpad.net/bugs/1897744 Summary: - we understand what happened - we have a workaround for users by changing a config - we have a systemd change that can be considered to be backported by the systemd package maintainers ** Tags removed: server-next -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to systemd in Ubuntu. https://bugs.launchpad.net/bugs/1898590 Title: Verify DNS fingerprints not working Status in systemd: Unknown Status in glibc package in Ubuntu: Invalid Status in openssh package in Ubuntu: Invalid Status in systemd package in Ubuntu: Fix Released Status in systemd source package in Focal: New Status in openssh package in Debian: Unknown Bug description: When setting in /etc/ssh/ssh_config VerifyHostKeyDNS to yes the fingerprints are fetched, but the result is always: debug1: found n insecure fingerprints in DNS With dig +dnssec -tsshfp hostname the result is ok: ad flg is set. To manage notifications about this bug go to: https://bugs.launchpad.net/systemd/+bug/1898590/+subscriptions -- Mailing list: https://launchpad.net/~touch-packages Post to : touch-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~touch-packages More help : https://help.launchpad.net/ListHelp
[Touch-packages] [Bug 1898590] Re: Verify DNS fingerprints not working
*** This bug is a duplicate of bug 1897744 *** https://bugs.launchpad.net/bugs/1897744 TL;DR: one affected by this upgrade triggered behavior change needs to set options edns0 trust-ad in /etc/resolv.conf to fix the issue. And as usual, once you already know what things are about - then (but only then :-/ ) you find the important related issues - reported by Laney \o/ Subscribing him to the bug FYI Debian bug https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=960023 I'll add a bug link to Debian. Discussed and declared to be systed issue and asked to file upstream. Which led to ... Systemd https://github.com/systemd/systemd/issues/15767 This is fixed in newer systemd and will set trust-ad I'll add a systemd task to consider backporting this to Focal -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to systemd in Ubuntu. https://bugs.launchpad.net/bugs/1898590 Title: Verify DNS fingerprints not working Status in systemd: Unknown Status in glibc package in Ubuntu: Invalid Status in openssh package in Ubuntu: Invalid Status in systemd package in Ubuntu: Fix Released Status in systemd source package in Focal: New Status in openssh package in Debian: Unknown Bug description: When setting in /etc/ssh/ssh_config VerifyHostKeyDNS to yes the fingerprints are fetched, but the result is always: debug1: found n insecure fingerprints in DNS With dig +dnssec -tsshfp hostname the result is ok: ad flg is set. To manage notifications about this bug go to: https://bugs.launchpad.net/systemd/+bug/1898590/+subscriptions -- Mailing list: https://launchpad.net/~touch-packages Post to : touch-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~touch-packages More help : https://help.launchpad.net/ListHelp
[Touch-packages] [Bug 1898590] Re: Verify DNS fingerprints not working
*** This bug is a duplicate of bug 1897744 *** https://bugs.launchpad.net/bugs/1897744 ok up/downgrading just "libc6" is enough to trigger. I also found that libc6 from Eoan version 2.30-0ubuntu2.2 is good. So it is new in 2.31! The changelog mentions soem DNSSEC https://sourceware.org/legacy-ml/libc-announce/2020/msg1.html "* The DNS stub resolver will optionally send the AD (authenticated data) bit in queries if the trust-ad option is set via the options directive in /etc/resolv.conf (or if RES_TRUSTAD is set in _res.options). In this mode, the AD bit, as provided by the name server, is available to applications which call res_search and related functions. In the default mode, the AD bit is not set in queries, and it is automatically cleared in responses, indicating a lack of DNSSEC validation. (Therefore, the name servers and the network path to them are treated as untrusted.)" Once I knew that it was a small step and I found that options edns0 trust-ad in /etc/resolv.conf indeed fixes the issue. I'm not sure if openssh would be entitled to set RES_TRUSTAD is set in _res.options. Maybe not as that is more a decision of the admin setting up and configuring the system than the openssh software. Therefore I think this is actually a little detail that upgraders that use dnssec for openssh (and maybe others) via libc6 resolv need to consider. ** Bug watch added: Debian Bug tracker #960023 https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=960023 ** Also affects: openssh (Debian) via https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=960023 Importance: Unknown Status: Unknown ** Bug watch added: github.com/systemd/systemd/issues #15767 https://github.com/systemd/systemd/issues/15767 ** Also affects: systemd via https://github.com/systemd/systemd/issues/15767 Importance: Unknown Status: Unknown ** Also affects: systemd (Ubuntu) Importance: Undecided Status: New ** Changed in: systemd (Ubuntu) Status: New => Fix Released ** Also affects: glibc (Ubuntu Focal) Importance: Undecided Status: New ** Also affects: openssh (Ubuntu Focal) Importance: Undecided Status: New ** Also affects: systemd (Ubuntu Focal) Importance: Undecided Status: New ** No longer affects: glibc (Ubuntu Focal) ** Changed in: openssh (Ubuntu) Status: Confirmed => Invalid ** No longer affects: openssh (Ubuntu Focal) ** Changed in: glibc (Ubuntu) Status: New => Invalid -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to openssh in Ubuntu. https://bugs.launchpad.net/bugs/1898590 Title: Verify DNS fingerprints not working Status in systemd: Unknown Status in glibc package in Ubuntu: Invalid Status in openssh package in Ubuntu: Invalid Status in systemd package in Ubuntu: Fix Released Status in systemd source package in Focal: New Status in openssh package in Debian: Unknown Bug description: When setting in /etc/ssh/ssh_config VerifyHostKeyDNS to yes the fingerprints are fetched, but the result is always: debug1: found n insecure fingerprints in DNS With dig +dnssec -tsshfp hostname the result is ok: ad flg is set. To manage notifications about this bug go to: https://bugs.launchpad.net/systemd/+bug/1898590/+subscriptions -- Mailing list: https://launchpad.net/~touch-packages Post to : touch-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~touch-packages More help : https://help.launchpad.net/ListHelp
[Touch-packages] [Bug 1898590] Re: Verify DNS fingerprints not working
As a first step I tried to make sure this actually is a change in openssh. Because my reading of the issues referred above has shown that not all of the verification is done inside ssh but partially in glibc. So I upgraded on the bionic test system step by step. The upgrade dependency list for openssh-client is "libc-bin libc6 libgssapi-krb5-2 libkrb5-3 libkrb5support0 locales" I found that the bug is reproducible with openssh-client 1:7.6p1-4ubuntu0.3 running against the newer glibc. I can switch in/out the reported bug be switching on a Bionic system between glibc 2.27-3ubuntu1.2 - Bionic - Good 2.31-0ubuntu9.1 - Focal - Bad ** Also affects: glibc (Ubuntu) Importance: Undecided Status: New -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to openssh in Ubuntu. https://bugs.launchpad.net/bugs/1898590 Title: Verify DNS fingerprints not working Status in glibc package in Ubuntu: New Status in openssh package in Ubuntu: Confirmed Bug description: When setting in /etc/ssh/ssh_config VerifyHostKeyDNS to yes the fingerprints are fetched, but the result is always: debug1: found n insecure fingerprints in DNS With dig +dnssec -tsshfp hostname the result is ok: ad flg is set. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/glibc/+bug/1898590/+subscriptions -- Mailing list: https://launchpad.net/~touch-packages Post to : touch-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~touch-packages More help : https://help.launchpad.net/ListHelp
[Touch-packages] [Bug 1898590] Re: Verify DNS fingerprints not working
Something helpful for anyone looking into this later I found what seems a good testcase without requiring too much a local setup (of a dnssec dns server): To get unbound (brute force) do: apt install unbound sudo systemctl stop systemd-resolved sudo systemctl disable systemd-resolved sudo systemctl enable unbound-resolvconf sudo systemctl enable unbound # set 127.0.0.1 vim /etc/resolv.conf Now this should show the ad flag as reported: $ dig salsa.debian.org -t sshfp ... ;; flags: qr rd ra ad; QUERY: 1, ANSWER: 4, AUTHORITY: 0, ADDITIONAL: 1 $ ssh -v -o "VerifyHostKeyDNS=yes" t...@salsa.debian.org This indeed (as reported), does show the changed behavior (clean LXD containers, just changes as mentioned above, edns0 set by default): Bionic: debug1: found 4 secure fingerprints in DNS debug1: matching host key fingerprint found in DNS Focal: debug1: found 4 insecure fingerprints in DNS debug1: matching host key fingerprint found in DNS The authenticity of host 'salsa.debian.org (209.87.16.44)' can't be established. ED25519 key fingerprint is SHA256:OAD3pGSwcODIthxF+zIRvPTZ8UCJAYI75E42XDfGr84. Matching host key fingerprint found in DNS. ** Changed in: openssh (Ubuntu) Status: New => Confirmed -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to openssh in Ubuntu. https://bugs.launchpad.net/bugs/1898590 Title: Verify DNS fingerprints not working Status in openssh package in Ubuntu: Confirmed Bug description: When setting in /etc/ssh/ssh_config VerifyHostKeyDNS to yes the fingerprints are fetched, but the result is always: debug1: found n insecure fingerprints in DNS With dig +dnssec -tsshfp hostname the result is ok: ad flg is set. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/openssh/+bug/1898590/+subscriptions -- Mailing list: https://launchpad.net/~touch-packages Post to : touch-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~touch-packages More help : https://help.launchpad.net/ListHelp
[Touch-packages] [Bug 1898590] Re: Verify DNS fingerprints not working
Turns out this seems to be a never ending story and you might have found a comeback of that issue for your particular configuration as you say this worked on 18.04 but fails on 20.04. This goes way back https://bugzilla.mindrot.org/show_bug.cgi?id=1455 Or half way back https://trac.macports.org/ticket/49007 https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=618863 https://bugzilla.mindrot.org/show_bug.cgi?id=2119 Other more recent similar issues were around "options edns0" being required to be set for this to work now: https://github.com/NixOS/nixpkgs/issues/12470 https://exanames.typepad.com/blog/2009/06/one-more-thing-to-do-with-dnssec-ssh.html https://bugzilla.redhat.com/show_bug.cgi?id=1630180 https://bugzilla.redhat.com/show_bug.cgi?id=1878166 Note: that option was the default for /etc/resolv.conf on Bionic/Focal for me. Various working setups seem to have been affected by 7.5 https://lists.mindrot.org/pipermail/openssh-bugs/2017-April/017631.html https://lists.mindrot.org/pipermail/openssh-unix-dev/2018-January/036600.html https://bugzilla.mindrot.org/show_bug.cgi?id=2708 But Bionic -> Focal is openssh version 7.6 -> 8.3 Multiple of the above and some other references refer to requiring ldns support. That clearly is in openssh since ~v6 but we don't enable it at build time libldns support: no Is that required and is it now more required than before - I don't know :-/ Sorry, all that I could provide so far was a collection of a (disturbing) history of that feature. ** Bug watch added: OpenSSH Portable Bugzilla #1455 https://bugzilla.mindrot.org/show_bug.cgi?id=1455 ** Bug watch added: trac.macports.org #49007 http://trac.macports.org/ticket/49007 ** Bug watch added: Debian Bug tracker #618863 https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=618863 ** Bug watch added: OpenSSH Portable Bugzilla #2119 https://bugzilla.mindrot.org/show_bug.cgi?id=2119 ** Bug watch added: github.com/NixOS/nixpkgs/issues #12470 https://github.com/NixOS/nixpkgs/issues/12470 ** Bug watch added: Red Hat Bugzilla #1630180 https://bugzilla.redhat.com/show_bug.cgi?id=1630180 ** Bug watch added: Red Hat Bugzilla #1878166 https://bugzilla.redhat.com/show_bug.cgi?id=1878166 ** Bug watch added: OpenSSH Portable Bugzilla #2708 https://bugzilla.mindrot.org/show_bug.cgi?id=2708 -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to openssh in Ubuntu. https://bugs.launchpad.net/bugs/1898590 Title: Verify DNS fingerprints not working Status in openssh package in Ubuntu: Confirmed Bug description: When setting in /etc/ssh/ssh_config VerifyHostKeyDNS to yes the fingerprints are fetched, but the result is always: debug1: found n insecure fingerprints in DNS With dig +dnssec -tsshfp hostname the result is ok: ad flg is set. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/openssh/+bug/1898590/+subscriptions -- Mailing list: https://launchpad.net/~touch-packages Post to : touch-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~touch-packages More help : https://help.launchpad.net/ListHelp
[Touch-packages] [Bug 1898590] Re: Verify DNS fingerprints not working
this is likely a dup of bug 1897744 -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to openssh in Ubuntu. https://bugs.launchpad.net/bugs/1898590 Title: Verify DNS fingerprints not working Status in openssh package in Ubuntu: New Bug description: When setting in /etc/ssh/ssh_config VerifyHostKeyDNS to yes the fingerprints are fetched, but the result is always: debug1: found n insecure fingerprints in DNS With dig +dnssec -tsshfp hostname the result is ok: ad flg is set. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/openssh/+bug/1898590/+subscriptions -- Mailing list: https://launchpad.net/~touch-packages Post to : touch-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~touch-packages More help : https://help.launchpad.net/ListHelp
[Touch-packages] [Bug 1898590] Re: Verify DNS fingerprints not working
** Tags added: server-next -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to openssh in Ubuntu. https://bugs.launchpad.net/bugs/1898590 Title: Verify DNS fingerprints not working Status in openssh package in Ubuntu: New Bug description: When setting in /etc/ssh/ssh_config VerifyHostKeyDNS to yes the fingerprints are fetched, but the result is always: debug1: found n insecure fingerprints in DNS With dig +dnssec -tsshfp hostname the result is ok: ad flg is set. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/openssh/+bug/1898590/+subscriptions -- Mailing list: https://launchpad.net/~touch-packages Post to : touch-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~touch-packages More help : https://help.launchpad.net/ListHelp
[Touch-packages] [Bug 1898590] Re: Verify DNS fingerprints not working
The DNS queries captured with wireshark ssh to unbound and unbound to world looking correct and allways the AD flag in the responses is set. -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to openssh in Ubuntu. https://bugs.launchpad.net/bugs/1898590 Title: Verify DNS fingerprints not working Status in openssh package in Ubuntu: New Bug description: When setting in /etc/ssh/ssh_config VerifyHostKeyDNS to yes the fingerprints are fetched, but the result is always: debug1: found n insecure fingerprints in DNS With dig +dnssec -tsshfp hostname the result is ok: ad flg is set. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/openssh/+bug/1898590/+subscriptions -- Mailing list: https://launchpad.net/~touch-packages Post to : touch-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~touch-packages More help : https://help.launchpad.net/ListHelp
[Touch-packages] [Bug 1898590] Re: Verify DNS fingerprints not working
With @localhost as parameter it will use the local resolver. Local resolver is unbound. The cr** systemd resolver is disabled. Configuration is exactly same like on another machine where it is working like expected. Only difference: Ubuntu 18.04 instead of 20.04. On 18.04 debug1: found 3 secure fingerprints in DNS With 20.04 debug1: found 3 insecure fingerprints in DNS Also when I change the resolver this does not change. So the resolver seems not to be the problem. -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to openssh in Ubuntu. https://bugs.launchpad.net/bugs/1898590 Title: Verify DNS fingerprints not working Status in openssh package in Ubuntu: New Bug description: When setting in /etc/ssh/ssh_config VerifyHostKeyDNS to yes the fingerprints are fetched, but the result is always: debug1: found n insecure fingerprints in DNS With dig +dnssec -tsshfp hostname the result is ok: ad flg is set. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/openssh/+bug/1898590/+subscriptions -- Mailing list: https://launchpad.net/~touch-packages Post to : touch-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~touch-packages More help : https://help.launchpad.net/ListHelp
[Touch-packages] [Bug 1898590] Re: Verify DNS fingerprints not working
Hello, dig will do dns lookups itself, it doesn't rely on the host resolver configuration. Does your host resolver configuration support dnssec? It might be worth using tcpdump or tshark or wireshark to see if the queries are properly formed, and if the replies are correct. Thanks -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to openssh in Ubuntu. https://bugs.launchpad.net/bugs/1898590 Title: Verify DNS fingerprints not working Status in openssh package in Ubuntu: New Bug description: When setting in /etc/ssh/ssh_config VerifyHostKeyDNS to yes the fingerprints are fetched, but the result is always: debug1: found n insecure fingerprints in DNS With dig +dnssec -tsshfp hostname the result is ok: ad flg is set. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/openssh/+bug/1898590/+subscriptions -- Mailing list: https://launchpad.net/~touch-packages Post to : touch-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~touch-packages More help : https://help.launchpad.net/ListHelp
[Touch-packages] [Bug 1898590] Re: Verify DNS fingerprints not working
ssh version is OpenSSH_8.2p1 Ubuntu-4ubuntu0.1, OpenSSL 1.1.1f 31 Mar 2020 -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to openssh in Ubuntu. https://bugs.launchpad.net/bugs/1898590 Title: Verify DNS fingerprints not working Status in openssh package in Ubuntu: New Bug description: When setting in /etc/ssh/ssh_config VerifyHostKeyDNS to yes the fingerprints are fetched, but the result is always: debug1: found n insecure fingerprints in DNS With dig +dnssec -tsshfp hostname the result is ok: ad flg is set. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/openssh/+bug/1898590/+subscriptions -- Mailing list: https://launchpad.net/~touch-packages Post to : touch-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~touch-packages More help : https://help.launchpad.net/ListHelp
