Re: ntpdate(8) and unbound(8) dependencies during boot
Hi Martin, I obviously have disabled DoH on my browser, as I run my own DNS infrastructure. What sort of information disclosure are you talking about though? Be specific. What are the better solutions? If you think manually editing individual daemon config files to cope with broken clocks is a viable solution, then color me surprised. The idea is that you shouldn't have to be a tough guy with your local NTP server etc to have a working clock lol, it should (within reason) "just work". Regards, Jordan On 2020-10-19 01:41, Martin Husemann wrote: On Sun, Oct 18, 2020 at 02:40:17PM -0700, Jordan Geoghegan wrote: [..] As I see it, it's just a couple TLS handshakes which look identical to DNS over HTTPS traffic (which use the ubiquitous port 443). Heh, that is kinda funny. If you haven't disabled DNS over HTTPS network wide you certainly will not care about this information disclosure. I am very glad that the Mozilla folks made this easy to do with DNS tricks (so I could do it even for remote networks w/o site visit or using remote hands on every windows machine). Unless there's something I'm missing (or that the paranoiacs failed to address) I'm pretty sure this is one of the only viable solutions for combating the chicken and egg clock problem TODAY. This thread had several (from my POV) better ones already, but they all have the downside of needing local setup / configuration. Which I don't consider a big deal (or even a plus). However, it it totaly fine to behave like you described for all users unable to provide the needed services localy or conciously choses not to - as long as the rope is provided to override things and go with a better (according to local metrics, for the local setup) solution. Martin
Re: ntpdate(8) and unbound(8) dependencies during boot
On 2020-10-17 05:54, Martin Husemann wrote: On Sat, Oct 17, 2020 at 01:16:57PM +0100, Sad Clouds wrote: I'm not an expert on NTP, but what sort of information do you think it could leak that could compromise your system security? There are ways for hackers to abuse NTP protocol, but that is where you should be using NTS extensions. No, this is not about NTP, but leaking the information that an RTC-less device booted and did the magic handshake to get time correct (by sending standard queries to cloudflare). I actually do run a local ntpd inside the protected network, which is why I'm happy with hard coded or dhcp provided (local) IPs. Martin Hi Martin, There is no "magic handshake", it's a standard TLS handshake, and the traffic should look the same as any old DNS over HTTPS query (as done by default by Firefox, Chrome and a number of other products now adays). There's also nothing that "leaks" the fact you have an RTC-less machine - by default, the HTTPS constraint checks are performed for all machines, in an attempt to mitigate possible NTP man-in-the-middle attacks (see my previous email that mentions the two complementary use case scenarios for this solution). Also, nobody seems to have a problem using fixed IP addresses for DNS servers (whats the alternative?) So I don't think including a couple trustworthy providers (that are easily changeable if desired) would be that big of a deal. The paranoiacs over in OpenBSD land seemed to think this solution was adequate, so I image any privacy or information disclosure possibilities would have been addressed. As I see it, it's just a couple TLS handshakes which look identical to DNS over HTTPS traffic (which use the ubiquitous port 443). Unless there's something I'm missing (or that the paranoiacs failed to address) I'm pretty sure this is one of the only viable solutions for combating the chicken and egg clock problem TODAY. Regards, Jordan
Re: ntpdate(8) and unbound(8) dependencies during boot
On 2020-10-15 00:55, Sad Clouds wrote: On Wed, 14 Oct 2020 16:28:22 -0700 Jordan Geoghegan wrote: 1) Have ntp daemon check various trusted http/https servers at boot to sanity check our clock and NTP data (no DNS needed, fall back to HTTP only if clock is too broken to negotiate TLS) 2) Enjoy not having everything break on boot due to unfortunate lack of RTC Regards, Jordan [1] https://man.openbsd.org/ntpd [2] https://marc.info/?l=openbsd-tech=142363400330522=2 Hi, you say working DNS is not needed, so are you saying that OpenBSD default ntpd config comes with a set of static IP addresses that point to NTP servers running via https protocol? Not exactly, there are no NTP servers running over HTTP, it's a similar concept to the tlsdate util [1]. Basically all it's doing is extracting datestamps from the handshakes with the web servers, and comparing it to the data it's receiving via NTP (if any). What's nice about this, is that because of DNS over HTTPS, there's a number of highly available IP endpoints that have had TLS certs issued to them, such as Quad9's 9.9.9.9 and Cloudflare's 1.1.1.1, 1.0.0.1 etc By having all this fancy footwork done in one daemon (ntpd), it avoids having to mess around with individual daemons like unbound in a vain attempt to cope with broken clocks. None of this is in any RFC, and may very well break in the future, but at least it's a working solution for right now until the big brains can engineer a proper, purpose-built solution. Regards, Jordan [1] https://github.com/ioerror/tlsdate
Re: ntpdate(8) and unbound(8) dependencies during boot
Hello, On 2020-10-11 08:47, Sad Clouds wrote: On Sun, 11 Oct 2020 09:40:36 -0400 Greg Troxel wrote: So, this is a request to explain how a 'default install' has this problem, or to clarify the problem statement. Well NetBSD-9 comes with "unbound" which is supposed to replace "bind" as a recursive/caching name server. If you care about security, then you will always use DNSSEC and DoT, which (in my opinion) should be configured by default. Think of it as http vs https and how most people are now using https by default. Whether NetBSD default install configures those features, is a completely different matter. There is a known issue (which is not exclusive to NetBSD, nor to unbound) that revolves around a circular dependency with ntpdate/ntpd and DNSSEC. There are several ways to work around this issue. The fact that NetBSD does not enable DNSSEC by default, should not preclude it from implementing or documenting a work around. The default install is relying on "XXX.netbsd.pool.ntp.org" hostnames in /etc/ntp.conf for both ntpdate and ntpd. This fails to work correctly when two conditions occur at the same time: a) DNSSEC is used and b) System time is incorrect as hostnames cannot be resolved, due to DNSSEC signature validation failures, I think. This failure is not very obvious and only noticeable when system time is wrong by some specific value, which depends on the configuration of the name server (could be minutes or hours or days). Ideally ntpdate/ntpd should have a backup list of servers that are not hostnames, but IP addresses and don't require functioning DNS. If this can be automated via rc scripts, then it's one less thing to remember for NetBSD users. Could be as simple as adding a few stable IP addresses to /etc/ntp.conf and then marking hostnames as "prefer". OpenBSD has already solved the circular dependency issue by doing constraint validation against trusted HTTP/HTTPS servers (9.9.9.9 and 2620:fe::fe by default). "|ntpd| makes efforts to verify and correct the time at boot if constraints are configured and satisfied or if trusted servers or sensors return results, and if the clock is not being moved backwards." [1] To avoid the chicken and egg problem and issues with ntpd+DNS etc, OpenNTPD does constraint validation against Quad9 which is one of the few sites that has a TLS cert issued to its IP address. The benefits of this solution are two-fold, namely fixing the chicken and the egg clock problem, as well as helping against NTP MITM issues. The idea here is that if your DNS and NTP is broken because of lack of an RTC, on boot ntpd can at least get some idea as to what time it is based on the dates it receives in HTTP(S) response headers from trusted sources. Quad9, Cloudflare, Google etc basically anybody with good availability and who you would trust not to mess with their clocks should suffice for constraint validation. If the big boys are all reporting similar times in their HTTP/HTTPS headers, then it should be safe to set the date accordingly at boot. What's nice about this is that we can use TLS if the clock isn't too out of whack to negotiate a TLS session, and can fall back to HTTP only if necessary. Under normal operating procedures, the HTTP date info is not used to set the clock, its only a last resort. Under normal circumstances the HTTP date info is only used for constraint validation against traditional NTP data. The other benefit of constraint validation is that we can sanity check the time data we receive over NTP. If the big boys are all saying the date is "A" and the NTP pool is saying its "B", if there's a big enough skew, then it's likely the NTP pool is either lying or being man in the middled. This solution has greatly helped on my Edgerouter Lites that do not have an RTC. Before these changes were added to ntpd, I had to call rdate from a cron job and/or use ntpd -s otherwise my unbound DoT setup would fail. tl;dr: 1) Have ntp daemon check various trusted http/https servers at boot to sanity check our clock and NTP data (no DNS needed, fall back to HTTP only if clock is too broken to negotiate TLS) 2) Enjoy not having everything break on boot due to unfortunate lack of RTC Regards, Jordan [1] https://man.openbsd.org/ntpd [2] https://marc.info/?l=openbsd-tech=142363400330522=2
Re: Drive ID changed
On 2020-09-28 15:25, Ottavio Caruso wrote: On 28/09/2020 09:47, Jordan Geoghegan wrote: I found this snippet in some NetBSD documentation: "You will the be asked if you want to use DUID notation in /etc/fstab, instead of traditional device names. You are strongly advised to use DUIDs, as they allow you to move your disks to different controllers, or change their bus identifiers, without having to modify /etc/fstab every time your configuration changes." You should be able to determine your drives DUID using the disklabel command. I've done some googling and the text above comes from some old version of OpenBSD, not NetBSD. You're right. Woops, my bad. Sorry for the noise.
Re: Drive ID changed
On 2020-09-27 18:23, Jordan Geoghegan wrote: On 2020-09-27 16:37, Todd Gruhn wrote: I recabled the SSD and mechanical hard drives. When I start NetBSD from the boot menu, NetBSD gets to the end and gives this message: Starting root file system check: fsck: no match for 'wd0a': No such process Automatic file system check failed, help! Its just cabling. Is there a difference between SCSI and SATA that I dont know about? Isn't this why duids exist? I'm not sure about NetBSD, but in OpenBSD land, /etc/fstab should reference drives based on their duid rather than raw device number to avoid scenarios exactly like this. I found this snippet in some NetBSD documentation: "You will the be asked if you want to use DUID notation in /etc/fstab, instead of traditional device names. You are strongly advised to use DUIDs, as they allow you to move your disks to different controllers, or change their bus identifiers, without having to modify /etc/fstab every time your configuration changes." You should be able to determine your drives DUID using the disklabel command.
Re: Drive ID changed
On 2020-09-27 16:37, Todd Gruhn wrote: I recabled the SSD and mechanical hard drives. When I start NetBSD from the boot menu, NetBSD gets to the end and gives this message: Starting root file system check: fsck: no match for 'wd0a': No such process Automatic file system check failed, help! Its just cabling. Is there a difference between SCSI and SATA that I dont know about? Isn't this why duids exist? I'm not sure about NetBSD, but in OpenBSD land, /etc/fstab should reference drives based on their duid rather than raw device number to avoid scenarios exactly like this.
Re: OT: User-friendly /bin/sh
On 2020-07-31 11:58, Bob Bernstein wrote: *** Mandatory Trigger Warning /begin *** Brain-dead noobie post dead ahead! *** Mandatory Trigger Warning /end *** I have learned the hard way not to mess with the shell superuser is supposed to use. I rarely go to su, but it would be pleasing if, when I do, I had two features available from the shell: filename completion, and some sort of command history and recall. Can some Good Samaritan please point me to a HOW-TO that would provide the needed basic instructions? Thank you. You shouldn't need to use su, as sudo and doas are available in the package repository/ports tree. If you really want to run a different shell as the root user when you use su, you can just call the shell you want to use: $ su # zsh % Regards, Jordan
pf-badhost + unbound adblock v4 adds support for NetBSD
Hey folks, just thought I'd share with you that I've released the latest
versions of pf-badhost and unbound-adblock which add support for NetBSD :)
pf-badhost webpage: https://www.geoghegan.ca/pfbadhost.html
unbound-adblock webage: https://www.geoghegan.ca/unbound-adblock.html
Key pf-badhost changes:
* pf-badhost goes portable, we now support {Open,Free,Net,Dragonfly}BSD
as well as MacOS!
* Support for IPv6 subnet aggregation added
* Greatly improved IPv6 handling in general
* User configuration section added for configuring whitelists and custom
blocklists
* Bogon filtering added
* Greatly improved error handling
Key unbound-adblock changes:
* unbound-adblock goes portable, we now support
{Open,Free,Net,Dragonfly}BSD as well as Linux!
* Greatly improved error handling and input sanitation
* User configuration section added for configuring whitelists and custom
blocklists
pf-badhost changelog:
https://www.geoghegan.ca/pub/pf-badhost/0.4/changelog.txt
unbound-adblock changelog:
https://www.geoghegan.ca/pub/unbound-adblock/0.4/changelog.txt
Re: sha-512 efficiency
On 2020-01-04 04:36, Sad Clouds wrote: Not tried ZFS on NetBSD yet, but I was wondering about the efficiency of the various hash functions. ZFS can use fletcher4, sha-256, sha-512, etc. in order to verify its data integrity. Some of them may or may not be implemented on different platforms, but the most secure hash function seems to be sha-512. The problem with sha-512 is that it is quite slow, even on 64-bit Intel CPUs. Not exactly what you want when checksumming large numbers of blocks of data on a file system. I'm not a cryptographer, but from my limited knowledge I assume that cryptographic hash function like sha-512 are designed around the following principles: 1. It should be extremely difficult to reverse the hash value in order to get the original data. 2. It should be extremely difficult to get collisions, where the same hash value maps to different blocks of data. So when looking at ZFS (or any other file system that checksums data) I would say that for the purpose of checksumming (and not encrypting data on disk) principle No 1 is completely unnecessary. I think principle No 2 is the most important, as the less collisions we get, the better we can detect data corruption/modification. And so, it makes me wonder if sha-512 is too complex, when it doesn't need to be. I think of hash functions as lottery ball machines, which mix numbers in random order. If you run the machine for 60 seconds it will mix all numbers pretty well, but I'm pretty sure you are not going to get any more randomness if you run the machine for say 6 hours. After a certain amount of work you reach the point of diminishing returns. Doe anyone know if this point of diminishing returns has been validated in practice for sha-512 vs other functions? Mathematicians can do theoretical analysis, but are there practical tests which show that after a set of mixing operations, the randomness remains static and does not increase any more? SHA512 tends to be faster on 64bit processors and when hashing larger block sizes. You can test your machines hashing speed using the command: "openssl speed sha256 sha512" I tested this on my laptop (HP Probook 640 G4 with 8th Gen i5 running OpenBSD 6.6-stable) and it seems that SHA512 is faster when hashing block sizes larger than 1024 bytes: probook$ openssl speed sha256 sha512 Doing sha256 for 3s on 16 size blocks: 7092346 sha256's in 3.01s Doing sha256 for 3s on 64 size blocks: 4914063 sha256's in 3.01s Doing sha256 for 3s on 256 size blocks: 1688215 sha256's in 3.02s Doing sha256 for 3s on 1024 size blocks: 179917 sha256's in 3.03s Doing sha256 for 3s on 8192 size blocks: 24795 sha256's in 3.04s Doing sha512 for 3s on 16 size blocks: 1407346 sha512's in 3.02s Doing sha512 for 3s on 64 size blocks: 1757133 sha512's in 3.01s Doing sha512 for 3s on 256 size blocks: 836394 sha512's in 3.01s Doing sha512 for 3s on 1024 size blocks: 320466 sha512's in 3.01s Doing sha512 for 3s on 8192 size blocks: 57414 sha512's in 2.98s LibreSSL 3.0.2 built on: date not available options:bn(64,64) rc4(16x,int) des(idx,cisc,16,int) aes(partial) idea(int) blowfish(idx) compiler: information not available The 'numbers' are in 1000s of bytes per second processed. type 16 bytes 64 bytes 256 bytes 1024 bytes 8192 bytes sha256 37700.18k 104485.06k 143106.97k 60803.63k 66816.00k sha512 7456.14k 37360.97k 71135.17k 109022.32k 157830.70k When I run the same test on a 2nd Gen i5 desktop and an Edgerouter Lite [featuring a MIPS64 cpu] (both also running OpenBSD 6.6 - stable) SHA512 is faster on block sizes over 16 bytes for both machines ( ie SHA512 is faster in all tests but one): # 2nd Gen i5 Desktop: LibreSSL 3.0.2 built on: date not available options:bn(64,64) rc4(16x,int) des(idx,cisc,16,int) aes(partial) idea(int) blowfish(idx) compiler: information not available The 'numbers' are in 1000s of bytes per second processed. type 16 bytes 64 bytes 256 bytes 1024 bytes 8192 bytes sha256 34938.55k 90543.08k 169867.23k 223988.26k 247599.08k sha512 28363.61k 112817.63k 210125.65k 320487.51k 381940.43k # MIPS64 Edgerouter Lite: LibreSSL 3.0.2 built on: date not available options:bn(64,64) rc4(ptr,int) des(ptr,risc2,2,int) aes(partial) idea(int) blowfish(ptr) compiler: information not available The 'numbers' are in 1000s of bytes per second processed. type 16 bytes 64 bytes 256 bytes 1024 bytes 8192 bytes sha256 2388.76k 5985.47k 11379.24k 14386.52k 15311.69k sha512 2012.78k 8230.70k 13971.58k 20825.82k 24665.81k So in short, hashing performance can vary quite largely depending on your machine. The only way to to know is to test for yourself, but a simple rule of thumb would be to assume that SHA512 will be faster for most workloads.
Re: About using NetBSD as a guest, why, how etc.
On 2019-12-09 23:24, Mayuresh wrote: Considering this because I got a new hardware on which a few things don't work on NetBSD (1. wifi: can live with using mobile with tethering; 2. touchpad: I anyway prefer normal mouse and use that also very less, so ok; 3. hdmi: that's a deal breaker as I need laptop to connect to hdmi for my normal usage). To solve all these, could I make NetBSD a guest and what are some good options? Is it Linux+VirtualBox, any specific Linux flavor would be good to act merely as host by and large retaining NetBSD feel of the system? Mayuresh If you need to run Linux as the host due to driver issues, I would recommend using a nice minimalist distro such as Void Linux or Alpine Linux and use QEMU/KVM as the hypervisor. You might also have some luck using FreeBSD/bhyve.
