Re: Proposal to enable WAPBL by default for 10.0
On Sun, Jul 26, 2020 at 11:20:37PM +, m...@netbsd.org wrote: > > To be explicit: > > > > It is the same underly problem either way, and it is worse in practice > > with WAPBL than without because WAPBL ffs runs faster than non-WAPBL > > ffs and thus accumulates more unwritten blocks. > > It looks like this difference is because FFS doesn't flush the disk > cache often, but if WAPBL is enabled, it does on every log write. That would cause WAPBL to generate fewer unwritten blocks, which isn't consistent with the observed results. (Or maybe it is, and without this effect WAPBL would be even worse.) But this is unlikely to be an issue in most cases, because data that makes it to the disk-level cache is not lost just because the kernel panics. You have to turn off the power for that. -- David A. Holland dholl...@netbsd.org
Re: /dev/crypto missing
j...@ziaspace.com (John Klos) writes: >I erroneously thought that if pseudo-device crypto wasn't in the kernel, >crypto would be done in userland. That's not the case: >openssl s_client -debug -connect 192.80.49.7:443 >Could not open /dev/crypto: Device not configured Crypto is done in userland. The error message comes from initializing all builtin crypto engines, whether they get used or not. % openssl engine (devcrypto) /dev/crypto engine (dynamic) Dynamic engine loading support This also means that openssl (the library, not the command) eats one file descriptor if /dev/crypto exists. Only ENOENT is suppressed, deleting the /dev/crypto entry makes the error message go away. You could expand that to also hide ENXIO or rewrite devcrypto as a dynamic engine. -- -- Michael van Elst Internet: mlel...@serpens.de "A potential Snark may lurk in every tree."
Re: /dev/crypto missing
On Tue, Jul 28, 2020 at 01:35:53AM +, Taylor R Campbell wrote: > > /dev/crypto is totally obsolete as it exists today. Really the only > reason it continues to exist is to test opencrypto drivers from > userland before using them in the kernel. This is not really the case. The OpenSSL project has *finally* made the changes to their core TLS state machine required to take advantage of asynchronous crypto via device driver in a performant way. It would now be possible, with a better /dev/crypto ENGINE in OpenSSL, to actually get a pretty good performance bump from hardware accelleration on a number of platforms. Unfortunately, roughly contemporaneously with so doing, they also managed to rewrite their own /dev/crypto engine to a weird variant Linux /dev/crypto API, ignoring the significant enhancements we added in NetBSD about 15 years ago (multiple request submission/retrieval and asynchronous operation). This is particularly frustrating to me since, back then, we (Coyote Point and NBMK) sent them patches for both parts of the puzzle... Anyhow, it's no longer the case that OpenSSL structurally _couldn't_ use /dev/crypto efficiently. But it'd take a second rewrite on their new devvcrypto ENGINE to make it do so. -- Thor Lancelot Simon t...@panix.com "Whether or not there's hope for change is not the question. If you want to be a free person, you don't stand up for human rights because it will work, but because it is right." --Andrei Sakharov
Re: /dev/crypto missing
I erroneously thought that if pseudo-device crypto wasn't in the kernel, crypto would be done in userland. That's not the case: What makes you think crypto isn't being done in userland? Just a bad guess that the reason for pseudo-device crypto was to do some things in the kernel. The problem looks to me like the server returns garbage on a TLS connection, which gets mixed up with an OpenSSL debugging message -- or possibly it is garbage _because_ it got mixed up with the OpenSSL debugging message. Maybe OpenSSL should handle ENXIO quietly like it handles ENOENT there, but it looks like there's a deeper problem if crap that OpenSSL printed got included in the TLS stream! If this is the case, then why isn't crypto in every kernel configuration by default, except perhaps special cases? /dev/crypto is totally obsolete as it exists today. Really the only reason it continues to exist is to test opencrypto drivers from userland before using them in the kernel. Hmmm... Then I wonder what's really going on. This is from trying to use bozohttpd with TLS on an Amiga with exactly the same configuration as used on ARM and amd64. I'll have to look in to this a bit more and perhaps open a PR. Thanks, John
Re: /dev/crypto missing
> Date: Tue, 28 Jul 2020 01:10:34 + (UTC) > From: John Klos > > I erroneously thought that if pseudo-device crypto wasn't in the kernel, > crypto would be done in userland. That's not the case: What makes you think crypto isn't being done in userland? The problem looks to me like the server returns garbage on a TLS connection, which gets mixed up with an OpenSSL debugging message -- or possibly it is garbage _because_ it got mixed up with the OpenSSL debugging message. Maybe OpenSSL should handle ENXIO quietly like it handles ENOENT there, but it looks like there's a deeper problem if crap that OpenSSL printed got included in the TLS stream! > If this is the case, then why isn't crypto in every kernel configuration > by default, except perhaps special cases? /dev/crypto is totally obsolete as it exists today. Really the only reason it continues to exist is to test opencrypto drivers from userland before using them in the kernel.
/dev/crypto missing
Hi, I erroneously thought that if pseudo-device crypto wasn't in the kernel, crypto would be done in userland. That's not the case: openssl s_client -debug -connect 192.80.49.7:443 Could not open /dev/crypto: Device not configured CONNECTED(0003) write to 0xe4f02d0 [0xe546000] (293 bytes => 293 (0x125)) - 16 03 01 01 20 01 00 01-1c 03 03 40 b2 73 a3 d5 ..@.s.. 0010 - 13 f4 91 bb ad cf 6b 49-f1 33 6f 86 ae 5b 1e 1e ..kI.3o..[.. 0020 - f5 cb db 10 5e 27 a5 07-10 97 8d 20 f6 9b 7c 26 ^'. ..|& 0030 - f3 52 e6 e5 19 1e 57 24-c2 ff c7 07 6d 34 23 74 .RW$m4#t 0040 - 6c 36 da 86 f8 39 f9 a8-7e 24 1b 6c 00 3e 13 02 l6...9..~$.l.>.. 0050 - 13 03 13 01 c0 2c c0 30-00 9f cc a9 cc a8 cc aa .,.0 0060 - c0 2b c0 2f 00 9e c0 24-c0 28 00 6b c0 23 c0 27 .+./...$.(.k.#.' 0070 - 00 67 c0 0a c0 14 00 39-c0 09 c0 13 00 33 00 9d .g.9.3.. 0080 - 00 9c 00 3d 00 3c 00 35-00 2f 00 ff 01 00 00 95 ...=.<.5./.. 0090 - 00 0b 00 04 03 00 01 02-00 0a 00 0c 00 0a 00 1d 00a0 - 00 17 00 1e 00 19 00 18-00 23 00 00 00 16 00 00 .#.. 00b0 - 00 17 00 00 00 0d 00 30-00 2e 04 03 05 03 06 03 ...0 00c0 - 08 07 08 08 08 09 08 0a-08 0b 08 04 08 05 08 06 00d0 - 04 01 05 01 06 01 03 03-02 03 03 01 02 01 03 02 00e0 - 02 02 04 02 05 02 06 02-00 2b 00 09 08 03 04 03 .+.. 00f0 - 03 03 02 03 01 00 2d 00-02 01 01 00 33 00 26 00 ..-.3.&. 0100 - 24 00 1d 00 20 74 f9 da-78 03 7e ab f9 52 6d da $... t..x.~..Rm. 0110 - cf 19 9b 11 0d 3c 24 c2-00 44 f1 bf 4b e8 92 33 .<$..D..K..3 0120 - dd 79 33 d7 1e.y3.. read from 0xe4f02d0 [0xe4e7003] (5 bytes => 5 (0x5)) - 43 6f 75 6c 64Could 4294967295:error:1408F10B:SSL routines:ssl3_get_record:wrong version number:/usr/src/crypto/external/bsd/openssl/dist/ssl/record/ssl3_record.c:332: --- no peer certificate available --- No client certificate CA names sent --- SSL handshake has read 5 bytes and written 293 bytes Verification: OK --- New, (NONE), Cipher is (NONE) Secure Renegotiation IS NOT supported Compression: NONE Expansion: NONE No ALPN negotiated Early data was not sent Verify return code: 0 (ok) --- read from 0xe4f02d0 [0xe558000] (8192 bytes => 45 (0x2D)) - 20 6e 6f 74 20 6f 70 65-6e 20 2f 64 65 76 2f 63not open /dev/c 0010 - 72 79 70 74 6f 3a 20 44-65 76 69 63 65 20 6e 6f rypto: Device no 0020 - 74 20 63 6f 6e 66 69 67-75 72 65 64 0at configured. If this is the case, then why isn't crypto in every kernel configuration by default, except perhaps special cases? John Klos
Re: Proposal to enable WAPBL by default for 10.0
On 27/07/2020 11:58, nia wrote: Of course, it would also be nice to have the option of more filesystems in sysinst (ZFS, LFS in 10 assuming the remaining deadlocks mainly effect removable media - taylor?), and a noatime option for flash media. Until we get bootloader support for ZFS we might need to consider teaching sysinst to create a small FFS root and pivot to the real ZFS root. https://wiki.netbsd.org/wiki/RootOnZFS/ That would be nice for 10 at least. Roy
Re: Proposal to enable WAPBL by default for 10.0
It feels like we could avoid the controversy of whether it should be enabled by default by making it an option in sysinst. Of course, it would also be nice to have the option of more filesystems in sysinst (ZFS, LFS in 10 assuming the remaining deadlocks mainly effect removable media - taylor?), and a noatime option for flash media.
