Re: [PATCH v36 00/13] /dev/random - a new approach
On Mon, Nov 02, 2020 at 02:44:35PM +0100, Torsten Duwe wrote: > > Ted, if you don't have the time any more to take care of /dev/random, > it's not a shame to hand over maintainership, especially given your > long history of Linux contributions. > > Please do seriously consider to hand it over to someone new. This would > be a good opportunity. I can see you are quite busy working on ext4, and there is a number of patches for drivers/char/random.c awaiting review. Wouldn't it be good to pass it on to someone more enthusiastic? At least some sort of reply would be appreciated. Or are you already pondering the request ;-) ? Torsten
Re: [PATCH v36 00/13] /dev/random - a new approach
Am Montag, 19. Oktober 2020, 21:28:50 CET schrieb Stephan Müller: Hi, > > * Performance > > - Faster by up to 75% in the critical code path of the interrupt handler >depending on data collection size configurable at kernel compile time - >the default is about equal in performance with existing /dev/random as >outlined in [2] section 4.2. By streamlining the implementation a bit, the LRNG interrupt handler now operates about 130% faster than the existing /dev/random (average of 97 cycles of the existing /dev/random code vs. an average of 42 cycles of the LRNG). This fast operation is the default now due to patch [2]. The conceptual data handling outlined in [3] section 2.2 remains unchanged. Even the addition of health tests applied to the noise source data would still result in a faster interrupt handling code (average of 97 cycles of the existing /dev/random code vs on average 78 cycles of the LRNG). [1] https://github.com/smuellerDD/lrng/commit/ 10b74b242950371273e38df78060e258d9d3ea40 [2] https://github.com/smuellerDD/lrng/commit/ 383b087653c21cf20984f5508befa57e96f685ba [3] https://chronox.de/lrng/doc/lrng.pdf Ciao Stephan
Re: [PATCH v36 00/13] /dev/random - a new approach
On Mon, Nov 02, 2020 at 02:44:35PM +0100, Torsten Duwe wrote: > On Wed, 28 Oct 2020 19:07:28 +0100 > Greg Kroah-Hartman wrote: > > > On Wed, Oct 28, 2020 at 06:51:17PM +0100, Torsten Duwe wrote: > > > On Mon, 19 Oct 2020 21:28:50 +0200 > > > Stephan Müller wrote: > > > [...] > > > > * Sole use of crypto for data processing: > > > [...] > > > > - The LRNG uses only properly defined and implemented > > > > cryptographic algorithms unlike the use of the SHA-1 > > > > transformation in the existing /dev/random implementation. > > > > > > > > - Hash operations use NUMA-node-local hash instances to benefit > > > > large parallel systems. > > > > > > > > - LRNG uses limited number of data post-processing steps > > > [...] > > > > * Performance > > > > > > > > - Faster by up to 75% in the critical code path of the interrupt > > > > handler depending on data collection size configurable at kernel > > > > compile time - the default is about equal in performance with > > > > existing /dev/random as outlined in [2] section 4.2. > > > > > > [...] > > > > - ChaCha20 DRNG is significantly faster as implemented in the > > > > existing /dev/random as demonstrated with [2] table 2. > > > > > > > > - Faster entropy collection during boot time to reach fully > > > > seeded level, including on virtual systems or systems with SSDs as > > > > outlined in [2] section 4.1. > > > > > > > > * Testing > > > [...] > > > > > > So we now have 2 proposals for a state-of-the-art RNG, and over a > > > month without a single comment on-topic from any `get_maintainer.pl` > > > > > > I don't want to emphasise the certification aspects so much. The > > > interrelation is rather that those certifications require certain > > > code features, features which are reasonable per se. But the > > > current code is lagging way behind. > > > > > > I see the focus namely on performance, scalability, testability and > > > virtualisation. And it certainly is an advantage to use the code > > > already present under crypto, with its optimisations, and not rely > > > on some home brew. > > > > > > Can we please have a discussion about how to proceed? > > > Ted, Greg, Arnd: which approach would you prefer? > > > > Greg and Arnd are not the random driver maintainers, as is now > > correctly shown in the 5.10-rc1 MAINTAINERS file, so I doubt we (well > > at least I) have any say here, sorry. > > No problem. get_maintainer (for the proposals) works on paths, not on > topics and I didn't want to leave anybody out. > > Ted, if you don't have the time any more to take care of /dev/random, > it's not a shame to hand over maintainership, especially given your > long history of Linux contributions. > > Please do seriously consider to hand it over to someone new. This would > be a good opportunity. > > Torsten > I'd like to help with any solution upstream decide to follow either testing or with code. I understand some of the concerns the community has regarding FIPS but that doesn't make it less relevant and it's totally possible to improve /dev/random while allowing it users to decide if they want to comply to SP 800 90B. I believe the main blocker now is the lack of direction. -- Regards, Marcelo signature.asc Description: PGP signature
Re: [PATCH v36 00/13] /dev/random - a new approach
On Wed, 28 Oct 2020 19:07:28 +0100 Greg Kroah-Hartman wrote: > On Wed, Oct 28, 2020 at 06:51:17PM +0100, Torsten Duwe wrote: > > On Mon, 19 Oct 2020 21:28:50 +0200 > > Stephan Müller wrote: > > [...] > > > * Sole use of crypto for data processing: > > [...] > > > - The LRNG uses only properly defined and implemented > > > cryptographic algorithms unlike the use of the SHA-1 > > > transformation in the existing /dev/random implementation. > > > > > > - Hash operations use NUMA-node-local hash instances to benefit > > > large parallel systems. > > > > > > - LRNG uses limited number of data post-processing steps > > [...] > > > * Performance > > > > > > - Faster by up to 75% in the critical code path of the interrupt > > > handler depending on data collection size configurable at kernel > > > compile time - the default is about equal in performance with > > > existing /dev/random as outlined in [2] section 4.2. > > > > [...] > > > - ChaCha20 DRNG is significantly faster as implemented in the > > > existing /dev/random as demonstrated with [2] table 2. > > > > > > - Faster entropy collection during boot time to reach fully > > > seeded level, including on virtual systems or systems with SSDs as > > > outlined in [2] section 4.1. > > > > > > * Testing > > [...] > > > > So we now have 2 proposals for a state-of-the-art RNG, and over a > > month without a single comment on-topic from any `get_maintainer.pl` > > > > I don't want to emphasise the certification aspects so much. The > > interrelation is rather that those certifications require certain > > code features, features which are reasonable per se. But the > > current code is lagging way behind. > > > > I see the focus namely on performance, scalability, testability and > > virtualisation. And it certainly is an advantage to use the code > > already present under crypto, with its optimisations, and not rely > > on some home brew. > > > > Can we please have a discussion about how to proceed? > > Ted, Greg, Arnd: which approach would you prefer? > > Greg and Arnd are not the random driver maintainers, as is now > correctly shown in the 5.10-rc1 MAINTAINERS file, so I doubt we (well > at least I) have any say here, sorry. No problem. get_maintainer (for the proposals) works on paths, not on topics and I didn't want to leave anybody out. Ted, if you don't have the time any more to take care of /dev/random, it's not a shame to hand over maintainership, especially given your long history of Linux contributions. Please do seriously consider to hand it over to someone new. This would be a good opportunity. Torsten
Re: [PATCH v36 00/13] /dev/random - a new approach
On Mon, 19 Oct 2020 21:28:50 +0200 Stephan Müller wrote: [...] > * Sole use of crypto for data processing: [...] > - The LRNG uses only properly defined and implemented cryptographic >algorithms unlike the use of the SHA-1 transformation in the > existing /dev/random implementation. > > - Hash operations use NUMA-node-local hash instances to benefit large >parallel systems. > > - LRNG uses limited number of data post-processing steps [...] > * Performance > > - Faster by up to 75% in the critical code path of the interrupt > handler depending on data collection size configurable at kernel > compile time - the default is about equal in performance with > existing /dev/random as outlined in [2] section 4.2. [...] > - ChaCha20 DRNG is significantly faster as implemented in the > existing /dev/random as demonstrated with [2] table 2. > > - Faster entropy collection during boot time to reach fully seeded >level, including on virtual systems or systems with SSDs as > outlined in [2] section 4.1. > > * Testing [...] So we now have 2 proposals for a state-of-the-art RNG, and over a month without a single comment on-topic from any `get_maintainer.pl` I don't want to emphasise the certification aspects so much. The interrelation is rather that those certifications require certain code features, features which are reasonable per se. But the current code is lagging way behind. I see the focus namely on performance, scalability, testability and virtualisation. And it certainly is an advantage to use the code already present under crypto, with its optimisations, and not rely on some home brew. Can we please have a discussion about how to proceed? Ted, Greg, Arnd: which approach would you prefer? Torsten
Re: [PATCH v36 00/13] /dev/random - a new approach
On Wed, Oct 28, 2020 at 06:51:17PM +0100, Torsten Duwe wrote: > On Mon, 19 Oct 2020 21:28:50 +0200 > Stephan Müller wrote: > [...] > > * Sole use of crypto for data processing: > [...] > > - The LRNG uses only properly defined and implemented cryptographic > >algorithms unlike the use of the SHA-1 transformation in the > > existing /dev/random implementation. > > > > - Hash operations use NUMA-node-local hash instances to benefit large > >parallel systems. > > > > - LRNG uses limited number of data post-processing steps > [...] > > * Performance > > > > - Faster by up to 75% in the critical code path of the interrupt > > handler depending on data collection size configurable at kernel > > compile time - the default is about equal in performance with > > existing /dev/random as outlined in [2] section 4.2. > > [...] > > - ChaCha20 DRNG is significantly faster as implemented in the > > existing /dev/random as demonstrated with [2] table 2. > > > > - Faster entropy collection during boot time to reach fully seeded > >level, including on virtual systems or systems with SSDs as > > outlined in [2] section 4.1. > > > > * Testing > [...] > > So we now have 2 proposals for a state-of-the-art RNG, and over a month > without a single comment on-topic from any `get_maintainer.pl` > > I don't want to emphasise the certification aspects so much. The > interrelation is rather that those certifications require certain code > features, features which are reasonable per se. But the current code is > lagging way behind. > > I see the focus namely on performance, scalability, testability and > virtualisation. And it certainly is an advantage to use the code > already present under crypto, with its optimisations, and not rely > on some home brew. > > Can we please have a discussion about how to proceed? > Ted, Greg, Arnd: which approach would you prefer? Greg and Arnd are not the random driver maintainers, as is now correctly shown in the 5.10-rc1 MAINTAINERS file, so I doubt we (well at least I) have any say here, sorry. good luck! greg k-h
[PATCH v36 00/13] /dev/random - a new approach
Hi, The following patch set provides a different approach to /dev/random which is called Linux Random Number Generator (LRNG) to collect entropy within the Linux kernel. It provides the same API and ABI and can be used as a drop-in replacement. The LRNG implements at least all features of the existing /dev/random such as NUMA-node-local DRNGs. Patches 1 through 3 provide the code that is feature- identical. The following advantages compared to the existing /dev/random implementation are present: * Sole use of crypto for data processing: - Exclusive use of a hash operation for conditioning entropy data with a clear mathematical description as given in [2] section 2.2 - non-cryptographic operations like LFSR are not used. - The LRNG uses only properly defined and implemented cryptographic algorithms unlike the use of the SHA-1 transformation in the existing /dev/random implementation. - Hash operations use NUMA-node-local hash instances to benefit large parallel systems. - LRNG uses limited number of data post-processing steps as documented in [2] section 2.2 compared to the large variation of different post-processing steps in the existing /dev/random implementation that have no apparent mathematical description (see [2] section 4.5). * Performance - Faster by up to 75% in the critical code path of the interrupt handler depending on data collection size configurable at kernel compile time - the default is about equal in performance with existing /dev/random as outlined in [2] section 4.2. - Configurable data collection sizes to accommodate small environments and big environments via CONFIG_LRNG_COLLECTION_SIZE. - Entropy collection using an almost never contended lock to benefit large parallel systems – worst case rate of contention is the number of DRNG reseeds, usually the number of potential contentions per 10 minutes is equal to number of NUMA nodes. - ChaCha20 DRNG is significantly faster as implemented in the existing /dev/random as demonstrated with [2] table 2. - Faster entropy collection during boot time to reach fully seeded level, including on virtual systems or systems with SSDs as outlined in [2] section 4.1. * Testing - Availablility of run-time health tests of the raw unconditioned noise source to identify degradation of the available entropy as documented in [2] section 2.5.4. Such health tests are important today due to virtual machine monitors reducing the resolution of or disabling the high-resolution timer. - Heuristic entropy estimation is based on quantitative measurements and analysis following SP800-90B and not on coincidental underestimation of entropy applied by the existing /dev/random as outlined in [4] section 4.4. - Power-on self tests for critical deterministic components (ChaCha20 DRNG, software hash implementation, and entropy collection logic) not already covered by power-up tests of the kernel crypto API as documented in [2] section 2.14. - Availability of test interfaces for all operational stages of the LRNG including boot-time raw entropy event data sampling as outlined in [2] section 2.15. - Fully testable ChaCha20 DRNG via a userspace ChaCha20 DRNG implementation [3]. - In case of using the kernel crypto API SHASH hash implementation, it is fully testable and tested via the NIST ACVP test framework, for example certificates A734, A737, and A738. - The LRNG offers a test interface to validate the used software hash implementation and in particular that the LRNG invokes the hash correctly, allowing a NIST ACVP-compliant test cycle - see [2] section 2.15. - Availability of stress testing covering the different code paths for data and mechanism (de)allocations and code paths covered with locks. * Entropy collection - The LRNG is shipped with test tools allowing the collection of raw unconditioned entropy during runtime and boot time available at [1]. - Full entropy assessment and description is provided with [2] chapter 3, specifically section 3.2.6. - Guarantee that entropy events are not credited with entropy twice (the existing /dev/random implementation credits HID/disk and interrupt events with entropy which are a derivative of each other) and guarantee that entropy data is not reused for two different use cases (as done in the existing /dev/random implementation when injecting a part of fast_pool into the net_rand_state). * Configurable - LRNG kernel configuration allows configuration that is functionally equivalent to the existing /dev/random. Non-compiled additional code is folded into no-ops. - The following additional functions are compile-time selectable independent of each other: + Enabling of switchable cryptographic implementation support. This allows enabling an SP800-90A DRBG. + Enabling of using Jitter RNG noise source. + Enabling of noise source
