Re: [openssl-project] DRBGs, threads and locking
We did a performance analysis for Oracle’s equivalent of Nginx, OTD, about two years ago. We were looking at the number of connections per second that could be established and the limiting factor was locking in ssleay_rand_bytes. Approximately a third of the CPU and over 90% of the lock wait time was there from memory. For this kind of workload (many threads, many connections) a non-locking RNG would have been an improvement. I’ll see if I can find the analyses and then find out what I can release. This was using 1.0.2 not 1.1 so things might have changed. Pauli -- Oracle Dr Paul Dale | Cryptographer | Network Security & Encryption Phone +61 7 3031 7217 Oracle Australia From: Tim Hudson [mailto:t...@cryptsoft.com] Sent: Wednesday, 14 March 2018 1:15 PM To: openssl-project@openssl.org Subject: Re: [openssl-project] DRBGs, threads and locking We have to keep in mind what threats we care about and their practicality. The security of a DRBG is dependent on both the secrecy of the seed material provided and the security of the algorithm in terms of its output not leaking information that materially leaks the internal state in a manner that enables it to be discovered or reversed in a manner to enable determination of previous or future outputs. For some of the arguments used to date there appears to be an assumption that there is a practical difference between a broken DRBG algorithm such that it is not such a security issue if we separate out the DRBG instances on a per SSL connection. In real terms if a DRBG is broken and its state is able to be determined remotely there is no practical difference in separating DRBG instances - they are all equally vulnerable in the same manner. In the case of the DualEC-DRBG this was clear - and no one I've seen has ever suggested that you were safer if you had separate instances of a broken algorithm for a DRBG - it makes no practical difference to the security at all. Sure there is a slight technical difference - but from a security perspective there is no difference - you are susceptible to the same attack - so the minor technical difference offers no actual meaningful security value - and everyone that has referenced this to date has also indicated that they don't think that there is actually any real practical value to the difference - it has been more of a "it cannot harm" sort of comment. In more general terms we need to have a clear view on what we think about our thread model - what is considered inside the scope of what we care to address - and what is frankly outside the scope (for our view). • We don't consider attacks from the same process against itself within our threat model. • Correspondingly we don't consider attacks from one thread against another thread without our threat model. • We don't consider privileged user attacks against the user in our threat model (i.e. root can read the memory of the process on most Unix-like systems). • We also don't actually consider a need to protect all secret information from every possible other bug that might leak arbitrary parts of memory. We could. But we don't. And if we did we would need to protect both the seeding material for the DRBG and its internal state and potentially its output. We don't do that - because that isn't within our threat model. Typical applications share an SSL_CTX between multiple SSL instances and we maintain the session cache against the SSL_CTX. This may be in a single process (thread) or shared across multiple threads - or even shared across multiple prcesses (which is simply the same as being in a single process from our perspective where the "magic" to coordinate the session id cache between processes is left to the developer/user). In a FIPS context, every DRBG has requirements on its inputs (seeding) and on maintaining a continuous RNG test (block-based compare for non-repeating outputs at a block level). All of these would be a per-instance requirement on the DRBG. They have to be factored in. There is also the argument that locking is bad and fewer locks are better - and that argument needs to be backed up by looking at the overall status - which particular application model are we concerned about? Have we measured it? Have we figured out where the bottlenecks are? Have we worked through optimising the actual areas of performance impact? Or are we just prematurely optimising? Excessive locking will have an impact for certain application models - but I don't think anyone is suggesting that what we had previously was excessive - and given the significant performance impact of the recent changes which went unmeasured and unaddressed I think it is clear we haven't been measuring performance related items for the DRBG at all to date - so there wasn't any "science" behind the choices made. Simple, clear, well documented code with good tests and known architectural assumptions is what we are trying to achieve - and
Re: [openssl-project] DRBGs, threads and locking
I think the intention is to ditch the drbg from the ssl object and then call the global function (either public or private) which has been changed to use the current thread's drbg rather than being global. I'm in favour of a single per ssl drbg still, I'm not sure what a clean way to hook it up to avoid locks is (yet). Pauli -- Oracle Dr Paul Dale | Cryptographer | Network Security & Encryption Phone +61 7 3031 7217 Oracle Australia -Original Message- From: Salz, Rich [mailto:rs...@akamai.com] Sent: Wednesday, 14 March 2018 11:27 AM To: openssl-project@openssl.org Subject: Re: [openssl-project] DRBGs, threads and locking So a major reason, as you explained, for having per-thread DRBG's is to reduce contention. When threadA creates an SSL object, the parent DRBG will be the threadA one. Therefore you have to introducing locking, since threadA might create two SSL objects and they could end up being used in threadB and threadC and each need to reseed from their parent. In order to do that safely, threadA also has to do the locking to avoid conflict. That defeats the major gain of per-thread. I think having the SSL object parent be whatever the *current* thread DRBG is seems like the best, if not only, way to go. ___ openssl-project mailing list openssl-project@openssl.org https://mta.openssl.org/mailman/listinfo/openssl-project ___ openssl-project mailing list openssl-project@openssl.org https://mta.openssl.org/mailman/listinfo/openssl-project
Re: [openssl-project] DRBGs, threads and locking
We have to keep in mind what threats we care about and their practicality. The security of a DRBG is dependent on both the secrecy of the seed material provided and the security of the algorithm in terms of its output not leaking information that materially leaks the internal state in a manner that enables it to be discovered or reversed in a manner to enable determination of previous or future outputs. For some of the arguments used to date there appears to be an assumption that there is a practical difference between a broken DRBG algorithm such that it is not such a security issue if we separate out the DRBG instances on a per SSL connection. In real terms if a DRBG is broken and its state is able to be determined remotely there is no practical difference in separating DRBG instances - they are all equally vulnerable in the same manner. In the case of the DualEC-DRBG this was clear - and no one I've seen has ever suggested that you were safer if you had separate instances of a broken algorithm for a DRBG - it makes no practical difference to the security at all. Sure there is a slight technical difference - but from a security perspective there is no difference - you are susceptible to the same attack - so the minor technical difference offers no actual meaningful security value - and everyone that has referenced this to date has also indicated that they don't think that there is actually any real practical value to the difference - it has been more of a "it cannot harm" sort of comment. In more general terms we need to have a clear view on what we think about our thread model - what is considered inside the scope of what we care to address - and what is frankly outside the scope (for our view). - We don't consider attacks from the same process against itself within our threat model. - Correspondingly we don't consider attacks from one thread against another thread without our threat model. - We don't consider privileged user attacks against the user in our threat model (i.e. root can read the memory of the process on most Unix-like systems). - We also don't actually consider a need to protect all secret information from every possible other bug that might leak arbitrary parts of memory. We could. But we don't. And if we did we would need to protect both the seeding material for the DRBG and its internal state and potentially its output. We don't do that - because that isn't within our threat model. Typical applications share an SSL_CTX between multiple SSL instances and we maintain the session cache against the SSL_CTX. This may be in a single process (thread) or shared across multiple threads - or even shared across multiple prcesses (which is simply the same as being in a single process from our perspective where the "magic" to coordinate the session id cache between processes is left to the developer/user). In a FIPS context, every DRBG has requirements on its inputs (seeding) and on maintaining a continuous RNG test (block-based compare for non-repeating outputs at a block level). All of these would be a per-instance requirement on the DRBG. They have to be factored in. There is also the argument that locking is bad and fewer locks are better - and that argument needs to be backed up by looking at the overall status - which particular application model are we concerned about? Have we measured it? Have we figured out where the bottlenecks are? Have we worked through optimising the actual areas of performance impact? Or are we just prematurely optimising? Excessive locking will have an impact for certain application models - but I don't think anyone is suggesting that what we had previously was excessive - and given the significant performance impact of the recent changes which went unmeasured and unaddressed I think it is clear we haven't been measuring performance related items for the DRBG at all to date - so there wasn't any "science" behind the choices made. Simple, clear, well documented code with good tests and known architectural assumptions is what we are trying to achieve - and my sense from the conversations on this topic to date was that we don't have a consensus as to what problem we are actually trying to solve - so the design approach shifts, and shifts again - all of which are the authors of the PRs responding to what is (in my view at least) conflicting suggestions based on different assumptions. That is what I put the -1 on the the PR - to have this discussion - and agree on what we are trying to solve - and also agree on what we are not trying to solve. And perhaps we can actually document some of our "threat model" - as I'm sure we have different views on that as well. I don't think we should have per-SSL DRBGs - it offers no meaningful security value. We could have a per-SSL_CTX - but I'm not sure that is needed. We could have a per-thread - but again that is unclear if we actually need that either. My thoughts are per-SSL_CTX
Re: [openssl-project] DRBGs, threads and locking
So a major reason, as you explained, for having per-thread DRBG's is to reduce contention. When threadA creates an SSL object, the parent DRBG will be the threadA one. Therefore you have to introducing locking, since threadA might create two SSL objects and they could end up being used in threadB and threadC and each need to reseed from their parent. In order to do that safely, threadA also has to do the locking to avoid conflict. That defeats the major gain of per-thread. I think having the SSL object parent be whatever the *current* thread DRBG is seems like the best, if not only, way to go. ___ openssl-project mailing list openssl-project@openssl.org https://mta.openssl.org/mailman/listinfo/openssl-project
Re: [openssl-project] DRBGs, threads and locking
On Wed, Mar 14, 2018 at 01:27:47AM +0100, Kurt Roeckx wrote: > My solution is to just have 1 master DRBG, and a public and > private DRBG per thread. The only lock that then is needed is when > the public or private DRBG needs to reseed. All the rest of the > code can stay just as it is, but we might want to change some > places to use the (thread local) private DRBG, which is what #4665 > is about. [...] > So the suggestion was to still have a per SSL public DRBG, but > then the problem is that that SSL object might have moved to a > different thread between creating and being used and so that the > parent DRBG might actually belong to a different thread. One > solution there is that we just take the current thread's public > DRBG as parent instead of the original threads public DRBG. This should be fine from a thread-safety point of view. I don't know whether it could potentially affect the standards compliance, for the intermediate DRBG to potentially change over time (even though it still chains to a common grandparent/master DRBG). Per-SSL DRBGs (especially if split to public and private) seem excessive to me, so architecture described in the quoted text seems like the best option, to me. -Ben ___ openssl-project mailing list openssl-project@openssl.org https://mta.openssl.org/mailman/listinfo/openssl-project
[openssl-project] DRBGs, threads and locking
So Tim has voted -1 on PR #5547 and wants us to discuss it here and vote on it. I don't know if it's clear to everybody what this is about. If something is not clear, please ask. PR #5461 contains a lot of documentation updates that is related to it, and it might be useful to read it as background information. There are many other related issues and pull request. I will try to make a basic summary here. The DRBGs is what we use to generate random numbers. They have the possibility to chain, where 1 DRBG gets it's entropy from it's parent. You can make long chains with this. The current state in master is that you have 1 master DRBG that gets it's entropy from the OS, and then 1 public and 1 private DRBG that gets the entropy from the master. Then there is an DRBG in the SSL struct that gets it's entropy from the public DRBG. I have 2 problems with the current setup that I would like to solve: - On SSL_new() we create a new DRBG. That needs to get initialized and that requires getting entropy from somewhere, where that's currently the global public DRBG. This requires taking a lock. - If we actually want to use the DRBG for everything related to that SSL connections to avoid having to lock a global DRBG everything that SSL code calls needs to be able to say with which DRBG it needs to generate random data. There was already 1 PR related to this merged. PR #5510 is still open that deals at least with mot of it that I know about. I find that a very ugly hack and really don't see an easy way improve it. My solution is to just have 1 master DRBG, and a public and private DRBG per thread. The only lock that then is needed is when the public or private DRBG needs to reseed. All the rest of the code can stay just as it is, but we might want to change some places to use the (thread local) private DRBG, which is what #4665 is about. Some people seem to have a desire to have a separate the DRBGs per SSL connection, at least for the public data. This is for cases where from that data it would be possible to get the internal state of the DRBG that is at least allegedly possible with the DUAL_EC_DRBG. I believe this is mitigated somewhat by our mixing in additional data when calling RAND_bytes() (or RAND_priv_bytes()), but I'm not an expert in this and will leave this to the others to comment on. So the suggestion was to still have a per SSL public DRBG, but then the problem is that that SSL object might have moved to a different thread between creating and being used and so that the parent DRBG might actually belong to a different thread. One solution there is that we just take the current thread's public DRBG as parent instead of the original threads public DRBG. I hope I at least covered most of it. Kurt ___ openssl-project mailing list openssl-project@openssl.org https://mta.openssl.org/mailman/listinfo/openssl-project
Re: [openssl-project] Copyrights, AUTHORS file, etc.
This discussion has been taken to the OMC mailing list (where it continues) rather than the openssl-project list as it goes across previous team decisions. An update once that discussion completes will be sent to the openssl-project list. Thanks, Tim. On Tue, Mar 13, 2018 at 11:22 AM, Salz, Richwrote: > For some background you can take a look at https://github.com/openssl/ > openssl/pull/5499 and the blog posts here: https://www.openssl.org/blog/ > blog/categories/license/ > > > > The OMC voted in 2014 to work to an eventual change of license to Apache > 2. That led to the CLA’s. We had calls with various open source folks who > we thought could help us. This include Max Sills from the Google open > source office, Gervais Markham who led the Mozilla relicensing effort, > someone from GitHub (to talk about CLA automation), and probably another > person who I am forgetting. We also talked with our legal counsel, Mishi > Choudhary of the Software Freedom Law Center. > > > > Around 2Q 3Q 2015 the discussions were completed, and we had coalesced > around a plan. There was no formal OMC vote (it was called the openssl-team > at that point). But there were no objections. OMC members can skim the > archives, starting around July 2015 if they need to refresh their memories. > > > > The key points of the plan are > >- Move to Apache license >- Require CLA’s from all contributors >- Reach out to everyone we can identify and get them to approve of the >change, or not >- Have a uniform copyright in all source files that points to the >license and authors separately, for easier maintenance > > > > The “removing some code” blog post gives more details about the scripts we > developed and what code we removed. Since then, nobody else has asked for > their code to be removed. > > > > The file/copyright changes happened during the 1.1.0 release. > > > > We’re on the verge of being able to change the license, and as we said in > our last press release, we are hoping and planning to do that for 1.1.1 > > > > The PR that marks part of this has a -1 from Tim, which is a hold. That > means we have to discuss and the OMC vote on this. This email is intended > to give the background and start the discussion. > > > > So what are your objections Tim, and what do you want to see done > differently? And also, please explain why it is better than the current > plan. > > > > ___ > openssl-project mailing list > openssl-project@openssl.org > https://mta.openssl.org/mailman/listinfo/openssl-project > ___ openssl-project mailing list openssl-project@openssl.org https://mta.openssl.org/mailman/listinfo/openssl-project