Re: On GitHub, Leaked Keys, and getting practical about revocation
On Thu, Jun 22, 2017 at 3:53 PM Jakob Bohm via dev-security-policy < dev-security-policy@lists.mozilla.org> wrote: > On 22/06/2017 15:02, Ryan Sleevi wrote: > > On Thu, Jun 22, 2017 at 1:59 PM Jakob Bohm via dev-security-policy < > > dev-security-policy@lists.mozilla.org> wrote: > > > > (Snip long repeat of the same opinion) > > You seem to argue: Your summary is neither accurate nor supported by what I said. However, as your reply doesn't seek to learn or understand, and rather chooses to belittle and misrepresent, nor does it provide any new or useful information to refute the generally accepted conclusions you appear to disagree with, it does seem that we should simply agree to disagree. If you should have interest in trying to provide persuasive arguments, factually supported, in a respectful way, perhaps there is an opportunity to learn from eachother, but as this is otherwise an unnecessarily combative, overly reducative, and textually unsupported strawman, it might be best if you were to take a break from this discussion. ___ dev-security-policy mailing list dev-security-policy@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-security-policy
Re: On GitHub, Leaked Keys, and getting practical about revocation
On 22/06/2017 15:02, Ryan Sleevi wrote: On Thu, Jun 22, 2017 at 1:59 PM Jakob Bohm via dev-security-policy < dev-security-policy@lists.mozilla.org> wrote: > (Snip long repeat of the same opinion) You seem to argue: - Because the recent research on efficient central CRL distribution was based on a novel optimization of a previously inefficient algorithm, then it is nothing new and should be ignored. - Operating a central CRL distribution service is a suspect commercial enterprise with a suspect business model, not a service to the community. - OCSP stapling of intermediary certificates is inefficient because we should all just use central CRL distribution in a form where not all revocations are included. - Because most/all browsers contain security holes in their revocation checking, making those holes bigger is not a problem. - Revocation by the issuing CA is not trustworthy, thus nothing is, therefore everybody should just trust compromised keys as if they were not compromised. - An attacker with a stolen/compromised key not doing OCSP stapling is the fault of the legitimate key holder. - Forcing people to use OCSP stapling will magically cause software that allows this to spring into existence overnight. If this doesn't happen it is the fault of the server operator, not because the demand was premature. Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded ___ dev-security-policy mailing list dev-security-policy@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-security-policy
Re: On GitHub, Leaked Keys, and getting practical about revocation
On Thu, Jun 22, 2017 at 1:59 PM Jakob Bohm via dev-security-policy < dev-security-policy@lists.mozilla.org> wrote: > Please note that Apache and NGINX are by far not the only TLS servers > that will need working OCSP stapling code before must-staple can become > default or the only method checked by Browsers and other TLS clients. > > What is needed is: > > 1. An alternative checking mechanism, such as the compact super-CRL >developed by some researchers earlier this year. This needs to be >deployed and used *before* turning off traditional CRL and OCSP >checking in relying party code, since leaving a relying party >application without any checking for revoked certificates is a pretty >obvious security hole. There is reasonable and well-argued disagreement with you on this. Without hardfail, it is not a security hole, and now mainstream client has ever shipped hardfail, so it is not a reasonable requirement to introduce into this discussion. There's equally a host of pragmatic and practical concerns, which you can find in the years of discussion on this topic and related solutions (as the "super-CRL" you refer to is just the application of one particular technology, which had previously been outlined years prior, and which alternatives of similar constraints also existed even before that). I suspect this may simply resolve to irreconcilable differences, as some people may incorrectly choose to believe that soft-fail revocation provides a defensible security boundary, or may believe that CAs revocation is a trustworthy reason to deny access, both of which ample evidence and arguments exist to show otherwise. I mention this largely to avoid rehashing the same conversations that have been had, but if you are unfamiliar with them, and are interested in learning of this other perspective, I would be happy to provide them. I just figure as shorthand that we disagree on this. > > 2. Full OCSP stapling support in all TLS libraries, including the LTS >branches of OpenSSL, mbedTLS, NSS, Java, Android etc. Providing this >only in the "next great release that is incompatible with many >existing users" is a common malpractice among security library >vendors, just as there are still systems that refuse to support TLS >1.2 and ubiquitous SHA-256 signatures in todays world, simply because >the original system/library vendor refuses to backport the needed >changes. There is zero reason to introduce this as a dependency beforehand. Perhaps your assumption is that it is unreasonable to require of the ecosystem what the ecosystem does not support, but equally realize, the ecosystem will not support it until it is required. It is already true that a sufficient and meaningful majority support most of the necessary work, and so the argument that ALL servers or libraries must support is to ignore both the market realities and to place the (unreasonable) perfect ahead of the achievable good. Further, you conflate OCSP Multi-Staple as necessary, when in fact it is entirely undesirable from a performance perspective and largely unnecessary from a deployment scenario, given the existence of CRLSets, CDLs, OneCRL-et-al. > > 3. Once #2 is achieved, actual TLS servers and clients using those >libraries can begin to enable OCSP stapling. > > 4. Once #3 is achieved and deployed, then OCSP stapling might become >mandatory by default. This is to completely upend the only action that has been seen to meaningfully improve the ecosystem, which is the mandate that thus spurs implementation or innovation. On a more pragmatic level, there is nothing wrong with saying "The only revocation supported will be stapling and OneCRL". There is no need to go above and beyond this, because collectively, this achieves the goal of providing a compelling revocation story. The disconnect that results in proposals like yours is that they presume revocation is for the benefit of the relying party, as opposed to being for the benefit of the site operator. A site operator cares about revocation for cases of key compromise and impersonation. A relying party may care about revocation for reasons like misrepresentation (which is not, despite some views contrary, an accepted concern of the Mozilla policies - c.f. malware and phishing), apathetic server compromise (that is, they did not enable stapling. However, the root cause/risk is the apathy, for which revocation does not fix), or should the user want to deny themselves access to a site (which no user does). If we focus on stapling, the position is that it is not necessary for the browser to protect the user from servers' apathy (in not enabling stapling), or from CAs' capricious opinions about certificates (which the so-called supercrls try to enable, as a business model), but to allow servers to protect themselves. There is similarly no concern given to CAs that want to use OCSP or CRLs to "rent a cert" (as some tried to in the past), beca
Re: On GitHub, Leaked Keys, and getting practical about revocation
On 21/06/2017 19:40, Matthew Hardeman wrote: Hi all, I'm sure questions of certificates leaked to the public via GitHub and other file sharing / code sharing / deployment repository hosting and sharing sites have come up before, but last night I spent a couple of hours constructing various search criteria which I don't think were even especially clever, but still I was shocked and amazed at what I found: At least 10 different Apple Development and Production environment certificates and key pairs. Of these minimum of 10 that I am counting, I validated that the certificate is within its validity period. Of these, I validated that the key I found matches the public key information in the certificate. Most of these certificates are TLS Client Authentication certificates which also have additional Apple proprietary extended key usages. These certificates are utilized for authenticating to the Apple Push Notification System. A couple of certificates were Apple Developer ID certificates appropriate for development and production environment deployment of executable code to Apple devices. (Those developer ID certificates I have reported to Apple for revocation.) There were more Apple Push authentication certificates than I cared to write up and send over. I was shocked at the level of improper distribution and leaking of these keys and certificates. Once in a while, the key was represented in encrypted form. In _every_ instance for which I found an encrypted key and dug further, either a piece of code, a configuration file, or sometimes a README-KEY-PASSWORD.txt (or similar) within the same repository successfully decrypted the encrypted key. Additionally, I did find some TLS server certificates. There were many more that I did not bother to carefully analyze. Some were expired. One was a in-validity-window DV certificate issued by Let's Encrypt. Utilizing the certificate's private key, I was able to successfully use the Let's Encrypt ACME API to automatically request revocation of that certificate. Minutes later, I verified that OCSP responses for that certificate were, in fact, indicating that the certificate was revoked. Of course, revocation even with a really nice OCSP responder system is not very effective today. I have this suspicion that human nature dictates that eliminating these kinds of key material leaks is not even a goal worth having. Disappointment, I suspect, lives down that road. Because live OCSP checks for certificates en-masse is not appealing to either the CAs or the browsers or the end users (consequences of network delay, reliability, etc.), revocation means very little pragmatically today. This only reinforces the value and importance of either/both: - Quite short lived certificates, automatically replaced and deployed, to reduce the risks associated with key compromise and/or - OCSP must-staple, which I believe is only pragmatically gated at the moment by a number of really poor server-side implementations of OCSP stapling. Servers must cache good responses. Servers must use those while awaiting a new good response further into the OCSP response validity period. Servers must validate the response and not server random garbage as if OCSP. Etc, etc. Ryan Sleevi's work documenting the core issues is clearly a step in the right direction. Both NGINX's and Apache HTTPD's implementations of OCSP stapling are lacking in several material respects. It would certainly be a significant undertaking, but I believe that organizations who are working to ensure a secure Web (and that reap the benefits of a secure and trustworthy web) could do much to achieve better deployment of OCSP stapling in relatively short time: 1. Direct contribution of funds / bounty to the core developers of each of those two web server projects for building a server-side OCSP stapling implementation which is trivial to configure and which meets the needs of an ideal implementation with respect to caching of good results, validating new responses to staple, scheduling the deployment of successful new responses or scheduling retries of fails, etc. Insist that the code be written with a view to maximal back-port capability for said implementations. 2. If such contributions are infeasible, funding competent external development of code which achieves the same as item 1 above. 3. High level engagement with major distributions. Tackle the technical and administrative hurdles to get these changes into the stable and development builds of all currently shipping versions of at least RedHat's, Canonical's, and Debian's distributions. Get these changes into the standard default version httpd and nginx updates. 4. Same as above but for common docker images, prevalent VM images, etc. 5. Ensure that the browsers are ready to support and enforce fail-hard on certificates which feature the OCSP must-staple extension. 6. Monitor progress in r
Re: On GitHub, Leaked Keys, and getting practical about revocation
On Wed, 21 Jun 2017 10:40:01 -0700 (PDT) Matthew Hardeman via dev-security-policy wrote: > Through a little Google digging, I find numerous comments and > references from well informed parties going back quite several years > lamenting the poor state of support of OCSP stapling in both Apache > HTTPD and NGINX. I'm well aware of the rising power that is Caddy, > but it's not there yet. The whole ecosystem could be greatly helped > by making the default shipping versions of those two daemons in the > major distros be ideal OCSP-stapling ready. There is some movement here for apache, see discussion over at the apache dev list: https://lists.apache.org/thread.html/1a61e9dfbd685c4102b097e8189bccb7d5da39bf9f32fcbe7407a760@%3Cdev.httpd.apache.org%3E I'm slightly optimistic that we'll have a better stapling implementation in apache soon. Also CII is interested in funding efforts that improve the state of ocsp stapling. -- Hanno Böck https://hboeck.de/ mail/jabber: ha...@hboeck.de GPG: FE73757FA60E4E21B937579FA5880072BBB51E42 ___ dev-security-policy mailing list dev-security-policy@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-security-policy
On GitHub, Leaked Keys, and getting practical about revocation
Hi all, I'm sure questions of certificates leaked to the public via GitHub and other file sharing / code sharing / deployment repository hosting and sharing sites have come up before, but last night I spent a couple of hours constructing various search criteria which I don't think were even especially clever, but still I was shocked and amazed at what I found: At least 10 different Apple Development and Production environment certificates and key pairs. Of these minimum of 10 that I am counting, I validated that the certificate is within its validity period. Of these, I validated that the key I found matches the public key information in the certificate. Most of these certificates are TLS Client Authentication certificates which also have additional Apple proprietary extended key usages. These certificates are utilized for authenticating to the Apple Push Notification System. A couple of certificates were Apple Developer ID certificates appropriate for development and production environment deployment of executable code to Apple devices. (Those developer ID certificates I have reported to Apple for revocation.) There were more Apple Push authentication certificates than I cared to write up and send over. I was shocked at the level of improper distribution and leaking of these keys and certificates. Once in a while, the key was represented in encrypted form. In _every_ instance for which I found an encrypted key and dug further, either a piece of code, a configuration file, or sometimes a README-KEY-PASSWORD.txt (or similar) within the same repository successfully decrypted the encrypted key. Additionally, I did find some TLS server certificates. There were many more that I did not bother to carefully analyze. Some were expired. One was a in-validity-window DV certificate issued by Let's Encrypt. Utilizing the certificate's private key, I was able to successfully use the Let's Encrypt ACME API to automatically request revocation of that certificate. Minutes later, I verified that OCSP responses for that certificate were, in fact, indicating that the certificate was revoked. Of course, revocation even with a really nice OCSP responder system is not very effective today. I have this suspicion that human nature dictates that eliminating these kinds of key material leaks is not even a goal worth having. Disappointment, I suspect, lives down that road. Because live OCSP checks for certificates en-masse is not appealing to either the CAs or the browsers or the end users (consequences of network delay, reliability, etc.), revocation means very little pragmatically today. This only reinforces the value and importance of either/both: - Quite short lived certificates, automatically replaced and deployed, to reduce the risks associated with key compromise and/or - OCSP must-staple, which I believe is only pragmatically gated at the moment by a number of really poor server-side implementations of OCSP stapling. Servers must cache good responses. Servers must use those while awaiting a new good response further into the OCSP response validity period. Servers must validate the response and not server random garbage as if OCSP. Etc, etc. Ryan Sleevi's work documenting the core issues is clearly a step in the right direction. Both NGINX's and Apache HTTPD's implementations of OCSP stapling are lacking in several material respects. It would certainly be a significant undertaking, but I believe that organizations who are working to ensure a secure Web (and that reap the benefits of a secure and trustworthy web) could do much to achieve better deployment of OCSP stapling in relatively short time: 1. Direct contribution of funds / bounty to the core developers of each of those two web server projects for building a server-side OCSP stapling implementation which is trivial to configure and which meets the needs of an ideal implementation with respect to caching of good results, validating new responses to staple, scheduling the deployment of successful new responses or scheduling retries of fails, etc. Insist that the code be written with a view to maximal back-port capability for said implementations. 2. If such contributions are infeasible, funding competent external development of code which achieves the same as item 1 above. 3. High level engagement with major distributions. Tackle the technical and administrative hurdles to get these changes into the stable and development builds of all currently shipping versions of at least RedHat's, Canonical's, and Debian's distributions. Get these changes into the standard default version httpd and nginx updates. 4. Same as above but for common docker images, prevalent VM images, etc. 5. Ensure that the browsers are ready to support and enforce fail-hard on certificates which feature the OCSP must-staple extension. 6. Monitor progress in readiness and incentivize deployment of OCSP mu
