RE: Use of Certificate/Public Key Pinning
So, pinning is an extremely complicated topic that I've always wanted to write a blog post about, but have never had the time to do it. It happens fairly regularly that we have to assist a company that has painted themselves into a corner with a poorly thought out pinning scheme. In my experience, it's mostly bad implementations of pinning that cause most of the problems, not pinning itself. I've seen people pin leaf certificate serial numbers, for example. That's guaranteed to end badly. If you do feel the need to pin, please make sure you've thought through all the possible certificate replacement use cases and make sure you have designed, implemented, AND TESTED them to make sure they work. All too often, a customer complains things stop working when they replace their certificate ... but no one at the company is aware what exactly is being pinned or why. Use pinning with care. -Tim > -Original Message- > From: dev-security-policy On > Behalf Of Ryan Sleevi via dev-security-policy > Sent: Wednesday, August 14, 2019 2:08 PM > To: Nuno Ponte > Cc: mozilla-dev-security-policy > > Subject: Re: Use of Certificate/Public Key Pinning > > On Tue, Aug 13, 2019 at 11:12 AM Nuno Ponte via dev-security-policy < dev- > security-pol...@lists.mozilla.org> wrote: > > > Dear m.d.s.p., > > > > I would like to bring into discussion the use of certificate/public > > key pinning and the impacts on the 5-days period for certificate > > revocation according to BR §4.9.1.1. > > > > Recently, we (Multicert) had to rollout a general certificate > > replacement due to the serial number entropy issue. Some of the most > > troubled cases to replace the certificates were customers doing > > certificate pinning on mobile apps. Changing the certificate in these > > cases required configuration changes in the code base, rebuild app, QA > > testing, submission to App stores, call for expedited review of each > > App store, wait for review to be completed and only then the new app > > version is made available for installation by end users (which is turn > > are required to update the app the soonest). > > > > Meeting the 5-days deadline with this sort of process is > > “challenging”, at best. > > > > A first approach is to move from certificate pinning to public key > > pinning (PKP). This prevents the need to update the app in many of the > > certificate replacement operations, where the public key is reused and > > the certificate can be replaced transparently to the app (generically, an > “User Agent” > > doing PKP). > > > > However, in the event of a serious security incident that requires > > re-key (such as key compromise), the certificate must be revoked in > > less than 24 hours (for the benefit of everyone – subscriber, relying > > parties, issuing CA, etc). It’s virtually impossible to release a new > > app version within this timeframe. And this, I think, make a very > > strong point against the use of PKI. > > > > On the other side, PKP is a simple yet powerful and effective > > technique to protect against MITM and other attacks. It seems to be > > widely used in apps with advanced threat models (mobile banking, > > sensitive personal information, etc) and there are many frameworks > > available (including native support in Android via Network Security > Configuration [1]). > > > > There are several possible mitigation actions, such as pinning more > > than one public key to have more than one certificate to quickly > > rollover in case of a revocation. Even then, it is very likely that > > all the redundant key pairs were generated and maintained by the same > > systems and procedures, and thus all of them will become effectively > compromised. > > > > Ultimately, it may become common practice that 1) PKP frameworks are > > set to bypass revocation checks or 2) PKP is done with private > > certificates (homemade, self-signed, managed ad-hoc with no CRL/OCSP > > services). Does any of this leads to a safer Internet? > > > > I don’t expect this thread to end up into an absolute conclusion > > advocating for or against, but opening it to discussion and > > contributions may help to document possible strategies, mitigations, > > alternatives, pros & cons, and hopefully provide guidance for an educated > decision. > > > > Best regards, > > > > Nuno Ponte > > Multicert SA > > > > Nuno, > > Thanks for raising this. I appreciate that a CA is attempting to provide > guidance > to their Subscribers, since, as you note, the CA is still beholden to the > Baseline > Requ
Re: Use of Certificate/Public Key Pinning
On Tue, Aug 13, 2019 at 11:12 AM Nuno Ponte via dev-security-policy < dev-security-policy@lists.mozilla.org> wrote: > Dear m.d.s.p., > > I would like to bring into discussion the use of certificate/public key > pinning and the impacts on the 5-days period for certificate revocation > according to BR §4.9.1.1. > > Recently, we (Multicert) had to rollout a general certificate replacement > due to the serial number entropy issue. Some of the most troubled cases to > replace the certificates were customers doing certificate pinning on mobile > apps. Changing the certificate in these cases required configuration > changes in the code base, rebuild app, QA testing, submission to App > stores, call for expedited review of each App store, wait for review to be > completed and only then the new app version is made available for > installation by end users (which is turn are required to update the app the > soonest). > > Meeting the 5-days deadline with this sort of process is “challenging”, at > best. > > A first approach is to move from certificate pinning to public key pinning > (PKP). This prevents the need to update the app in many of the certificate > replacement operations, where the public key is reused and the certificate > can be replaced transparently to the app (generically, an “User Agent” > doing PKP). > > However, in the event of a serious security incident that requires re-key > (such as key compromise), the certificate must be revoked in less than 24 > hours (for the benefit of everyone – subscriber, relying parties, issuing > CA, etc). It’s virtually impossible to release a new app version within > this timeframe. And this, I think, make a very strong point against the use > of PKI. > > On the other side, PKP is a simple yet powerful and effective technique to > protect against MITM and other attacks. It seems to be widely used in apps > with advanced threat models (mobile banking, sensitive personal > information, etc) and there are many frameworks available (including native > support in Android via Network Security Configuration [1]). > > There are several possible mitigation actions, such as pinning more than > one public key to have more than one certificate to quickly rollover in > case of a revocation. Even then, it is very likely that all the redundant > key pairs were generated and maintained by the same systems and procedures, > and thus all of them will become effectively compromised. > > Ultimately, it may become common practice that 1) PKP frameworks are set > to bypass revocation checks or 2) PKP is done with private certificates > (homemade, self-signed, managed ad-hoc with no CRL/OCSP services). Does any > of this leads to a safer Internet? > > I don’t expect this thread to end up into an absolute conclusion > advocating for or against, but opening it to discussion and contributions > may help to document possible strategies, mitigations, alternatives, pros & > cons, and hopefully provide guidance for an educated decision. > > Best regards, > > Nuno Ponte > Multicert SA > Nuno, Thanks for raising this. I appreciate that a CA is attempting to provide guidance to their Subscribers, since, as you note, the CA is still beholden to the Baseline Requirements, and the Subscriber still beholden to their Subscriber Agreement. As one of the co-authors of the HTTP Public Key Pinning RFC, RFC 7469, I would readily encourage your Subscribers not to pin at all. However, if they are going to pin to anything, then it: 1) Should be to the public key, not certificate 2) Should only be to the root 3) Should require an agreement with the root that the Root (i.e. Multicert) will disclose all possible certificate paths and root certificates that are unexpired and may exist. - This includes cross-signed certificates Note that mobile devices, the scenario you discuss for pinning, already largely do not implement revocation checks and do not offer a way for applications to control it, so I don't think that should be a concern. Of course, many of these techniques predate improvements to the ecosystem, such as CAA and Certificate Transparency. Many of the concerns that motivated HPKP, which subsequently evolved into the aforementioned OS APIs, were predicated on an assumption of a CA that has lax validation methods. Yet techniques such as CAA exist to give site operators meaningful control - and to ensure that any CA that violates such checks, and ignores CAA, is unambiguously at risk of being distrusted by the entire platform. If there's concern about the DNS registrar being hacked, the Subscriber has the option of choosing a better registrar before hand, without resorting to PKP, and/or transitioning to registries with better security practices overall. In general, this means avoiding ccTLDs, which are known lax due to their lack of any formal IANA or ICANN agreements with respect to operations. Certificate Transparency offers a compliment to this, ensuring that even if the Registrar or Registry
Re: Use of Certificate/Public Key Pinning
I feel that there's a great deal of consultancy and assistance that CAs and PKI professionals could bring to their more sophisticated customers with scenarios such as these where public key pinning an a field-deployed application may present problems for certificates being revoked. A best practices document explaining to the application developers and server-side teams that: 1. An app which calls a server-side API under your control should _always_ do so on a TLS endpoint at a different hostname & SNI label than any browser-facing websites. 2. Following step 1's guidance means that you can control the lifecycle of the certificate for the services accessed by your own application(s) separate from WebPKI facing certificates meant to facilitate a TLS authenticated session to a modern browser. 3. It also means that the endpoints serving the application CAN but don't have to be from a publicly trusted PKI. For compatibility reasons, they generally should be, if there are any external consumers of the API, but ultimately if their own application wishes to PIN, they should pre-create several certificates with distinct keys and write their app to override the platform trust decisioning and pin on the set of keys that their API endpoint certificates will have, ignoring revocation and requiring that the presented leaf certificate be a signature over one of the set of pinned public keys. This is essentially free in virtually all deployment models today. Oversubscribing TLS endpoints (for our purposes let's say a DNS based hostname and TLS SNI label define a TLS endpoint) for different target audiences, especially when those audiences are modern browsers in combination with anything else, is one of the most significant causes of compatibility issues and legacy cruft which have historically hindered the agility of the WebPKI. On Tue, Aug 13, 2019 at 10:12 AM Nuno Ponte via dev-security-policy < dev-security-policy@lists.mozilla.org> wrote: > Dear m.d.s.p., > > I would like to bring into discussion the use of certificate/public key > pinning and the impacts on the 5-days period for certificate revocation > according to BR §4.9.1.1. > > Recently, we (Multicert) had to rollout a general certificate replacement > due to the serial number entropy issue. Some of the most troubled cases to > replace the certificates were customers doing certificate pinning on mobile > apps. Changing the certificate in these cases required configuration > changes in the code base, rebuild app, QA testing, submission to App > stores, call for expedited review of each App store, wait for review to be > completed and only then the new app version is made available for > installation by end users (which is turn are required to update the app the > soonest). > > Meeting the 5-days deadline with this sort of process is “challenging”, at > best. > > A first approach is to move from certificate pinning to public key pinning > (PKP). This prevents the need to update the app in many of the certificate > replacement operations, where the public key is reused and the certificate > can be replaced transparently to the app (generically, an “User Agent” > doing PKP). > > However, in the event of a serious security incident that requires re-key > (such as key compromise), the certificate must be revoked in less than 24 > hours (for the benefit of everyone – subscriber, relying parties, issuing > CA, etc). It’s virtually impossible to release a new app version within > this timeframe. And this, I think, make a very strong point against the use > of PKI. > > On the other side, PKP is a simple yet powerful and effective technique to > protect against MITM and other attacks. It seems to be widely used in apps > with advanced threat models (mobile banking, sensitive personal > information, etc) and there are many frameworks available (including native > support in Android via Network Security Configuration [1]). > > There are several possible mitigation actions, such as pinning more than > one public key to have more than one certificate to quickly rollover in > case of a revocation. Even then, it is very likely that all the redundant > key pairs were generated and maintained by the same systems and procedures, > and thus all of them will become effectively compromised. > > Ultimately, it may become common practice that 1) PKP frameworks are set > to bypass revocation checks or 2) PKP is done with private certificates > (homemade, self-signed, managed ad-hoc with no CRL/OCSP services). Does any > of this leads to a safer Internet? > > I don’t expect this thread to end up into an absolute conclusion > advocating for or against, but opening it to discussion and contributions > may help to document possible strategies, mitigations, alternatives, pros & > cons, and hopefully provide guidance for an educated decision. > > Best regards, > > Nuno Ponte > Multicert SA > > [1] https://developer.android.com/training/articles/security-config > > > > > >
Re: Use of Certificate/Public Key Pinning
PKP is a footgun. Deploying it without being prepared for the situations you've described is ill-advised. There's a few options available for organizations who want to pin, in increasing order of sophistication: Enforce Certificate Transparency. You're not locked into any CA or key, only that the certificate has been published publicly. Pin to a CA or a couple of CAs - this reduces the operational/availability risk while increasing the security risk. (Although still a reduction from the entire set of CAs of course.) Pin to leaf *keys*, as you suggest, and ensure that they cannot all be compromised at once through the use of offline storage and careful key mangement. Use the keys to get certificates when needed. As you note, if you can't manage these keys securely and separately, you need to go to something less sophisticated, like pinning to CAs. Pin to a locally managed trust anchor, and operate a root CA oneself, managing it as one would a public CA (offline root, possibly offline intermediates, etc) -tom On Tue, 13 Aug 2019 at 15:12, Nuno Ponte via dev-security-policy wrote: > > Dear m.d.s.p., > > I would like to bring into discussion the use of certificate/public key > pinning and the impacts on the 5-days period for certificate revocation > according to BR §4.9.1.1. > > Recently, we (Multicert) had to rollout a general certificate replacement due > to the serial number entropy issue. Some of the most troubled cases to > replace the certificates were customers doing certificate pinning on mobile > apps. Changing the certificate in these cases required configuration changes > in the code base, rebuild app, QA testing, submission to App stores, call for > expedited review of each App store, wait for review to be completed and only > then the new app version is made available for installation by end users > (which is turn are required to update the app the soonest). > > Meeting the 5-days deadline with this sort of process is “challenging”, at > best. > > A first approach is to move from certificate pinning to public key pinning > (PKP). This prevents the need to update the app in many of the certificate > replacement operations, where the public key is reused and the certificate > can be replaced transparently to the app (generically, an “User Agent” doing > PKP). > > However, in the event of a serious security incident that requires re-key > (such as key compromise), the certificate must be revoked in less than 24 > hours (for the benefit of everyone – subscriber, relying parties, issuing CA, > etc). It’s virtually impossible to release a new app version within this > timeframe. And this, I think, make a very strong point against the use of PKI. > > On the other side, PKP is a simple yet powerful and effective technique to > protect against MITM and other attacks. It seems to be widely used in apps > with advanced threat models (mobile banking, sensitive personal information, > etc) and there are many frameworks available (including native support in > Android via Network Security Configuration [1]). > > There are several possible mitigation actions, such as pinning more than one > public key to have more than one certificate to quickly rollover in case of a > revocation. Even then, it is very likely that all the redundant key pairs > were generated and maintained by the same systems and procedures, and thus > all of them will become effectively compromised. > > Ultimately, it may become common practice that 1) PKP frameworks are set to > bypass revocation checks or 2) PKP is done with private certificates > (homemade, self-signed, managed ad-hoc with no CRL/OCSP services). Does any > of this leads to a safer Internet? > > I don’t expect this thread to end up into an absolute conclusion advocating > for or against, but opening it to discussion and contributions may help to > document possible strategies, mitigations, alternatives, pros & cons, and > hopefully provide guidance for an educated decision. > > Best regards, > > Nuno Ponte > Multicert SA > > [1] https://developer.android.com/training/articles/security-config > > > > > > ___ > dev-security-policy mailing list > dev-security-policy@lists.mozilla.org > https://lists.mozilla.org/listinfo/dev-security-policy ___ dev-security-policy mailing list dev-security-policy@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-security-policy
Re: Use of Certificate/Public Key Pinning
On Mon, 12 Aug 2019, Nuno Ponte via dev-security-policy wrote: Recently, we (Multicert) had to rollout a general certificate replacement due to the serial number entropy issue. Some of the most troubled cases to replace the certificates were customers doing certificate pinning on mobile apps. Changing the certificate in these cases required configuration changes in the code base, rebuild app, QA testing, submission to App stores, call for expedited review of each App store, wait for review to be completed and only then the new app version is made available for installation by end users (which is turn are required to update the app the soonest). Meeting the 5-days deadline with this sort of process is “challenging”, at best. The OS and/or App should look at Certificate Transparency, instead of hacks that hardcode the certificate serial number. Paul ___ dev-security-policy mailing list dev-security-policy@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-security-policy
