Re: Certificate with Debian weak key issued by Let's Encrypt
A report regarding this incident has been published on the Let's Encrypt community site: https://community.letsencrypt.org/t/2017-09-09-late-weak-key-revocation/42519 The text is copied here: On July 16, 2017 it was reported to Let’s Encrypt by researcher Hanno Böck that it was possible to get a certificate using a key known to be generated using the weak Debian random number generator. A specific certificate was given as an example. It so happens that Let’s Encrypt was already working on enhanced weak key checking which would have prevented the issuance in questions and deployment was imminent. Those mitigations were deployed to our production infrastructure on July 27, 2017. Let’s Encrypt was already checking for some types of weak keys as required by the Baseline Requirements, but we were not checking for the particular type of weak key that was reported to us on July 16, 2017. The Baseline Requirements specify that weak key checking must be done but they do not specify a particular algorithm, therefore Let’s Encrypt weak key checking was formally compliant both before and after the weak key mitigations deployed on July 27, 2017. However, we are always happy to improve the quality of our weak key checker. The Baseline Requirements do, however, require Let’s Encrypt to ensure that certificates are revoked if the associated private key is known to be compromised. We should have revoked the certificate referenced in the report from July 16, 2017, within 24 hours of receiving the report. We did not revoke the certificate within 24 hours of the report due to two contributing factors: the team was focused on improving weak key checking and the certificate was issued to a security researcher for testing purposes only. It was revoked on September 9, 2017, at 23:49 UTC, after the reporter posted publicly about the issue. As a result of this late revocation we have reviewed and improved our processes for handling incoming reports. ___ dev-security-policy mailing list dev-security-policy@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-security-policy
Re: Certificate with Debian weak key issued by Let's Encrypt
I'd like to push a bit harder on searching for more systemic remediations. "We forgot to get around to revoking it" is a pretty common element of CAs' post-mortems, I think it'd be good for us to dig deeper. For example, does Let's Encrypt have a runbook that gets used on misissuance reports? Is one the steps creating a persistent tracking item (e.g. a bug/ticket/issue) to revoke as required? I created https://misissued.com/ to assist the mdsp community in tracking these types of issues. It doesn't have any auth, so obviously anyone can put whatever garbage they want in (please don't though :-)), but if someone would find exposing data on a per-CA basis, or as JSON, or something else useful, I'd be more than happy to. These are just some off the cuff thoughts, but IMO "human error" should be a root cause of last resort. Alex On Mon, Sep 11, 2017 at 11:08 AM, josh--- via dev-security-policy < dev-security-policy@lists.mozilla.org> wrote: > This was simple human error. There isn't a programmatic fix. > > Our team is planning to scan our database for weak keys again early this > week. In any case, any weak key certs issued prior to our July 20 fix will > expire in at most 37 days. > > On Monday, September 11, 2017 at 8:24:49 AM UTC-5, Alex Gaynor wrote: > > Hi Josh, > > > > Does Let's Encrypt plan to implement any systematic or programmatic fixes > > to ensure certificates are promptly revoked in the future? > > > > Did you perform a scan of all your issued certificates to see if any > others > > were effected? > > > > Alex > > > > On Sat, Sep 9, 2017 at 8:14 PM, josh--- via dev-security-policy < > > dev-security-policy@lists.mozilla.org> wrote: > > > > > Thank you for bringing this oversight to our attention. The > certificate in > > > question has been revoked. > > > > > > The original incident report from July 16 was accidentally considered > > > closed on the basis of a fix for our infrastructure without actually > > > revoking the certificate that led to the report. > > > > > > Reading the recorded conversation, it seems we got overly focused on > fix > > > for our infrastructure and lost sight of the fact that the certificate > > > itself needed to be revoked. I imagine our guard was let down a bit by > the > > > fact that the cert was issued specifically to test us, it wasn't a > weak key > > > "in the wild." > > > > > > Let’s Encrypt has checked for some forms of weak keys since we > launched, > > > and we added additional checks that would have caught this on July 20, > > > 2017. We were already in the process of developing and deploying the > > > additional checks before we received the original report from Hanno. > > > > > > On Saturday, September 9, 2017 at 2:22:07 PM UTC-5, Hanno Böck wrote: > > > > Hi, > > > > > > > > A while ago I tested how some CAs would react to certificate requests > > > > with debian weak keys. > > > > > > > > I was able to get a certificate from Let's Encrypt with a debian weak > > > > key. Here is it: > > > > https://crt.sh/?id=173588030 > > > > > > > > I reported this to Let's Encrypt. They told me that they are aware > they > > > > weren't checking debian weak keys, but they were in the process of > > > > deploying a check: > > > > https://github.com/letsencrypt/boulder/pull/2765 > > > > > > > > I don't know if this is active by now, but I assume so. > > > > > > > > Maybe notable: The certificate hasn't been revoked, despite me > > > > reporting it. However I haven't explicitely asked for revocation > (and I > > > > could revoke it myself, given that I have the private key). > > > > > > > > > > > > I have also tried to get a cert with a debian weak key from the > > > > free trial offerings from Comodo and Symantec. Both rejected the > > > > request. > > > > > > > > -- > > > > Hanno Böck > > > > https://hboeck.de/ > > > > > > > > mail/jabber: ha...@hboeck.de > > > > GPG: FE73757FA60E4E21B937579FA5880072BBB51E42 > > > > > > ___ > > > dev-security-policy mailing list > > > dev-security-policy@lists.mozilla.org > > > https://lists.mozilla.org/listinfo/dev-security-policy > ___ > dev-security-policy mailing list > dev-security-policy@lists.mozilla.org > https://lists.mozilla.org/listinfo/dev-security-policy > ___ dev-security-policy mailing list dev-security-policy@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-security-policy
Re: Certificate with Debian weak key issued by Let's Encrypt
This was simple human error. There isn't a programmatic fix. Our team is planning to scan our database for weak keys again early this week. In any case, any weak key certs issued prior to our July 20 fix will expire in at most 37 days. On Monday, September 11, 2017 at 8:24:49 AM UTC-5, Alex Gaynor wrote: > Hi Josh, > > Does Let's Encrypt plan to implement any systematic or programmatic fixes > to ensure certificates are promptly revoked in the future? > > Did you perform a scan of all your issued certificates to see if any others > were effected? > > Alex > > On Sat, Sep 9, 2017 at 8:14 PM, josh--- via dev-security-policy < > dev-security-policy@lists.mozilla.org> wrote: > > > Thank you for bringing this oversight to our attention. The certificate in > > question has been revoked. > > > > The original incident report from July 16 was accidentally considered > > closed on the basis of a fix for our infrastructure without actually > > revoking the certificate that led to the report. > > > > Reading the recorded conversation, it seems we got overly focused on fix > > for our infrastructure and lost sight of the fact that the certificate > > itself needed to be revoked. I imagine our guard was let down a bit by the > > fact that the cert was issued specifically to test us, it wasn't a weak key > > "in the wild." > > > > Let’s Encrypt has checked for some forms of weak keys since we launched, > > and we added additional checks that would have caught this on July 20, > > 2017. We were already in the process of developing and deploying the > > additional checks before we received the original report from Hanno. > > > > On Saturday, September 9, 2017 at 2:22:07 PM UTC-5, Hanno Böck wrote: > > > Hi, > > > > > > A while ago I tested how some CAs would react to certificate requests > > > with debian weak keys. > > > > > > I was able to get a certificate from Let's Encrypt with a debian weak > > > key. Here is it: > > > https://crt.sh/?id=173588030 > > > > > > I reported this to Let's Encrypt. They told me that they are aware they > > > weren't checking debian weak keys, but they were in the process of > > > deploying a check: > > > https://github.com/letsencrypt/boulder/pull/2765 > > > > > > I don't know if this is active by now, but I assume so. > > > > > > Maybe notable: The certificate hasn't been revoked, despite me > > > reporting it. However I haven't explicitely asked for revocation (and I > > > could revoke it myself, given that I have the private key). > > > > > > > > > I have also tried to get a cert with a debian weak key from the > > > free trial offerings from Comodo and Symantec. Both rejected the > > > request. > > > > > > -- > > > Hanno Böck > > > https://hboeck.de/ > > > > > > mail/jabber: ha...@hboeck.de > > > GPG: FE73757FA60E4E21B937579FA5880072BBB51E42 > > > > ___ > > dev-security-policy mailing list > > dev-security-policy@lists.mozilla.org > > https://lists.mozilla.org/listinfo/dev-security-policy ___ dev-security-policy mailing list dev-security-policy@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-security-policy
Re: Certificate with Debian weak key issued by Let's Encrypt
Thank you for bringing this oversight to our attention. The certificate in question has been revoked. The original incident report from July 16 was accidentally considered closed on the basis of a fix for our infrastructure without actually revoking the certificate that led to the report. Reading the recorded conversation, it seems we got overly focused on fix for our infrastructure and lost sight of the fact that the certificate itself needed to be revoked. I imagine our guard was let down a bit by the fact that the cert was issued specifically to test us, it wasn't a weak key "in the wild." Let’s Encrypt has checked for some forms of weak keys since we launched, and we added additional checks that would have caught this on July 20, 2017. We were already in the process of developing and deploying the additional checks before we received the original report from Hanno. On Saturday, September 9, 2017 at 2:22:07 PM UTC-5, Hanno Böck wrote: > Hi, > > A while ago I tested how some CAs would react to certificate requests > with debian weak keys. > > I was able to get a certificate from Let's Encrypt with a debian weak > key. Here is it: > https://crt.sh/?id=173588030 > > I reported this to Let's Encrypt. They told me that they are aware they > weren't checking debian weak keys, but they were in the process of > deploying a check: > https://github.com/letsencrypt/boulder/pull/2765 > > I don't know if this is active by now, but I assume so. > > Maybe notable: The certificate hasn't been revoked, despite me > reporting it. However I haven't explicitely asked for revocation (and I > could revoke it myself, given that I have the private key). > > > I have also tried to get a cert with a debian weak key from the > free trial offerings from Comodo and Symantec. Both rejected the > request. > > -- > Hanno Böck > https://hboeck.de/ > > mail/jabber: ha...@hboeck.de > GPG: FE73757FA60E4E21B937579FA5880072BBB51E42 ___ dev-security-policy mailing list dev-security-policy@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-security-policy
Certificate with Debian weak key issued by Let's Encrypt
Hi, A while ago I tested how some CAs would react to certificate requests with debian weak keys. I was able to get a certificate from Let's Encrypt with a debian weak key. Here is it: https://crt.sh/?id=173588030 I reported this to Let's Encrypt. They told me that they are aware they weren't checking debian weak keys, but they were in the process of deploying a check: https://github.com/letsencrypt/boulder/pull/2765 I don't know if this is active by now, but I assume so. Maybe notable: The certificate hasn't been revoked, despite me reporting it. However I haven't explicitely asked for revocation (and I could revoke it myself, given that I have the private key). I have also tried to get a cert with a debian weak key from the free trial offerings from Comodo and Symantec. Both rejected the request. -- Hanno Böck https://hboeck.de/ mail/jabber: ha...@hboeck.de GPG: FE73757FA60E4E21B937579FA5880072BBB51E42 ___ dev-security-policy mailing list dev-security-policy@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-security-policy
