Let's Encrypt likes to learn about the ecosystem from the experiences of the community. Today we want to bring some information about an event that may be interesting or useful to other CAs in precisely that way.
One of our engineers discovered a bug in Boulder <https://github.com/letsencrypt/boulder/pull/8096> (our Certificate Authority software) which caused revoked certificates included in one of our CRL shards to retain their original revocation reason code despite a subsequent ACME revoke-certificate request which "upgraded" the reason code to keyCompromise. In succinct technical detail, our engineer described the bug in this way: > For explicitly sharded certificates, CRL status is read from the > revokedCertificates table. This table gets written at revocation time. At > re-revocation time (for key compromise), it only gets written by the SA if > the caller passes a nonzero ShardIdx to UpdateRevokedCertificate. The RA > was never passing a nonzero ShardIdx to UpdateRevokedCertificate. Note: “SA” and “RA” here are referencing our Boulder components, and “RA” is not a typical “Registration Authority”. This prompted an investigation covering two primary points: 1) What language in the Baseline Requirements and Root Programs covers the "upgrade" of reason codes in CRLs, or the expectations around re-revocation? 2) Since this bug did not affect our OCSP infrastructure, what are the implications of CRLs and OCSP responses containing different reason codes for a single revoked certificate? For the first point, we found what we believe to be clear language in Mozilla's Root Store Policy and the BRs: - First paragraph of Mozilla's Root Store Policy 6.1.1 <https://www.mozilla.org/en-US/about/governance/policies/security-group/certs/policy/#611-end-entity-tls-certificate-crlrevocation-reasons> > When an end entity TLS certificate (i.e. a certificate capable of being > used for TLS-enabled servers) is revoked for one of the reasons below, the > specified CRLReason MUST be included in the reasonCode extension of the CRL > entry corresponding to the end entity TLS certificate, as described in > sections 4.9.1 and 7.2.2 of the TLS BRs. - Final paragraph of Baseline Requirements 7.2.2 <https://cabforum.org/working-groups/server/baseline-requirements/documents/CA-Browser-Forum-TLS-BR-2.1.4.pdf#51> > When a CA obtains verifiable evidence of Key Compromise for a Certificate > whose CRL entry does not contain a reasonCode extension or has a reasonCode > extension with a non‐keyCompromise reason, the CA SHOULD update the CRL > entry to enter keyCompromise as the CRLReason in the reasonCode extension. For the second point, language specifically discussing the possibility of a mismatch between CRL and OCSP revocation reasons was harder to find. What we did find seemed to represent a design of eventual consistency, which makes operational sense given the very different nextUpdate requirements for CRLs vs OCSP, and possible variability for software that is publishing CRLs vs software responding to OCSP requests. Even if "SHOULD update the CRL entry" appears to leave room for this sort of thing, it did not match our expectations on our system. We fixed the bug, deployed the new version, and we tracked-down three certificates in our database which would have shown as Revoked in both CRLs and OCSP, but which would have shown reason code "keyCompromise" in OCSP while retaining their original reason code in CRLs (all three happened to be "unspecified"). Re-revocation is pretty rare, and re-revocation with the new reason being keyCompromise is even more rare. We started using the previously-mentioned revokedCertificates table in early March, and only accumulated these three. We updated the affected rows, and all is consistent. Thank you for reading, we welcome questions and discussion. -- Kiel C SRE at Let's Encrypt -- You received this message because you are subscribed to the Google Groups "CCADB Public" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion visit https://groups.google.com/a/ccadb.org/d/msgid/public/CABK0L0038%2Bo235fz7oM2YkQDNhdBVdF8DQmKsQ%2B--bq%3DHbtC-Q%40mail.gmail.com.
