On October 17, Apple submitted an incident report: https://bugzilla.mozilla.org/show_bug.cgi?id=1588001#c3, which is reposted below.
Incident Report 1. How your CA first became aware of the problem (e.g. via a problem report submitted to your Problem Reporting Mechanism, a discussion in mozilla.dev.security.policy, a Bugzilla bug, or internal self-audit), and the time and date. On 03-October-2019 at 13:51 PT, we were notified via a problem report submitted to our Problem Reporting Mechanism that our OCSP responders were returning signed responses with incorrect issuer. 2. A timeline of the actions your CA took in response. A timeline is a date-and-time-stamped sequence of all relevant events. This may include events before the incident was reported, such as when a particular requirement became applicable, or a document changed, or a bug was introduced, or an audit was done. 03-October-2019 at 13:51 PT - We were notified via a problem report submitted to our Problem Reporting Mechanism that our OCSP responders were returning signed responses with incorrect issuer. 03-October-2019 at 13:52 PT - The compliance team read the problem report and began pulling the appropriate people together to begin the investigation. 03-October-2019 at 14:15 PT - Investigation began to confirm the reported behavior. 03-October-2019 at 16:15 PT - Based on an initial investigation, we determined that in some cases when the OCSP service receives a request it can’t process, it returns a status of unknown signed with a default OCSP responder, which is not always signed by the CA that issued the certificate whose revocation status is being checked. 03-October-2019 at 19:21 - Notified DigiCert (Root vendor). 03-October-2019 at 20:10 - We responded to the reporter with an initial acknowledgement and a commitment to investigate further and respond with more details within 24 hours of the report being submitted. 04-October-2019 at 10:10 PT - Notified Sectigo (Root vendor). 04-October-2019 at 11:26 PT - Notified Ernst & Young (WebTrust assessors). 04-October-2019 at 13:48 PT - We provided a preliminary report on our findings to the individual who filed the problem report. 07-October-2019 at 14:15 PT - We began rolling out a fix to our OCSP service. The fix was to disable the default OCSP responder so that the responses are always signed by the CA that issued the certificate whose revocation status is being checked. Disabling the default OCSP responder ensures that the responder will reply ‘unauthorized’ (as per RFC 6960) for all unknown issuers. The issuer may be unknown if the OCSP service cannot identify the issuer, such as when an OCSP client uses a hash algorithm for CertID that the OCSP service does not support or when the request indicates an unrecognized issuer that is not served by our OCSP service. 10-October-2019 at 19:38 PT - Posted initial incident report to Bugzilla. 17-October-2019 - The fix has been pushed out to the majority of our production OCSP service and scheduled for completion by 18-October-2019. 3. Whether your CA has stopped, or has not yet stopped, issuing certificates with the problem. A statement that you have will be considered a pledge to the community; a statement that you have not requires an explanation. No non-compliant certificates were issued. The OCSP fix has been pushed out to the majority of our production OCSP service and scheduled for completion by 18-October-2019. 4. A summary of the problematic certificates. For each problem: number of certs, and the date the first and last certs with that problem were issued. No non-compliant certificates were issued. 5. The complete certificate data for the problematic certificates. The recommended way to provide this is to ensure each certificate is logged to CT and then list the fingerprints or crt.sh IDs, either in the report or as an attached spreadsheet, with one list per distinct problem. No non-compliant certificates were issued. 6. Explanation about how and why the mistakes were made or bugs introduced, and how they avoided detection until now. When the OCSP service was first set up in 2012, the OCSP service software did not allow the default OCSP responder to be disabled. We began issuing publicly trusted TLS certificates in 2014 and the default OCSP responder was not disabled. We did not identify this as an issue because our test cases did not address scenarios in which our OCSP service could not identify the issuer and thus signs the response with the default OCSP responder. The default OCSP responder is only used when the OCSP service cannot identify the issuer. This may occur when an OCSP client uses a hash algorithm for CertID that the OCSP service does not support or when the request indicates an unrecognized issuer that is not served by our OCSP service. We were aware of section 4.9.9 of the Baseline Requirements that states that OCSP responses must “Be signed by an OCSP Responder whose Certificate is signed by the CA that issued the Certificate whose revocation status is being checked.” but we were unaware that our OCSP responder configuration was violating that requirement when the issuer could not be identified due to the default OCSP responder. 7. List of steps your CA is taking to resolve the situation and ensure such issuance will not be repeated in the future, accompanied with a timeline of when your CA expects to accomplish these things. i. As mentioned in the timeline above, we are in the process of changing our OCSP service configurations to ensure we respond with an unsigned OCSP error of ‘unauthorized’ for all unknown issuers by disabling the default OCSP responder. This is scheduled for completion by 18-October-2019. ii. We are enhancing our OCSP service test cases to include additional OCSP request scenarios to ensure responses are compliant with Baseline Requirements. This is scheduled for completion by 22-November-2019. _______________________________________________ dev-security-policy mailing list dev-security-policy@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-security-policy
