On October 17, Apple submitted an incident report: 
https://bugzilla.mozilla.org/show_bug.cgi?id=1588001#c3, which is reposted 
below. 

Incident Report

1. How your CA first became aware of the problem (e.g. via a problem report 
submitted to your Problem Reporting Mechanism, a discussion in 
mozilla.dev.security.policy, a Bugzilla bug, or internal self-audit), and the 
time and date.

On 03-October-2019 at 13:51 PT, we were notified via a problem report submitted 
to our Problem Reporting Mechanism that our OCSP responders were returning 
signed responses with incorrect issuer. 

2. A timeline of the actions your CA took in response. A timeline is a 
date-and-time-stamped sequence of all relevant events. This may include events 
before the incident was reported, such as when a particular requirement became 
applicable, or a document changed, or a bug was introduced, or an audit was 
done.

03-October-2019 at 13:51 PT - We were notified via a problem report submitted 
to our Problem Reporting Mechanism that our OCSP responders were returning 
signed responses with incorrect issuer.

03-October-2019 at 13:52 PT - The compliance team read the problem report and 
began pulling the appropriate people together to begin the investigation.

03-October-2019 at 14:15 PT - Investigation began to confirm the reported 
behavior. 

03-October-2019 at 16:15 PT -  Based on an initial investigation, we determined 
that in some cases when the OCSP service receives a request it can’t process, 
it returns a status of unknown signed with a default OCSP responder, which is 
not always signed by the CA that issued the certificate whose revocation status 
is being checked.

03-October-2019 at 19:21 - Notified DigiCert (Root vendor).

03-October-2019 at 20:10 - We responded to the reporter with an initial 
acknowledgement and a commitment to investigate further and respond with more 
details within 24 hours of the report being submitted.

04-October-2019 at 10:10 PT - Notified Sectigo (Root vendor).

04-October-2019 at 11:26 PT - Notified Ernst & Young (WebTrust assessors).

04-October-2019 at 13:48 PT - We provided a preliminary report on our findings 
to the individual who filed the problem report.

07-October-2019 at 14:15 PT - We began rolling out a fix to our OCSP service. 
The fix was to disable the default OCSP responder so that the responses are 
always signed by the CA that issued the certificate whose revocation status is 
being checked. Disabling the default OCSP responder ensures that the responder 
will reply ‘unauthorized’  (as per RFC 6960) for all unknown issuers. The 
issuer may be unknown if the OCSP service cannot identify the issuer, such as 
when an OCSP client uses a hash algorithm for CertID that the OCSP service does 
not support or when the request indicates an unrecognized issuer that is not 
served by our OCSP service.

10-October-2019 at 19:38 PT  - Posted initial incident report to Bugzilla.

17-October-2019 - The fix has been pushed out to the majority of our production 
OCSP service and scheduled for completion by 18-October-2019.

3. Whether your CA has stopped, or has not yet stopped, issuing certificates 
with the problem. A statement that you have will be considered a pledge to the 
community; a statement that you have not requires an explanation.

No non-compliant certificates were issued. The OCSP fix has been pushed out to 
the majority of our production OCSP service and scheduled for completion by 
18-October-2019.

4. A summary of the problematic certificates. For each problem: number of 
certs, and the date the first and last certs with that problem were issued.

No non-compliant certificates were issued.

5. The complete certificate data for the problematic certificates. The 
recommended way to provide this is to ensure each certificate is logged to CT 
and then list the fingerprints or crt.sh IDs, either in the report or as an 
attached spreadsheet, with one list per distinct problem.

No non-compliant certificates were issued.

6. Explanation about how and why the mistakes were made or bugs introduced, and 
how they avoided detection until now.

When the OCSP service was first set up in 2012, the OCSP service software did 
not allow the default OCSP responder to be disabled. We began issuing publicly 
trusted TLS certificates in 2014 and the default OCSP responder was not 
disabled. We did not identify this as an issue because our test cases did not 
address scenarios in which our OCSP service could not identify the issuer and 
thus signs the response with the default OCSP responder.

The default OCSP responder is only used when the OCSP service cannot identify 
the issuer. This may occur when an OCSP client uses a hash algorithm for CertID 
that the OCSP service does not support or when the request indicates an 
unrecognized issuer that is not served by our OCSP service. 

We were aware of section 4.9.9 of the Baseline Requirements that states that 
OCSP responses must “Be signed by an OCSP Responder whose Certificate is signed 
by the CA that issued the Certificate whose revocation status is being 
checked.” but we were unaware that our OCSP responder configuration was 
violating that requirement when the issuer could not be identified due to the 
default OCSP responder.

7. List of steps your CA is taking to resolve the situation and ensure such 
issuance will not be repeated in the future, accompanied with a timeline of 
when your CA expects to accomplish these things.

i. As mentioned in the timeline above, we are in the process of changing our 
OCSP service configurations to ensure we respond with an unsigned OCSP error of 
‘unauthorized’ for all unknown issuers by disabling the default OCSP responder. 
This is scheduled for completion by 18-October-2019.

ii. We are enhancing our OCSP service test cases to include additional OCSP 
request scenarios to ensure responses are compliant with Baseline Requirements. 
This is scheduled for completion by 22-November-2019.
_______________________________________________
dev-security-policy mailing list
dev-security-policy@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-security-policy

Reply via email to