Thanks for reporting this, David. I've filed https://bugzilla.mozilla.org/show_bug.cgi?id=1522975 to track both discussion and remediation.
It would be useful if you can add to the timeline both the changes you've made in response and when you anticipate the remaining remediation steps to be taken, so that we can track them. On Fri, Jan 25, 2019 at 3:19 PM David Kluge via dev-security-policy < dev-security-policy@lists.mozilla.org> wrote: > Summary > During a signing ceremony in October 2018, Google Trust Services generated > OCSP responses for five of its subordinate CAs and published them > afterwards. On 11 January 2019 it was discovered that one of these > responses was not created accurately. > > The incorrect OCSP response did not have an impact on subscribers or > relying parties because the concerned CA GTSY1 is not in operation yet. By > publishing this report we nevertheless want to share our lessons learned > and hope that they will help other CAs improve their systems and processes. > > Cause and Detection > A key ceremony tool was used to generate the OCSP responses based on a > config file which specifies all relevant parameters of the output files to > be created. > The config file for the October ceremony was prepared, reviewed and tested > following established procedures to ensure quality and conformity with the > Baseline Requirements. Subsequently, it was submitted to a version control > system. In a later review, a CA engineer discovered that some of the serial > numbers in the file were in decimal and others in hex format. To make the > number formats consistent across the file, they submitted a change list > replacing the decimal numbers with their hex representation. The change > list contained a paste error, which assigned the GTSY1 OCSP response a > serial number which does not correspond to the serial number of the CA > certificate. > > The change list was reviewed before it was submitted but the review did > not catch the mismatch. On 11 January 2019 a CA engineer identified it > while doing work to prepare for use of the previously generated certificate. > > Timeline > 2018-10-18 Config file is created and submitted to version control > system > 2018-10-18 Change list is submitted against config file (convert > decimal to hex serial numbers) > 2018-10-29 OCSP responses are created and signed during a ceremony > 2018-11-03 OCSP responses are published > 2019-01-11 Serial mismatch is discovered and the root cause > investigated. An impact assessment is performed and it yields that relying > parties and subscribers are not affected. > 2019-01-14 Generation of new OCSP response is planned and tested > 2019-01-17 New OCSP response is generated > 2019-01-18 New OCSP response is published > > > Analysis and Findings > The serial number mismatch was the result of a human error made when > composing the change list. As a control against such errors we enforce the > performance of code reviews before change lists can be submitted. The > reviews are supported by a set of linters for various purposes. Tests for > serial number consistency were not implemented at the time of the error, > but have been added since that time. > Additionally, OCSP responders are monitored and alerts are generated if > certain conditions are met. CA specific alerting for GTSY1 was not > configured when the OCSP response was published because the CA was not in > an issuing state. > > Remediation > -Correct OCSP response for GTSY1 has been prepared and published. > -CA specific alerting was configured on the OCSP responder for all newly > created subCAs regardless of their operating status. > -Instructions for generating subCAs were improved to update the OCSP > monitoring configuration without waiting for the subCA to be productionized. > -Additional linters are being implemented which test the accuracy and > plausibility of ceremony config files. > > _______________________________________________ > dev-security-policy mailing list > dev-security-policy@lists.mozilla.org > https://lists.mozilla.org/listinfo/dev-security-policy > _______________________________________________ dev-security-policy mailing list dev-security-policy@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-security-policy
