I posted this to Bugzilla last night. Basically, we had an issue with validation that resulted in some certs issuing without proper (post-Aug 1) domain verification. Still working out how many. The major reason was lack of training by the validation staff combined with a lack of strict document controls in the early part of the Symantec-DigiCert transition.
1. How your CA first became aware of the problem (e.g. via a problem report submitted to your Problem Reporting Mechanism, a discussion in mozilla.dev.security.policy, a Bugzilla bug, or internal self-audit), and the time and date. On 2018/08/07 at 17:00 UTC, a customer submitted a request for information about our validation process for the verification of four of their domains. Upon investigation, we found that the four domains were not properly validated using a post-Aug 1 domain validation method. When attempting to revalidated the domains prior to August 1, the random value was sent to an address other than the WHOIS contact. This launched a broader investigation into our overall revalidation efforts. This investigation is ongoing. 2. A timeline of the actions your CA took in response. A timeline is a date-and-time-stamped sequence of all relevant events. This may include events before the incident was reported, such as when a particular requirement became applicable, or a document changed, or a bug was introduced, or an audit was done. >From approximately February through April 2018, DigiCert permitted some legacy Symantec customers to use Method 1 to validate their domains. Use of the method was subject to manager approval and reserved only for those companies that had urgent replacement deadlines that could not be met with an alternative validation method. Under this process, prior to approval, the validation staff was required to match the WHOIS company information and obtain approval using the WHOIS email address. Around April, this process was modified to include a BR-compliant Random Value that the validation staff sent using the WHOIS contact information. Use of the random value indicated acceptance. Adding the random value effectively transformed the validation from Method 1 to Method 2. The email could include multiple domains with the understanding that the WHOIS contact information had to match each domain listed. We believe that in some cases either the validation staff failed to match the WHOIS contact information for each domain listed, approving the certificate solely based on the existing verified registrant info, or the system did not check whether the WHOIS contact information matched the email address used in the original confirmation. On Aug 1, 2018, Ballot 218 took effect, deprecating Method 1. On, August 7, 2018, a customer requested the audit trail of a certificate issued using our new process. Upon review, validation management discovered the validation was improper because the previously verified email contact information did not match the WHOIS contact information. This discovery created an escalation up to management. On August 13, 2018, we stopped all issuance based on the process that converted Method 1 validations to Method 2 validations. We're currently investigating and will post an update when we know the number of certificates and more about what went wrong. For now, we know the number of impacted certificates is just under 2,500. We should have a clearer picture shortly, after we have conducted a manual review of all 2,500 certificates. 3. Whether your CA has stopped, or has not yet stopped, issuing certificates with the problem. A statement that you have will be considered a pledge to the community; a statement that you have not requires an explanation. On August 13, although most of these validations were likely properly completed, we stopped issuance using information converted from Method 1 until completing a more thorough investigation. 4. A summary of the problematic certificates. For each problem: number of certs, and the date the first and last certs with that problem were issued. Approximately 2,500 certificates are under review for validation issues. We wanted to get the incident report out quickly as an FYI while our investigation continues. We'll update this section in a final report. 5. The complete certificate data for the problematic certificates. The recommended way to provide this is to ensure each certificate is logged to CT and then list the fingerprints or crt.sh IDs, either in the report or as an attached spreadsheet, with one list per distinct problem. Still under review. We will upload them to the bug once we have a complete list. Because the error was human, we are reviewing each validation to determine whether Method 2 was correctly used. Once we complete our review, we'll post a Bugzilla attachment with the links for revoked certificates. 6. Explanation about how and why the mistakes were made or bugs introduced, and how they avoided detection until now. Preliminarily, we believe that the revalidation process that relied on previously verified information failed because there were not technical controls in place to ensure information integrity and because that the validation staff received insufficient training on the process of uploading WHOIS information., This resulted in validation staff adding some domains to the approval email where the WHOIS contact information didn't always match all the other domains listed. When we performed revalidation of domains in preparation for Ballot 218's effective date, we considered the existing domain information to be correct and reusable until the normal expiration date. However, this inadvertently blended poor quality validations due to the failure to check WHOIS contact information with properly completed Method 2 validations, allowing certs to issue post Aug 1 without proper domain validation. 7. List of steps your CA is taking to resolve the situation and ensure such issuance will not be repeated in the future, accompanied with a timeline of when your CA expects to accomplish these things. We stopped all issuance relying on impacted domains until the domain control is revalidated. We are also investigating each impacted domain's validation to determine whether the email was sent to the appropriate WHOIS contact. Any certs issued based on an improper validation will be revoked or revalidated using an approved method. We will update this issue and post to the Mozilla list when we have a precise number of impacted certs. We're still considering what additional action to take and will post an update when we figure out more about what went wrong. Jeremy NEW
smime.p7s
Description: S/MIME cryptographic signature
_______________________________________________ dev-security-policy mailing list [email protected] https://lists.mozilla.org/listinfo/dev-security-policy
