Issue T: RA Program Misissuances (January 2010 - January 2017)

Program Background:

Symantec has operated an RA program designed to deliver a superior customer 
experience in global markets where language skills, understanding of local 
business requirements, and physical local presence are necessary. RA partners 
have supported various certificate types, including those for publicly trusted 

The RA program for publicly trusted SSL/TLS authorized appropriately trained 
personnel at select RA partners to complete all steps for authentication, 
review, and certificate issuance.

In 2011, prior to ratification of the CA/Browser Forum Baseline Requirements, 
Symantec scaled back the scope of the RA program for publicly trusted SSL/TLS 
to support only those partners whose scale of business and investment in the 
future success of that business warranted the additional cost associated with 
supporting the then-new BRs. Since 2013, there have been only 4 RA partners 
still capable of processing and issuing publicly trusted SSL/TLS certificates: 
CrossCert, Certisign, Certsuperior, and Certisur.

Symantec has had multiple controls in place to ensure these RAs' compliance 
with the BRs:

1. Symantec operates an internal Knowledge Base ("KB") for its authentication 
staff and RA partners that contains detailed step-by-step procedures for 
performing each of the tasks required to validate the identity asserted in a 
certificate request.
2. The KB reinforces acceptable and unacceptable sources of validation 
information and processes using a subset of the information in the BRs.
3. The KB explains request flagging, flag reasons, and flag clearing procedures.

Training & Exams:
1. Topics include BR changes, CPS changes, process changes resulting from 
industry incidents regardless of the CA involved, and a review of Symantec's 
procedures that extend the Baseline Requirements.
2. Exams are modified and retaken annually as criteria to renew individual 
access certificates or after significant internal or external process changes.

Technology During Authentication:
1. Each request is screened for trade compliance, high-risk names, potential 
phishing (strings used in scam domains, high-profile brands), and other 
potentially risky content such as "test". Potential failures are flagged, 
preventing RA issuance, until and unless further review and analysis is 
2. Risk flags require manual override by authentication personnel - internal or 
RA personal as appropriate - for certificate issuance to proceed. Flag clearing 
privileges are only granted to personnel who are have completed the requisite 
training and passed appropriate exams.

Technology Pre- and Post- Issuance:
1. Each request is screened to ensure elements outside of the subject 
information are BR compliant (e.g. SAN fields are complete, proper validity 
limits are in place, 2048 bit or higher key lengths are used, etc.). This scan 
is done after Authentication personnel approve the request and before it is 
issued. These checks cannot be overridden.
2. Daily, we rescan all certificates issued on the prior day using these same 

1. We requested independent WebTrust audit reports from RAs and assessed them 
for material findings pursuant to BR 8.4 regarding WebTrust audited Delegated 
Third Parties. See issue V addressing audits.

Customer-Facing Controls:
1. Symantec supports Certification Authority Authorization, putting control of 
authorized CAs in the hands of customers.
2. Symantec logs publicly trusted certificates to Certificate Transparency Logs 
and offers a CT monitor to provide visibility for all customers to enable 
detection of suspect certificates.

CrossCert Test Certificate Issue:

On January 19, 2017, Andrew Ayer, an independent researcher posted the results 
of an analysis of public Certificate Transparency logs through which he 
identified roughly 270 instances of suspicious certificates issued by multiple 
CAs, including Symantec, containing the words "test" or "example" in the 
subject information.

Symantec determined that 127 of these certificates were issued from Symantec 
operated CAs and that all 127 had been issued by the RA CrossCert. All but 31 
had already expired or been revoked.

Immediate Response
Andrew Ayer's report was a certificate problem report under BR 4.9.5 which 
required us to begin an investigation within 24 hours, which we did. We 
determined that 127 certificates were in scope of the problem report.

1. On January 19, 2017, after becoming aware of this issue, Symantec disabled 
issuance privileges for all CrossCert staff.
2. On January 20, 2017, Symantec revoked the 31 still valid and active 
certificates. These certificates had been issued between December 28, 2016 and 
January 18, 2017.
3. Symantec promptly took over validation and issuance for all pending and new 
orders submitted through CrossCert. Since then, Symantec's authentication team 
has continued to fully process these orders independently, without relying on 
any previous authentication work completed by CrossCert.
4. Symantec disabled customer issuance from enterprise accounts originally 
authenticated by CrossCert. New issuance from these accounts was blocked until 
the accounts were re-validated by Symantec personnel.

Root Cause
Following discussions with CrossCert and review of their audit logs, Symantec 
confirmed that all 127 certificates were issued as part of either internal or 
customer-specific testing. In mid-2016, CrossCert created a customer-focused 
feature with good intent to help customers with testing, but they did so 
without following the proper guidelines.

CrossCert personnel had completed the Symantec-required training mandating that 
only fully validated certificates be issued. At no point was CrossCert 
authorized to perform less vetting on certificates for their own internal use 
or for customer testing. The personnel at CrossCert had also completed exams 
designed to verify comprehension of this training. Nonetheless, CrossCert 
management authorized overrides for certificates being used for internal and 
customer-specific testing, and CrossCert authentication personnel complied with 
that authorization.

Audit logs confirm that our controls to flag these certificates for compliance 
failures worked appropriately but CrossCert management overrode those flags. 
CrossCert did not consult with Symantec on the significance of the compliance 
failure flags or their decisions to override the flags for any of the 
certificates. Our flagging process is extensive and a normal part of every 
business day. For example, our process catches popular domains/trademarks and 
variants using fuzzy logic that might otherwise be issued by a CA that doesn't 
test or block trademark abuse. Our flag/hold rate is conservative - as a 
result, the false positive rate is high and, following manual analysis, most 
flags are cleared. We appreciate the suggestion that additional flag monitoring 
and trend analysis would provide greater insight and increase the opportunity 
to catch issues like this more quickly in the future.

Review of the 127 certificates showed that all issuances occurred in two time 
frames: Before June, 2012 or after June, 2016. Symantec received unqualified 
WebTrust for CAs and WebTrust for CAs Baseline and Network Security audits from 
Ernst & Young KR for the period July 1, 2015-June 30, 2016. Each of the 127 
certificates issued for testing purposes were issued either years before these 
audits or after these most recent audits and so not detectable as part of these 
independent audits.

Subsequent Investigation
As part of investigating this incident, Symantec began a direct review of the 
authentication practices at CrossCert and their engagement with Ernst & Young 
KR. Through that investigation, Symantec identified two additional issues:

1. There were deficiencies in CrossCert's documentation of the validation step 
in the authentication process, failing to meet both Symantec and Baseline 
2. The CrossCert 2015-2016 audit did not include in its scope all of the CAs 
from which CrossCert was authorized to issue certificates. E&Y KR stated that 
CrossCert provided a list of all their issuing CAs but reduced the list of 
issuing CAs in scope for sampling for budgetary reasons. E&Y KR did not call 
out this known limit in its report.

These additional findings caused Symantec to seriously question not only 
CrossCert, but also the adequacy of E&Y KR's WebTrust for CAs Baseline and 
Network Security audits. This episode also reinforced that we need to more 
closely monitor and review the scope, methodology, and practices of external 

1. Symantec fully terminated CrossCert as an RA partner. They no longer 
participate in the authentication process or issuance of publicly trusted 
SSL/TLS certificates.
2. Symantec has hired a team of Korean speaking authentication personnel who 
have completed the requisite training and now independently process all new 
orders from this market.
3. Symantec is in the process of fully and independently revalidating active 
SSL/TLS certificates previously approved by CrossCert, and if necessary, 
revoking and replacing certificates if we detect any errors. As we posted in 
response to the assertion about reusing RA validation in m.d.s.policy, we 
reiterate that we are not doing this.  For CrossCert, we are validating each 
existing certificate as if it was a new request.
4. Symantec announced the decision to wind down the RA program for publicly 
trusted SSL/TLS.
5. Symantec is in the process of reviewing 100% of the authentication records 
for the active SSL/TLS certificates approved and issued by Certisign, 
Certsuperior, and Certisur. If we become aware of a certificate that is 
misleading, we follow the Baseline Requirements regarding revocation.

It has been suggested that the issues identified above and those associated 
with Issue V warrant immediate revocation of all certificates issued by each of 
these RAs. Symantec disagrees with that assessment. Revocation by a CA of 
certificates, outside of a customer request, can be materially disruptive to 
the individuals and the organizations that rely on them, and so should be 
reserved for cases where there are clear security risks.

In the case of CrossCert, Symantec immediately revoked the clearly problematic 
certificates mis-issued for testing. With respect to individual certificates, 
the key additional issue identified with CrossCert related to proper 
record-keeping.  Accordingly, using a risk-informed approach, Symantec is fully 
re-validating these certificates.

In the cases of Certisign, Certsuperior, and Certisur, we relied on at least 
one unqualified audit for their operations. With respect to their individual 
certificates, there was no evidence of any problems with the certificates they 
approved and issued. Accordingly, and again using a risk-informed approach, 
Symantec is reviewing all of these certificates, which amounts to conducting an 
audit with 100% sampling.

Symantec will provide updates to the community regarding our revalidation of 
active CrossCert-issued SSL/TLS certificates and our review of the other RA 

dev-security-policy mailing list

Reply via email to