Thanks, Kathleen, for raising these issues. At a high level, this highlights an interesting concern. If we, as the broader community, lack the expertise to appropriate review and consume the audit reports as intended, it may signal a question about whether or not we should consider consuming ETSI reports. Thus, to ensure ETSI reports continue to be viable for CAs to provide, it would behove those supporters and professionals to ensure there is a robust understanding about how to consume such reports, much as there is similar ongoing discussion (and much better expertise) towards the consumption of WebTrust reports.
The text you've described seems to align with the process outlined in https://assets.kpmg.com/content/dam/kpmg/ch/pdf/kpmg-certification-compliance-and-methodology-en.pdf However, that process notably differs from the expectations of the Mozilla Root Certificate Policy and expectations. As you've noted, the letter calls out that this is a main certification audit (good!), but then notes surveillance audits are planned. While permissible under the appropriate ETSI guidelines, for purposes of inclusion in the Mozilla store, the expectation is that a full assessment audit is performed each year. That is, http://www.etsi.org/deliver/etsi_en/319400_319499/319403/02.02.02_60/en_319403v020202p.pdf Section 7.9 permits (limited scope) assessments, but Mozilla does not. Thus, despite being a main certification audit (which is expected), the fact that it's called out that other items are planned is concerning. I think this also aligns with why "agreed upon procedures" is concerning. An AUP could indicate that the CA has gone above and beyond the set of controls, or it may indicate that the CA has self-limited the scope of the engagement to a subset of activities. It may be that the activities are not performed, as we've seen similarly in WebTrust audits (for example, with respect to certificate suspension). Unfortunately, this is not clear - and there's not enough information in the report, as best I can tell, to distinguish this case. I am very concerned with respect to the "point in time" audit. There should not be any ambiguity here - both with respect to Mozilla expectations and to the general understanding, as previously discussed in the CA/Browser Forum with the respective auditor communities. To an extent, this aligns with my understanding of a "main certification audit" - that is, that they may be inclined to be "point in time" audit, with the surveillance audit being a review of the CA's adherence to that. I agree there's some ambiguity with the statement of "Note that the corresponding certification report was written in German and is only intended for the client". On one sense, we want to ensure that the report provided is consistent with and 'binding' of the auditor (conformance assessment body) and a statement they'll stand by, hence the concern. That said, the report also asserts KPMG's SCESm registration, so presumably, this is an official statement. The scope is noted as discussed further at https://www.seco.admin.ch/sas/PKI , except that URL isn't valid. Similarly, I agree, it's concerning that it's noted as conforming to the EVCP profile, but similarly notes "We were not engaged to and did not conduct an examination, the objective of which would be the expression of an opinion on the Application for Extended Validation (EV) Certificate." It's not clear how to reconcile this difference. Another element of uncertainty is that the policies are listed as compliance to "DVCP and PTC-BR", except that with respect to EN 319 411-1, it states that "NOTE: Within the context of the present document PTC is used synonymously with EVC, DVC and OVC as per CAB Forum documents. ". So is this indicative that they evaluated DVCP, OVCP, and EVCP? Or something else? As best I can tell, "PTC-BR" is an artifact from the predecessor document, TS 102 042, and not relevant in the context of 411-1. One other thing that I'm unclear with - it's notable that in audits performed by other CABs provide certificates of compliance. For example, TUVIT provides https://www.tuvit.de/en/services/certification/certification-authorities-according-to-etsi/ . My understanding from the ETSI representatives to the CA/B Forum is that such certifications are standard - but it's unclear that this represents such a certification. Perhaps this is only unique to TUVIT? On Mon, Jun 19, 2017 at 4:57 PM, Kathleen Wilson via dev-security-policy < dev-security-policy@lists.mozilla.org> wrote: > On Monday, June 19, 2017 at 12:21:46 PM UTC-7, Peter Bowen wrote: > > It seems there is some confusion. The document presented would appear > > to be a Verified Accountant Letter (as defined in the EV Guidelines) > > and can used as part of the process to validate a request for an EV > > certificate. It is not an audit report and is not something normally > > submitted to browsers. > > Yet, it is the document that was provided to root store operators as the > annual audit statement. And there has been plenty of time in Bug #1142323 > for that to have been rectified. > > As reference, here is the audit statement that was provided in 2016: > https://bug343756.bmoattachments.org/attachment.cgi?id=8781268 > It says: "KPMG has executed a main certification audit in year 2013, and > surveillance certification audits in 2014 and 2015..." > "We were engaged to conduct the annual examinations, with the objective of > which would be the expression of an opinion on the application for Extended > Validation (EV) Certificates. Accordingly we do express our positive > opinion and provide you confirmation that the requirements were fulfilled > during the annual certification audits... " > > > In the audit statement in question (https://bug1142323. > bmoattachments.org/attachment.cgi?id=8853299) it says: > "KPMG has executed a main certification audit in year 2017..." So I took > that to mean that this was intended to be their annual audit statement, and > the format is very similar to the audit statement from the previous year. > But as I read through it I noticed phrases like "point in time audit". And > then it said: > "We were not engaged to and did not conduct an examination, the objective > of which would be the expression of an opinion on the Application for > Extended Validation (EV) Certificate. Accordingly, we do not express such > an opinion. Had we performed additional procedures, other matters might > have come to our attention that would have been reported to you." > This is very different from the statement the previous year. > > Thanks, > Kathleen > > _______________________________________________ > dev-security-policy mailing list > dev-security-policy@lists.mozilla.org > https://lists.mozilla.org/listinfo/dev-security-policy > _______________________________________________ dev-security-policy mailing list dev-security-policy@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-security-policy
