Am Montag, 30. Oktober 2017 22:19:31 UTC+1 schrieb Ryan Sleevi: > On Mon, Oct 30, 2017 at 3:50 PM, Kathleen Wilson via dev-security-policy < > [email protected]> wrote: > > > > How do we get all auditors to start meeting our audit statement > > requirements? > > > > Why haven't all included CAs communicated these requirements to their > > auditors? > > > > Why am I seeing so many audit statements (particularly ETSI audit > > statements) that do not meet our requirements? > > > > I will greatly appreciate thoughtful and constructive ideas on this. > > > > Thanks, > > Kathleen > > > > Kathleen, > > Thanks for raising this matter! I think it's an important highlighting of > the very different approaches to auditing employed by WebTrust and ETSI, > and the underlying reliability and assurance of those audits. > > Below is my attempt to summarize my understanding so far: > - ETSI EN 319 411-1 specifies generally applicable policy and security > requirements for trust service providers (TSPs) - including website > certificates. > - The purpose of EN 319 411-1 is to provide a framework for assessment of a > TSP, but does not specify how that assessment is to be caried out (c.f. > Section 1 of EN 319 411-1) > - EN 319 411-1 mentions EN 319 403 for guidance on such assessments > - EN 319 403 provides the framework for conformity assessment bodies (CABs) > to evaluate TSPs. It's based on/extends ISO/IEC 17065 specific to TSPs. > - ISO/IEC 17065 is incorporated due to Regulation (EC) No 765/2008 to > ensure consistency in CABs in evaluating TSPs > > As noted within 319 403 (Introduction), several other documents are > incorporated as well - ISO/IEC 17021 (common requirements for conformity > assessment bodies evaluating management systems) and ISO/IEC 27006 (common > requirements for CABs evaluating information security management systems), > for example. > > If we put the layer cake together and simplify: > * ISO/IEC 17065 - Common requirements for conformity assessment bodies > looking at products/services (e.g. "What should all auditors do") > * ISO/IEC 17021 - Common requirements for conformity assessment bodies > looking at management systems > * ISO/IEC 27006 - Common requirements for conformity assessment bodies > looking at information security management systems > * EN 319 403 - Common requirements for conformity assessment bodies > evaluating TSPs (e.g. "What makes an auditor qualified to be a CA auditor") > * EN 319 411-1 - Common requirements on the TSP for websites (combined with > EN 319 401, which 319 411-1 incorporates-and-builds-on) > > In trying to understand why the reports are what they are, we need to look > in particular at 17021 and 17065, and the framework they use for both audit > engagements and reporting. 17065 describes a certification scheme. > Reproducing a paragraph from the introduction: > > """ > Certification of products, processes or services is a means of providing > assurance that they comply with > specified requirements in standards and other normative documents. Some > product, process or service > certification schemes may include initial testing or inspection and > assessment of its suppliers' quality > management systems, followed by surveillance that takes into account the > quality management system and > the testing or inspection of samples from the production and the open > market. Other schemes rely on initial > testing and surveillance testing, while still others comprise type testing > only. > """ > > 319 411-1 certification describes a system that is based on an initial > testing or inspection, along with periodic surveillance. Importantly, > within 17065 and 17021, the way of ensuring continued compliance is done > based on contracts and reporting - that is, the client is responsible for > reporting changes to their conformity assessment body, and the CAB may > determine to revoke the certification or indicate corrective actions should > be taken (see ISO/IEC 17065:2012(E) 4.1.2.2, ISO/IEC 17021:2006(E) 8.6.3). > ISO/IEC 17065:2012(E), Section 7.7 describes the general reporting: the > name of the CAB, the date the certification is granted, the name and > address of the client, the scope of the certification, the validity > period/expiration date of the certificate, and anything else required by > the certification scheme (319 411-1). > > Within the WebTrust scheme, the reports are based on either US (AICPA) > Standards of AT101, Canadian (CPA Canada) Standards of Section 5025, or > IFAC's ISAE3000 standards. What is important and notable about these is > that they are reports over information, with a scope and norm (e.g. > WebTrust for CAs) applied. This is why there's a consistent period of time > - information is collected and the auditor evaluates, on the basis of that > information, the level of assurance that is being met. > > I'm still working to have a better understanding here (and this is all in a > personal capacity), but my conclusion is this: > - "ETSI" audits reflect an engagement at a particular point in time, where > a series of system controls are evaluated, and the result is a certificate > indicating the process and products comply with the relevant criteria (319 > 411-1). This is similar to a "Point in Time Readiness Assessment". After > the certification is granted, the TSP is responsible for notifying the CAB > of changes (in the future), and the CAB evaluates whether those changes > should cause it to revoke certification or institute some corrective action > (which may not be noted in the certification). An ETSI audit is thus meant > to indicate a CA "is" or "will be" conforming. > - "WebTrust" audits reflect an engagement looking over the past, by the CA > providing documentation and evidence that their policies were consistent > with the requirements. That is, a WebTrust audit is meant to assess whether > or not a CA is being accurate when it says it "was" conforming, but makes > no guarantees about whether or not a CA "is" conforming, nor whether or not > the CA disclosed all relevant information. > > > The challenge is in determining whether this is a correct understanding of > the respective differences, and then determining whether or not the > certification-based approach provides the desired or sufficient level of > assurance - in particular, its 'forward-looking' approach with > self-reporting, rather than the WebTrust-style assurance engagement that is > more 'backward-looking' without future promises.
Hello, there is a problem with the auditor qualification and the national accreditation of the auditing body. We´ll ask ACABc to suggest a solution to take care about proper education of "qualified" auditors and "good practise" audit statements as suggested by Mozilla. Maybe a "white listing" of qualified audit bodies is the best way. Best regards Arno Fiedler _______________________________________________ dev-security-policy mailing list [email protected] https://lists.mozilla.org/listinfo/dev-security-policy
