I am particularly disturbed by three points made by TUVIT during this discussion:
1. A malformed qcStatement extension is a minor non-conformity because there is no known security risk - This argument is incredibly dangerous and harmful. It implies that all sorts of well-defined requirements can be ignored if the CA and/or auditor decide that there is no risk in doing so. 2. Citing ISO 17065 as requiring non-conformities be kept confidential - adding on Ryan's comments, we're seeing increasing disclosure of audit findings (another example: https://netlock.hu/download/etsi-certification-netlock/?wpdmdl=57340), leading me to believe that this is TUVIT's own interpretation of the confidentiality requirements. I don't believe that TUVIT needs "the establishment of rules with in the CAB/Forum for making such kind of incidents public" in order to begin making these disclosures. 3. The argument that T-Systems has 3-months to revoke these certificates - while I understand that under ETSI TSPs have 3 months to correct minor non-conformities, using that as an excuse to ignore CAB Forum revocation requirements is unacceptable, and perhaps explains why we see such poor compliance with this requirement. If this is indeed the accepted interpretation (please confirm), then I will look for ways to fix this via Mozilla policy. - Wayne On Fri, Nov 2, 2018 at 8:25 AM Ryan Sleevi via dev-security-policy < dev-security-policy@lists.mozilla.org> wrote: > On Fri, Nov 2, 2018 at 10:24 AM Wiedenhorst, Matthias via > dev-security-policy <dev-security-policy@lists.mozilla.org> wrote: > > > Auditor and Reviewer, as stated on > > > https://www.tuvit.de/fileadmin/Content/TUV_IT/zertifikate/en/AA2018072001_Audit_Attestation_E_Deutsche-Telekom-Root-CA-2_20180718_s.pdf > > - the parties tasked with ensuring that the audit is meaningfully able to > > ensure the criteria were met and the testing procedures were able to meet > > those requirements. > > > > Auditors and reviewers need to be distinguished: ISO/IEC 17065 §7.5.1 > > forbids that the person(s) performing the review is involved in the audit > > process. > > > > Indeed - and yet do you agree that the Reviewer is tasked with reviewing > the audit methodology and artifacts to ensure it appropriately meets the > objectives expected and required? A multi-party failure to assure the > necessary assurance is just that - a multi-party failure. > > >Issue A) As part of their initial response to my complaint, TUVIT, by way > > >of Matthias Wiedenhorst (Head of Certification Division TSP) stated "As > a > > >very first, quick cross check with our audit evidence record, I can only > > >say that we did check issued certificates from within the declared > period > > >and that they did contain the proper qcStatement-6.3 / > id-etsi-qcs-QcType > > >3". However, this statement was in direct conflict with the TSP's own > > >investigation and incident report, provided at > > >https://bugzilla.mozilla.org/show_bug.cgi?id=1498463#c3 , which states > > the > > >mistake was introduced during the development of support - that is, no > > such > > >properly issued certificates were issued. > > > > We do not understand why the important fact that Matthias was not in > > office and replied what he remembered from an audit that was month ago is > > not mentioned here. In addition, he replied that he would verify and come > > back as soon as possible (when he is in office again). That actually > > happened, see below. > > > > Wrong or misleading information - which was only corrected upon specific > > questioning and a request for proof or evidence of the claim - has been > > used to disqualify CAs in the past. This statement was made after the TSP > > had themselves already investigated and confirmed this was not possible. > > > > The same standard being applied to CA incident reports is being proposed > > here - that incomplete and improper investigations raise serious > questions. > > > > Let’s stay with present case (not with the past): In his email, Matthias > > said that he is not in his office and will begin to investigate the > > situation as soon as possible. We think that it is clear, that further > > information contained in the email cannot be based on the result of the > > investigation. Back in his office after looking into the audit logs and > > verifying the qcStatement he gave the proper answer. > > > > I appreciate the attempt to narrowly scope the issue, but that is equally > an attempt to deflect or ignore the ample set of past precedent and > expectations. As such, I reject the premise that this should be considered > without regard to past failures by CAs or other auditors - as an auditor > being entrusted to report truthfully and faithfully to the community about > a CAs compliance with its own CP, CPS, and the appropriate supervisory > framework, auditors are expected to consider the best practices and > precedents in their activities and actions. > > With respect to your suggestion that the information can not be relied > upon, I'm more than happy to provide the full e-mail chain if there's some > consideration given to misinterpretation. However, I do not believe the > reply at all indicates that the information contained was not reliable or > may be counter-factual and to be corrected later. Yes, Matthias stated they > were out of office - but then immediately began with a remark that "As a > very first, quick cross check with our audit evidence record, I can only > say that we did check issued certificates". > > I can understand the argument being made here that, on a more detailed > examination of your audit evidence record, you discovered that it was not, > in fact, checked. However, that raises significant concerns that a quick > check can lead to a completely opposite conclusion, particularly for a > supposedly-skilled practitioner. > > > > As said before, we are using tools in the audit process. The sentence > > about lint tools should be seen as additional information and nothing > else. > > > > Yet you've failed to describe what these tools encompass, beyond what is > readily available off the shelf. Further, in your description of the > methodology used, it was clear that human visual inspection without regard > to the actual specification was performed. While you may have used a tool > to, say, dump the DER-encoded contents into a structural representation, > the procedures for examining that structural representation against the > profile are clearly and significantly deficient. > > > > Considering the significance of misencoding of profiles - which has lead > > to critical misissuance and security risk (see, for example, Turktrust) - > > the question about whether or not TUVIT's testing procedures are adequate > > to ensure compliance are extremely relevant and appropriate. TUVIT could, > > for example, attempt to resolve these concerns by providing documentation > > about their approach to assessing compliance with the relevant > certificate > > profiles, including methodologies for sampling (not normatively specified > > by the relevant documents), their testing and training procedures, and > > other evidence that would feel suitable to highlight that this was, > indeed, > > a 'one off' approach. > > > > Let’s stay with the present case: We have not seen any security breach > due > > to the miscoding of the qcStatement and have not seen any arguments how > > certificates with wrong qcStatement could be used to breach the security > in > > future. That is the reason why believe the non-conformity is a > non-critical > > non-conformity with respect to the performed assessment. In addition, a > > correct qcStatement is not required by the Baseline Requirements (but by > > ETSI EN 319 412-5). Having this in mind, we expect that Browser Ecosystem > > should not be directly affected. > > > > This response most perfectly fundamentally why I believe that TUVIT should > no longer be recognized as a suitable auditor. > > The community has, for the better part of a decade now, recognized that the > failure to appropriately encode certificates minimally represents a > significant barrier to interoperability. The need to support and > interoperate with such broken certificates has resulted in multiple > security vulnerabilities being introduced into the ecosystem, such as the > processing of public key encodings that were improperly DER encoded or the > improper encoding of negative serial numbers. > > Similarly, the failure to adhere to the specified profile has resulted in a > number of high-profile CA security events. For example, the inclusion of a > "cA:True" within the basicConstraints of a certificate intended for > subscribers, or the misuse of the EV certificate profile for testing, have > lead to the dissolution of CA services due to the critical blows to trust > in those providers. > > One of the most core, and fundamental, expectations of auditors is that > they have the necessary technical skill to review the configured profiles, > software, and systems to ensure the correct production of certificates. > That these produced certificates align with the stated CP and CPS, and any > relevant specifications incorporated therein. That they have the necessary > technical judgement to recognize the critical nature of these functions, > and its fundamental impact on the trust ecosystem. > > One would expect the degree of auditor competency to review the produced > certificates on an aggressive basis, to ensure that the level of confidence > in the certification is appropriate. Any sampling methodology used - even > to the degree of just a single certificate - would and should have revealed > this critical misconfiguration. That no such detection or discovery was > made fundamentally calls into question the level of supervision and > oversight by the CAB, which is a critical function to ensure those relying > on the certification do not face security or compatibility risks. > > Furthermore, that within the framework of the nominally more rigorous, more > critical trust objectives of qualified certificates, one would expect that > the level of scrutiny and supervision is beyond reproach, so as not to call > into disrepute the entire concept and framework. > > TUVIT has not met these expectations, and in doing so, fundamentally calls > into question the necessary competencies of the auditors and the > methodology employed. This is fundamentally no different than the rejection > of those WebTrust auditors who offer letters ascribing high confidence that > the principles and criteria are met, while readily available public > information actively contradicts this, and even simple sampling would have > revealed these flaws. This is not singling out ETSI-using auditors; this is > recognizing an auditor that is failing to meet the requirements of the > community. > > > > 1) TUVIT's assessment methodologies are fundamentally flawed and do not > > meet the level of assurance expected of the community > > > > This is an exaggerated conclusion based on the only fact that a wrong > > coded qcStatement was not detected. > > > > TUVIT - and Matthias Wiedenhorst - have been the auditor of note for > T-Systems for some time, as demonstrated through > > https://www.t-systems.com/blob/776674/ec9fa6212c7a1c5537cfb57ad33af6bb/DL_Trust-Center_ETSI_Audit.pdf > > During those same periods, T-Systems had multiple issues of material > non-compliance, as demonstrated through > https://bug1391074.bmoattachments.org/attachment.cgi?id=8915934 > > This is not an exaggerated conclusion by any means, but rather, a > systematic failure to apply sound audit methodology and best practices. > > > > 2) TUVIT's and T-Systems' incident handling approach does not meet the > > level of expectations of the community. This includes both the oversight > of > > revocation, but also the approach to auditing for when such a critical > > non-conformity has been found. A resolution of "They promise to fix it" > > does not provide a reasonable level of assurance, given that at > fundamental > > question is what other issues TUVIT failed to consider or assess. The > lack > > of introspection, by TUVIT, as to its auditing process and procedures is > > concerning. > > > > How one single person (with his private hat on) can speak for the > > community? > > > I do not speak to the community decisions in the future. However, I am more > than happy to detail amply the community expectations, captured in bugs and > in discussions, about the handling of revocation and errors. In this > regard, consider this a summary of the past discussions setting forth the > expectations. > > > > We explained before why we believe that the wrong qcStatement coding is a > > non-critical nonconformity (the security of ecosystems is not affected). > > Where did you copy "They promise to fix it" from? We have not used this > > expression. > > > > With respect to your previous messages: > "We apply this requirement in the context of a certificate decision (for > the issuance of a ETSI certificate) also for this case of a non-conformity > found for an issued ETSI certificate: Correction of the bug and revocation > of all concerned website certificates till end of 2018 will fulfill the > requirement." > > and > "Failure to comply with the Baseline Requirements due to the non-revocation > of the concerned website certificates within the required time line is > analogously regarded as minor non-conformity, that shall be fixed within > three month." > > Both of these assert that remediation within three months time is suitable > to satisfy the needs. While that is certainly the minimum required, that > does not align with the expectations for resolution and/or revocation. > _______________________________________________ > dev-security-policy mailing list > dev-security-policy@lists.mozilla.org > https://lists.mozilla.org/listinfo/dev-security-policy > _______________________________________________ dev-security-policy mailing list dev-security-policy@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-security-policy
