On Thu, Aug 22, 2019 at 12:46 AM Jeremy Rowley via dev-security-policy < [email protected]> wrote:
> Hey all, > > An interesting issue came up recently with audits. Because the Mozilla > policy includes some requirements that diverge from the BRs, the audit > criteria don't necessarily cover everything Mozilla cares about. Thus, it's > possible to have an incident that doesn't show up on an audit. It's also > possible that the auditor determines the incident is not sufficiently > important/risky(?) to include it in an audit. For example: > https://bugzilla.mozilla.org/show_bug.cgi?id=1458024. Auditors aren't > controlled by the CA and operate independently which means the CA can't > dictate what goes into the opinion. One solution is to require CAs to list > all of the incidents that occur during their audit in the management > assertion letter. I posted an addendum to the management assertion on that > thread. Going forward, we'll just include it as part of the main body. I > need to look into whether I can get our existing audit reissued to the > appendix is part of the seal as well. > > What do you think about just requiring that as part of the Mozilla policy? > Ie - the management assertion letter must include a list of the incidents > active/opened during the audit period. Something like that could ensure > transparency and make sure all incidents are disclosed to the auditor, > distinguishing the CA's disclosures from the auditors. > > Jeremy > _______________________________________________ > dev-security-policy mailing list > [email protected] > https://lists.mozilla.org/listinfo/dev-security-policy > Hey Jeremy, Huge thanks for raising this. Sadly, you’re about to be eaten by a grue [1] Browsers have definitely had conversations with both WebTrust and ETSI on this, and I recall some conversations to that effect during the CA/Browser Forum F2F discussions. One of the challenges is that ETSI and WebTrust are highly divergent in this area, in that an ETSI audit is not based on Management’s Assertions in the way that WebTrust is, but against a set of criteria (largely) CAs (via ETSI) developed for themselves, in order to meet certain European legal obligations. To date, the policy has left it flexible for that reason. Instead, we’ve tried to work very closely to explain the concerns to both WebTrust and to ETSI. To address these concerns, the WebTrust TF went back and examined their relevant professional obligations (e.g. under ISAE 3000 or AICPA’s AT-C standards) and developed illustrative reports that WebTrust-licensed auditors could use to opine on and disclose other relevant matters, such as incidents. While it’s certainly true that the WebTrust TF does not presently place an obligation on the auditor to disclose these matters, either via the (“externally” maintained) aforementioned professional standards or via the (TF-maintained) audit criteria, a competent and capable auditor should be familiar both with the illustrative reports and the policy requirements of the relevant report recipients. CAs contracting with their auditors can certainly ensure that the auditor understands the requirements placed upon the CA by its report recipients, and thus ensure the auditor will disclose information they are professionally empowered to disclose, and as demonstrated via the illustrative report. There are several subtle differences in the implications of the reporting, as to whether it’s disclosed via Management’s Assertion or disclosed as other findings, and much more stark differences with respect to ETSI reporting, so I’m not sure a one-size-fits-all approach works here. That said, I absolutely entirely welcome any and all additional disclosures that are made by CAs in their management letters, and that’s something commonly discussed with WebTrust in trying to find angles for formalizing various reporting and disclosure. This community is wholly open, so perhaps the best thing CAs can do is ensure their auditors actively follow, and perhaps even engage with, this community. We’ve definitely had folks from the audit community involved and seeking insight and input (without violating their professional obligations re: confidentiality or their professional opinion forming requirements), so perhaps it’s worth making sure your auditor does or commits to do so. I know Scott Perry was one of the original firms involved here, before the CPA licensing, as they were recognized by Mozilla as a competent auditor independent from Mozilla when they first proposed becoming an auditor many years ago (15+?) Perhaps they should revisit and be more closely following? I realize that answer is a bit “No, but yes, but no” - but it’s exactly the sort of discussion worth having, and it’s been a conversations browsers have been having for years now with the auditors, in trying to address auditor quality control issues. Hopefully, some of the auditors, and particularly the WTTF members who I do know lurk here, may want to chime in, especially if I’ve botched any of the summarization :) [1] http://mentalfloss.com/article/29885/eaten-grue-brief-history-zork _______________________________________________ dev-security-policy mailing list [email protected] https://lists.mozilla.org/listinfo/dev-security-policy
