On Wednesday, August 21, 2019 at 11:46:37 PM UTC-5, Jeremy Rowley wrote: > Hey all, > > An interesting issue came up recently with audits. Because the Mozilla policy > includes some requirements that diverge from the BRs, the audit criteria > don't necessarily cover everything Mozilla cares about. Thus, it's possible > to have an incident that doesn't show up on an audit. It's also possible that > the auditor determines the incident is not sufficiently important/risky(?) to > include it in an audit. For example: > https://bugzilla.mozilla.org/show_bug.cgi?id=1458024. Auditors aren't > controlled by the CA and operate independently which means the CA can't > dictate what goes into the opinion. One solution is to require CAs to list > all of the incidents that occur during their audit in the management > assertion letter. I posted an addendum to the management assertion on that > thread. Going forward, we'll just include it as part of the main body. I need > to look into whether I can get our existing audit reissued to the appendix is > part of the seal as well. > > What do you think about just requiring that as part of the Mozilla policy? Ie > - the management assertion letter must include a list of the incidents > active/opened during the audit period. Something like that could ensure > transparency and make sure all incidents are disclosed to the auditor, > distinguishing the CA's disclosures from the auditors. > > Jeremy
Reposting as I don't see my original response to this thread: Sorry for the delay in responding, but I first wanted to gather feedback from the members of the WebTrust Task Force. Keep in mind any audit/assurance engagement entails professional judgment, which of course may vary from firm to firm. We are certainly seeing a trend in audit reports whereby “Other Matters” are described that do not modify/qualify the auditor’s opinion but do allow the auditor to make mention of the issues. Some firms use this section to list items noted in Bugzilla, for instance, to demonstrate that these types of issues were addressed in the course of the audit. Depending on the firm, some firms have been listing all issues noted in Bugzilla, while others are listing only those that are significant. As a Task Force, it is not our position, or even a possibility, for us to mandate how these should be handled as professional judgment will dictate their treatment based on the respective firms’ risk management policies. That being said, we have as part of our draft practitioner guidance commentary that the browser community prefers seeing all these types of issues listed as either modifications/qualifications, or described in “Other Matters” and encourage practitioners to use this approach. In most cases, management’s Assertion accompanies the audit report, and there is greater flexibility for what goes into them. Our professional standards require certain items be present in the Assertion, but other items that management wants to add are permissible to include. Except in the rare reporting scenario when an assertion by management is not provided, most of the Task Force members agree there would not be any issue with your proposed requirement to require a CA's management to detail Bugzilla or similar issues relevant to the current audit period in their assertion to the auditors. As far as the management representation letter, which is a required written communication to the auditors at the conclusion to the audit, regulatory non-compliance would normally form part of the management representation that is needed for each audit, and the assertion can form part of that representation. Keep in mind that the management representation letter is not part of the deliverable that is viewed publicly as is the case with the auditor’s opinion and management assertion. To demonstrate that the auditor has addressed the significance of the relevant issues, we can propose that the standard WebTrust reports have an Exhibit that sets out the issues identified by reference to Bugzilla, as an example. The audit report can then reflect the relevant significance either as a report qualification or through an "Other Matters" paragraph that provides further commentary. The main issue is that, even by identifying and addressing them in the audit report, there is still the potential for disagreements to exist as to significance due to auditor’s professional judgment. Of course, in our new SOC-like report, these issues will be discussed in the system description, audit report and potentially an unaudited management comment section. Thanks, Jeff Ward _______________________________________________ dev-security-policy mailing list [email protected] https://lists.mozilla.org/listinfo/dev-security-policy
