On Thu, Aug 22, 2019 at 12:46 AM Jeremy Rowley via dev-security-policy <
dev-security-policy@lists.mozilla.org> wrote:

> Hey all,
>
> An interesting issue came up recently with audits. Because the Mozilla
> policy includes some requirements that diverge from the BRs, the audit
> criteria don't necessarily cover everything Mozilla cares about. Thus, it's
> possible to have an incident that doesn't show up on an audit. It's also
> possible that the auditor determines the incident is not sufficiently
> important/risky(?) to include it in an audit. For example:
> https://bugzilla.mozilla.org/show_bug.cgi?id=1458024. Auditors aren't
> controlled by the CA and operate independently which means the CA can't
> dictate what goes into the opinion. One solution is to require CAs to list
> all of the incidents that occur during their audit in the management
> assertion letter. I posted an addendum to the management assertion on that
> thread. Going forward, we'll just include it as part of the main body. I
> need to look into whether I can get our existing audit reissued to the
> appendix is part of the seal as well.
>
> What do you think about just requiring that as part of the Mozilla policy?
> Ie - the management assertion letter must include a list of the incidents
> active/opened during the audit period. Something like that could ensure
> transparency and make sure all incidents are disclosed to the auditor,
> distinguishing the CA's disclosures from the auditors.
>
> Jeremy
> _______________________________________________
> dev-security-policy mailing list
> dev-security-policy@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-security-policy
>
Hey Jeremy,

Huge thanks for raising this. Sadly, you’re about to be eaten by a grue [1]

Browsers have definitely had conversations with both WebTrust and ETSI on
this, and I recall some conversations to that effect during the CA/Browser
Forum F2F discussions.

One of the challenges is that ETSI and WebTrust are highly divergent in
this area, in that an ETSI audit is not based on Management’s Assertions in
the way that WebTrust is, but against a set of criteria (largely) CAs (via
ETSI) developed for themselves, in order to meet certain European legal
obligations.

To date, the policy has left it flexible for that reason. Instead, we’ve
tried to work very closely to explain the concerns to both WebTrust and to
ETSI. To address these concerns, the WebTrust TF went back and examined
their relevant professional obligations (e.g. under ISAE 3000 or AICPA’s
AT-C standards) and developed illustrative reports that WebTrust-licensed
auditors could use to opine on and disclose other relevant matters, such as
incidents. While it’s certainly true that the WebTrust TF does not
presently place an obligation on the auditor to disclose these matters,
either via the (“externally” maintained) aforementioned professional
standards or via the (TF-maintained) audit criteria, a competent and
capable auditor should be familiar both with the illustrative reports and
the policy requirements of the relevant report recipients. CAs contracting
with their auditors can certainly ensure that the auditor understands the
requirements placed upon the CA by its report recipients, and thus ensure
the auditor will disclose information they are professionally empowered to
disclose, and as demonstrated via the illustrative report.

There are several subtle differences in the implications of the reporting,
as to whether it’s disclosed via Management’s Assertion or disclosed as
other findings, and much more stark differences with respect to ETSI
reporting, so I’m not sure a one-size-fits-all approach works here. That
said, I absolutely entirely welcome any and all additional disclosures that
are made by CAs in their management letters, and that’s something commonly
discussed with WebTrust in trying to find angles for formalizing various
reporting and disclosure.

This community is wholly open, so perhaps the best thing CAs can do is
ensure their auditors actively follow, and perhaps even engage with, this
community. We’ve definitely had folks from the audit community involved and
seeking insight and input (without violating their professional obligations
re: confidentiality or their professional opinion forming requirements), so
perhaps it’s worth making sure your auditor does or commits to do so. I
know Scott Perry was one of the original firms involved here, before the
CPA licensing, as they were recognized by Mozilla as a competent auditor
independent from Mozilla when they first proposed becoming an auditor many
years ago (15+?) Perhaps they should revisit and be more closely following?

I realize that answer is a bit “No, but yes, but no” - but it’s exactly the
sort of discussion worth having, and it’s been a conversations browsers
have been having for years now with the auditors, in trying to address
auditor quality control issues. Hopefully, some of the auditors, and
particularly the WTTF members who I do know lurk here, may want to chime
in, especially if I’ve botched any of the summarization :)

[1]
http://mentalfloss.com/article/29885/eaten-grue-brief-history-zork
_______________________________________________
dev-security-policy mailing list
dev-security-policy@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-security-policy

Reply via email to