Re: Policy 2.7.1: MRSP Issue #192: Require information about auditor qualifications in the audit report

[Ryan Sleevi via dev-security-policy]

On Sat, Nov 7, 2020 at 9:21 AM Jeff Ward via dev-security-policy <
dev-security-policy@lists.mozilla.org> wrote:

> Sure Ryan, the answer is quite simple.  When I used the word "public" in
> my post, I should have been more clear as to the nuance of this concept.
> Public reports by definition are generally distributed (available to
> anyone).  When reports are restricted in distribution and only intended for
> a certain user or users as specified in the report, they are no longer
> public.  In each of the EY examples you reference, they all state in the
> last paragraph before the EY signature, "This report is intended solely for
> the information and use of GPO-CA and the Federal PKI Policy Authority and
> is not intended to be, and should not be, used by anyone other than GPO-CA
> and the Federal PKI Policy Authority."  When reports are not generally
> distributed and available to the general public, the specifics of
> individuals performing the audit will not be present.   When my firm issues
> reports for FPKI, we also provide the listing of individuals in a letter
> that is not public information.
>

Thanks Jeff,

This is useful for confirming that there is a clearly viable path forward
here. As you know, I appreciate the nuance here regarding public reporting,
as well as reports that are restricted in distribution but still made
public (e.g. as part of a regulatory function, such as the OIG in this
case). I think we agree in the substance: that this is possible, and are
merely working out the details here with regards to reporting.

For example, Mozilla could require that, in addition to the "traditional"
WebTrust reporting , Mozilla be named as part of a restricted distribution
report that contains these details. Alternatively, Mozilla could require
that, as part of participating within their root program, CAs ensure such
reports include as restricted distribution those Subscribers, Relying
Parties, and Application Vendors that would rely upon the CAs' services,
much like widely practiced in industry today with respect to cloud
providers.

Would you agree that it's fair to say that it is not fundamentally that
auditors cannot report such information, but focused here on the details of
how that report is delivered to Mozilla?
___
dev-security-policy mailing list
dev-security-policy@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-security-policy

Re: Policy 2.7.1: MRSP Issue #192: Require information about auditor qualifications in the audit report

[Jeff Ward via dev-security-policy]

On Friday, November 6, 2020 at 1:13:43 PM UTC-6, Ryan Sleevi wrote:
> On Fri, Nov 6, 2020 at 12:31 PM Jeff Ward via dev-security-policy < 
> dev-secur...@lists.mozilla.org> wrote: 
> 
> > Audit reports, whether for WebTrust, financial statements, or other forms 
> > of engagement reports providing assurance to users of the information, do 
> > not include specific audit team members’ names. Simply stated, this desire 
> > to include individual auditor’s qualifications in a public report is not 
> > consistent with any other compliance reporting methods or reporting 
> > requirements for CAs, or any other auditee for that matter.
> Hi Jeff, 
> 
> Could you help me square this statement with the practical examples? For 
> example, here's a report [1] from a WebTrust TF member, Ernst and Young, 
> demonstrating how this works in practice. You can see there hasn't been an 
> issue for years [2][3], even with the direct involvement of individuals on 
> the WebTrust TF in preparing such an audit. 
> 
> So I'm having difficulty squaring your statement that they "do not", when 
> we've got examples from long-standing members of the WebTrust TF that 
> demonstrate that, in practice, they do. Could you help highlight what's 
> inconsistent here? 
> 
> Alternatively, and as mentioned to ETSI, here's an example of an ISAE 3000 
> based audit scheme, similar to WebTrust, that also discloses the relevant 
> professional qualifications and skills to clients [4], as discussed in 
> 4.4.8 and 4.4.9. 
> 
> [1] https://www.oversight.gov/sites/default/files/oig-reports/18-19.pdf 
> [2] 
> https://www.oversight.gov/sites/default/files/oig-reports/Assessment%20Report%2019-12%20GPO%20Federal%20PKI%20Compliance.pdf
>  
> [3] https://www.oversight.gov/sites/default/files/oig-reports/17-27.pdf 
> [4] 
> https://www.bsi.bund.de/EN/Topics/CloudComputing/Compliance_Criteria_Catalogue/Compliance_Criteria_Catalogue_node.html

Sure Ryan, the answer is quite simple.  When I used the word "public" in my 
post, I should have been more clear as to the nuance of this concept.  Public 
reports by definition are generally distributed (available to anyone).  When 
reports are restricted in distribution and only intended for a certain user or 
users as specified in the report, they are no longer public.  In each of the EY 
examples you reference, they all state in the last paragraph before the EY 
signature, "This report is intended solely for the information and use of 
GPO-CA and the Federal PKI Policy Authority and is not intended to be, and 
should not be, used by anyone other than GPO-CA and the Federal PKI Policy 
Authority."  When reports are not generally distributed and available to the 
general public, the specifics of individuals performing the audit will not be 
present.   When my firm issues reports for FPKI, we also provide the listing of 
individuals in a letter that is not public information.  

Jeff
___
dev-security-policy mailing list
dev-security-policy@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-security-policy

Re: Policy 2.7.1: MRSP Issue #192: Require information about auditor qualifications in the audit report

[Ryan Sleevi via dev-security-policy]

On Sat, Nov 7, 2020 at 4:52 AM Dimitris Zacharopoulos 
wrote:

>
> I will try to further explain my thoughts on this. As we all know,
> according to Mozilla Policy "CAs MUST follow and be aware of discussions in
> the mozilla.dev.security.policy
>  forum, where
> Mozilla's root program is coordinated". I believe Mozilla Root store
> managers' minimum expectations from CAs are to *read the messages and
> understand the content of those messages*. Right now, we have [1], [2],
> [3], [4], [5], [6], [7], [8], [9] policy-related threads opened up for
> discussion since October 15th.
>
> If every post in these threads contained as much information and
> complexity as your recent reply to Clemens,
>

This seems like a strawman argument,  ht I don’t think it’s intentional.

You’re arguing that “if things were like this hypothetical situation, that
would be bad”. However, they aren’t like that situation, as the evidence
you provided shows. This also goes back to the “what is your desired
outcome from your previous mail”, and trying to work out what a clear call
to action to address your concerns. Your previous message, especially in
the context of your (hypothetical) concern, reads like you’re suggesting
“Mozilla shouldn’t discuss policy changes with the community”. I think
we’re all sensitive and aware of the desire not to have too many parallels
discussions, which is exactly why Ben’s been only introducing a few points
a week, to facilitate that and make progress without overwhelming.

As it relates to this thread, or any other thread, it seems the first order
evaluation for any CA is “Will the policy change”, followed by “What do I
need to do to meet the policy?”, both of which are still very early in this
discussion. You’re aware of the policy discussion, and you’re aware a
decision has not been made yet: isn’t that all you need at this point?
Unlike some of the other proposals, which require action by CAs, this is a
proposal that largely requires action by auditors, because it touches on
the audit framework and scheme. It seems like, in terms of expectations for
CAs to participate, discussing this thread with your auditor is the
reasonable step, and working with them to engage here.

Hopefully that helps. Your “but what if” is easily answered as “but we’re
not”, and the “this is a lot, what do I need to do” is simply “talk with
your auditor and make sure they’re aware of discussions here”. That seems a
very simple, digestible call to action?
___
dev-security-policy mailing list
dev-security-policy@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-security-policy

Re: Policy 2.7.1: MRSP Issue #192: Require information about auditor qualifications in the audit report

[Dimitris Zacharopoulos via dev-security-policy]

I will try to further explain my thoughts on this. As we all know, 
according to Mozilla Policy "CAs MUST follow and be aware of discussions 
in the mozilla.dev.security.policy 
 forum, where 
Mozilla's root program is coordinated". I believe Mozilla Root store 
managers' minimum expectations from CAs are to _read the messages and 
understand the content of those messages_. Right now, we have [1], [2], 
[3], [4], [5], [6], [7], [8], [9] policy-related threads opened up for 
discussion since October 15th.


If every post in these threads contained as much information and 
complexity as your recent reply to Clemens, I think it eventually 
"abuses" the requirement that CAs must follow discussions in m.d.s.p. 
and leads to fatigue. Understanding the complicated English language 
used, especially for non-Native English speakers, is a very challenging 
and difficult task of its own. Therefore, I think it is unreasonable for 
Mozilla Root store managers to expect that CAs will follow and 
understand all of these discussions if these threads are bombarded with 
long and complicate emails that only very few will be able to read and 
understand.


I think sending specific questions is a good advice and I will try to do 
that next week, but please try to also consider and respect the fact 
that CAs have a finite set of resources to work on these issues, among 
other duties. An unexpected increase in the volume of information CAs 
must follow, creates a risk that something critical might be missed, 
despite the good efforts of CAs having allocated the necessary resources 
to monitor these lists and Bugzilla incidents.


I obviously can't suggest anyone to post more or less, each person has 
the right to post whatever he/she deems necessary. I just wanted you to 
know, as a peer to this Module, that some participants of this Root 
Program want to contribute and continue to do so, and it would help 
tremendously if some messages were shorter and simpler to read. Perhaps 
breaking down your long reply into more than one messages might make 
them easier to process, I don't know.


Thanks for listening :-)


Dimitris.



[1]: 
https://groups.google.com/g/mozilla.dev.security.policy/c/4fhP4iV4ut4/m/WQknrWbhAAAJ
[2]: 
https://groups.google.com/g/mozilla.dev.security.policy/c/ZFLsguJyFDo/m/Tmn5rcXhAAAJ
[3]: 
https://groups.google.com/g/mozilla.dev.security.policy/c/oJiMmvAJXdI/m/ZhH6oLwpAAAJ
[4]: 
https://groups.google.com/g/mozilla.dev.security.policy/c/3sW3_cRBrfo/m/ErldH8JWAQAJ
[5]: 
https://groups.google.com/g/mozilla.dev.security.policy/c/Oqd2iKCFELI/m/f9Kfs0M0BAAJ
[6]: 
https://groups.google.com/g/mozilla.dev.security.policy/c/DChXLJrMwag/m/uGpEqiEcBgAJ
[7]: 
https://groups.google.com/g/mozilla.dev.security.policy/c/nMrORsPPcds/m/hVahATyTBwAJ
[8]: 
https://groups.google.com/g/mozilla.dev.security.policy/c/rbSFMYKlfI4/m/3kvOhydWAQAJ
[9]: 
https://groups.google.com/g/mozilla.dev.security.policy/c/xk3BanrcljY/m/8dFyM-5pAQAJ




On 2020-11-07 1:40 π.μ., Ryan Sleevi via dev-security-policy wrote:

On Fri, Nov 6, 2020 at 6:08 PM Dimitris Zacharopoulos via
dev-security-policy  wrote:


Can other people, except Ryan, follow this thread? I certainly can't. Too
much information, too much text, too many assumptions, makes it impossible
to meaningfully participate in the discussion.


These are complex topics, for sure, but that’s unavoidable. Participation
requires a degree of understanding both about the goals to be achieved by
auditing, as well as the relevant legal and institutional frameworks for
these audits. So, admittedly, that’s not the easiest to jump into.

Could you indicate what you’re having trouble following? I don’t know that
we can do much about “too much information”, since that can be said about
literally anything unfamiliar, but perhaps if you would simply ask
questions, or highlight what you’d like to more about, it could be more
digestible?

What would you say your desired outcome from your email to be? Accepting,
for a second that this is a complex topic, and so discussion will
inherently be complex, and so a response such as “make it simpler for me”
is a bit unreasonable.
___
dev-security-policy mailing list
dev-security-policy@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-security-policy



___
dev-security-policy mailing list
dev-security-policy@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-security-policy