On Friday, March 6, 2020 at 12:13:49 PM UTC-6, Ryan Sleevi wrote: > Thanks Jeff, > > This is incredibly helpful to understand the approach (and limitations) > that are relevant in the context of a WebTrust report. I'm hoping our ETSI > colleagues might provide a similar level of detail, as I suspect this is > hardly "just" a WebTrust problem at this point. > > On Fri, Mar 6, 2020 at 11:51 AM jwardcpa--- via dev-security-policy < > [email protected]> wrote: > > > If the potential threat of a scope limitation is primarily due do an > > auditor not being able to travel to perform necessary testing, as with the > > Coronavirus, there are potential remedies for the auditor to consider, > > including, but not limited to: > > > > • Using the work of another auditor, whereby the lead auditor > > verifies the independence, qualifications and technical competency of > > another firm that can do a portion of the work, and the lead auditor > > directs the work, plans, supervises and reviews the other auditor’s work, > > taking ultimate responsibility. In this case, no mention of the other firm > > is made in the report as the lead auditor is taking responsibility for the > > other firm’s work. > > • Using technology to observe physical controls and underlying > > documents/artifacts via remote means, such as video. In this case, the > > auditor must ensure the authenticity, integrity, security and > > confidentiality of the transmission. > > > > This is incredibly helpful to understand the possibilities! Thank you for > including this. While I can understand this might not be a universal > solution, especially as the virus continues to spread, it's incredibly > helpful to know what options might be possible and available. > > If the auditor is able to design the audit plan in a manner that overcomes > > the challenges present from what otherwise would be a scope limitation, and > > can obtain satisfaction through adequate testing procedures, the auditor > > will be able to express an unqualified/unmodified (clean) opinion resulting > > in the ability to obtain the WebTrust seal. Otherwise, the auditor will > > explain what gave rise to the scope limitation and no seal will be able to > > be issued. > > CAs should work with their auditors as early as possible to identify any > > impact on the scope of their audit and communicate any issues with the > > browsers. It looks like from this thread any impact on the scope and the > > timing of the release of the audit should be documented in Bugzilla, which > > should also include the CAs incident response plan. > > > > As a follow-up: would the use of such methods (a supervised auditor, > technology based controls) be something that would be available as part of > the Detailed Reporting? My understanding is that it would be noted in such > a report, but perhaps I'm assuming too much. > > > > So what happens if a modified opinion is provided by an auditor, for > > example, because a data center in China could not be tested in the normal > > course of the examination? Then say, six months later, the data center > > becomes accessible and available for audit. Since the audit for the year > > was already issued with the qualification, as required, you would have the > > option of waiting for the next annual audit to include the data center in > > question and proceed as normal. Once again, a WebTrust audit cannot > > include a carve out of the data center, nor can a WebTrust audit be > > performed later on just the data center. Depending on the significance of > > the operations not able to be included in the scope of the most current > > audit, and depending on the needs and requirements of the users (browsers), > > a CA could undergo specified/agreed-up procedures in a separate engagement, > > or conduct a full scope WebTrust audit when possible. There ae no hard and > > fast rules for this situation and each should be treated on a case by case > > basis, with discussions including the CA, the browsers, and the auditor. > > > > Thanks again for this. It's incredibly helpful to know the limitations here > as well, such as a limited-physical-scope audit being non-viable. > > Are there limitations as to how long an audit in the past can be conducted? > That is, I'm imagining a scenario where a report is delivered, potentially > with the issues you note (qualified, adverse, disclaimer). Assuming there > comes a point in the future where the factors leading to that opinion are > eventually addressed, and access to the location again becomes possible, is > it possible to perform a full audit for the original period of time? That > is, for a CA that might have an audit period of March-to-March (and thus, > likely affected by these considerations), and the auditor is unable to > sufficiently determine to an unqualified opinion during that assessment > period (of March ~ June), is it possible to examine after-the-fact (e.g. if > the virus is contained by July, in July) for the original March-to-March > period? What challenges may exist for such audits that are further than 3 > months from the audit period? > > Realizing there's no hard and fast rules, I'm aware it's perhaps obvious > the line of thinking I'm going towards, which is to require provisions or > contracts for a full audit once/if the virus is contained and travel > restrictions are lifted, as part of accepting such a qualified or adverse > report. I'm also thinking of situations beyond the immediate incident, such > as natural disasters that may impact a region, and trying to understand if > there is an upper-bound or limit as to how far retroactively things may go. > > In the situation of the Mar-Mar period, I can also understand that some CAs > and their auditors may be waiting, during the BR alloted 90 days, to see if > it may be possible to inspect the location during that period (such as > waiting to see if by May they can visit). I would still encourage full > transparency now, in such cases, as they still signal an awareness of > potential delays, and we can work with the CA to understand the challenges > and limitations, and work towards reasonable solutions for the worst case.
Great follow on questions Ryan. As far as the detailed report, whether the end product is in the current form, or in the detailed version, the lead auditor is taking full responsibility and does not make mention of the other auditor in both the opinion, and the detailed section of the controls tested for a detailed report. That being said, nothing prohibits a CA from creating a Bug to draw attention to the fact and explain the auditors obtained the assistance of another firm to complete the scope of the testing. If WebTrust did allow a carve out approach, there would be more flexibility to allow the reference to another firm, but since it is inclusive and the lead auditor takes full responsibility, that is not an option. As far as your question on the situation where the issue giving rise to a qualification of the report, such as the data center that could not be visited during the audit period subsequently becomes available, the auditor could consider if it is appropriate and not misleading to perform additional testing in the areas that were not able to be tested originally, and if so, what reporting is permissible. I could see a situation where the 90 day deadline was near and the report was issued with a scope limitation, and then after some time passes, testing could be performed to complete the scope. The report would then be unmodified/unqualified/clean assuming the results of the testing are favorable. It could be possible for a firm to re-issue it's report and "dual-date" the opinion, which basically means the date at the bottom of the report would indicate two dates, one for the original issuance, and the second date to extend to the completion of the testing for those areas to complete the scope. For example, in a 12/31 period end, the report would be signed by the audit firm, and dated in a manner to say something like XYZ CPAs March 27, XX, except for the subsequent event as detailed in the report, which is dates July 20t. This example demonstrates the firm was able to complete the scope of the audit testing on July 20th. It is up to the auditor's judgment as to how far the opinion can be dual dated/extended. Once too much time passes, this option is no longer viable. I hope this is helpful. Thanks, Jeff _______________________________________________ dev-security-policy mailing list [email protected] https://lists.mozilla.org/listinfo/dev-security-policy
