Re: Incident Reporting Guidance
Having received no comments on this proposal, I went ahead and added it to the wiki [1]. - Wayne [1] https://wiki.mozilla.org/CA/Responding_To_An_Incident#Incident_Report On Wed, Dec 11, 2019 at 4:45 PM Wayne Thayer wrote: > While thinking about different ways to solve the problem of disclosing > missed revocation deadlines, we devised a solution for searching and > reporting on delayed revocations separately from other incidents. We've > begun to add a new Bugzilla "whiteboard" label to delayed revocation > incident bugs. We can then display those incidents separately on our > Incident Dashboard, as follows: > https://wiki.mozilla.org/CA/Incident_Dashboard#Revocation_Delays > > I've come to the conclusion that Mozilla should begin to require that CAs > submit a separate bug when a revocation deadline is missed. I dislike the > extra overhead of creating and managing more bugs, but doing so allows us > to provide clearer guidance to CAs, and it helps to ensure that we are > seeking out the root causes of revocation delays when they occur, rather > than potentially missing those learnings amongst the issue(s) that > triggered the revocation(s). To be clear, this proposal is not intended to > create more work or somehow punish CAs for missing revocation deadlines. > > This results in the following new guidance: > > CAs should submit a separate incident report when: > * Mozilla policy requires that the CA revoke one or more certificates by a > certain deadline, such as those in BR section 4.9, but that deadline is not > met by the CA > * In the process of researching one incident, another incident with a > distinct root cause and/or remediation is discovered > * After an incident bug is marked resolved, the incident reoccurs > > I will appreciate everyone's feedback on these proposed improvements to > the Incident Report section [1] of our Responding to an Incident wiki page. > If no responses are received over the next few days, I'll go ahead and make > these changes. > > - Wayne > > [1] https://wiki.mozilla.org/CA/Responding_To_An_Incident#Incident_Report > > On Thu, Nov 21, 2019 at 10:48 AM Ryan Sleevi > wrote: > >> >> >> On Thu, Nov 21, 2019 at 10:54 AM Wayne Thayer via dev-security-policy < >> dev-security-policy@lists.mozilla.org> wrote: >> >>> During the recent CA/Browser Forum meeting, I was asked to provide better >>> guidance on Mozilla's expectations for incident reporting. We're adding a >>> requirement for incident reporting to the new version of our policy [1], >>> but in this message I'm focused on the guidance provided on our wiki [2]. >>> The question was when to add information to an existing bug versus >>> creating >>> a new one. I'd like to propose adding the following guidance to the wiki >>> to >>> address this question: >>> >>> CAs should create a separate bug and file a new incident report when: >>> * In the process of researching one incident another incident with a >>> distinct root cause and/or remediation is discovered >>> * After an incident bug is marked resolved, the incident reoccurs >>> >>> A third possible addition would be: >>> * When a CA accidentally or intentionally misses a revocation deadline, a >>> separate bug should/must be filed examining the root cause and >>> remediation >>> for missing the deadline >>> >>> I believe the argument for this is that tracking revocation issues >>> separately will help us to focus on improving the agility of the web PKI. >>> On the other hand, Mozilla has not generally required separate reports in >>> the past, and doing so certainly creates more work for everyone involved. >>> It's not clear to me that the benefit of this outweighs the cost. >> >> >> Yeah, I've been thinking about this in light of the feedback Kathleen >> offered on >> https://groups.google.com/d/msg/mozilla.dev.security.policy/M7NGwCh14DI/jZft5ZH_AgAJ >> >> My primary objectives in treating it as two distinct issues are: >> 1) Ensuring the timely remediation of the root causes of whatever >> 'caused' the incident. In general, this should be 'sooner' than 'later' >> 2) Ensuring transparency about when CAs are having repeat "delayed >> revocation" issues; buried within issues themselves makes it difficult to >> track when there are repeat offenders, but also limits visibility into >> systemic issues that all CAs can/should be aware of when contemplating the >> design and maintenance of their PKI >> 3) Ensuring progress is made towards revocation, by not having bugs >> prematurely closed when the 'root' incident is resolved but the CA has >> taken no meaningful steps to bring their PKI back into compliance (yet) >> > ___ dev-security-policy mailing list dev-security-policy@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-security-policy
Re: Incident Reporting Guidance
While thinking about different ways to solve the problem of disclosing missed revocation deadlines, we devised a solution for searching and reporting on delayed revocations separately from other incidents. We've begun to add a new Bugzilla "whiteboard" label to delayed revocation incident bugs. We can then display those incidents separately on our Incident Dashboard, as follows: https://wiki.mozilla.org/CA/Incident_Dashboard#Revocation_Delays I've come to the conclusion that Mozilla should begin to require that CAs submit a separate bug when a revocation deadline is missed. I dislike the extra overhead of creating and managing more bugs, but doing so allows us to provide clearer guidance to CAs, and it helps to ensure that we are seeking out the root causes of revocation delays when they occur, rather than potentially missing those learnings amongst the issue(s) that triggered the revocation(s). To be clear, this proposal is not intended to create more work or somehow punish CAs for missing revocation deadlines. This results in the following new guidance: CAs should submit a separate incident report when: * Mozilla policy requires that the CA revoke one or more certificates by a certain deadline, such as those in BR section 4.9, but that deadline is not met by the CA * In the process of researching one incident, another incident with a distinct root cause and/or remediation is discovered * After an incident bug is marked resolved, the incident reoccurs I will appreciate everyone's feedback on these proposed improvements to the Incident Report section [1] of our Responding to an Incident wiki page. If no responses are received over the next few days, I'll go ahead and make these changes. - Wayne [1] https://wiki.mozilla.org/CA/Responding_To_An_Incident#Incident_Report On Thu, Nov 21, 2019 at 10:48 AM Ryan Sleevi wrote: > > > On Thu, Nov 21, 2019 at 10:54 AM Wayne Thayer via dev-security-policy < > dev-security-policy@lists.mozilla.org> wrote: > >> During the recent CA/Browser Forum meeting, I was asked to provide better >> guidance on Mozilla's expectations for incident reporting. We're adding a >> requirement for incident reporting to the new version of our policy [1], >> but in this message I'm focused on the guidance provided on our wiki [2]. >> The question was when to add information to an existing bug versus >> creating >> a new one. I'd like to propose adding the following guidance to the wiki >> to >> address this question: >> >> CAs should create a separate bug and file a new incident report when: >> * In the process of researching one incident another incident with a >> distinct root cause and/or remediation is discovered >> * After an incident bug is marked resolved, the incident reoccurs >> >> A third possible addition would be: >> * When a CA accidentally or intentionally misses a revocation deadline, a >> separate bug should/must be filed examining the root cause and remediation >> for missing the deadline >> >> I believe the argument for this is that tracking revocation issues >> separately will help us to focus on improving the agility of the web PKI. >> On the other hand, Mozilla has not generally required separate reports in >> the past, and doing so certainly creates more work for everyone involved. >> It's not clear to me that the benefit of this outweighs the cost. > > > Yeah, I've been thinking about this in light of the feedback Kathleen > offered on > https://groups.google.com/d/msg/mozilla.dev.security.policy/M7NGwCh14DI/jZft5ZH_AgAJ > > My primary objectives in treating it as two distinct issues are: > 1) Ensuring the timely remediation of the root causes of whatever 'caused' > the incident. In general, this should be 'sooner' than 'later' > 2) Ensuring transparency about when CAs are having repeat "delayed > revocation" issues; buried within issues themselves makes it difficult to > track when there are repeat offenders, but also limits visibility into > systemic issues that all CAs can/should be aware of when contemplating the > design and maintenance of their PKI > 3) Ensuring progress is made towards revocation, by not having bugs > prematurely closed when the 'root' incident is resolved but the CA has > taken no meaningful steps to bring their PKI back into compliance (yet) > ___ dev-security-policy mailing list dev-security-policy@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-security-policy
Re: Incident Reporting Guidance
On Thu, Nov 21, 2019 at 10:54 AM Wayne Thayer via dev-security-policy < dev-security-policy@lists.mozilla.org> wrote: > During the recent CA/Browser Forum meeting, I was asked to provide better > guidance on Mozilla's expectations for incident reporting. We're adding a > requirement for incident reporting to the new version of our policy [1], > but in this message I'm focused on the guidance provided on our wiki [2]. > The question was when to add information to an existing bug versus creating > a new one. I'd like to propose adding the following guidance to the wiki to > address this question: > > CAs should create a separate bug and file a new incident report when: > * In the process of researching one incident another incident with a > distinct root cause and/or remediation is discovered > * After an incident bug is marked resolved, the incident reoccurs > > A third possible addition would be: > * When a CA accidentally or intentionally misses a revocation deadline, a > separate bug should/must be filed examining the root cause and remediation > for missing the deadline > > I believe the argument for this is that tracking revocation issues > separately will help us to focus on improving the agility of the web PKI. > On the other hand, Mozilla has not generally required separate reports in > the past, and doing so certainly creates more work for everyone involved. > It's not clear to me that the benefit of this outweighs the cost. > Yeah, I've been thinking about this in light of the feedback Kathleen offered on https://groups.google.com/d/msg/mozilla.dev.security.policy/M7NGwCh14DI/jZft5ZH_AgAJ My primary objectives in treating it as two distinct issues are: 1) Ensuring the timely remediation of the root causes of whatever 'caused' the incident. In general, this should be 'sooner' than 'later' 2) Ensuring transparency about when CAs are having repeat "delayed revocation" issues; buried within issues themselves makes it difficult to track when there are repeat offenders, but also limits visibility into systemic issues that all CAs can/should be aware of when contemplating the design and maintenance of their PKI 3) Ensuring progress is made towards revocation, by not having bugs prematurely closed when the 'root' incident is resolved but the CA has taken no meaningful steps to bring their PKI back into compliance (yet) ___ dev-security-policy mailing list dev-security-policy@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-security-policy
Incident Reporting Guidance
During the recent CA/Browser Forum meeting, I was asked to provide better guidance on Mozilla's expectations for incident reporting. We're adding a requirement for incident reporting to the new version of our policy [1], but in this message I'm focused on the guidance provided on our wiki [2]. The question was when to add information to an existing bug versus creating a new one. I'd like to propose adding the following guidance to the wiki to address this question: CAs should create a separate bug and file a new incident report when: * In the process of researching one incident another incident with a distinct root cause and/or remediation is discovered * After an incident bug is marked resolved, the incident reoccurs A third possible addition would be: * When a CA accidentally or intentionally misses a revocation deadline, a separate bug should/must be filed examining the root cause and remediation for missing the deadline I believe the argument for this is that tracking revocation issues separately will help us to focus on improving the agility of the web PKI. On the other hand, Mozilla has not generally required separate reports in the past, and doing so certainly creates more work for everyone involved. It's not clear to me that the benefit of this outweighs the cost. Are there other examples that would provide helpful guidance to CAs? I will appreciate everyone's input on this proposal. - Wayne [1] https://github.com/mozilla/pkipolicy/blob/2.7/rootstore/policy.md#24-incidents [2] https://wiki.mozilla.org/CA/Responding_To_An_Incident#Incident_Report ___ dev-security-policy mailing list dev-security-policy@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-security-policy
