Re: Misissuance/non-compliance remediation timelines
On February 9, 2018 at 1:24:12 AM, Wayne Thayer (wtha...@mozilla.com) wrote: On Tue, Feb 6, 2018 at 6:03 PM, Paul Kehrer via dev-security-policy < dev-security-policy@lists.mozilla.org> wrote: > > So, how long is too long? > This is the crux of the issue for me. If a CA (that really should have stopped responding 'good' for unknown certs back in 2013) needs to select, purchase, and deploy an entirely new OCSP system, is 5 months a really long time? From their perspective, probably not. I agree that from their perspective that's a short period of time. However, I strongly believe that asking the public to bear the burden of a CA's own incompetence is, while historically what has been done, not tenable moving forward. In the specific case of the OCSP issues I question why we should give CAs half a year to remediate a fault that had already been a requirement for 4 years when it was discovered. In many ways a CA's primary job is knowing and following the rules, so why are we giving CAs who fail in such colossal fashion a free pass? I don't believe there is a standard answer to this question that can apply to a whole class of issues, but I do think we could do a better job of communicating our expectations when a situation like this arises by making a statement such as 'being a CA that has been granted the public's trust, Mozilla expects problem X to be resolved in Y days'. Responsible CAs will meet the deadline and thus distinguish themselves from CAs that simply aren't taking the problem seriously. If Mozilla provides a deadline and a CA misses it, what's the penalty? I believe a graduated notion of penalties and risk mitigation would make it easier for Mozilla to push CAs. If the only penalty is distrust then "little" things like a slow but steady trickle of misissued certificates, operating your OCSP responder out of compliance for 4 years, or failing to get a BR audit for 3 years after they became required never rise to the level of a distrust conversation. If, on the other hand, there exists a set of penalty tiers a CA can be placed on that path relatively quickly. Instead of a "sudden" (from the perspective of the CA or subscribers who aren't engaged with policy discussions on mdsp) distrust thread, there is an escalation that makes everyone aware of a CA's need to shape up. -Paul ___ dev-security-policy mailing list dev-security-policy@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-security-policy
Re: Misissuance/non-compliance remediation timelines
On Tue, Feb 6, 2018 at 6:03 PM, Paul Kehrer via dev-security-policy < dev-security-policy@lists.mozilla.org> wrote: > > So, how long is too long? > This is the crux of the issue for me. If a CA (that really should have stopped responding 'good' for unknown certs back in 2013) needs to select, purchase, and deploy an entirely new OCSP system, is 5 months a really long time? From their perspective, probably not. I don't believe there is a standard answer to this question that can apply to a whole class of issues, but I do think we could do a better job of communicating our expectations when a situation like this arises by making a statement such as 'being a CA that has been granted the public's trust, Mozilla expects problem X to be resolved in Y days'. Responsible CAs will meet the deadline and thus distinguish themselves from CAs that simply aren't taking the problem seriously. Wayne ___ dev-security-policy mailing list dev-security-policy@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-security-policy
Re: Misissuance/non-compliance remediation timelines
On 07/02/18 15:14, Alex Gaynor wrote: > That said, given the issues Paul highlighted in his original mail (which I > wholeheartedly concur with), it seems the place to focus is the folks who > are getting Ds right now. Therefore I think the essential part of your > email is your agreement that CAs which are persistently low performing need > to be recognized and potentially penalized for the sum total of their > behaviors. This is, in a reasonably strong sense, what happens now. We require each incident in which a CA is involved to be documented in a public bug, so all can see the timeline, outcomes, how the CA reacted and other factors which might be taken into account when determining a CA's competence. Occasionally, we decide that some CA's list of recent[0] problems is sufficiently serious[0] that we need to run an investigation. We do so, and invite the CA to more formally comment on the sum total of the problems. We assess the responses and the style and level of response, and make a determination[0]. This is what happened to WoSign, Symantec and PROCERT: https://wiki.mozilla.org/CA:WoSign_Issues https://wiki.mozilla.org/CA:Symantec_Issues https://wiki.mozilla.org/CA:PROCERT_Issues I therefore expect and hope that CAs in our program have noted what happened in those cases, particularly PROCERT (which is probably the clearest case of simple "general incompetence" that we have had), and want to make sure they are not next. Gerv [0] Yes, this is vague. But so is the concept of "trust". ___ dev-security-policy mailing list dev-security-policy@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-security-policy
RE: Misissuance/non-compliance remediation timelines
The idea of a grading system being used to judge CAs compliance will be a total
disaster. We should instead be focusing our efforts on more transparency.
James
-Original Message-
From: dev-security-policy
[mailto:dev-security-policy-bounces+jb=0.me...@lists.mozilla.org] On Behalf Of
Tim Hollebeek via dev-security-policy
Sent: 07 February 2018 16:11
To: Alex Gaynor <agay...@mozilla.com>
Cc: mozilla-dev-security-pol...@lists.mozilla.org; Paul Kehrer
<paul.l.keh...@gmail.com>
Subject: RE: Misissuance/non-compliance remediation timelines
Alex,
Most CAs probably wouldn’t aim for an A. I don’t think doing this would be a
game changer.
However there are some CAs that would. And I think that would be a positive
thing, and lead to more innovation in best practices that could become
mandatory for everyone over time.
And I don’t disagree with you that action is needed on those who are currently
getting Ds. I’m very disturbed by the behavior of about half of the CAs in the
industry.
-Tim
From: Alex Gaynor [mailto:agay...@mozilla.com]
Sent: Wednesday, February 7, 2018 8:15 AM
To: Tim Hollebeek <tim.holleb...@digicert.com>
Cc: Paul Kehrer <paul.l.keh...@gmail.com>;
mozilla-dev-security-pol...@lists.mozilla.org
Subject: Re: Misissuance/non-compliance remediation timelines
Hey Tim,
A piece I think I'm missing is what you see as the incentive for CAs to aim for
an "A" rather than being happy to have a "B". It reminds me of the old joke:
What do you call the Dr^W CA who graduated with a C average? Dr.^W trusted to
assert www-wide identity :-)
That said, given the issues Paul highlighted in his original mail (which I
wholeheartedly concur with), it seems the place to focus is the folks who are
getting Ds right now. Therefore I think the essential part of your email is
your agreement that CAs which are persistently low performing need to be
recognized and potentially penalized for the sum total of their behaviors.
Alex
On Tue, Feb 6, 2018 at 8:30 PM, Tim Hollebeek via dev-security-policy
<dev-security-policy@lists.mozilla.org
<mailto:dev-security-policy@lists.mozilla.org> > wrote:
Paul,
I understand your frustration. I've read some of the recent threads about "how
long does it take to update a CPS?" and clearly there needs to be some stronger
compliance language in either the BRs or Mozilla policy ("CAs MUST be able to
update their CPS within 90 days"). And as you note such policies need to have
teeth otherwise there will be some who will just ignore them.
However negative penalties are not the only thing that should be considered.
Mozilla should also have some way of recognizing CAs that are performing above
and beyond the minimum requirements. I would love to see Mozilla encourage CAs
to compete to be the best CA in Mozilla's program.
To satisfy both goals, I'd like to suggest an idea I've had for a while: at
some point in time (annually?), Mozilla should assess their opinion of how well
each CA in the program is performing, and give them a letter grade. This could
include policy improvements like "Two consecutive failing grades, or three
consecutive C or lower grades and you're out of the Mozilla program."
This would not preclude other actions as Mozilla deems necessary. But it would
provide a regular checkpoint for CAs to understand either "Hey, you're great,
keep up the good work!" or "Meh, we think you're ok." or "Your performance to
date is unacceptable. Get your sh*t together or you're gone."
-Tim
> -Original Message-
> From: dev-security-policy [mailto:dev-security-policy-
> <mailto:dev-security-policy->
> bounces+tim.hollebeek=digicert@lists.mozilla.org
> bounces+<mailto:digicert@lists.mozilla.org> ] On Behalf Of Paul
> Kehrer via dev-security-policy
> Sent: Tuesday, February 6, 2018 6:03 PM
> To: mozilla-dev-security-pol...@lists.mozilla.org
> <mailto:mozilla-dev-security-pol...@lists.mozilla.org>
> Subject: Misissuance/non-compliance remediation timelines
>
> A bit over 5 months ago I reported a series of OCSP responders that
> were violating BRs (section 4.9.10) by returning GOOD on unknown
> serial
numbers.
> Since that time the majority of those responder endpoints have been
> fixed,
but
> several are still non-compliant; either with little communication or
continuing
> assurances that it will be fixed "soon", where soon is a date that
continuously
> slides into the future.
>
> At the moment Mozilla possesses very few options when it comes to
> punitive action and the lesson some CAs appear to be learning is that
> as long as
you
> don't rise to PROCERT levels of malfeasance/incompetence then the
> maximum penalty is censure on bugzilla and email threads. Clearly t
RE: Misissuance/non-compliance remediation timelines
Alex,
Most CAs probably wouldn’t aim for an A. I don’t think doing this would be a
game changer.
However there are some CAs that would. And I think that would be a positive
thing, and lead to more innovation in best practices that could become
mandatory for everyone over time.
And I don’t disagree with you that action is needed on those who are currently
getting Ds. I’m very disturbed by the behavior of about half of the CAs in the
industry.
-Tim
From: Alex Gaynor [mailto:agay...@mozilla.com]
Sent: Wednesday, February 7, 2018 8:15 AM
To: Tim Hollebeek <tim.holleb...@digicert.com>
Cc: Paul Kehrer <paul.l.keh...@gmail.com>;
mozilla-dev-security-pol...@lists.mozilla.org
Subject: Re: Misissuance/non-compliance remediation timelines
Hey Tim,
A piece I think I'm missing is what you see as the incentive for CAs to aim for
an "A" rather than being happy to have a "B". It reminds me of the old joke:
What do you call the Dr^W CA who graduated with a C average? Dr.^W trusted to
assert www-wide identity :-)
That said, given the issues Paul highlighted in his original mail (which I
wholeheartedly concur with), it seems the place to focus is the folks who are
getting Ds right now. Therefore I think the essential part of your email is
your agreement that CAs which are persistently low performing need to be
recognized and potentially penalized for the sum total of their behaviors.
Alex
On Tue, Feb 6, 2018 at 8:30 PM, Tim Hollebeek via dev-security-policy
<dev-security-policy@lists.mozilla.org
<mailto:dev-security-policy@lists.mozilla.org> > wrote:
Paul,
I understand your frustration. I've read some of the recent threads about
"how long does it take to update a CPS?" and clearly there needs to be
some stronger compliance language in either the BRs or Mozilla policy
("CAs MUST be able to update their CPS within 90 days"). And as you note
such policies need to have teeth otherwise there will be some who will
just ignore them.
However negative penalties are not the only thing that should be
considered.
Mozilla should also have some way of recognizing CAs that are performing
above and beyond the minimum requirements. I would love to see Mozilla
encourage CAs to compete to be the best CA in Mozilla's program.
To satisfy both goals, I'd like to suggest an idea I've had for a while: at
some
point in time (annually?), Mozilla should assess their opinion of how well
each CA in the program is performing, and give them a letter grade. This
could include policy improvements like "Two consecutive failing grades,
or three consecutive C or lower grades and you're out of the Mozilla
program."
This would not preclude other actions as Mozilla deems necessary. But it
would provide a regular checkpoint for CAs to understand either "Hey,
you're great, keep up the good work!" or "Meh, we think you're ok." or
"Your performance to date is unacceptable. Get your sh*t together or
you're gone."
-Tim
> -Original Message-
> From: dev-security-policy [mailto:dev-security-policy-
> <mailto:dev-security-policy->
> bounces+tim.hollebeek=digicert@lists.mozilla.org
> <mailto:digicert@lists.mozilla.org> ] On Behalf Of Paul
> Kehrer via dev-security-policy
> Sent: Tuesday, February 6, 2018 6:03 PM
> To: mozilla-dev-security-pol...@lists.mozilla.org
> <mailto:mozilla-dev-security-pol...@lists.mozilla.org>
> Subject: Misissuance/non-compliance remediation timelines
>
> A bit over 5 months ago I reported a series of OCSP responders that were
> violating BRs (section 4.9.10) by returning GOOD on unknown serial
numbers.
> Since that time the majority of those responder endpoints have been fixed,
but
> several are still non-compliant; either with little communication or
continuing
> assurances that it will be fixed "soon", where soon is a date that
continuously
> slides into the future.
>
> At the moment Mozilla possesses very few options when it comes to punitive
> action and the lesson some CAs appear to be learning is that as long as
you
> don't rise to PROCERT levels of malfeasance/incompetence then the maximum
> penalty is censure on bugzilla and email threads. Clearly this is not a
deterrent.
>
> So, how long is too long? At what point should a CA incur consequences
(and
> what form can those consequences take) for failure to remediate despite
being
> given such immense latitude?
>
> As a straw man: what if we developed a set of technical constraints
related to
> minimizing risk associated with CAs that are deemed to be acting poorly?
> For example, CAs deemed a risk would have their certificate maximum
lifetime
> constrained to some amount less than the BRs currently allow.
> Additional penalties could include removal of EV trust indicat
RE: Misissuance/non-compliance remediation timelines
That’s pretty much exactly not what I said. From: Ryan Sleevi [mailto:r...@sleevi.com] Sent: Tuesday, February 6, 2018 10:38 PM To: Tim Hollebeek <tim.holleb...@digicert.com> Cc: Paul Kehrer <paul.l.keh...@gmail.com>; mozilla-dev-security-pol...@lists.mozilla.org; r...@sleevi.com Subject: Re: Misissuance/non-compliance remediation timelines So your view is the “carrot” is getting to use Mozilla’s brand as an endorsement, and the “stick” being that if you don’t get that endorsement for a while, you get kicked out? The assumption is that the branding of “best” is valuable - presumably, through the indirect benefit of being able to appeal to customers as “the highest rated (by Mozilla) CA”. In practice, much like the CA/Browser Forum indirectly gave birth to the CA “Security” Council, or the existence of firms like Netcraft or NSS Labs, the more common outcome seems to be that if you don’t like the rules of the game you’re playing, you make up your own/redefine them and try to claim equivalency (much lol “alternative facts”). That is, I’m skeptical of approaches that attempt to say “most good,” because those seem to encourage all sorts of games of coming up with their own schemes, while “least bad” is more actionable - as “most bad” is more likely to receive sanctions. On Tue, Feb 6, 2018 at 10:03 PM Tim Hollebeek via dev-security-policy <dev-security-policy@lists.mozilla.org <mailto:dev-security-policy@lists.mozilla.org> > wrote: Absolutely not. I view the competition as being based as the “most best”. You cannot get an “A” (or even A- or B+) without significantly exceeding the minimum requirements, or demonstrating behaviors and practices that, while not required, are behaviors Mozilla wants to encourage. Sticks are good. Carrots are tasty. -Tim Do you see the competition based on being the 'least bad' (i.e. more likely to get an A because of no issues than a B because of some?) ___ dev-security-policy mailing list dev-security-policy@lists.mozilla.org <mailto:dev-security-policy@lists.mozilla.org> https://lists.mozilla.org/listinfo/dev-security-policy smime.p7s Description: S/MIME cryptographic signature ___ dev-security-policy mailing list dev-security-policy@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-security-policy
Re: Misissuance/non-compliance remediation timelines
So your view is the “carrot” is getting to use Mozilla’s brand as an endorsement, and the “stick” being that if you don’t get that endorsement for a while, you get kicked out? The assumption is that the branding of “best” is valuable - presumably, through the indirect benefit of being able to appeal to customers as “the highest rated (by Mozilla) CA”. In practice, much like the CA/Browser Forum indirectly gave birth to the CA “Security” Council, or the existence of firms like Netcraft or NSS Labs, the more common outcome seems to be that if you don’t like the rules of the game you’re playing, you make up your own/redefine them and try to claim equivalency (much lol “alternative facts”). That is, I’m skeptical of approaches that attempt to say “most good,” because those seem to encourage all sorts of games of coming up with their own schemes, while “least bad” is more actionable - as “most bad” is more likely to receive sanctions. On Tue, Feb 6, 2018 at 10:03 PM Tim Hollebeek via dev-security-policy < dev-security-policy@lists.mozilla.org> wrote: > Absolutely not. I view the competition as being based as the “most best”. > > > > You cannot get an “A” (or even A- or B+) without significantly exceeding > the minimum requirements, or demonstrating behaviors and practices that, > while not required, are behaviors Mozilla wants to encourage. > > > > Sticks are good. Carrots are tasty. > > > > -Tim > > > > Do you see the competition based on being the 'least bad' (i.e. more > likely to get an A because of no issues than a B because of some?) > > ___ > dev-security-policy mailing list > dev-security-policy@lists.mozilla.org > https://lists.mozilla.org/listinfo/dev-security-policy > ___ dev-security-policy mailing list dev-security-policy@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-security-policy
RE: Misissuance/non-compliance remediation timelines
Absolutely not. I view the competition as being based as the “most best”. You cannot get an “A” (or even A- or B+) without significantly exceeding the minimum requirements, or demonstrating behaviors and practices that, while not required, are behaviors Mozilla wants to encourage. Sticks are good. Carrots are tasty. -Tim Do you see the competition based on being the 'least bad' (i.e. more likely to get an A because of no issues than a B because of some?) smime.p7s Description: S/MIME cryptographic signature ___ dev-security-policy mailing list dev-security-policy@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-security-policy
