Re: Globalsign accidental intermediate revocation incident
On 18/10/2016 20:50, douglas.beat...@gmail.com wrote: On Monday, October 17, 2016 at 4:19:34 PM UTC-7, Jakob Bohm wrote: On 16/10/2016 09:59, Adrian R. wrote: Hello i read in the news (but not here on m.d.s.p) that a few days ago Globalsign revoked one of their intermediary roots and then un-revoked it (well, the revocation is accidental, but it was still a properly announced revocation, via signed CRL and OCSP). The official report is available here: https://www.globalsign.com/en/customer-revocation-error/ Yes, I read the full report, I was nitpicking the bad communication skills of whomever GlobalSign assigned to write the contents of the web page (as it existed at that time). Seems only fair to nitpick a western CA in perfect standing when we are being so harsh on a Chinese CA in much worse standing. To be clear, an intermediate root (not sure what this is tbh) was never revoked - we revoked a cross certificate between roots R2 and R1 and an unused/EOL CA under R2. These correctly appeared and continues to appear on the CRL, in fact it was included on the October 7th Root R1 CRL well before the OCSP incident. http://www.theregister.co.uk/2016/10/15/globalsign_incident_report/ They rolled back the revocation, but i thought that the BRs explicitly forbid that a suspended/revoked certificate be un-suspended/un-revoked. https://www.globalsign.com/en/customer-revocation-error/ is this revival/un-revocation of an intermediary CA allowed by the BRs? I have just read that page, and find it utterly confusing and badly written. Lot's of formal tone of voice, but no precision or clarity. What I would have expected was a much clearer statement (on the page, not in some linked document) as to: I hope my responses help clarify the situation: 1. Which Intermediary CA certificates were affected (because certificate holders cannot see the cache state affecting relying parties, we need to check our certificates against something specific to see if we are affected). All CAs under Root R1 were effected. And the page should have said that. Or more precisely: - All certificates whose chain goes to Root R1 before going to Root R2. - Maybe (unclear in retrospect) certificates under root R2 on machines that only trust root R1 (such as Windows 8.x without auto-root updates), and which checked the OCSP status of the R1 to R2 cross cert or R1-root, at a time affected by the OCSP responder bug. - Maybe (unclear in retrospect) certificates under root R2 if the web server sends the R1 to R2 cross certificate to support R1-only trusting machines, and the web browser uses certificates sent be the web server in preference to checking that its root store already contains R2 (a common implementation limitation), while actually checking for revocation of R1-to-R2 cross or R1-root, at a time affected by the OCSP responder bug. 2. If this affects all AlphaSSL and CloudSSL certificates or only some of them. This affects the CAs under Root R1 only (AlphaSSL, DV, OV, CloudSSL). None of the actual end entity SSL certificates were ever revoked or marked as revoked either via CRL or OCSP. The scope of the users impacted by this depends on many factors, which are outlined in the report. Not all users were blocked from accessing sites with SSL certificates under R1. Yes, the report explains the cache issue clearly, but the web page could have said something like: AlphaSSL SHA-1 certificates issued before/after some dates (I guess all of them). AlphaSSL SHA-256 certificates issued before some date CloudSSL ... GlobalSign branded DV ... GlobalSign branded OV ... 3. If this was an "exercise" (training/experimental etc.) or an actual operational decision. We intentionally revoked 2 CA certificates as listed in the incident report - One old CA that had no live certificates issued under it, and the one cross certificate referenced above. So the word "exercise" was badly chosen (an exotic use of the word, which I am fully aware of, but which is not clear in a context where the meaning could just as well have been for example "as part of a disaster readiness test"). We did not take actions to revoke any CAs other than these two; however as pointed out, the OCSP responders incorrectly created and provided OCSP status messages for CAs under R1 indicating that the CA was revoked. These CAs never appeared in any CRLs. 4. If the revoked cross certificate was one of the cross certificates previously provided to certificate holders to allow us to make our servers compatible with older clients, and if so, which one. The report explains it was not, the web page was 100% vague. 5. If this was e-mailed to all potentially affected certificate holders, or just dumped in some public forums which certificate holders might not see in time to take necessary action. GlobalSign contacted as many of our customers as possible with SSL
Re: Globalsign accidental intermediate revocation incident
On 18/10/16 19:15, Ryan Sleevi wrote: > On Tuesday, October 18, 2016 at 10:52:19 AM UTC-7, Rob Stradling wrote: >> AIUI, it's permissible to "un-revoke" any certificate via OCSP, but it's >> only permissible to "un-revoke" a certificate via CRL if it was revoked >> with the reason code certificateHold. > > Which "permissible" are we talking about? BRs or 5280? I was talking just about 5280 there. i.e. I believe my statement was correct insofar as it went. > While 5280 allows certificateHold, the BRs do not - see 4.9.13: > The Repository MUST NOT include entries that indicate that a Certificate is > suspended Thanks for finding that, Ryan. (I was pretty sure the BRs prohibited certificateHold, but...insufficient PDF searching fu ;-) ). -- Rob Stradling Senior Research & Development Scientist COMODO - Creating Trust Online ___ dev-security-policy mailing list dev-security-policy@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-security-policy
Re: Globalsign accidental intermediate revocation incident
On Monday, October 17, 2016 at 4:19:34 PM UTC-7, Jakob Bohm wrote: > On 16/10/2016 09:59, Adrian R. wrote: > > Hello > > > > i read in the news (but not here on m.d.s.p) that a few days ago Globalsign > > revoked one of their intermediary roots and then un-revoked it (well, the > > revocation is accidental, but it was still a properly announced revocation, > > via signed CRL and OCSP). The official report is available here: https://www.globalsign.com/en/customer-revocation-error/ To be clear, an intermediate root (not sure what this is tbh) was never revoked - we revoked a cross certificate between roots R2 and R1 and an unused/EOL CA under R2. These correctly appeared and continues to appear on the CRL, in fact it was included on the October 7th Root R1 CRL well before the OCSP incident. > > > > http://www.theregister.co.uk/2016/10/15/globalsign_incident_report/ > > > > They rolled back the revocation, but i thought that the BRs explicitly > > forbid that a suspended/revoked certificate be un-suspended/un-revoked. > > > > https://www.globalsign.com/en/customer-revocation-error/ > > > > is this revival/un-revocation of an intermediary CA allowed by the BRs? > > > I have just read that page, and find it utterly confusing and badly > written. Lot's of formal tone of voice, but no precision or clarity. > > What I would have expected was a much clearer statement (on the page, > not in some linked document) as to: I hope my responses help clarify the situation: > 1. Which Intermediary CA certificates were affected (because >certificate holders cannot see the cache state affecting relying >parties, we need to check our certificates against something >specific to see if we are affected). All CAs under Root R1 were effected. > 2. If this affects all AlphaSSL and CloudSSL certificates or only some >of them. This affects the CAs under Root R1 only (AlphaSSL, DV, OV, CloudSSL). None of the actual end entity SSL certificates were ever revoked or marked as revoked either via CRL or OCSP. The scope of the users impacted by this depends on many factors, which are outlined in the report. Not all users were blocked from accessing sites with SSL certificates under R1. > 3. If this was an "exercise" (training/experimental etc.) or an actual >operational decision. We intentionally revoked 2 CA certificates as listed in the incident report - One old CA that had no live certificates issued under it, and the one cross certificate referenced above. We did not take actions to revoke any CAs other than these two; however as pointed out, the OCSP responders incorrectly created and provided OCSP status messages for CAs under R1 indicating that the CA was revoked. These CAs never appeared in any CRLs. > 4. If the revoked cross certificate was one of the cross certificates >previously provided to certificate holders to allow us to make our >servers compatible with older clients, and if so, which one. > 5. If this was e-mailed to all potentially affected certificate >holders, or just dumped in some public forums which certificate >holders might not see in time to take necessary action. GlobalSign contacted as many of our customers as possible with SSL certificates issued under Root R1. Note that EV certificates are issued under Root R2 and were not impacted by this issue. > > Enjoy > > Jakob > -- > Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com > Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10 > This public discussion message is non-binding and may contain errors. > WiseMo - Remote Service Management for PCs, Phones and Embedded ___ dev-security-policy mailing list dev-security-policy@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-security-policy
Re: Globalsign accidental intermediate revocation incident
On Tuesday, October 18, 2016 at 10:52:19 AM UTC-7, Rob Stradling wrote: > AIUI, it's permissible to "un-revoke" any certificate via OCSP, but it's > only permissible to "un-revoke" a certificate via CRL if it was revoked > with the reason code certificateHold. Which "permissible" are we talking about? BRs or 5280? While 5280 allows certificateHold, the BRs do not - see 4.9.13: The Repository MUST NOT include entries that indicate that a Certificate is suspended This tries to make it very clear that you shouldn't "un-revoke" a certificate, using 5280 language. This frequently comes up when performing CP/CPS reviews for European CAs, since many use a unified CP/CPS to cover both TLS/SSL and other forms (e.g. e-signature certificates), and is flagged as a Bad (because it may be a violation of the BRs) But let's take a little step back here - to me, this suggests a bit of ambiguity with respect to the BRs' notion of revocation. That is, we might think of revocation as a conceptual thing - a certificate is either revoked or it's not, and once it's revoked, it should not be unrevoked. However, how that revocation is communicated is done via Repositories - provided via CRLs and OCSP, for sure, but we've certainly seen CAs provide other forms of repositories. If you think back to India CCA, for example, they had a repository where you could look up by serial number and see the status, via a web page. And we see request CAs revoke by entering revocation information into Salesforce - ostensibly, another form of Repository. So what we see with GlobalSign incident is that the Repository had issues, not reflective of the conceptual Revocation that was performed. That is, the software hosting the Repository improperly returned some certificates as revoked, even though GlobalSign had not "revoked" them. That is, at least based on my understanding of the incident report, it seems to be an issue with an OCSP responder fed via CRLs, and the issue was the responder software. Every CA is required to retain an unmodifiable audit log of these Revocations, which may be independent of the Repository representation. That is, *something* has to feed the CRL for generation - most frequently a 'backing' database, rather than 'chaining' CRLs from the past. This is all subtle semantics, and I'm concerned that it may allow CAs to 'game' the system, but at least with respect to GlobalSign, that's how I view this incident and it's relation to the BRs. In my view, if a CA "revokes" a certificate in their underlying system, they MUST NOT unrevoke, on the basis of 4.9.13. Similarly, they're expected to communicate that revocation detail via the Repository. It's possible in the future that future Repositories will encounter similar desync events with the backing store, much like GlobalSign's did. I think each of those represent an Incident - a failure of the Repository is potentially a failure of 2.1 of the BRs. I do not believe it's in the best interest of the ecosystem to suggest that CAs can "unrevoke" certificates, but I also understand issues and bugs happen - and we should treat such incidents as such, and evaluate on a case-by-case basis and evaluate the CA's communications, explanations, frequency, and practices to determine whether this represents a risk to Mozilla users. ___ dev-security-policy mailing list dev-security-policy@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-security-policy
Re: Globalsign accidental intermediate revocation incident
On 17/10/16 15:36, Gervase Markham wrote: > On 16/10/16 08:59, Adrian R. wrote: >> is this revival/un-revocation of an intermediary CA allowed by the >> BRs? > > I agree that the wording is a little loose but I think the intended > purpose of the clause in question was as Peter interprets it - don't > remove things from OCSP or CRLs before their expiry date because relying > parties may want to continue to check their revocation status at any > time up to then. > > I don't think it was intended to forbid the "un-revoking" of a > certificate. Whether or not that will even work properly in a given > situation is another question, but I think that's outside the scope of > the BRs. AIUI, it's permissible to "un-revoke" any certificate via OCSP, but it's only permissible to "un-revoke" a certificate via CRL if it was revoked with the reason code certificateHold. -- Rob Stradling Senior Research & Development Scientist COMODO - Creating Trust Online ___ dev-security-policy mailing list dev-security-policy@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-security-policy
Re: Globalsign accidental intermediate revocation incident
On 16/10/2016 09:59, Adrian R. wrote: Hello i read in the news (but not here on m.d.s.p) that a few days ago Globalsign revoked one of their intermediary roots and then un-revoked it (well, the revocation is accidental, but it was still a properly announced revocation, via signed CRL and OCSP). http://www.theregister.co.uk/2016/10/15/globalsign_incident_report/ They rolled back the revocation, but i thought that the BRs explicitly forbid that a suspended/revoked certificate be un-suspended/un-revoked. https://www.globalsign.com/en/customer-revocation-error/ is this revival/un-revocation of an intermediary CA allowed by the BRs? I have just read that page, and find it utterly confusing and badly written. Lot's of formal tone of voice, but no precision or clarity. What I would have expected was a much clearer statement (on the page, not in some linked document) as to: 1. Which Intermediary CA certificates were affected (because certificate holders cannot see the cache state affecting relying parties, we need to check our certificates against something specific to see if we are affected). 2. If this affects all AlphaSSL and CloudSSL certificates or only some of them. 3. If this was an "exercise" (training/experimental etc.) or an actual operational decision. 4. If the revoked cross certificate was one of the cross certificates previously provided to certificate holders to allow us to make our servers compatible with older clients, and if so, which one. 5. If this was e-mailed to all potentially affected certificate holders, or just dumped in some public forums which certificate holders might not see in time to take necessary action. Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded ___ dev-security-policy mailing list dev-security-policy@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-security-policy
Re: Globalsign accidental intermediate revocation incident
Here is my understanding, according to the wording in GlobalSign's incident report ( https://downloads.globalsign.com/acton/attachment/2674/f-06d2/1/-/-/-/-/globalsign-incident-report-13-oct-2016.pdf ): -Revocation of the certificate was intended. GlobalSign writes: "In a revocation exercise which should have been 'business as usual.'" The certificate they intended to revoke is referred to as a "Cross Certificate." -They first published the revocation to their CRL. No adverse effects. -They then published the revocation to their OCSP database. This is where the issue occurred. It seems that, due to the subject information of the revoked certificate, it was inferred that a separate intermediate certificate, which had NOT been revoked (at least, in GlobalSign's opinion) HAD been revoked. It appears that this error is a bug in the third-party OCSP software that GlobalSign uses. They wrote: "the logic within the responder code base determined that the revocation of the Cross Certificate, identified by its Public Key and Subject Name in a lookup table, was effectively an instruction to also identify all other subordinate certificate authorities including DomainSSL and AlphaSSL as ‘bad’." -This incident caused the widespread revocation errors that The Register reported on. -GlobalSign remediated the issue in multiple ways: They "unrevoked" the Cross Certificate, so that the CDNs they were using for OCSP availability would "refresh" and stop reporting the revocation. They also issued a new set of intermediate certificates that server admins could configure their server to use. Either option would hopefully allow affected sites to solve the problem. Some users were never affected because the CDNs that provided the OCSP information did not cache at those CDNs. Some sites were never affected because GlobalSign has separate roots for some certs (EV, etc). - So, on to the question of if this is a violation of the Baseline Requirements. It seems like it may indeed be a violation. Looking at the BRs ( https://cabforum.org/wp-content/uploads/CA-Browser-Forum-BR-1.4.1.pdf ). Section 4.10.1 says: "Revocation entries on a CRL or OCSP Response MUST NOT be removed until after the Expiry Date of the revoked Certificate." In GlobalSign's Incident Report, within the section "Incident Timeline", they write: "13-Oct-16 12:20 Removal of cross signing certificate from the OCSP Database" So it seems that be un-publishing the revocation information for the Cross Certificate would indeed be a BR violation. But this is not an area of compliance I am familiar with. Hopefully someone else can chime in and say for sure if this is a violation. ___ dev-security-policy mailing list dev-security-policy@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-security-policy
Re: Globalsign accidental intermediate revocation incident
Sound to me like they probably still want that particular certificate revoked as soon as the bug has been fixed. CU Hans ___ dev-security-policy mailing list dev-security-policy@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-security-policy
Globalsign accidental intermediate revocation incident
So that explains why our URL checking batch job was logging certificate invalid errors for some 700 links to the Wikipedia we have on our website for two days. I checked with a browser but couldn't see anything wrong. Make more sense knowing this... ;-)t CU Hans ___ dev-security-policy mailing list dev-security-policy@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-security-policy
Globalsign accidental intermediate revocation incident
Hello i read in the news (but not here on m.d.s.p) that a few days ago Globalsign revoked one of their intermediary roots and then un-revoked it (well, the revocation is accidental, but it was still a properly announced revocation, via signed CRL and OCSP). http://www.theregister.co.uk/2016/10/15/globalsign_incident_report/ They rolled back the revocation, but i thought that the BRs explicitly forbid that a suspended/revoked certificate be un-suspended/un-revoked. https://www.globalsign.com/en/customer-revocation-error/ is this revival/un-revocation of an intermediary CA allowed by the BRs? Adrian R. (p.s. can we call this revived certificate a zombie? :) ) (off-topic note: sigh, was intending not to post here again because news relay servers that strip DKIM signatures will generate a lot of DMARC failure reports for rejected messages. oh well.) ___ dev-security-policy mailing list dev-security-policy@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-security-policy
