Thanks for the info, and for the quick fix!
Jacob
On Wed, May 4, 2016 at 3:48 AM, Florian Apolloner <f.apollo...@gmail.com>
wrote:
> Hi,
>
> earlier today (roughly 9:30 UTC) I deployed a wrong (local) branch from
> our ansible repository to dp.com. This branch included our old (now
> expired) gandi SSL certificate. Once I realized what I did (which took a
> few minutes since I was preparing to push a commit) I switched to the
> proper branch and redeployed (+ notified ops and IRC). This redeployment
> caused our new and shiny letsencrypt.org key to be redeployed (which we
> have been running successfully the last few weeks) but left the old gandi
> crt (the cert not the key!) in place, since letsencrypt automatically
> generates that. At this point nginx would no longer start:
>
> SSL_CTX_use_PrivateKey_file("/etc/nginx/ssl/djangoproject.com.key") failed
> (SSL: error:0B080074:x509 certificate routines:X509_check_private_key:key
> values mismatch)
>
> This is kind of an obvious error, as the private key and the cert no
> longer match, which I verified with "openssl x509 -in djangoproject.com.crt
> -text -noout". The solution to this problem is simple, regenerate the cert
> and be done with it -- sadly not that simple since nginx refused to start
> :D Setting ssl to off in nginx.conf had no effect, apparently nginx is
> really picky about unusable ssl certs. Knowing that we configure nginx only
> via ansible I quickly reduced the config to the default server without ssl,
> started it and regenerated the cert. An ansible run restored the original
> config and allowed the server to start again -- executed twice just to
> ensure that the second run would not yield any changes in files (finally at
> 9:56 UTC).
>
> Going forward I think we will do the following:
> * Monitor SSL certs more closely, it shouldn't take me minutes to notice
> that the cert is broken
> * See if we can ensure that we push from the correct ansible branch
> * Find a config that allows nginx to start with invalid certs if possible
> (at least the http listeners), so we can easily reissue certs.
>
> Sorry for the outage,
> Florian
>
> P.S.: On a non-related note: Google should display search results in the
> correct language now :D
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at https://groups.google.com/group/django-developers.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/992b6ec7-fa82-4392-ba4b-d0f74f09b865%40googlegroups.com
> <https://groups.google.com/d/msgid/django-developers/992b6ec7-fa82-4392-ba4b-d0f74f09b865%40googlegroups.com?utm_medium=email&utm_source=footer>
> .
> For more options, visit https://groups.google.com/d/optout.
>
--
You received this message because you are subscribed to the Google Groups
"Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-developers/CAK8PqJEVqPpEXXzOAaGJEaRYbua%2BJQ4YTrjKk%3DWtoGb7QJUxRw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
