Thanks, Cedrick. A question that is, perhaps, obvious... are you able to take the human component out of this? If 2 reminders were not enough to get the humans to act, I'm not sure the current methodology is sustainable.
Mark. On 8/Apr/19 17:46, Cedrick Adrien Mbeyet wrote: > > Dear AFRINIC community, > > > Find below postmortem report on the incident that happen on 06 April > 2019. > > > > The AFRINIC RPKI engine has an offline part that has to be renewed on > a monthly bases. The process is known, documented and automated > reminders set. The system is set to send 2 reminders each month, one > 15 days prior to the expiry date and the second one 7 days before > expiry. On the 2nd half of March, the monitoring system sent a > reminder to perform the offline refresh but this was not acted upon. > > > > > > On Saturday 06 April 2019, Certificate revocation List (CRL) and the > manifest file of AFRINIC RPKI repository expired (around 07:24AM UTC). > Our monitoring system picked this up. The immediate action was to > generate new certificates and manifest file and upload them onto RPKI > engine system. > > > > The failure was as a result of human error, no changes were made on > the system but we have taken additional steps to the existing process > to ensure that this does not happen again. We do acknowledge that it > is unacceptable to have such a failure with critical infrastructure > and necessary done in this regard. > > > > > > We do apologize for the inconvenience caused and thank you for your > patience in this regard. > > -- > _______________________________________________________________ > Cedrick Adrien Mbeyet > Infrastructure Unit Manager, AFRINIC Ltd. > t: +230 403 5100 / 403 5115 | f: +230 466 6758 | tt: @afrinic | w: > www.afrinic.net > facebook.com/afrinic | flickr.com/afrinic | youtube.com/afrinicmedia > ______________________________________________________ >
_______________________________________________ Community-Discuss mailing list [email protected] https://lists.afrinic.net/mailman/listinfo/community-discuss
