Hi all, Etherpad [0], our real-time colaborative editing tool suffered an outage due to what we only know for now was database corruption. This was detected shortly after it happened 14:27 UTC and we (ops in charge of the service and the database) worked to reestablish the service.
As the service continued crashing despite our efforts, we decided to recover a database backup from 2016-06-22 01:00:01 UTC. The service is now back up and working since 16:11 UTC, but that means that you may have lost a day and a half of edits in the current available etherpad [0]. I understand that that may cause a lot of inconveniences, specially for the people at Wikimania. *We are now trying to recover more than that*, but as the corruption could come back, or not all could be recovered, and people need the service the plan is the following: - Keep the current pads as is, not delete or add anything from now. You can continue using etherpad now as usual. - If possible, recover the last days of edits on a separate location. See [1] for progress if you are affected. Sorry for the inconveniences. Please, more than ever, follow the recommendation we added at the beginning of every empty pad: > "Keep in mind as well that there is no guarantee that a pad's contents will > always be available. A pad may be corrupted, deleted or similar. Please keep > a copy of important data somewhere else as well" The reason for this is that wiki content has proper HA and redundancy, etherpad does not. Again, my most sincere apologies, [0] <https://etherpad.wikimedia.org/> [1] <https://phabricator.wikimedia.org/T138516> -- Jaime Crespo <http://wikimedia.org> _______________________________________________ Wikitech-l mailing list [email protected] https://lists.wikimedia.org/mailman/listinfo/wikitech-l
