On Wed, Mar 6, 2013 at 4:54 PM, Petr Bena <[email protected]> wrote: > okay this is third time when we have same outage... bastion2 and 3 > were accessible for short time after bastion1's gluster died, then > they died as well. public keys weren't accessible on any of them so > basically labs were inaccessible for anyone.
citation needed. I was just able to log in to both of bastion[23].wmflabs.org on the first try. [removed garbage about password auth being wonderful...] > Set up a cron script that sync a local folder on bastion with > /public/keys so that when gluster is down or that folder isn't working > login to bastion's still works. That might be feasible. But really the solution is don't let people kill the bastion. idk how we do that. and idk why the past social restrictions aren't sufficient. maybe we need ulimit or cgroups or something. :-( -Jeremy _______________________________________________ Labs-l mailing list [email protected] https://lists.wikimedia.org/mailman/listinfo/labs-l
