Earlier today labsdb1003 hit a segfault. It recovered and has caught up replication. We have a stack trace and a lead on a bug for it. The others, labsdb100[12], were not affected.
Paging didn't happen correctly due to an Icinga check broken since labsdb upgrades last year. That has been fixed along with some other puppet cleanup. The labsdbs deserve a restart for maintenance and package upgrade which will happen within the next few hours, and should take a few minutes each. If you experience disconnects, try one of the others. They won't all be down simultaneously. Sean -- DBA @ WMF
_______________________________________________ Labs-l mailing list [email protected] https://lists.wikimedia.org/mailman/listinfo/labs-l
