We had some kind of glitch in our internal network last night. This impacted
(failed) all jobs that were running around 10:30pm - 11:30pm PST. At the time,
the queue of reviews in CI was very large.
Throughout the night, our CI was limping along, running at about 15% capacity
(slaves were offline due to dangling ssh connections, stuck slave VMs, etc).
Early this morning I restarted jenkins and many slave VMs that were stuck. CI
back up and seems to be working well. However, due to the huge backlog, it
will take several days until CI catches up.
Dev mailing list