Currently manage-projects is frequently failing on a mysql error: http://paste.openstack.org/show/134438/
Our operating theory is that if new project creation takes a long time (definition to be determined) the open mysql port goes away and then it isn't open when we need it, so the update to the groups file for the new project fails and then so do all subsequent actions. This creates a repo in gerrit without a default branch which then breaks any image creation actions for us and for many third parties implementing their own CI as well. This is not directly related to manage-projects but is one of the effects we are seeing when we have a manage-projects failure. The frequency of the failures on the mysql error is quite high, most recent projects have hit it and have had to be respun manually, which is less than efficient since manage-projects is triggered on average about 3 times a day currently. The story for this can be found here: https://storyboard.openstack.org/#!/story/2000017 Course of action: - cease approving new project repos until further notice - help us to remedy the situation (calling any mysql folks with advice on how to keep the port open, or re-open it when we need it) - share this information with anyone who you feel needs to know Thank you, Anita. _______________________________________________ OpenStack-Infra mailing list [email protected] http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
