Le 10/02/2016 22:06, Greg Grossmeier a écrit : > Hello all, > > I want to apologize for the current slowness in our CI stack. Everything > is working fine in the sense that nothing is broken. > > We simply did not adequately prepare for the switch to php5.5 being the > new minimum version for MediaWiki. This resulted in a large influx of > jobs on a set of our Jenkins executers that was not a large enough pool. > Timo and Antoine have created more executers to help mitigate [0]. > > More explicitly: due to package version availability in Ubuntu, our > php5.3 jobs run on our pool of Ubuntu Precis hosts, while the > php5.5/hhvm jobs run on our pool of Ubuntu Trusty hosts. > > Both sets of jobs are in our pipeline to migrate to the new Nodepool > system that allows burstable availability of executers. > > Again, apologies for the delays but know that the jobs will complete > just not as fast as you are accustomed to. > > Greg > > [0] https://phabricator.wikimedia.org/T126423
Hello, As a follow, yesterday we created new instance turns out 2GB memory is not enough to run some of the MediaWiki jobs. Since yesterday you might have encountered errors such as: * Access denied for user 'jenkins_mw0'@'localhost' * Disk I/O Error * File that went away mysteriously * Long delays in jobs etc A few minutes ago I have deleted all those 2GB slaves and spawned bigger ones. Provisioning is ongoing as I am writing. Will get them pooled in a couple/three hours. I do not have proof: but PHP 5.5 on Trusty might have its share of issue as well when stress tests with our test suites. After all we never massively ran tests against 5.5 ;-} cheers and be careful! -- Antoine "hashar" Musso _______________________________________________ Wikitech-l mailing list [email protected] https://lists.wikimedia.org/mailman/listinfo/wikitech-l
