It would be nice if we had data indicating how often tests fail on just one version of MacOS, so we didn't have guess how useful having 10.6, 10.7, and 10.8 tests are. That's bug 860870. It's currently blocked on treeherder, but maybe it should be re-prioritized, since we keep running into cases where this data would be helpful.
Anyway, disabling the 10.7 tests sounds reasonable to me given no data, but maybe we continue running these tests on m-c? Maybe we also deprecate the 10.7 tests on tryserver, so you only get the tests if you really really want them? On Thu, Apr 25, 2013 at 1:40 PM, Andreas Gal <g...@mozilla.com> wrote: > > How many 10.7 machines do we operate in that pool? > > Andreas > > On Apr 25, 2013, at 10:30 AM, "Armen Zambrano G." <arme...@mozilla.com> wrote: > >> (please follow up through mozilla.dev.planning) >> >> Hello all, >> I have recently been looking into our Mac OS X test wait times which have >> been bad for many months and progressively getting worst. >> Less than 80% of test jobs on OS X 10.6 and 10.7 are able to start >> within 15 minutes of being requested. >> This slows down getting tests results for OS X and makes tree closures >> longer if we have Mac OS X test back logs. >> Unfortunately, we can't buy any more revision 4 Mac minis (they're not sold >> anymore) as Apple discontinues old hardware as new ones comes out. >> >> In order to improve the turnaround time for Mac testing, we have to look >> into reducing our test load in one of these two OSes (both of them run on >> revision 4 minis). >> We have over a third of our OS X users running 10.6. Eventually, down the >> road, we could drop 10.6 but we still have a significant amount of our users >> there; even though Mac stopped serving them major updates since July 2011 >> [1]. >> >> Our current Mac OS X distribution looks like this: >> * 10.6 - 43% >> * 10.7 - 30% >> * 10.8 - 27% >> OS X 10.8 is the only version that is growing. >> >> In order to improve our wait times, I propose that we stop testing on tbpl >> per-checkin [2] on OS X 10.7 and re-purpose the 10.7 machines as 10.6 to >> increase our capacity. >> >> Please let us know if this plan is unacceptable and needs further discussion. >> >> best regards, >> Armen Zambrano - Mozilla's Release Engineering >> _______________________________________________ >> dev-platform mailing list >> dev-platform@lists.mozilla.org >> https://lists.mozilla.org/listinfo/dev-platform > > _______________________________________________ > dev-platform mailing list > dev-platform@lists.mozilla.org > https://lists.mozilla.org/listinfo/dev-platform _______________________________________________ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
