On Tuesday 2013-11-05 14:44 +0000, David Burns wrote: > We appear to be doing 1 backout for every 15 pushes on a rough > average[4]. This number I am sure you can all agree is far too high > especially if we think about the figures that John O'Duinn > suggests[5] for the cost of each push for running and testing. With > the offending patch + backout we are using 508 computing hours for > essentially doing no changes to the tree and then we do another 254 > computing hours for the fixed reland. Note the that the 508 hours > doesn't include retriggers done by the Sheriffs to see if it is > intermittent or not. > > This is a lot of wasted effort when we should be striving to get > patches to stick first time. Let's see if we can try make this > figure 1 in 30 patches getting backed out.
If your goal is saving compute time, I suspect a message like this could actually worsen our use of compute time, since the increase in try server compute time usage as a result of people being more careful may well be larger than the savings in mozilla-inbound compute time from fewer backouts, at least for many people. That said, I think there are other reasons to want to improve this number, because broken trees have other effects on developer productivity. However, with that there's also definitely a point where it's not worth an individual to spend more time for a small reduction in the probability of wasting others' time. I think it's worth tracking both the resource consumption rates and backout rates of individual committers, because they're substantially different, and if we want people working at the correct optimum we should be giving opposite advice to different committers. (I'm aware we track resource consumption on try at [1].) I think some committers are too careful and overconsume compute resources on try, and some are not careful enough and overconsume resources (computer and human) on inbound. So we should gather data so that we can give the correct advice to different committers rather than giving blanket advice that's going to be correct for some people and wrong for others. I also don't buy the argument raised elsewhere in this thread that testing on developer machines scales better than testing on consolidated hardware. Purchasing and issuing machines to developers, and then having those developers spend time setting up a development environment on those machines takes time, just as increasing our build and test infrastructure does. I think if it's cheaper than making an equivalent increase in our build infrastructure (of identically-configured machines) then it seems to me that there's something wrong with the way we build up that infrastructure. -David [1] https://secure.pub.build.mozilla.org/builddata/reports/reportor/daily/highscores/highscores.html -- 𝄞 L. David Baron http://dbaron.org/ 𝄂 𝄢 Mozilla https://www.mozilla.org/ 𝄂 Before I built a wall I'd ask to know What I was walling in or walling out, And to whom I was like to give offense. - Robert Frost, Mending Wall (1914)
signature.asc
Description: Digital signature
_______________________________________________ dev-platform mailing list [email protected] https://lists.mozilla.org/listinfo/dev-platform
