I wanted to give an update on our work for reducing intermittents. Last month when we posted there was a lot of discussion and concerns around disabling tests, likewise some of the terminology and process.
I have outlined many pieces of data about the progress and rates of bugs and disabling tests in my blog post earlier today: https://elvis314.wordpress.com/2017/04/11/project-stockwell-reduce-intermittents-april-2017/ Given the push for disabling more tests, we ended up disabling 30 tests, but fixing >70- so a much higher percentage of tests are being fixed than previous months! Overall, the Orange Factor is hanging out around 10.0, but we are adding a lot of new tests and test jobs- so this is great news. While we would have like to end Q1 with something closer to 8.0, we remain optimistic that we can still make a great difference going forward and continue to drive this down. Our focus will be on proving our process, and working towards shifting this to be easier to self manage for dev teams and other sheriffs. Thanks for fixing so many intermittent failures! _______________________________________________ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
