Michael and others are making good progress fixing intermittent failures (see his "Orange is the new bad" thread.) He's doing this with a patch in bug 1222215 that turns off automatic retries for failing tests. Hopefully he will be able to actually land that soon.
But until that is landed, if you are working on a test, it is worth trying out your test with that patch applied so you can see if it is failing intermittently. You can probably just cherry-pick this commit: https://github.com/nullaus/gaia/commit/735dd67cdb565f152e86f43506836a5f6e134b71 The thing about intermittent test failures is that they seem to show up on treeherder but not when you run the tests locally. So the process of fixing them can require lots of pushes to github to trigger new treeherder test runs. And since test runs take a long time, it can be hard to iterate quickly on these bugs. I worked yesterday on one of these bugs and realized that I could make the tests runs go faster by turning off the tests I didn't care about. If you edit tests/taskcluster/tasks.yml you can comment out (with #) almost all of the lines in that file, leaving only marionette_js_tests: chunks: 2 Note that I changed from 40 chunks to 2 chunks. This is because the next useful change is to tests/taskcluster/tasks/marionette_js_tests.yml where you can add: APP: gallery # or whatever app you're working on tests for at line 23, as part of the `env:` section. When you push those changes to your PR, the test run that is triggered will run the usual "Gaia decision opt" and "Bootstrap" tests, then it will only the Gij integration tests, in two chunks, for your one app. It still isn't fast, but it should be a lot faster than running a full set of tests. David
_______________________________________________ dev-fxos mailing list [email protected] https://lists.mozilla.org/listinfo/dev-fxos
