David, thank you for this information. This is extremely helpful for debugging these test. I also found that you can make the harness run only one test on taskcluster by adding the TEST_FILES environment variable here [1]. You can see the full run here [2].
Also, Aleh, thank you for this information! _slow_ VM sounds like an excellent way to reproduce those pesky failures. Cheers! 1.) https://github.com/mozilla-b2g/gaia/commit/3e02b10b905c 2.) https://treeherder.mozilla.org/#/jobs?repo=gaia&revision=d7aa5f2bce9b On Wed, Nov 11, 2015 at 8:34 PM, David Flanagan <[email protected]> wrote: > Michael and others are making good progress fixing intermittent failures > (see his "Orange is the new bad" thread.) He's doing this with a patch in > bug 1222215 that turns off automatic retries for failing tests. Hopefully > he will be able to actually land that soon. > > But until that is landed, if you are working on a test, it is worth trying > out your test with that patch applied so you can see if it is failing > intermittently. You can probably just cherry-pick this commit: > https://github.com/nullaus/gaia/commit/735dd67cdb565f152e86f43506836a5f6e134b71 > > The thing about intermittent test failures is that they seem to show up on > treeherder but not when you run the tests locally. So the process of fixing > them can require lots of pushes to github to trigger new treeherder test > runs. And since test runs take a long time, it can be hard to iterate > quickly on these bugs. I worked yesterday on one of these bugs and realized > that I could make the tests runs go faster by turning off the tests I > didn't care about. > > If you edit tests/taskcluster/tasks.yml you can comment out (with #) > almost all of the lines in that file, leaving only > > marionette_js_tests: > chunks: 2 > > Note that I changed from 40 chunks to 2 chunks. This is because the next > useful change is to tests/taskcluster/tasks/marionette_js_tests.yml > > where you can add: > > APP: gallery # or whatever app you're working on tests for > > at line 23, as part of the `env:` section. > > When you push those changes to your PR, the test run that is triggered > will run the usual "Gaia decision opt" and "Bootstrap" tests, then it will > only the Gij integration tests, in two chunks, for your one app. It still > isn't fast, but it should be a lot faster than running a full set of tests. > > David > > _______________________________________________ > dev-fxos mailing list > [email protected] > https://lists.mozilla.org/listinfo/dev-fxos > >
_______________________________________________ dev-fxos mailing list [email protected] https://lists.mozilla.org/listinfo/dev-fxos
