On Wed, Feb 10, 2016, at 03:04 PM, Robert O'Callahan wrote: > Background: > http://robert.ocallahan.org/2016/02/introducing-rr-chaos-mode.html > > I just landed on rr master support for a "-h" option which enables a > chaos > mode for rr recording. This is designed to help reproduce intermittent > test > failures under rr. We already have a few reports of people using this > successfully to find difficult bugs. Even though rr works only on desktop > Linux (including VMs), I've reproduced a bug that only showed up in > automation on Android, and khuey reproduced a bug that only showed up on > OSX 10.6. > > I'm continuing to do experiments to try to reproduce more of our top > intermittents, but you may already find rr chaos mode useful. I recommend > running a single test or a small group of tests continuously; one of my > bugs only had a few failing runs out of a thousand. I'm sure there are > still bugs rr can't reproduce, and I'm very interested in hearing about > bugs that eventually get fixed but that rr was not able to reproduce. By > studying such bugs we can improve rr chaos mode so it can find them. > > Obviously, once rr chaos mode has proved itself, we should get some > automation around it. I'd like a bit more experience with it before we > have > that discussion.
This is great! I've kept holding out hope that rr can help us fix intermittent test failures, but so far we've failed to actually prove this out. BenWa tried doing some work on this but kept getting hung up on hitting test failures unrelated to the ones we see in production, possibly due to environment issues. jmaher and armenzg and others have been doing some great work lately standing up Linux tests in Taskcluster, as a side effect of which means we now have a Docker image for running Linux tests. If anyone wants to prototype reproducing failures from CI running rr inside that image would be a good place to start. -Ted _______________________________________________ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
