Gregory Szorc wrote on 07/15/2014 09:04 PM: > On 7/15/14, 11:49 AM, Dave Townsend wrote: >> Since forever Jetpack tests in the Firefox trees have been run using our >> custom python CFX tool which is based on a fork of an ancient version of >> mozrunner. This causes us a number of problems. Keeping up with tree >> visibility rules is hard. Some features from newer versions of mozrunner >> like crash stack handling aren't available and our attempts to update to >> the newer mozbase have been blocked on trying to get some of our forked >> code accepted. It also makes it hard for Mozilla other developers to run >> our tests as CFX has a very different syntax to the other test suites. >> >> We've started investigating switching away from CFX and instead using the >> python automation that the mochitests use. This would work somewhat >> similarly to browser-chrome tests, runtests.py will startup Firefox and >> overlay some XUL and JS on the main window from where we can run the >> existing JS parts of the Jetpack test suites. >> >> There are many benefits here. The runtests.py code is well used and known >> to be resilient. It supports things like screenshots on failures and crash >> stacks that Jetpack tests don't currently handle. We'll use manifest files >> like the other test suites so disabling tests per platform will be easy. >> Excellent mach integration will make running individual tests simple. It >> also makes it possible to use commonjs style tests elsewhere in the tree. >> Release engineering should find managing the Jetpack tests a lot easier as >> they behave just like other mochitests. >> >> My initial experiment last week shows that this will work. The first part >> of our tests (package tests) is running and passing on my local machine and >> I expect to have the add-on tests working this week. >> >> I wanted to give everyone a heads up about this work to give you all a >> chance to ask questions or raise objections. The changes to runtests and >> the build system are minimal, just adding support for new manifest types >> really but I will be needing reviews for those. We'll also have to make the >> buildbot changes to switch over to use these new tests but I expect that to >> be pretty straightforward. > > Was Marionette considered? From what little I know (jgriffin and others > can correct me), Marionette seems like the logical base for this test suite.
Adding the tools mailing list, so that members of the A-team are aware of this thread, and can answer appropriately. -- Henrik Skupin Senior Test Engineer Mozilla Corporation _______________________________________________ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
