Henrik Skupin wrote on 01/09/16 17:37: After a longer time without a response I would like to give a follow-up on where we stand at the moment and what the future will be for those tests.
But first, I want to say thanks to everyone who participated in this thread. The received feedback was very helpful and showed that a couple of topics are questionable and needed further discussion. With all that in mind we decided to get away with the original idea of moving the firefox-ui tests to the appropriate components in the state they are in right now. This will ensure that: * no other subdirectory for a new kind of test has to be added * don't make it harder for developers to decide which harness to use Instead our team decided to work towards the goal in transforming the firefox-ui-functional tests into plain Marionette tests. The work as such is not that trivial given that a couple of things have to be obeyed, and not everything is clear yet - especially not how we want to continue in testing nightly and release builds via mozmill-ci. But that shouldn't stop us to make writing Marionette tests for Firefox easier. Just to give an overview, here are the sub-projects I have to work on: * Decouple the Firefox Puppeteer package from FirefoxTestCase and make it an optional (mixin) class, which then could even be used with MarionetteTestCase if wanted. * Get rid of large portions of the firefox-ui harness except for update tests which would still require a separate harness due to their own command line arguments. * Refactor Marionette tests in domain specific jobs. As you know we run all the existing tests (unit, browser, layout, toolkit) in a single job and report as `Mn` to Treeherder. This should be split up into chunks. There will be a follow-up email on this topic soon. * Ensure that those jobs which report as Tier-1 will not use remote testcase data, and consider a new Tier-2 group for others (necessary for lot of the fx-ui security tests) * Refactor existing firefox-ui-functional tests into plain Marionette tests and get browser peer review before moving them to the appropriate tests folders. * Keep automation (mozharness, mozmill-ci) intact in parallel so we do not loose test coverage as needed by QE. I'm sure that the above outlined goals will be in a better alignment with you all. If not, or if there are questions, please let me know. Thanks, -- Henrik Skupin Senior Software Engineer Mozilla Corporation _______________________________________________ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
