Gij on device is in a bit of a "weird" state for the moment. Hub did a bit of work at the end of 2013/beginning of 2014 to create faculties within the js framework for running tests on device. Later in 2014, the A-team came to us hoping to write a little service to expose the python runner's device capabilities to our js framework so that we could benefit from the work they were doing related to interfacing with devices and recovering crash data from gecko. We went for that and the python service they created mostly worked wrt running tests on b2g-desktop and device at the time, so we landed the thing.
Then a re-org happened and no one maintained the python service or ran jsmarionette tests on device for about a year. The python codebase for managing gecko processes has changed but the changes haven't been downstreamed to the python service that we're using in gaia. Aus and John Dorlus have been working on resurrecting the python service, but they're both frustrated by the complexity and architecture of the python codebase. I think that what our team needs is to throw away the unmaintained python goop and replace it with a web service for interfacing with devices. That API will expose methods to, among other things, 1. return metadata about device state (whether there is a device connected, whether b2g is running, device type and gonk & gecko versions) 2. restart the phone or b2g 3. push a parameter gecko profile to the device 4. open a socket connection to a device port and return metadata that will allow the service client to connect to it 5. stream device logs 6. recover a crash dump This is a non-trivial undertaking, but I can't imagine our ui testing efforts progressing without that investment. The list you made sounds reasonable to me, but I think interfacing with devices in a sane, tested, scalable way is the biggest piece of the puzzle IMHO. On Tue, Oct 6, 2015 at 8:54 AM, David Scravaglieri < [email protected]> wrote: > Hi, > > I put together all the conversation I had regarding quality improvements > and I'd like to know if it sounds good to you. > If so, we need an epm assigned and start executing > *.* > > *▾ Continuous Integration plan* > Goal : Every commit on Gecko and Gaia must trigger full test run, > failures and performance issues to be reported on Treeherder for backout > *▾ Automation* > • Create a test matrix to define which tests are > running on which platform (Device, Mulet, Emulator) > *▾ Engineering* > • Port Gip tests to Gij > *QA to provide the list of tests to port > and create the related bugs in bugzilla* > • Create new Gij tests from moztrap scenarios > *QA to select the moztrap scenario that > can be valuable to automate and technically feasible* > *▾ Device* > • Replace Jenkins module by Treeherder for > sending jobs to Bitbar > • Marionette.js to run on device > • Gecko Unit tests an Gij tests to run on devices > • Add Aries devices to Bitbar (30 units - 15 KK > and 15 L) > * • Mulet* > *Run Gaia Unit tests and Gij tests on Mulet* > *• Emulator-x86-KK* > *Activate emulator-x86-kk on Treeherder, run Gecko > unit tests and Gij tests* > *• Emulator-x86-L* > *Start porting emulator to L and activate it on > Treeherder* > *• Raptor* > *Integrate Raptor performance tests on Treeherder* > > Thanks, > --- > David Scravaglieri > Mozilla - Firefox OS > > _______________________________________________ > dev-fxos mailing list > [email protected] > https://lists.mozilla.org/listinfo/dev-fxos > >
_______________________________________________ dev-fxos mailing list [email protected] https://lists.mozilla.org/listinfo/dev-fxos
