Background: Mochitests are currently run against emulator and they are not actually run against specific device targets. The effort I am engaging in is to find mechanisms for doing this in a reliable manner. Along the way I thought it might be pertinent to provide some data on where I am seeing crashers, and see if the data can be actionable by the various dev teams.
Information on how to run mochitests on device is provided at the end of the e-mail. How each test was run: The mochitests that were run were trunk based, and the variation was only in the build that was running on the target Buri device. The buri device was either running the 1.3 branch or the trunk on BuildID: 20140226040201. Currently the run results are fairly reliable on the device, but the harness needs more work to make the beginning of the run be more reliable. Results (Test Pass / Failures): Buri Device Trunk: https://docs.google.com/spreadsheet/ccc?key=0Astsg__lnySwdFFWRXVuSUVvQzl6R3Jsd1A0R0ZYTnc&usp=sharing Buri Device 1.3: https://docs.google.com/spreadsheet/ccc?key=0Astsg__lnySwdDdzUkgyV0dHc1dUejVlUjZGcVJxdHc&usp=sharing The expectation is that there are several more failures on 1.3 vs Trunk. The second expectation would be that there are more crashers on 1.3 vs Trunk, primarily because we are running tests from the trunk, and that overtime with new interface / fixes the 1.3 branch would be a bit stale. The reason I ran things this way was to see the progression of failures -> passing over time. Crashers: content/media/webaudio content/base/test/ - a few crashers that need to be explored dom/workers/ - several errors / one known crasher test_errorPropagation.html layout/generic/test/test_plugin_clipping_transformed.xhtml -- crashers with all plugin tests dom/settings/tests/test_settings* - crashers, because the settings are required for wifi* dom/camera/test - crashes dom/tests/mochitest/general/test_vibrator.html - crashes tests that run after How to run: from B2G root cd objdir-gecko/_tests/testing/mochitest python runtestsb2g.py --b2gpath /home/onecyrenus/B2G/ --xre-path /home/onecyrenus/B2G/gecko/obj-x86_64-unknown-linux-gnu/dist/bin/ --console-level INFO --test-path dom/datastore/tests You can optionally provide the wifi information in the command line. python runtestsb2g.py --b2gpath /home/onecyrenus/B2G/ --xre-path /home/onecyrenus/B2G/gecko/obj-x86_64-unknown-linux-gnu/dist/bin/ --console-level INFO --test-path dom/datastore/tests --wifi "{\"ssid\": \"Mozilla Guest\"}" **The specified wifi network needs to be the same wifi network that your host computer is on. More information: https://developer.mozilla.org/en-US/Firefox_OS/Platform/Automated_testing/Mochitests There is a bunch of ongoing work to stabilize the runner for running mochitest on device. I'll ping the team later in the week with a status update on that. -David _______________________________________________ dev-b2g mailing list [email protected] https://lists.mozilla.org/listinfo/dev-b2g
