We now have basic B2G performance tests reporting to
https://datazilla.mozilla.org/b2
There are a few things worth noting:
1. The version number is taken from the device (these are running
against nightly unagi engineering builds with latest master Gaia flashed
on top). Until recently this was 1.0.0-prerelease but now appears to be
1.0.0.0-prerelease and is the reason you will see two charts.
2. The revision identifier for these tests in DataZilla is the git
commit hash for Gaia, however DataZilla is currently only set up to work
with Mercurial, so any links to the commit are broken. One of the next
steps is to switch this to use github and leverage the API to detect and
report performance regressions.
3. Clicking the summary chart at the top of the results page will
display a larger version of the chart below. In this chart, each point
represents a test run, and clicking data points will display detailed
results for the test run in an additional chart below. If you then click
a data point in this chart, a final chart showing the data submitted for
that application/metric will be displayed.
It's also worth a brief explanation of what these tests do. For now they
simply open the Phone, Messages, and Settings apps and take a
measurement of the time before the first paint event is fired. Each app
is launched 30 times, and these tests are triggered when a new nightly
build is available, or when Gaia master branch is updated.
I encourage and look forward to your questions, feedback, and
suggestions. :)
For those interested, the source code for these tests can be found here:
http://hg.mozilla.org/users/tmielczarek_mozilla.com/b2gperf
Cheers,
Dave Hunt
Automation Development
_______________________________________________
dev-b2g mailing list
[email protected]
https://lists.mozilla.org/listinfo/dev-b2g
