That is great! There is a script travis.sh[1] in GAIA_DIR/tools/ci/unit which is used for running gaia unit tests on travis-ci.org [2] (we still have some problems which I'm fixing... like 864795 & 863691). I think you can also use this script for TBPL.
[1] https://github.com/mozilla-b2g/gaia/blob/master/tools/ci/unit/travis.sh [2] https://travis-ci.org/mozilla-b2g/gaia/builds 2013/4/30 Jonathan Griffin <[email protected]> > TL;DR - The a-team and rel-eng are currently preparing to get gaia unit > tests running in TBPL (https://bugzilla.mozilla.org/** > show_bug.cgi?id=833666<https://bugzilla.mozilla.org/show_bug.cgi?id=833666>), > on both trunk gecko branches and gaia branches (e.g., > https://tbpl.mozilla.org/?**tree=Gaia-Master<https://tbpl.mozilla.org/?tree=Gaia-Master>). > We have a plan to integrate these tests into the "main" TBPL pages for > mozilla-inbound, etc, and to provide try access to gaia developers. We'd > like feedback to ensure that this plan will satisfy developers before we > begin implementation. > > == What is already in progress == > All the harness work has been done to get gaia unit tests into > buildbot/TBPL > (https://bugzilla.mozilla.org/**show_bug.cgi?id=833666<https://bugzilla.mozilla.org/show_bug.cgi?id=833666>), > running on b2g desktop builds. There is some additional buildbot work that > is being done to enable these to be turned on for the "cedar" project > branch > (https://bugzilla.mozilla.org/**show_bug.cgi?id=865379<https://bugzilla.mozilla.org/show_bug.cgi?id=865379>). > Once we shake out any remaining bugs in production there, we will turn > these tests on for both gecko trunk branches, like mozilla-inbound, and > gaia branches, like gaia-master. > > We are additionally endeavoring to get gaia ui tests (in Python) stable on > b2g desktop builds > (https://bugzilla.mozilla.org/**show_bug.cgi?id=857622<https://bugzilla.mozilla.org/show_bug.cgi?id=857622>). > We have been running them on pandas for some time in buildbot, but the > panda build is not well-maintained enough for large-scale automation, and > it is hard for developers to debug panda failures, since most of them do > not have panda boards, and working with panda boards is not as easy as > working with other b2g builds. Is there any objection to turning these > panda tests off? > > WebQA is also currently running gaia ui tests on unagi's in a Jenkins > cluster. We can't run these in buildbot, nor have them report to TBPL in > the near term, but we do have a longer-term plan to allow them to report to > TBPL > (https://wiki.mozilla.org/**Auto-tools/Projects/**TreeHerder_2013Q2<https://wiki.mozilla.org/Auto-tools/Projects/TreeHerder_2013Q2> > ). > > After work for gaia unit and ui tests is completed, we will turn our > attention to the gaia integration tests (in JS) ( > https://bugzilla.mozilla.org/**show_bug.cgi?id=866909<https://bugzilla.mozilla.org/show_bug.cgi?id=866909>), > but the gaia developers will first have to get these working reliably. > > == What we are planning to do == > The current state of gaia tests in TBPL is not ideal. Among other things, > sheriffs can't always easily identify gaia regressions, and gaia developers > have no access to try. Having gaia results on separate pages from gecko > activity makes them harder (or sometimes impossible) to sheriff > effectively, and reduces their visibility and relevance. There have been > multiple plans floated to deal with this, but at the recent b2g work week, > a few of the B2G leads proposed an alternative. > > There are two primary pieces to this proposal: > > 1) - couple gecko and gaia commits in buildbot so that there is no > uncertainty about which commits are used to create a build, and so that > gaia test results appear on "main" TBPL pages like mozilla-inbound and > mozilla-central > 2) - create a gaia-try repo that would allow gaia developers to push > patches to try > > This proposal initially will only target b2g desktop builds running > against mozilla-central and gaia's master branch. > > == Coupling gecko and gaia commits == > This would be accomplished by adding a file to mozilla-central that > contains a pointer to a gaia commit. Whenever buildbot produces a b2g > desktop build, it would use the corresponding gaia commit in the build. > > When a gaia developer commits a patch to gaia's master branch, a > corresponding commit would be automatically submitted to gecko > (mozilla-inbound, or possibly birch) that would change gecko's gaia > pointer. This auto-commit would automatically trigger a full set of test > runs, as happens with any other gecko commit. > > The gecko auto-commit would use the same commit message as the original > gaia commit. For example, if the original gaia commit was "Bug 865091 - > Enable systemXhr so youtube bypasses CORS. r=daleharvey", the gecko > auto-commit might be something like "Bug 865091 - Enable systemXhr so > youtube bypasses CORS. r=daleharvey, [gaia auto-commit]". > > If an auto-commit caused a test to go orange, it would be easily > identifiable by the sheriffs, and they would back out the original gaia > commit. > > == Gaia and tryserver == > We will provide a gaia-try repo, and gaia developers may submit a patch to > this if they would like to kick off a tryserver run. > > We realize that gaia developers would prefer to commit to a github repo > for this purpose, but that would require significant changes to our > build/test infrastructure such that it isn't feasible this quarter. What > we propose as a first iteration for try support is as follows: > > * provide a gaia-try repo in hg > * when a gaia developer wishes to push to try, he would commit his gaia > patch to this hg repo (we would provide very easy-to-follow instructions), > and then commit another patch to the "regular" hg gecko try repo ( > http://hg.mozilla.org/try/) that updates the gaia pointer; this gecko > commit would use normal trychooser syntax (http://trychooser.pub.build.** > mozilla.org/ <http://trychooser.pub.build.mozilla.org/>). > > In subsequent iterations of gaia-try (to be done post Q2), we could > improve the process by moving to a git-based gaia-try repo, and providing a > mechanism that would allow gaia developers to omit the gecko try push if > they were only interested in testing gaia changes using mozilla-central's > head revision. Developers that have coupled gecko/gaia patches would still > need to perform two commits, one to gaia-try and one to gecko-try. > > We believe this plan (sans future gaia-try improvements) is achievable in > a reasonable amount of time. Does this plan meet the needs of b2g > developers, and does it provide enough benefit beyond our current ability > to display separate pages for gecko and gaia in TBPL that it merits the > time it will take to implement? > > We'd like to arrive at a decision by the end of the week, so that we may > begin implementation if there is general agreement. > > Jonathan, on behalf of rel-eng and the a-team > > > > > > > ______________________________**_________________ > dev-gaia mailing list > [email protected] > https://lists.mozilla.org/**listinfo/dev-gaia<https://lists.mozilla.org/listinfo/dev-gaia> > _______________________________________________ dev-b2g mailing list [email protected] https://lists.mozilla.org/listinfo/dev-b2g
