Regarding TBPL tests, the removal of nightly will be problematic for sheriffing.
Currently, we run Gaia UI tests per gecko commit, not per gaia commit. (We have plans to do the latter, but it's a little complicated getting it hooked up.) On each gecko commit, we run the tests twice, once against gaia's master branch and once against gaia's nightly branch. If a test breaks against both, the sheriffs can assume it is a gecko problem. If a test breaks only against gaia's master branch, but not against gaia's nightly branch, then sheriffs can assume it's a gaia problem. Without gaia's nightly branch, we won't be able to make that distinction; sheriffs will have no way to know if a bustage is caused by a gecko or gaia commit, which will effectively make them unsheriffable, at least until we get tests running per-commit to gaia. Can we postpone this change until buildbot is capable of running Gaia tests per-Gaia-commit? Jonathan _______________________________________________ dev-b2g mailing list [email protected] https://lists.mozilla.org/listinfo/dev-b2g
