For years it has been a common pattern for local builds and Firefox
automation to copy test files to the object directory (or some other
staging area) and run them from there. This creates several inefficiencies:

* The build system must symlink or copy thousands of test related files
during the build
* Running tests requires checking if test files in the objdir are up to
date (can add significant latency to the edit-test loop)
* Builds in automation must create and upload archives containing thousands
of test files
  * This makes it harder to skip build jobs if all that has changed is a
test file
* Test jobs in automation must download and extract archives containing
test files
* Test jobs in automation typically don't have access to a full source
checkout and can't easily piggyback on existing infrastructure (such as
mach commands, vendored Python packages, etc). This leads to wheel
reinvention and inconsistency between local dev environments and automation.

The historical reasons for doing things this way were valid. But with, other improvements to the build system, automation scheduling
and configuration living in-tree, and better scaling of Mercurial in
automation, these historical reasons are largely no longer valid.

Going forward, I'm requesting we change our policy regarding tests and
automation to be source checkout first. This means:

* Tests should be designed to run from a source checkout with minimal
"build" actions required (no file copying/symlinking, no preprocessing into
a new file, etc)
* Automation should run everything from a source checkout (as opposed to
downloading archives containing files derived from source checkouts)

Does anyone have any concerns or objections to this?
dev-builds mailing list

Reply via email to