On 4/3/13 3:36 PM, L. David Baron wrote:
On Wednesday 2013-04-03 17:31 -0400, Kartikaya Gupta wrote:
1. Take the latest green m-c change, commit your patch(es) on top of
it, and push it to try.
2. If your try push is green, flag it for eventual merge to m-c and
you're done.
3. If your try push is not green, update your patch(es) and go back
to step 1.
This seems like it would lead to a substantial increase in
build/test load -- one that I suspect we don't currently have the
hardware to support. This is because it would require a full
build/test run for every push, which we avoid today because many
builds and tests get merged on inbound when things are behind. (I
also don't feel like we need a full build/test run for every push,
so it feels like unnecessary use of resources to me.)
Let me ask a related question: why does my push to change a .js file in
/services incur 40+ hours of machine time to run the full reftest suite
- something which it is practically guaranteed to not interfere with?
IMO we should have a mechanism in place that audits changesets for
likely impact, chooses a reasonable set of builds/tests to perform, and
runs with it. We can supplement this with "no changeset lands in m-c
until all builds/tests have ran on it" (i.e. a later push on that tree
or a manual or timer triggered full gamut run). This general idea is
being considered in bug 849385.
Sure, regressions will fall through the cracks. But, the cost to not
doing it this way is continued and perpetual run-ins with automation
capacity.
At the same time, we need to make our automation infrastructure more
efficient. http://brasstacks.mozilla.com/gofaster/#/overhead/build says
our win32 builders spend 1 hour (out of their 3 hour total wall time) in
"setup and teardown." 33% lost to overhead on every push. Sadness.
Failing to use all available cores for xpcshell tests, reftests, etc
wastes even more CPU cycles.
Our current approach is not sustainable (unless we are content with
throwing a lot of money and resources at the problem).
_______________________________________________
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform
