On 03/01/2019 16:17, jmaher wrote:
I would like to propose that we do not run tests on linux64-opt, windows7-opt,
and windows10-opt.
Why am I proposing this:
1) All test regressions that were found on trunk are mostly on debug, and in
fewer cases on PGO. There are no unique regressions found in the last 6 months
(all the data I looked at) that are exclusive to OPT builds.
2) On mozilla-beta, mozilla-release, and ESR, we only build/test PGO builds, we
do not run tests on plan OPT builds
3) This will reduce the jobs (about 16%) we run which in turn reduces, cpu
time, money spent, turnaround time, intermittents, complexity of the taskgraph.
4) PGO builds are very similar to OPT builds, but we add flags to generate
profile data and small adjustments to build scripts behind MOZ_PGO flag
in-tree, then we launch the browser, collect data, and repack our binaries for
faster performance.
5) We ship PGO builds, not OPT builds
What are the risks associated with this?
1) try server build times will increase as we will be testing on PGO instead of
OPT
2) we could miss a regression that only shows up on OPT, but if we only ship
PGO and once we leave central we do not build OPT, this is a very low risk.
It's not just tryserver build times. Presumably this will also tend to
increase the time between a patch landing on inbound or autoland and any
resulting test failures showing up.
This seems like a negative in that it means more patches are likely to
have landed on top of the regressing one in the meantime, potentially
complicating backouts, and the original developer may be less likely to
still be around for a quick investigation/fix.
How long does it typically take for full PGO test results to be
available for a push? How does that compare to full Opt test results?
ISTM that if the increase is quite marginal, this may be OK, but if the
latency becomes substantially greater, there will be a continual cost in
increased developer and/or sheriff pain.
JK
_______________________________________________
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform
