Do we have a policy on perf/memory regressions? I've been told that there's a "zero-regression" policy for page load tests. Is that actually the case? What about cases that are clear perf regressions that don't show up in PLT (e.g. they show up in one of the Chromium page load tests and/or on a real-world web page)?
Perf and memory regressions seem much worse to me than test failures. Unlike test failures, there's isn't a binary right or wrong. If a patch regresses performance and then other patches come in that further improve/regress performance, it becomes nearly impossible to tell if the fix for the original patch actually addresses the entirety of the original regression. The only case I can think of where perf regressions might be ok are: 1. The fix will be checked in reasonably soon (hours, not days/weeks). This matches our policy with failing tests. 2. We think there's no way to address the perf/memory regression and that the new functionality justifies it. This is extremely rare. I'm asking because I've had considerable pushback dealing with a recent perf/memory regression on OS X that's been sitting in the tree for 3 weeks. I'm not linking to the bug in question because that issue is resolved. This is forward looking. I'd like to see us have a written down policy. Ideally, one day, we'll also have bots that run perf/memory tests and turn red when there is a regression. Ojan
_______________________________________________ webkit-dev mailing list [email protected] http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
