Leandro's recent message reminded me that there was some discussion on this list about the burden caused by storing and maintaining pixel baselines for an ever-growing list of configurations. I've been working on a proposal for moving pixel baselines to a "branch" so that they don't bother most folks.
Unfortunately, I haven't been able to work out all the details yet. Here's a somewhat rough, work-in-progress proposal. If you like this approach, I can spend more time trying to solve the remaining problems. == Motivation == Maintaining pixel test results in svn.webkit.org for the various Chromium configuration imposes a number of costs on the WebKit community: All members of the community need to store these results locally when they create a checkout of WebKit. Updating the pixel results creates churn in the project that increases the time required to update a working copy and makes it more difficult to understand what is actually changing in the project. As the Chromium port grows in complexity, there is pressure from the Chromium project to expand the number of test configurations, increasing these costs on the WebKit community. == Proposal (Work-in-progress) == One option is to stop running pixel test altogether, but that decreases test coverage and costs correctness. Another possibility is to store the pixel results in a location where they are largely invisible to the rest of the WebKit community. For example, we could store the pixel results for chromium-linux the following location: http://svn.webkit.org/repository/webkit/PixelResults/chromium-linux Chromium contributors could then use gclient and DEPS to map these results into their working copies and non-chromium contributors could safely ignore any changes in the PixelResults “branch”. FIXME: What about non-platform specific results? == Problems == The main reason this proposal is half-baked is I haven't quite been able to work out how folks can do some common operations: 1) Land patch with expected image failures (+ revert) 2) Land patch with new image baselines (+ revert) 3) Land new baselines Because the pixel results and trunk are in the same SVN repo, we should be able to do these things atomically, but dealing with the sparse checkout is kind of tricky. Another big question mark is how this would work for contributors who use git. When git mirrors an SVN repro, it only mirrors "trunk", so the PixelResults directory wouldn't exist in the git version of the repository. Thoughts? Adam _______________________________________________ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
