Re: [webkit-dev] Putting layout tests on a separate repository (was Supporting w3c ref tests and changing our convention)
On Nov 4, 2011 4:59 PM, Dirk Pranke dpra...@chromium.org wrote: On Fri, Nov 4, 2011 at 3:21 PM, Alan Stearns stea...@adobe.com wrote: Once we figure out how to support imported reftests, we should be encouraging people to use reftests internally (even for tests we have no intention of pushing to the W3C) instead of dumprendertree or pixel tests (where possible - I assume reftests won't always work for visual tests). Actually, we should be encouraging people to use reftests now, since every port but two supports them, and we should be moving the last two over ASAP. I'd argue that we should not encourage people from writing reftests until we migrate those two ports to NRWT and https://bugs.webkit.org/show_bug.cgi?id=60605 is fixed given all but Chromium port's bots run pixel tests. In fact, I'd strongly discourage people from writing reftests for features that are enabled on all ports until the above two conditions are met. - Ryosuke ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Re: [webkit-dev] Putting layout tests on a separate repository (was Supporting w3c ref tests and changing our convention)
On Nov 7, 2011 8:00 PM, Ryosuke Niwa rn...@webkit.org wrote: I'd argue that we should not encourage people from writing reftests until we migrate those two ports to NRWT and https://bugs.webkit.org/show_bug.cgi?id=60605 is fixed given all but Chromium port's bots run pixel tests. I meant to say only Chromium port's bots run pixel tests. ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Re: [webkit-dev] Putting layout tests on a separate repository (was Supporting w3c ref tests and changing our convention)
On Mon, Nov 7, 2011 at 8:00 PM, Ryosuke Niwa rn...@webkit.org wrote: On Nov 4, 2011 4:59 PM, Dirk Pranke dpra...@chromium.org wrote: Actually, we should be encouraging people to use reftests now, since every port but two supports them, and we should be moving the last two over ASAP. I'd argue that we should not encourage people from writing reftests until we migrate those two ports to NRWT... It sounds like those last two port don't run pixel tests, so there wouldn't be a reduction in coverage by using reftests even before they are migrated, right? dave ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Re: [webkit-dev] Putting layout tests on a separate repository (was Supporting w3c ref tests and changing our convention)
On Mon, Nov 7, 2011 at 8:47 PM, David Levin le...@chromium.org wrote: On Mon, Nov 7, 2011 at 8:00 PM, Ryosuke Niwa rn...@webkit.org wrote: On Nov 4, 2011 4:59 PM, Dirk Pranke dpra...@chromium.org wrote: Actually, we should be encouraging people to use reftests now, since every port but two supports them, and we should be moving the last two over ASAP. I'd argue that we should not encourage people from writing reftests until we migrate those two ports to NRWT... It sounds like those last two port don't run pixel tests, so there wouldn't be a reduction in coverage by using reftests even before they are migrated, right? They don't generate or compare ping images but they DO generate and compare render tree dump tests for non tests so converting the existing tests to reftests will reduce their test coverage. - Ryosuke ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
[webkit-dev] Putting layout tests on a separate repository (was Supporting w3c ref tests and changing our convention)
On Fri, Nov 4, 2011 at 2:39 PM, Ryan Leavengood leaveng...@gmail.comwrote: I would also like to throw out the idea of pulling the layout tests out into their own repo, maybe even per platform. How do we keep webkit/layout repositories in sync? Currently the huge number of layout tests in WebKit make many git operations unbearably slow, such as git status, which basically does a stat() on every file in a repo when the plain git status is used. Even on Linux where stat() is mega-fast it takes many seconds to show git status even on a clean checkout (at least on my laptop, which admittedly isn't the fastest.) It is worse on other platforms with a slower stat(). Even if we put in a separate repo, you'll likely need to checkout all of them anyway because if your patch ever require baselines in some port that you don't work on, you'll still need to rebaseline them. But I expect there might be a lot of pushback on such an idea, especially just to make one tool run faster. Indeed, I'm against this idea. - Ryosuke ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Re: [webkit-dev] Putting layout tests on a separate repository (was Supporting w3c ref tests and changing our convention)
On Fri, Nov 4, 2011 at 5:58 PM, Ryosuke Niwa rn...@webkit.org wrote: Indeed, I'm against this idea. You make good points. There may be solutions to your objections, but they might not be much better than having a slow git status right now. Do other people who use WebKit with Git see this problem? If so, what do you do to stay productive? If not then I'll just be sure to only develop WebKit on a faster machine (which I do have and look forward to testing soon.) Sorry for hijacking the other thread, thanks for changing the subject. -- Regards, Ryan ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Re: [webkit-dev] Putting layout tests on a separate repository (was Supporting w3c ref tests and changing our convention)
On Fri, Nov 4, 2011 at 3:09 PM, Ryan Leavengood leaveng...@gmail.comwrote: You make good points. There may be solutions to your objections, but they might not be much better than having a slow git status right now. Do other people who use WebKit with Git see this problem? If so, what do you do to stay productive? If not then I'll just be sure to only develop WebKit on a faster machine (which I do have and look forward to testing soon.) I use even slower svn and never had an issue with it except when I revert/merge commits. - Ryosuke ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Re: [webkit-dev] Putting layout tests on a separate repository (was Supporting w3c ref tests and changing our convention)
On 11/4/11 3:09 PM, Ryan Leavengood leaveng...@gmail.com wrote: On Fri, Nov 4, 2011 at 5:58 PM, Ryosuke Niwa rn...@webkit.org wrote: Indeed, I'm against this idea. You make good points. There may be solutions to your objections, but they might not be much better than having a slow git status right now. Do other people who use WebKit with Git see this problem? If so, what do you do to stay productive? If not then I'll just be sure to only develop WebKit on a faster machine (which I do have and look forward to testing soon.) Note that supporting reftests has the possibility of slowing LayoutTests folder growth for WebKit-specific tests. If you have the choice of writing a reftest with a single html reference as the expected result, you should not get a proliferation of dumprendertree or png baselines in the platform folder. Once we figure out how to support imported reftests, we should be encouraging people to use reftests internally (even for tests we have no intention of pushing to the W3C) instead of dumprendertree or pixel tests (where possible - I assume reftests won't always work for visual tests). Alan ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Re: [webkit-dev] Putting layout tests on a separate repository (was Supporting w3c ref tests and changing our convention)
On Fri, Nov 4, 2011 at 3:21 PM, Alan Stearns stea...@adobe.com wrote: Once we figure out how to support imported reftests, we should be encouraging people to use reftests internally (even for tests we have no intention of pushing to the W3C) instead of dumprendertree or pixel tests (where possible - I assume reftests won't always work for visual tests). Actually, we should be encouraging people to use reftests now, since every port but two supports them, and we should be moving the last two over ASAP. -- Dirk ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
