Hi Ojan. I wonder if it would help to distinguish --exit-after-n-failures from --exit-after-n-crashes.
I think that crashing tests are the biggest problem, since they can cause a bot to lag behind quite a bit. Geoff On Jun 16, 2010, at 1:57 PM, Ojan Vafai wrote: > Currently, --exit-after-n-failures on the bots is set to 20. I like the idea > of exiting early, but I think 20 is too low. We should up it to 100. Is > anyone opposed to that? > > There are some straightforward, mechanical patches that cause more than 20 > tests to fail where they just need new expected results (e.g. changing form > control or scrollbar metrics). Right now, to make such a change you need > access to every platform so you can create new results or you need to get > someone who has access to that platform to pull in your change and create new > results. > > The problem that confounds this is that many people have trouble ever getting > all the tests to pass on Windows. I've never succeeded. There are always ~50 > tests that fail for me and it's not due to lack of trying. So, even though I > have access to Windows, it's hard for me to get new expected results for > Windows changes. > > Long-term we really need a solution that lets you get expected results for a > platform you don't have access to without committing code, e.g., the EWS > archiving results for failing tests. > > Ojan > _______________________________________________ > webkit-dev mailing list > [email protected] > http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev _______________________________________________ webkit-dev mailing list [email protected] http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
