I think --exit-after-N-failures is actually very useful as-is. I think peopel just use it for different things. For teh commit-queue --exit-after-N-failures is great for keeping it quick.
Perhaps people want to use the bots more like try-bots and have --exit-after-N-failures higher. I think these are all symptoms of WebKit's total lack of real trybots. ;) -eric On Wed, Jun 16, 2010 at 3:27 PM, Maciej Stachowiak <[email protected]> wrote: > > On Jun 16, 2010, at 2:04 PM, Eric Seidel wrote: > >> We could add a separate option to DumpRenderTree to disable >> ReportCrash (sign up for all the crashing signals and simply exit(2) >> or similar). That would be useful in many instances besides the bots. >> >> Yes, --exit-after-N-failures was designed to prevent crashers from >> eating the bots. > > I think --exit-after-n-crashes is probably the most expedient solution to the > problem. > > I think when only a few tests crash, having the crash report is very useful, > particularly if the developer of the patch cannot reproduce the crash off the > bot. All we need to do is limit the number of crashes to prevent the bots > falling way behind. > > Regards, > Maciej > >> >> On Wed, Jun 16, 2010 at 1:59 PM, Geoffrey Garen <[email protected]> wrote: >>> Hi Ojan. >>> >>> I wonder if it would help to distinguish --exit-after-n-failures from >>> --exit-after-n-crashes. >>> >>> I think that crashing tests are the biggest problem, since they can cause a >>> bot to lag behind quite a bit. >>> >>> Geoff >>> >>> On Jun 16, 2010, at 1:57 PM, Ojan Vafai wrote: >>> >>>> Currently, --exit-after-n-failures on the bots is set to 20. I like the >>>> idea of exiting early, but I think 20 is too low. We should up it to 100. >>>> Is anyone opposed to that? >>>> >>>> There are some straightforward, mechanical patches that cause more than 20 >>>> tests to fail where they just need new expected results (e.g. changing >>>> form control or scrollbar metrics). Right now, to make such a change you >>>> need access to every platform so you can create new results or you need to >>>> get someone who has access to that platform to pull in your change and >>>> create new results. >>>> >>>> The problem that confounds this is that many people have trouble ever >>>> getting all the tests to pass on Windows. I've never succeeded. There are >>>> always ~50 tests that fail for me and it's not due to lack of trying. So, >>>> even though I have access to Windows, it's hard for me to get new expected >>>> results for Windows changes. >>>> >>>> Long-term we really need a solution that lets you get expected results for >>>> a platform you don't have access to without committing code, e.g., the EWS >>>> archiving results for failing tests. >>>> >>>> Ojan >>>> _______________________________________________ >>>> webkit-dev mailing list >>>> [email protected] >>>> http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev >>> >>> _______________________________________________ >>> webkit-dev mailing list >>> [email protected] >>> http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev >>> >> _______________________________________________ >> webkit-dev mailing list >> [email protected] >> http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev > > _______________________________________________ webkit-dev mailing list [email protected] http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
