Re: [webkit-dev] Defaulting --exit-after-n-failures to 0
On Fri, Jun 8, 2012 at 12:05 PM, Ojan Vafai o...@chromium.org wrote: We'll still default --exit-after-n-crashes-or-timeouts to 20. That seems more useful to me since you'd rarely hit this case locally and want to continue running tests. We should also increase this number. Non-chromium bots hit this limit all the time, and it's one of most frequent complaints about new-run-webkit-tests. - Ryosuke ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Re: [webkit-dev] Defaulting --exit-after-n-failures to 0
On Fri, Jun 8, 2012 at 12:08 PM, Ryosuke Niwa rn...@webkit.org wrote: On Fri, Jun 8, 2012 at 12:05 PM, Ojan Vafai o...@chromium.org wrote: We'll still default --exit-after-n-crashes-or-timeouts to 20. That seems more useful to me since you'd rarely hit this case locally and want to continue running tests. We should also increase this number. Non-chromium bots hit this limit all the time, and it's one of most frequent complaints about new-run-webkit-tests. I have no objection to upping this number. We picked 20 fairly arbitrarily. I think we picked it to match the number that the build.webkit.org bots were using at the time. That said, every non-chromium bot is passing in 20 via the commandline as best I can tell, so changing the default won't change anything there. Is it really true that non-chromium bots are hitting this frequently? Ojan ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Re: [webkit-dev] Defaulting --exit-after-n-failures to 0
On Fri, Jun 8, 2012 at 12:12 PM, Ojan Vafai o...@chromium.org wrote: On Fri, Jun 8, 2012 at 12:08 PM, Ryosuke Niwa rn...@webkit.org wrote: On Fri, Jun 8, 2012 at 12:05 PM, Ojan Vafai o...@chromium.org wrote: We'll still default --exit-after-n-crashes-or-timeouts to 20. That seems more useful to me since you'd rarely hit this case locally and want to continue running tests. We should also increase this number. Non-chromium bots hit this limit all the time, and it's one of most frequent complaints about new-run-webkit-tests. I have no objection to upping this number. We picked 20 fairly arbitrarily. I think we picked it to match the number that the build.webkit.org bots were using at the time. That said, every non-chromium bot is passing in 20 via the commandline as best I can tell, so changing the default won't change anything there. Is it really true that non-chromium bots are hitting this frequently? Yes, particularly in debug builds. Jessie can tell you all the glory story about how she had to disable lots of tests to make Windows bots work :( - Ryosuke ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Re: [webkit-dev] Defaulting --exit-after-n-failures to 0
I have no objection either to increasing the defaults for either of these numbers or making it possible to have different defaults per port. Do you want to suggest different defaults? Should we use ORWT's (infinite failures and infinite crashes by default)? -- Dirk On Fri, Jun 8, 2012 at 12:31 PM, Ryosuke Niwa rn...@webkit.org wrote: On Fri, Jun 8, 2012 at 12:12 PM, Ojan Vafai o...@chromium.org wrote: On Fri, Jun 8, 2012 at 12:08 PM, Ryosuke Niwa rn...@webkit.org wrote: On Fri, Jun 8, 2012 at 12:05 PM, Ojan Vafai o...@chromium.org wrote: We'll still default --exit-after-n-crashes-or-timeouts to 20. That seems more useful to me since you'd rarely hit this case locally and want to continue running tests. We should also increase this number. Non-chromium bots hit this limit all the time, and it's one of most frequent complaints about new-run-webkit-tests. I have no objection to upping this number. We picked 20 fairly arbitrarily. I think we picked it to match the number that the build.webkit.org bots were using at the time. That said, every non-chromium bot is passing in 20 via the commandline as best I can tell, so changing the default won't change anything there. Is it really true that non-chromium bots are hitting this frequently? Yes, particularly in debug builds. Jessie can tell you all the glory story about how she had to disable lots of tests to make Windows bots work :( - Ryosuke ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Re: [webkit-dev] Defaulting --exit-after-n-failures to 0
On Fri, Jun 8, 2012 at 12:37 PM, Dirk Pranke dpra...@chromium.org wrote: I have no objection either to increasing the defaults for either of these numbers or making it possible to have different defaults per port. Do you want to suggest different defaults? Should we use ORWT's (infinite failures and infinite crashes by default)? Unbounded failures crashes sound good to me although I can definitely see a value in passing some fixed value on bots so as to reduce the cycle time when 10,000+ tests start to crash. Something like 100, 300, or 500 would probably be acceptable. - Ryosuke ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Re: [webkit-dev] Defaulting --exit-after-n-failures to 0
On Fri, Jun 8, 2012 at 12:44 PM, Ryosuke Niwa rn...@webkit.org wrote: On Fri, Jun 8, 2012 at 12:37 PM, Dirk Pranke dpra...@chromium.org wrote: I have no objection either to increasing the defaults for either of these numbers or making it possible to have different defaults per port. Do you want to suggest different defaults? Should we use ORWT's (infinite failures and infinite crashes by default)? Unbounded failures crashes sound good to me although I can definitely see a value in passing some fixed value on bots so as to reduce the cycle time when 10,000+ tests start to crash. Something like 100, 300, or 500 would probably be acceptable. NRWT defaults to 500 failures, so you're suggesting we just up the default # of crashes to 500? Anyone have any other numbers jumping out at them? -- Dirk ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Re: [webkit-dev] Defaulting --exit-after-n-failures to 0
On Fri, Jun 8, 2012 at 1:38 PM, Ojan Vafai o...@chromium.org wrote: On Fri, Jun 8, 2012 at 12:46 PM, Dirk Pranke dpra...@chromium.org wrote: On Fri, Jun 8, 2012 at 12:44 PM, Ryosuke Niwa rn...@webkit.org wrote: On Fri, Jun 8, 2012 at 12:37 PM, Dirk Pranke dpra...@chromium.org wrote: I have no objection either to increasing the defaults for either of these numbers or making it possible to have different defaults per port. Do you want to suggest different defaults? Should we use ORWT's (infinite failures and infinite crashes by default)? Unbounded failures crashes sound good to me although I can definitely see a value in passing some fixed value on bots so as to reduce the cycle time when 10,000+ tests start to crash. Something like 100, 300, or 500 would probably be acceptable. NRWT defaults to 500 failures, so you're suggesting we just up the default # of crashes to 500? Anyone have any other numbers jumping out at them? I'm OK with the ORWT defaults of infinite for both. Then we never have to have this discussion again. :) It can be on the buildbot maintainers to pass in the right flags. All of the non-chromium ports already do anyways as best I can tell. That's an argument I can buy. - Ryosuke ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
