Re: [webkit-dev] are fuzzer tests appropriate layout tests?

2012-06-13 Thread Maciej Stachowiak
On Jun 13, 2012, at 1:32 PM, Geoffrey Garen wrote: >> These tests regularly timeout on the Chromium debug bots and occasionally >> timeout on the Apple Lion bots. > > WebKit has a clear policy about this: Tests must be fast enough not to time > out. We can fix this issue by making these tests

Re: [webkit-dev] are fuzzer tests appropriate layout tests?

2012-06-13 Thread Geoffrey Garen
> These tests regularly timeout on the Chromium debug bots and occasionally > timeout on the Apple Lion bots. WebKit has a clear policy about this: Tests must be fast enough not to time out. We can fix this issue by making these tests shorter. I don't really see the connection to an abstract de

Re: [webkit-dev] are fuzzer tests appropriate layout tests?

2012-06-13 Thread Dirk Pranke
I guess I was saying two slightly different things ... 1) I have a strong bias for individual tests that are fast 2) I have a strong bias for individual tests that are simple, focused, easy to understand, and are predictable. All other things being equal (which of course they never are), I would p

Re: [webkit-dev] are fuzzer tests appropriate layout tests?

2012-06-13 Thread Filip Pizlo
Are we sure that we want to make this a general rule? We have two profitable fuzzers in fast/js that I believe deserve to be in LayoutTests and should be run every time you make any JSC change: LayoutTests/fast/js/dfg-double-vote-fuzz.html LayoutTests/fast/js/dfg-poison-fuzz.html Both are somew

Re: [webkit-dev] are fuzzer tests appropriate layout tests?

2012-06-13 Thread Darin Adler
On Jun 13, 2012, at 12:12 PM, Dirk Pranke wrote: > On Wed, Jun 13, 2012 at 12:05 PM, Darin Adler wrote: >> On Jun 12, 2012, at 5:17 PM, Ojan Vafai wrote: >> >>> It's great to use a fuzzer in order to find cases where we're broken and >>> then make reduced layout tests from those. >> >> Gener

Re: [webkit-dev] are fuzzer tests appropriate layout tests?

2012-06-13 Thread Dirk Pranke
On Wed, Jun 13, 2012 at 12:05 PM, Darin Adler wrote: > On Jun 12, 2012, at 5:17 PM, Ojan Vafai wrote: > >> It's great to use a fuzzer in order to find cases where we're broken and >> then make reduced layout tests from those. > > > Generally we do require a test each time we fix a bug. So it’s a

Re: [webkit-dev] are fuzzer tests appropriate layout tests?

2012-06-13 Thread Darin Adler
On Jun 12, 2012, at 5:17 PM, Ojan Vafai wrote: > It's great to use a fuzzer in order to find cases where we're broken and then > make reduced layout tests from those. Generally we do require a test each time we fix a bug. So it’s a strategy for the project to always make reduced tests when we

Re: [webkit-dev] are fuzzer tests appropriate layout tests?

2012-06-13 Thread Dirk Pranke
I agree that the fuzzer should be used to create dedicated layout tests, but we shouldn't run the fuzzer itself as part of the layout test regression. I would have no objection to it being a separate test step. -- Dirk On Tue, Jun 12, 2012 at 5:17 PM, Ojan Vafai wrote: > See https://bugs.webkit.

Re: [webkit-dev] are fuzzer tests appropriate layout tests?

2012-06-13 Thread Dan Bernstein
On Jun 12, 2012, at 5:17 PM, Ojan Vafai wrote: > See https://bugs.webkit.org/show_bug.cgi?id=87772. > > It's great to use a fuzzer in order to find cases where we're broken and then > make reduced layout tests from those. The viewspec-parser tests are > themselves just a fuzzer though. Grante

[webkit-dev] are fuzzer tests appropriate layout tests?

2012-06-12 Thread Ojan Vafai
See https://bugs.webkit.org/show_bug.cgi?id=87772. It's great to use a fuzzer in order to find cases where we're broken and then make reduced layout tests from those. The viewspec-parser tests are themselves just a fuzzer though. Granted, they are deterministic by avoiding using an actual random f