Re: [webkit-dev] run-bindings-tests

2011-09-12 Thread Darin Adler
On Sep 12, 2011, at 4:23 PM, Maciej Stachowiak wrote: > Notwithstanding all the later discussion, I think run-bindings-tests would > still be more effective as a build step that updates a source file rather > than a test step. I see, a build step that updates a checked-in source file. Sounds li

Re: [webkit-dev] run-bindings-tests

2011-09-12 Thread Maciej Stachowiak
On Sep 8, 2011, at 12:25 PM, Darin Adler wrote: > On Sep 8, 2011, at 11:49 AM, Alexey Proskuryakov wrote: > >> As discussed on IRC, I do not think that bots should run this test at all. >> It has a non-trivial maintenance cost, but provides very little benefit. >> Even the time spent by multip

Re: [webkit-dev] run-bindings-tests

2011-09-09 Thread Dirk Pranke
On Fri, Sep 9, 2011 at 10:17 AM, Geoffrey Garen wrote: > I also think it’s good to have a WebKit-specific or specific-enough word in > script names when possible so you can have the scripts in your path even > when not working on WebKit. That’s why run-webkit-tests has the word WebKit > in it, and

Re: [webkit-dev] run-bindings-tests

2011-09-09 Thread Ryosuke Niwa
On Fri, Sep 9, 2011 at 7:59 AM, Adam Roben wrote: > > An alternate path to success would be: > > 1) Decide what to call the script that just runs tests from LayoutTests > 2) Rename run-webkit-tests to that new name and add a new script called > run-webkit-tests that just calls the renamed script >

Re: [webkit-dev] run-bindings-tests

2011-09-09 Thread Geoffrey Garen
> I also think it’s good to have a WebKit-specific or specific-enough word in > script names when possible so you can have the scripts in your path even when > not working on WebKit. That’s why run-webkit-tests has the word WebKit in it, > and run-safari does not. I'd suggest one script -- run-

Re: [webkit-dev] run-bindings-tests

2011-09-09 Thread Darin Adler
On Sep 9, 2011, at 9:07 AM, Adam Barth wrote: > run-layout-tests? Sorry in advance for bikeshed'ing this: That would be a good name if we thought LayoutTests was the right name for our main regression test suite. Since I think it’s not, I would love to figure out that new name and have the scr

Re: [webkit-dev] run-bindings-tests

2011-09-09 Thread Jarred Nicholls
On Fri, Sep 9, 2011 at 12:07 PM, Adam Barth wrote: > run-layout-tests? > too obvious. as an ode to Paul, I propose run-the-mother-effing-layout-tests http://mothereffinghsl.com/ ;) > On Sep 9, 2011 8:04 AM, "Adam Roben" wrote: > > On Sep 9, 2011, at 10:59 AM, Adam Roben wrote: > > > >> On

Re: [webkit-dev] run-bindings-tests

2011-09-09 Thread Adam Barth
run-layout-tests? On Sep 9, 2011 8:04 AM, "Adam Roben" wrote: > On Sep 9, 2011, at 10:59 AM, Adam Roben wrote: > >> On Sep 9, 2011, at 10:52 AM, Eric Seidel wrote: >> >>> On Thu, Sep 8, 2011 at 12:44 PM, Darin Adler wrote: On Sep 8, 2011, at 12:29 PM, Eric Seidel wrote: > I'm happy

Re: [webkit-dev] run-bindings-tests

2011-09-09 Thread Adam Roben
On Sep 9, 2011, at 10:59 AM, Adam Roben wrote: > On Sep 9, 2011, at 10:52 AM, Eric Seidel wrote: > >> On Thu, Sep 8, 2011 at 12:44 PM, Darin Adler wrote: >>> On Sep 8, 2011, at 12:29 PM, Eric Seidel wrote: >>> I'm happy to write a run-all-tests script which runs all known tests that

Re: [webkit-dev] run-bindings-tests

2011-09-09 Thread Adam Roben
On Sep 9, 2011, at 10:52 AM, Eric Seidel wrote: > On Thu, Sep 8, 2011 at 12:44 PM, Darin Adler wrote: >> On Sep 8, 2011, at 12:29 PM, Eric Seidel wrote: >> >>> I'm happy to write a run-all-tests script which runs all known tests that >>> platform can handle. :) >> >> I think run-webkit-tests s

Re: [webkit-dev] run-bindings-tests

2011-09-09 Thread Eric Seidel
On Thu, Sep 8, 2011 at 12:44 PM, Darin Adler wrote: > On Sep 8, 2011, at 12:29 PM, Eric Seidel wrote: > >> I'm happy to write a run-all-tests script which runs all known tests that >> platform can handle. :) > > I think run-webkit-tests should be this. We can come up with a new name for > the “j

Re: [webkit-dev] run-bindings-tests

2011-09-08 Thread Adam Barth
2011/9/8 Oliver Hunt : > On Sep 8, 2011, at 7:21 PM, Alexey Proskuryakov wrote: >> 08.09.2011, в 12:25, Darin Adler написал(а): >>> I find the bindings tests quite helpful. Because the perl script is so hard >>> to read, it’s the changes in bindings script test results that I look at >>> when rev

Re: [webkit-dev] run-bindings-tests

2011-09-08 Thread Oliver Hunt
On Sep 8, 2011, at 7:21 PM, Alexey Proskuryakov wrote: > > 08.09.2011, в 12:25, Darin Adler написал(а): > >> I find the bindings tests quite helpful. Because the perl script is so hard >> to read, it’s the changes in bindings script test results that I look at >> when reviewing changes to the

Re: [webkit-dev] run-bindings-tests

2011-09-08 Thread Alexey Proskuryakov
08.09.2011, в 12:25, Darin Adler написал(а): > I find the bindings tests quite helpful. Because the perl script is so hard > to read, it’s the changes in bindings script test results that I look at when > reviewing changes to the bindings scripts. The fact that the results are > checked in hel

Re: [webkit-dev] run-bindings-tests

2011-09-08 Thread Darin Adler
On Sep 8, 2011, at 12:29 PM, Eric Seidel wrote: > I'm happy to write a run-all-tests script which runs all known tests that > platform can handle. :) I think run-webkit-tests should be this. We can come up with a new name for the “just run the tests in the LayoutTests directory” tool. A bigger

Re: [webkit-dev] run-bindings-tests

2011-09-08 Thread Ryosuke Niwa
On Thu, Sep 8, 2011 at 12:19 PM, Oliver Hunt wrote: > > The other problem is "people check in without looking at the bots". I do > try to watch the bots, but the time between me landing and the bots actually > going red can literally be hours. Of course i'm away from irc/email > whatever when I

Re: [webkit-dev] run-bindings-tests

2011-09-08 Thread Eric Seidel
If the objection against run-bindings-tests is that they're not part of some larger test script which developers can run locally, it's very easy to add a wrapper script which runs all known testing harnesses. The test tests which currently run on the bots include: run-webkit-tests (minutes) run-ja

Re: [webkit-dev] run-bindings-tests

2011-09-08 Thread Adam Barth
2011/9/8 Darin Adler : > I find the bindings tests quite helpful. Because the perl script is so hard > to read, it’s the changes in bindings script test results that I look at > when reviewing changes to the bindings scripts. The fact that the results > are checked in helps me review patches. > It

Re: [webkit-dev] run-bindings-tests

2011-09-08 Thread Darin Adler
On Sep 8, 2011, at 11:49 AM, Alexey Proskuryakov wrote: > As discussed on IRC, I do not think that bots should run this test at all. It > has a non-trivial maintenance cost, but provides very little benefit. Even > the time spent by multiple engineers on IRC today discussing bot complaints > is

Re: [webkit-dev] run-bindings-tests

2011-09-08 Thread Oliver Hunt
On Sep 8, 2011, at 11:55 AM, James Robinson wrote: > > We used to not run these tests on the bots. This meant that people would > change the bindings code and not update the expected results, so the expected > results were always massively out of date. This meant when patching the > bindings

Re: [webkit-dev] run-bindings-tests

2011-09-08 Thread Eric Seidel
I'm not currently working on bindings, so I don't have very strong opinions for or against the script. I added it to the bots back in June so that the results would stop breaking. It's possible such script would be more useful w/o checked in results, unclear. I will point out that Darin Adler,

Re: [webkit-dev] run-bindings-tests

2011-09-08 Thread James Robinson
On Thu, Sep 8, 2011 at 11:49 AM, Alexey Proskuryakov wrote: > > 08.09.2011, в 11:32, Eric Seidel написал(а): > > > FYI: As many of you already know, the build.webkit.org bots run > > "run-bindings-tests" on (almost) all platforms. > > > > They've been running (mostly w/o incident) on the bots si

Re: [webkit-dev] run-bindings-tests

2011-09-08 Thread Alexey Proskuryakov
08.09.2011, в 11:32, Eric Seidel написал(а): > FYI: As many of you already know, the build.webkit.org bots run > "run-bindings-tests" on (almost) all platforms. > > They've been running (mostly w/o incident) on the bots since 6/20: > http://trac.webkit.org/changeset/89267 > > > These just mak

[webkit-dev] run-bindings-tests

2011-09-08 Thread Eric Seidel
FYI: As many of you already know, the build.webkit.org bots run "run-bindings-tests" on (almost) all platforms. They've been running (mostly w/o incident) on the bots since 6/20: http://trac.webkit.org/changeset/89267 These just make sure that our generated bindings look sane, by comparing the