Re: [webkit-dev] Switching to new-run-webkit-tests

2011-07-06 Thread Eric Seidel
Update: Leopard Release, Gtk and Qt have been successfully transitioned. The Leopard Debug bot has a stray httpd process (without corresponding /tmp/WebKit/httpd.pid) and will require manual intervention:

Re: [webkit-dev] Switching to new-run-webkit-tests

2011-07-06 Thread Xan Lopez
[Sending with the right address...] On Wed, Jul 6, 2011 at 11:10 AM, Eric Seidel e...@webkit.org wrote: Update: Leopard Release, Gtk and Qt have been successfully transitioned. What do we exactly consider successfully transitioned? The GTK+ bots were still failing, so I reverted to the old

Re: [webkit-dev] Switching to new-run-webkit-tests

2011-07-06 Thread Eric Seidel
No problem. I leave it in your hands to re-transition, since you're much more familiar with the platform than I. It was faling before the move, and failing again after. :) The bots have simply been red today. I'm happy to work with you to update the skipped lists. -eric On Wed, Jul 6, 2011

Re: [webkit-dev] Does NRWT let you indicate that a test should fail with a particular failure diff?

2011-07-06 Thread Maciej Stachowiak
On Jul 5, 2011, at 4:51 PM, Dirk Pranke wrote: On Tue, Jul 5, 2011 at 3:46 PM, Ojan Vafai o...@chromium.org wrote: We could simplify the syntax somewhat to not require the = PASS at the end. We could also change the bug format to be actual links instead (e.g. webkit.org/b/12345 and

Re: [webkit-dev] Switching to new-run-webkit-tests

2011-07-06 Thread Eric Seidel
Xan and I found the issue regarding the timeouts, and Xan is trying NRWT again on the machine: https://bugs.webkit.org/show_bug.cgi?id=63983 On Wed, Jul 6, 2011 at 2:20 AM, Xan xan.lo...@gmail.com wrote: On Wed, Jul 6, 2011 at 11:10 AM, Eric Seidel e...@webkit.org wrote: Update: Leopard

Re: [webkit-dev] Switching to new-run-webkit-tests

2011-07-06 Thread Adam Roben
Now that more and more ports are switching to NRWT, it would be great for someone to explain what the best practices are for dealing with failing and flaky tests. -Adam ___ webkit-dev mailing list webkit-dev@lists.webkit.org

Re: [webkit-dev] Switching to new-run-webkit-tests

2011-07-06 Thread Adam Roben
On Jul 6, 2011, at 11:53 AM, Adam Roben wrote: Now that more and more ports are switching to NRWT, it would be great for someone to explain what the best practices are for dealing with failing and flaky tests. Two specific questions I have: 1) Are the ports that have switched to NRWT no

[webkit-dev] Best practices for failing a flaky tests (was Re: Switching to new-run-webkit-tests)

2011-07-06 Thread Adam Barth
I'm not sure we've quite figured that out yet. NRWT supports both Skipped lists and test_expectations.txt, which is a more expressive (but also more complex) version of Skipped lists. IMHO, we should wait for the dust to settle on the transition before changing our practices. Adam On Wed, Jul

Re: [webkit-dev] Best practices for failing a flaky tests (was Re: Switching to new-run-webkit-tests)

2011-07-06 Thread Adam Roben
On Jul 6, 2011, at 11:58 AM, Adam Barth wrote: I'm not sure we've quite figured that out yet. NRWT supports both Skipped lists and test_expectations.txt, which is a more expressive (but also more complex) version of Skipped lists. IMHO, we should wait for the dust to settle on the

Re: [webkit-dev] Best practices for failing a flaky tests (was Re: Switching to new-run-webkit-tests)

2011-07-06 Thread Ryosuke Niwa
On Wed, Jul 6, 2011 at 9:00 AM, Adam Roben aro...@apple.com wrote: OK. Then I have another question: What should I do to make the Leopard and SnowLeopard bots green, now that they have switched to NRWT? Looking at

Re: [webkit-dev] Using Skipped vs. test_expectations.txt, WAS Switching to new-run-webkit-tests

2011-07-06 Thread Eric Seidel
We've intentionally left that decision up to the ports. Mac has a stop-gap test_expectations.txt file, which depending on the result of this discussion will likely be expanded, or removed: http://trac.webkit.org/browser/trunk/LayoutTests/platform/mac/test_expectations.txt That exists solely to

Re: [webkit-dev] Best practices for failing a flaky tests (was Re: Switching to new-run-webkit-tests)

2011-07-06 Thread Adam Barth
On Wed, Jul 6, 2011 at 9:00 AM, Adam Roben aro...@apple.com wrote: On Jul 6, 2011, at 11:58 AM, Adam Barth wrote: I'm not sure we've quite figured that out yet.  NRWT supports both Skipped lists and test_expectations.txt, which is a more expressive (but also more complex) version of Skipped

Re: [webkit-dev] Using Skipped vs. test_expectations.txt, WAS Switching to new-run-webkit-tests

2011-07-06 Thread Eric Seidel
Nm, Adam Barth already split the thread in a nicer name too. Folks can reply there. :) On Wed, Jul 6, 2011 at 9:12 AM, Eric Seidel e...@webkit.org wrote: We've intentionally left that decision up to the ports. Mac has a stop-gap test_expectations.txt file, which depending on the result of

Re: [webkit-dev] Best practices for failing a flaky tests (was Re: Switching to new-run-webkit-tests)

2011-07-06 Thread Xan Lopez
On Wed, Jul 6, 2011 at 6:29 PM, Eric Seidel e...@webkit.org wrote: NRWT uses both!  It will read in all the port's Skipped files, covert them to SKIP text_expectations, and add them to your test_expectations file.

Re: [webkit-dev] Best practices for failing a flaky tests (was Re: Switching to new-run-webkit-tests)

2011-07-06 Thread Eric Seidel
On Wed, Jul 6, 2011 at 10:18 AM, Xan Lopez x...@gnome.org wrote: On Wed, Jul 6, 2011 at 6:29 PM, Eric Seidel e...@webkit.org wrote: NRWT uses both!  It will read in all the port's Skipped files, covert them to SKIP text_expectations, and add them to your test_expectations file.

Re: [webkit-dev] Switching to new-run-webkit-tests

2011-07-06 Thread Dirk Pranke
On Wed, Jul 6, 2011 at 9:04 AM, Xan Lopez x...@gnome.org wrote: On Wed, Jul 6, 2011 at 5:58 PM, Adam Roben aro...@apple.com wrote: On Jul 6, 2011, at 11:53 AM, Adam Roben wrote: Now that more and more ports are switching to NRWT, it would be great for someone to explain what the best

Re: [webkit-dev] Does NRWT let you indicate that a test should fail with a particular failure diff?

2011-07-06 Thread Dirk Pranke
On Tue, Jul 5, 2011 at 6:12 PM, Ryosuke Niwa rn...@webkit.org wrote: On Tue, Jul 5, 2011 at 5:40 PM, Dirk Pranke dpra...@chromium.org wrote: I keep hearing that the syntax is excessively complicated. It's a pretty simple syntax, but even you think that it is complicated, but in what way is it

Re: [webkit-dev] Does NRWT let you indicate that a test should fail with a particular failure diff?

2011-07-06 Thread Ryosuke Niwa
On Wed, Jul 6, 2011 at 3:03 PM, Dirk Pranke dpra...@chromium.org wrote: I personally find your examples to be much harder to parse visually. Partially the advantage to putting the test in the middle of the line is that it makes it easy to separate the stuff on the left from the expectations

Re: [webkit-dev] parallel painting

2011-07-06 Thread Gustavo Noronha Silva
On Mon, 2011-06-06 at 17:59 +0530, Monil Parmar wrote: How to use it for gtk launcher...I think it is for safari. A bit late for this answer, but for completeness sake: GtkLauncher is not a full browser, so it doesn't expose many features available in WebKitGTK+. You should use Epiphany or

Re: [webkit-dev] Inconsistency in logging approach

2011-07-06 Thread Gustavo Noronha Silva
On Thu, 2011-06-23 at 22:26 +0200, Łukasz Ślachciak wrote: They warn user with messaage sth like: WEBKIT_DEBUG is not empty, but this is a release build. Notice that many log messages will only appear in a debug build. Of course to have logging working in GTK you need to turn off

[webkit-dev] Is there special setup needed to run the WebKit2 tests?

2011-07-06 Thread Eric Seidel
old-run-webkit-tests -2 --debug produces: 107 test cases (1%) had incorrect layout 1 test case (1%) crashed 8 test cases (1%) Web process crashed 24 test cases (1%) had stderr output new-run-webkit-tests -2 --debug (which is what I'm trying to make work better) produces: Regressions: Unexpected