date:20120608


On Jun 8, 2012, at 4:38 AM, Balazs Kelemen kbal...@webkit.org wrote:

 On 06/08/2012 09:46 AM, Osztrogonac Csaba wrote:
 Hi,
 
 Dirk Pranke írta:
 I believe most if not all of the ports have started using either
 TestExpectations files or a combination of TestExpectations files
 (except for the Apple Win port).
 
 Can we explicitly switch to the TestExpectations files at this point
 and drop support for Skipped files on the other ports (and perhaps
 disable old-run-webkit-tests for all but apple win)?
 
 Until NRWT can't handle cascaded TestExpectations - 
 https://bugs.webkit.org/show_bug.cgi?id=65834,
 Qt port can't drop supporting Skipped files. We have many tests skipped in 
 qt-5.0, qt-5.0-wk1,
 qt-5.0-wk2, wk2 Skipped lists. We can't migrate all of them to the only one 
 TestExpectations.
 
 And I disagree with disabling ORWT at all. Qt port still support using ORWT 
 locally.
 It is better for gardening than NRWT. NRWT regularly has problems with 
 generating
 new results for a given platform dir (qt,qt-5.0,qt-5.0-wk1,...), it doesn't 
 support
 the good --skipped=only option . If folks don't want to use it, just not 
 use, but
 disabling for everyone by fiat isn't a friendly thing.
 
 1. These are real weaknesses of nrwt, we should fix them. If gardening is 
 better with orwt (i doubt that is the case, but I don't do gardening 
 regularly), we should improve nrwt, i.e. reimplement features from orwt.

I applaud your enthusiasm. 

 2. I believe basically everybody agrees that we should drop orwt, except you 
 Ossy. Maybe I'm wrong. So, is there anybody still want to have support for 
 orwt? If so, why?

I'm with Ossy on this. 

Getting rid of ORWT would be a show stopper for me. 

This talk of fixing bugs in NRWT is really great but until such time as those 
bugs are fixed, let's keep ORWT. 

 
 -kbalazs
 
 ___
 webkit-dev mailing list
 webkit-dev@lists.webkit.org
 http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev

Re: [webkit-dev] can we stop using Skipped files?

2012-06-08 Thread Balazs Kelemen


On 06/08/2012 05:21 PM, Filip Pizlo wrote:

On Jun 8, 2012, at 4:38 AM, Balazs Kelemenkbal...@webkit.org  wrote:


On 06/08/2012 09:46 AM, Osztrogonac Csaba wrote:

Hi,

Dirk Pranke írta:

I believe most if not all of the ports have started using either
TestExpectations files or a combination of TestExpectations files
(except for the Apple Win port).

Can we explicitly switch to the TestExpectations files at this point
and drop support for Skipped files on the other ports (and perhaps
disable old-run-webkit-tests for all but apple win)?

Until NRWT can't handle cascaded TestExpectations - 
https://bugs.webkit.org/show_bug.cgi?id=65834,
Qt port can't drop supporting Skipped files. We have many tests skipped in 
qt-5.0, qt-5.0-wk1,
qt-5.0-wk2, wk2 Skipped lists. We can't migrate all of them to the only one 
TestExpectations.

And I disagree with disabling ORWT at all. Qt port still support using ORWT 
locally.
It is better for gardening than NRWT. NRWT regularly has problems with 
generating
new results for a given platform dir (qt,qt-5.0,qt-5.0-wk1,...), it doesn't 
support
the good --skipped=only option . If folks don't want to use it, just not use, 
but
disabling for everyone by fiat isn't a friendly thing.

1. These are real weaknesses of nrwt, we should fix them. If gardening is 
better with orwt (i doubt that is the case, but I don't do gardening 
regularly), we should improve nrwt, i.e. reimplement features from orwt.

I applaud your enthusiasm.


2. I believe basically everybody agrees that we should drop orwt, except you 
Ossy. Maybe I'm wrong. So, is there anybody still want to have support for 
orwt? If so, why?

I'm with Ossy on this.

Getting rid of ORWT would be a show stopper for me.

This talk of fixing bugs in NRWT is really great but until such time as those 
bugs are fixed, let's keep ORWT.



Understandable. Let me make it clear: I don't prefer one over the other. 
I believe it's contra-productive that some people/bots use this, and 
others use that. It adds overhead to bot maintainance, it's bad for 
developer experience, and it blocks the evolution of the one and only 
tool - because some people still make efforts on fixing/improving the 
other instead.


___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev

Re: [webkit-dev] can we stop using Skipped files?


On Jun 8, 2012, at 9:16 AM, Balazs Kelemen wrote:

 On 06/08/2012 05:21 PM, Filip Pizlo wrote:
 On Jun 8, 2012, at 4:38 AM, Balazs Kelemenkbal...@webkit.org  wrote:
 
 On 06/08/2012 09:46 AM, Osztrogonac Csaba wrote:
 Hi,
 
 Dirk Pranke írta:
 I believe most if not all of the ports have started using either
 TestExpectations files or a combination of TestExpectations files
 (except for the Apple Win port).
 
 Can we explicitly switch to the TestExpectations files at this point
 and drop support for Skipped files on the other ports (and perhaps
 disable old-run-webkit-tests for all but apple win)?
 Until NRWT can't handle cascaded TestExpectations - 
 https://bugs.webkit.org/show_bug.cgi?id=65834,
 Qt port can't drop supporting Skipped files. We have many tests skipped in 
 qt-5.0, qt-5.0-wk1,
 qt-5.0-wk2, wk2 Skipped lists. We can't migrate all of them to the only 
 one TestExpectations.
 
 And I disagree with disabling ORWT at all. Qt port still support using 
 ORWT locally.
 It is better for gardening than NRWT. NRWT regularly has problems with 
 generating
 new results for a given platform dir (qt,qt-5.0,qt-5.0-wk1,...), it 
 doesn't support
 the good --skipped=only option . If folks don't want to use it, just not 
 use, but
 disabling for everyone by fiat isn't a friendly thing.
 1. These are real weaknesses of nrwt, we should fix them. If gardening is 
 better with orwt (i doubt that is the case, but I don't do gardening 
 regularly), we should improve nrwt, i.e. reimplement features from orwt.
 I applaud your enthusiasm.
 
 2. I believe basically everybody agrees that we should drop orwt, except 
 you Ossy. Maybe I'm wrong. So, is there anybody still want to have support 
 for orwt? If so, why?
 I'm with Ossy on this.
 
 Getting rid of ORWT would be a show stopper for me.
 
 This talk of fixing bugs in NRWT is really great but until such time as 
 those bugs are fixed, let's keep ORWT.
 
 
 Understandable. Let me make it clear: I don't prefer one over the other. I 
 believe it's contra-productive that some people/bots use this, and others use 
 that. It adds overhead to bot maintainance, it's bad for developer 
 experience, and it blocks the evolution of the one and only tool - because 
 some people still make efforts on fixing/improving the other instead.

I think one issue that ought to be fixed with NRWT is the level of 
infrastructural complexity that it has, and the extent to which its algorithmic 
design (say, load balancing technique) is ossified through the use of entirely 
unnecessary object oriented pattern overhead.

In short, NRWT is overengineered.

As such, there will be those of us, which prefer solutions that are not 
overengineered, and who thus end up hacking ORWT when we need to get work done.

-F

 
 ___
 webkit-dev mailing list
 webkit-dev@lists.webkit.org
 http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev

___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev

Re: [webkit-dev] can we stop using Skipped files?

On Fri, Jun 8, 2012 at 12:46 AM, Osztrogonac Csaba o...@inf.u-szeged.huwrote:

Hi,

Dirk Pranke írta:

I believe most if not all of the ports have started using either
TestExpectations files or a combination of TestExpectations files
(except for the Apple Win port).

Can we explicitly switch to the TestExpectations files at this point
and drop support for Skipped files on the other ports (and perhaps
disable old-run-webkit-tests for all but apple win)?

Until NRWT can't handle cascaded TestExpectations -
https://bugs.webkit.org/show_**bug.cgi?id=65834https://bugs.webkit.org/show_bug.cgi?id=65834
,
Qt port can't drop supporting Skipped files. We have many tests skipped in
qt-5.0, qt-5.0-wk1,
qt-5.0-wk2, wk2 Skipped lists. We can't migrate all of them to the only
one TestExpectations.

And I disagree with disabling ORWT at all. Qt port still support using
ORWT locally.
It is better for gardening than NRWT. NRWT regularly has problems with
generating
new results for a given platform dir (qt,qt-5.0,qt-5.0-wk1,...), it
doesn't support
the good --skipped=only option . If folks don't want to use it, just not
use, but
disabling for everyone by fiat isn't a friendly thing.

Can you file bugs for these explaining where it does something different
from ORWT? Dirk is actively working on the cascading issue, but I don't
think these other issues are known.

I don't see why it would make sense to keep two parallel tools for this
once all the workflow bugs people have are addressed.

If not, what needs to happen so that we can do that? Do we need to
land more of the discussed changes to the syntax of the
TestExpectations file? Anything else?

It would be nice to get rid of the complexity (both in the code and in
our heads) if we can.

-- Dirk

br,
Ossy

__**_
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/**mailman/listinfo.cgi/webkit-**devhttp://lists.webkit.org/mailman/listinfo.cgi/webkit-dev

___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev

Re: [webkit-dev] can we stop using Skipped files?

On Fri, Jun 8, 2012 at 9:25 AM, Filip Pizlo fpi...@apple.com wrote:


 On Jun 8, 2012, at 9:16 AM, Balazs Kelemen wrote:

  On 06/08/2012 05:21 PM, Filip Pizlo wrote:
  On Jun 8, 2012, at 4:38 AM, Balazs Kelemenkbal...@webkit.org  wrote:
 
  On 06/08/2012 09:46 AM, Osztrogonac Csaba wrote:
  Hi,
 
  Dirk Pranke írta:
  I believe most if not all of the ports have started using either
  TestExpectations files or a combination of TestExpectations files
  (except for the Apple Win port).
 
  Can we explicitly switch to the TestExpectations files at this point
  and drop support for Skipped files on the other ports (and perhaps
  disable old-run-webkit-tests for all but apple win)?
  Until NRWT can't handle cascaded TestExpectations -
 https://bugs.webkit.org/show_bug.cgi?id=65834,
  Qt port can't drop supporting Skipped files. We have many tests
 skipped in qt-5.0, qt-5.0-wk1,
  qt-5.0-wk2, wk2 Skipped lists. We can't migrate all of them to the
 only one TestExpectations.
 
  And I disagree with disabling ORWT at all. Qt port still support
 using ORWT locally.
  It is better for gardening than NRWT. NRWT regularly has problems
 with generating
  new results for a given platform dir (qt,qt-5.0,qt-5.0-wk1,...), it
 doesn't support
  the good --skipped=only option . If folks don't want to use it, just
 not use, but
  disabling for everyone by fiat isn't a friendly thing.
  1. These are real weaknesses of nrwt, we should fix them. If gardening
 is better with orwt (i doubt that is the case, but I don't do gardening
 regularly), we should improve nrwt, i.e. reimplement features from orwt.
  I applaud your enthusiasm.
 
  2. I believe basically everybody agrees that we should drop orwt,
 except you Ossy. Maybe I'm wrong. So, is there anybody still want to have
 support for orwt? If so, why?
  I'm with Ossy on this.
 
  Getting rid of ORWT would be a show stopper for me.
 
  This talk of fixing bugs in NRWT is really great but until such time as
 those bugs are fixed, let's keep ORWT.
 
 
  Understandable. Let me make it clear: I don't prefer one over the other.
 I believe it's contra-productive that some people/bots use this, and others
 use that. It adds overhead to bot maintainance, it's bad for developer
 experience, and it blocks the evolution of the one and only tool - because
 some people still make efforts on fixing/improving the other instead.

 I think one issue that ought to be fixed with NRWT is the level of
 infrastructural complexity that it has, and the extent to which its
 algorithmic design (say, load balancing technique) is ossified through the
 use of entirely unnecessary object oriented pattern overhead.

 In short, NRWT is overengineered.

 As such, there will be those of us, which prefer solutions that are not
 overengineered, and who thus end up hacking ORWT when we need to get work
 done.


Do you really do quick local hacks to ORWT? Or are you talking about adding
actual features to ORWT?

While NRWT is more code than ORWT, it's also thoroughly unittested. It's a
little harder to dive into, but it's much easier to maintain IMO.

The complexity in the codebase is partially because it was written before
webkitpy existed and then ported to work on top of webkitpy. So there is
still cleanup that could happen to make it less complex. But the specific
complexity you're talking about for the parallelism was necessary to make
NRWT use multiple processes, which itself was necessary because of Python
multithreading bugs.
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev

Re: [webkit-dev] can we stop using Skipped files?

2012-06-08 Thread Zoltan Herczeg

Hi,

 I don't see why it would make sense to keep two parallel tools for this
 once all the workflow bugs people have are addressed.

The reason is easy. In the past when people tried to add new features to
NRWT, they were not allowed to because the feature is not useful for NRWT
devs. Eventually people got tired of NRWT and use ORWT instead since its
development is not as restricted. Is this changed?

Regards,
Zoltan


___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev

Re: [webkit-dev] can we stop using Skipped files?

On Fri, Jun 8, 2012 at 10:14 AM, Zoltan Herczeg zherc...@webkit.org wrote:

 Hi,

  I don't see why it would make sense to keep two parallel tools for this
  once all the workflow bugs people have are addressed.

 The reason is easy. In the past when people tried to add new features to
 NRWT, they were not allowed to because the feature is not useful for NRWT
 devs. Eventually people got tired of NRWT and use ORWT instead since its
 development is not as restricted. Is this changed?


Can you point to a case of this? There's no policy restricting adding
features to NRWT, so I'm not really sure what you're referring to. A
specific example might help.
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev

Re: [webkit-dev] can we stop using Skipped files?


On Jun 8, 2012, at 9:52 AM, Ojan Vafai wrote:

 On Fri, Jun 8, 2012 at 9:25 AM, Filip Pizlo fpi...@apple.com wrote:
 
 On Jun 8, 2012, at 9:16 AM, Balazs Kelemen wrote:
 
  On 06/08/2012 05:21 PM, Filip Pizlo wrote:
  On Jun 8, 2012, at 4:38 AM, Balazs Kelemenkbal...@webkit.org  wrote:
 
  On 06/08/2012 09:46 AM, Osztrogonac Csaba wrote:
  Hi,
 
  Dirk Pranke írta:
  I believe most if not all of the ports have started using either
  TestExpectations files or a combination of TestExpectations files
  (except for the Apple Win port).
 
  Can we explicitly switch to the TestExpectations files at this point
  and drop support for Skipped files on the other ports (and perhaps
  disable old-run-webkit-tests for all but apple win)?
  Until NRWT can't handle cascaded TestExpectations - 
  https://bugs.webkit.org/show_bug.cgi?id=65834,
  Qt port can't drop supporting Skipped files. We have many tests skipped 
  in qt-5.0, qt-5.0-wk1,
  qt-5.0-wk2, wk2 Skipped lists. We can't migrate all of them to the only 
  one TestExpectations.
 
  And I disagree with disabling ORWT at all. Qt port still support using 
  ORWT locally.
  It is better for gardening than NRWT. NRWT regularly has problems with 
  generating
  new results for a given platform dir (qt,qt-5.0,qt-5.0-wk1,...), it 
  doesn't support
  the good --skipped=only option . If folks don't want to use it, just not 
  use, but
  disabling for everyone by fiat isn't a friendly thing.
  1. These are real weaknesses of nrwt, we should fix them. If gardening is 
  better with orwt (i doubt that is the case, but I don't do gardening 
  regularly), we should improve nrwt, i.e. reimplement features from orwt.
  I applaud your enthusiasm.
 
  2. I believe basically everybody agrees that we should drop orwt, except 
  you Ossy. Maybe I'm wrong. So, is there anybody still want to have 
  support for orwt? If so, why?
  I'm with Ossy on this.
 
  Getting rid of ORWT would be a show stopper for me.
 
  This talk of fixing bugs in NRWT is really great but until such time as 
  those bugs are fixed, let's keep ORWT.
 
 
  Understandable. Let me make it clear: I don't prefer one over the other. I 
  believe it's contra-productive that some people/bots use this, and others 
  use that. It adds overhead to bot maintainance, it's bad for developer 
  experience, and it blocks the evolution of the one and only tool - because 
  some people still make efforts on fixing/improving the other instead.
 
 I think one issue that ought to be fixed with NRWT is the level of 
 infrastructural complexity that it has, and the extent to which its 
 algorithmic design (say, load balancing technique) is ossified through the 
 use of entirely unnecessary object oriented pattern overhead.
 
 In short, NRWT is overengineered.
 
 As such, there will be those of us, which prefer solutions that are not 
 overengineered, and who thus end up hacking ORWT when we need to get work 
 done.
 
 Do you really do quick local hacks to ORWT? Or are you talking about adding 
 actual features to ORWT?

Both.

 
 While NRWT is more code than ORWT, it's also thoroughly unittested. It's a 
 little harder to dive into, but it's much easier to maintain IMO.

It's a lot harder to dive into, a lot more cumbersome to improve, and not any 
easier to maintain.

The fact that it is unittested is part of the problem.  It restricts what you 
can do to the interfaces used internally in the code, which makes larger 
changes much harder to pull off.  It also steepens the learning curve.

Look, the test harness should be a thing that helps us get work done, not an 
end unto itself.  The harness should err on the side of understandability, 
hackability, and, in short, simplicity.

 
 The complexity in the codebase is partially because it was written before 
 webkitpy existed and then ported to work on top of webkitpy. So there is 
 still cleanup that could happen to make it less complex. But the specific 
 complexity you're talking about for the parallelism was necessary to make 
 NRWT use multiple processes, which itself was necessary because of Python 
 multithreading bugs.

I look forward to seeing what such a cleanup would look like.

Python feels like the wrong technology to use for multithreading.  I'm curious 
if, in the future, next time we rewrite RWT (and I do believe that there will 
be such a next time), we could pick a language that allows this more 
naturally.  It's sad that most languages have bugs in either the library code 
for starting and managing processes (Java, I'm glaring at you!) or bugs in 
threads (as you say, Python, and in my experience, Ruby and Perl, and probably 
others as well).

-F___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev

[webkit-dev] Defaulting --exit-after-n-failures to 0

Speaking of differences between NRWT and ORWT, ORWT doesn't limit to n
failures by default. NRWT limits to 500 by default. It looks like all the
bots pass in --exit-after-n-failures=500. Any objection to making NRWT
match ORWT here? I don't think having this limit is useful/necessary for
local development.

We'll still default --exit-after-n-crashes-or-timeouts to 20. That seems
more useful to me since you'd rarely hit this case locally and want to
continue running tests.

Ojan
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev

Re: [webkit-dev] Defaulting --exit-after-n-failures to 0

On Fri, Jun 8, 2012 at 12:05 PM, Ojan Vafai o...@chromium.org wrote:

 We'll still default --exit-after-n-crashes-or-timeouts to 20. That seems
 more useful to me since you'd rarely hit this case locally and want to
 continue running tests.


We should also increase this number. Non-chromium bots hit this limit all
the time, and it's one of most frequent complaints about
new-run-webkit-tests.

- Ryosuke
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev

Re: [webkit-dev] Defaulting --exit-after-n-failures to 0

On Fri, Jun 8, 2012 at 12:08 PM, Ryosuke Niwa rn...@webkit.org wrote:

 On Fri, Jun 8, 2012 at 12:05 PM, Ojan Vafai o...@chromium.org wrote:

 We'll still default --exit-after-n-crashes-or-timeouts to 20. That seems
 more useful to me since you'd rarely hit this case locally and want to
 continue running tests.


 We should also increase this number. Non-chromium bots hit this limit all
 the time, and it's one of most frequent complaints about
 new-run-webkit-tests.


I have no objection to upping this number. We picked 20 fairly arbitrarily.
I think we picked it to match the number that the build.webkit.org bots
were using at the time. That said, every non-chromium bot is passing in 20
via the commandline as best I can tell, so changing the default won't
change anything there. Is it really true that non-chromium bots are hitting
this frequently?

Ojan
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev

Re: [webkit-dev] can we stop using Skipped files?

Hi Ossy,

Thanks for your reply ...

On Fri, Jun 8, 2012 at 12:46 AM, Osztrogonac Csaba o...@inf.u-szeged.hu wrote:
 Hi,

 Dirk Pranke írta:

 I believe most if not all of the ports have started using either
 TestExpectations files or a combination of TestExpectations files
 (except for the Apple Win port).

 Can we explicitly switch to the TestExpectations files at this point
 and drop support for Skipped files on the other ports (and perhaps
 disable old-run-webkit-tests for all but apple win)?


 Until NRWT can't handle cascaded TestExpectations -
 https://bugs.webkit.org/show_bug.cgi?id=65834,
 Qt port can't drop supporting Skipped files. We have many tests skipped in
 qt-5.0, qt-5.0-wk1,
 qt-5.0-wk2, wk2 Skipped lists. We can't migrate all of them to the only one
 TestExpectations.


Sorry, you're right, I should have mentioned that we would need to
support cascaded files before we can do anything. As Ojan mentioned, I
am actively working on that.

One thing that would be helpful would be to how each port would like
to support using cascading files and modifiers. Assuming each of you
is free to do as you like, you can pick from several paths, e.g. you
could use one file, or one file per operating system, or one file per
os plus one file per version plus maybe a webkit-2 specific file
(roughly what most ports do today, I think), or something else?


 And I disagree with disabling ORWT at all. Qt port still support using ORWT
 locally.
 It is better for gardening than NRWT. NRWT regularly has problems with
 generating
 new results for a given platform dir (qt,qt-5.0,qt-5.0-wk1,...), it doesn't
 support
 the good --skipped=only option .

I am not aware of these issues ... can you explain further, or file
bugs? In particular, I'm not sure I fully understand how all of the
different Qt version/platform combinations work, so maybe there are
issues there that can be improved?

If ORWT is easier for you to use than NRWT, I would love to understand
why, so that hopefully I can improve NRWT for you as well.

 If folks don't want to use it, just not use, but disabling for everyone by 
 fiat isn't a friendly thing.

I'm sorry, I was under the impression that no one (apart from Apple
Win) was using ORWT any more, and I thought we had reached feature
parity between the two tools (or at least that NRWT supported
everything that ORWT did, obviously ORWT doesn't support reftests or a
bunch of other things that NRWT does).

I'm not about to remove a tool that people are still using :). That
said, having two tools that are both widely used and do essentially
the same thing is unfortunate, so it would be good to converge
where/when/if we can, I think.

-- Dirk
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev

[webkit-dev] Running layout tests for the chromium port, using the multi-processing architecture and fully sandboxed

2012-06-08 Thread Jochen Eisinger

Hey,

I've implemented initial support for running layout tests using chromium's
content_shell instead of test_shell as the current DRT implementation does.
It's still a very experimental, but might already be interesting for some
of you to try.

The main advantage is that content_shell includes chromium's full storage
implementation, so layout tests for indexed db are running against the
multi-processed IDB implementation instead of the simple stub used for DRT.

Here's how to build webkit with content_shell:

1. Make sure your WebKit is at least at r119852 (see
http://trac.webkit.org/wiki/Chromium for prerequisites)
2. Apply the attachment from https://bugs.webkit.org/show_bug.cgi?id=87045
3. In Source/WebKit/chromium run gclient sync
4. build webkit as usual

E.g. for a debug build on Linux, this should give you
out/Debug/content_shell

You can now run layout tests like this:

new-run-webkit-tests --chromium --debug --driver-name=content_shell
--additional-drt-flag=--dump-render-tree LayoutTests/storage/indexeddb/*

You'll notice that not all tests are passing yet, mainly because not all
(or actually, almost none of the) layoutTestController features are
implemented yet.

Another missing feature is producing pixel results. However, I'm currently
concentrating on text results, as I think the biggest benefit is the
ability to run storage tests on the real storage implementation.

Let me know if you have any feedback, comments, or questions. If somebody
is implemented in helping adding features (e.g. to get your favorite part
of the layout tests passing), I'm happy to point you at the code locations!

best
-jochen
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev

Re: [webkit-dev] can we stop using Skipped files?

On Fri, Jun 8, 2012 at 8:21 AM, Filip Pizlo fpi...@apple.com wrote:

 On Jun 8, 2012, at 4:38 AM, Balazs Kelemen kbal...@webkit.org wrote:

 On 06/08/2012 09:46 AM, Osztrogonac Csaba wrote:
 Hi,

 Dirk Pranke írta:
 I believe most if not all of the ports have started using either
 TestExpectations files or a combination of TestExpectations files
 (except for the Apple Win port).

 Can we explicitly switch to the TestExpectations files at this point
 and drop support for Skipped files on the other ports (and perhaps
 disable old-run-webkit-tests for all but apple win)?

 Until NRWT can't handle cascaded TestExpectations - 
 https://bugs.webkit.org/show_bug.cgi?id=65834,
 Qt port can't drop supporting Skipped files. We have many tests skipped in 
 qt-5.0, qt-5.0-wk1,
 qt-5.0-wk2, wk2 Skipped lists. We can't migrate all of them to the only one 
 TestExpectations.

 And I disagree with disabling ORWT at all. Qt port still support using ORWT 
 locally.
 It is better for gardening than NRWT. NRWT regularly has problems with 
 generating
 new results for a given platform dir (qt,qt-5.0,qt-5.0-wk1,...), it doesn't 
 support
 the good --skipped=only option . If folks don't want to use it, just not 
 use, but
 disabling for everyone by fiat isn't a friendly thing.

 1. These are real weaknesses of nrwt, we should fix them. If gardening is 
 better with orwt (i doubt that is the case, but I don't do gardening 
 regularly), we should improve nrwt, i.e. reimplement features from orwt.

 I applaud your enthusiasm.

 2. I believe basically everybody agrees that we should drop orwt, except you 
 Ossy. Maybe I'm wrong. So, is there anybody still want to have support for 
 orwt? If so, why?

 I'm with Ossy on this.

 Getting rid of ORWT would be a show stopper for me.

Can you file bugs or give me examples of why this is true? Are there
specific features missing, or bugs you're hitting? You mentioned the
architecture of the code and how easy it was for you to modify things
- I'll follow up with you on this separately (although I'd be happy to
discuss this on webkit-dev, just in another thread to avoid
confusion).

-- Dirk
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev

Re: [webkit-dev] can we stop using Skipped files?

On Fri, Jun 8, 2012 at 10:14 AM, Zoltan Herczeg zherc...@webkit.org wrote:
 Hi,

 I don't see why it would make sense to keep two parallel tools for this
 once all the workflow bugs people have are addressed.

 The reason is easy. In the past when people tried to add new features to
 NRWT, they were not allowed to because the feature is not useful for NRWT
 devs. Eventually people got tired of NRWT and use ORWT instead since its
 development is not as restricted. Is this changed?

Hi Zoltan,

I don't know who gave you that impression in the past; if it was me, I
apologize and that certainly wasn't my general intent. It might be
that I had preferred one approach to another, but perhaps that's a
different issue (I'd be happy to look at or revisit any specific
examples).

At any rate, I believe we are definitely open to adding new features;
feel free to suggest them or work on them!

-- Dirk
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev

Re: [webkit-dev] can we stop using Skipped files?


On Jun 8, 2012, at 12:19 PM, Dirk Pranke wrote:

 On Fri, Jun 8, 2012 at 8:21 AM, Filip Pizlo fpi...@apple.com wrote:
 
 On Jun 8, 2012, at 4:38 AM, Balazs Kelemen kbal...@webkit.org wrote:
 
 On 06/08/2012 09:46 AM, Osztrogonac Csaba wrote:
 Hi,
 
 Dirk Pranke írta:
 I believe most if not all of the ports have started using either
 TestExpectations files or a combination of TestExpectations files
 (except for the Apple Win port).
 
 Can we explicitly switch to the TestExpectations files at this point
 and drop support for Skipped files on the other ports (and perhaps
 disable old-run-webkit-tests for all but apple win)?
 
 Until NRWT can't handle cascaded TestExpectations - 
 https://bugs.webkit.org/show_bug.cgi?id=65834,
 Qt port can't drop supporting Skipped files. We have many tests skipped in 
 qt-5.0, qt-5.0-wk1,
 qt-5.0-wk2, wk2 Skipped lists. We can't migrate all of them to the only 
 one TestExpectations.
 
 And I disagree with disabling ORWT at all. Qt port still support using 
 ORWT locally.
 It is better for gardening than NRWT. NRWT regularly has problems with 
 generating
 new results for a given platform dir (qt,qt-5.0,qt-5.0-wk1,...), it 
 doesn't support
 the good --skipped=only option . If folks don't want to use it, just not 
 use, but
 disabling for everyone by fiat isn't a friendly thing.
 
 1. These are real weaknesses of nrwt, we should fix them. If gardening is 
 better with orwt (i doubt that is the case, but I don't do gardening 
 regularly), we should improve nrwt, i.e. reimplement features from orwt.
 
 I applaud your enthusiasm.
 
 2. I believe basically everybody agrees that we should drop orwt, except 
 you Ossy. Maybe I'm wrong. So, is there anybody still want to have support 
 for orwt? If so, why?
 
 I'm with Ossy on this.
 
 Getting rid of ORWT would be a show stopper for me.
 
 Can you file bugs or give me examples of why this is true? Are there
 specific features missing, or bugs you're hitting?

Here's a really nasty one, that has been sitting untouched since November:

https://bugs.webkit.org/show_bug.cgi?id=71634

 You mentioned the
 architecture of the code and how easy it was for you to modify things
 - I'll follow up with you on this separately (although I'd be happy to
 discuss this on webkit-dev, just in another thread to avoid
 confusion).
 
 -- Dirk

___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev

Re: [webkit-dev] can we stop using Skipped files?

2012-06-08 Thread Rafael Brandao

I'm all for getting rid of ORWT. I've observed some wrong code paths there
that are probably not even used anymore. It makes more difficult to hack on
a code which almost nobody uses and whose part of it is wrong and
misleading.

NRWT is not that easy thought, but I see the unittests as an improvement.
So we can keep the same standard of the rest of webkit contributions of
writing a test to keep the behavior you're introducing.

I would like to help on removing qt dependency of ORWT.

Regards,

On Fri, Jun 8, 2012 at 4:15 PM, Dirk Pranke dpra...@chromium.org wrote:

 Hi Ossy,

 Thanks for your reply ...

 On Fri, Jun 8, 2012 at 12:46 AM, Osztrogonac Csaba o...@inf.u-szeged.hu
 wrote:
  Hi,
 
  Dirk Pranke írta:
 
  I believe most if not all of the ports have started using either
  TestExpectations files or a combination of TestExpectations files
  (except for the Apple Win port).
 
  Can we explicitly switch to the TestExpectations files at this point
  and drop support for Skipped files on the other ports (and perhaps
  disable old-run-webkit-tests for all but apple win)?
 
 
  Until NRWT can't handle cascaded TestExpectations -
  https://bugs.webkit.org/show_bug.cgi?id=65834,
  Qt port can't drop supporting Skipped files. We have many tests skipped
 in
  qt-5.0, qt-5.0-wk1,
  qt-5.0-wk2, wk2 Skipped lists. We can't migrate all of them to the only
 one
  TestExpectations.
 

 Sorry, you're right, I should have mentioned that we would need to
 support cascaded files before we can do anything. As Ojan mentioned, I
 am actively working on that.

 One thing that would be helpful would be to how each port would like
 to support using cascading files and modifiers. Assuming each of you
 is free to do as you like, you can pick from several paths, e.g. you
 could use one file, or one file per operating system, or one file per
 os plus one file per version plus maybe a webkit-2 specific file
 (roughly what most ports do today, I think), or something else?


  And I disagree with disabling ORWT at all. Qt port still support using
 ORWT
  locally.
  It is better for gardening than NRWT. NRWT regularly has problems with
  generating
  new results for a given platform dir (qt,qt-5.0,qt-5.0-wk1,...), it
 doesn't
  support
  the good --skipped=only option .

 I am not aware of these issues ... can you explain further, or file
 bugs? In particular, I'm not sure I fully understand how all of the
 different Qt version/platform combinations work, so maybe there are
 issues there that can be improved?

 If ORWT is easier for you to use than NRWT, I would love to understand
 why, so that hopefully I can improve NRWT for you as well.

  If folks don't want to use it, just not use, but disabling for everyone
 by fiat isn't a friendly thing.

 I'm sorry, I was under the impression that no one (apart from Apple
 Win) was using ORWT any more, and I thought we had reached feature
 parity between the two tools (or at least that NRWT supported
 everything that ORWT did, obviously ORWT doesn't support reftests or a
 bunch of other things that NRWT does).

 I'm not about to remove a tool that people are still using :). That
 said, having two tools that are both widely used and do essentially
 the same thing is unfortunate, so it would be good to converge
 where/when/if we can, I think.

 -- Dirk
 ___
 webkit-dev mailing list
 webkit-dev@lists.webkit.org
 http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev




-- 
Rafael Brandao @ INdT
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev

Re: [webkit-dev] Defaulting --exit-after-n-failures to 0

On Fri, Jun 8, 2012 at 12:12 PM, Ojan Vafai o...@chromium.org wrote:

 On Fri, Jun 8, 2012 at 12:08 PM, Ryosuke Niwa rn...@webkit.org wrote:

 On Fri, Jun 8, 2012 at 12:05 PM, Ojan Vafai o...@chromium.org wrote:

 We'll still default --exit-after-n-crashes-or-timeouts to 20. That seems
 more useful to me since you'd rarely hit this case locally and want to
 continue running tests.


 We should also increase this number. Non-chromium bots hit this limit all
 the time, and it's one of most frequent complaints about
 new-run-webkit-tests.


 I have no objection to upping this number. We picked 20 fairly
 arbitrarily. I think we picked it to match the number that the
 build.webkit.org bots were using at the time. That said, every
 non-chromium bot is passing in 20 via the commandline as best I can tell,
 so changing the default won't change anything there. Is it really true that
 non-chromium bots are hitting this frequently?


Yes, particularly in debug builds. Jessie can tell you all the glory story
about how she had to disable lots of tests to make Windows bots work :(

- Ryosuke
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev

Re: [webkit-dev] can we stop using Skipped files?

On Fri, Jun 8, 2012 at 10:56 AM, Filip Pizlo fpi...@apple.com wrote:

 It's a lot harder to dive into, a lot more cumbersome to improve, and not
 any easier to maintain.


I definitely agree that NRWT is more complicated than it seems like it
should be; it got contorted as we added all the features we needed to
add, and I have been in a simplify mode over the past few months. I
would welcome any feedback where you think things are overly complex.

 The fact that it is unittested is part of the problem.  It restricts what
 you can do to the interfaces used internally in the code, which makes larger
 changes much harder to pull off.  It also steepens the learning curve.

This is true. Of course, like all tests, it's a tradeoff. If it's not
tested, it's very easy for one to change things and break some other
feature that people need.

 Look, the test harness should be a thing that helps us get work done, not an
 end unto itself.  The harness should err on the side of understandability,
 hackability, and, in short, simplicity.

I definitely agree that the harness should be a means and not an end.
I of course like those ilities, too, but we also  include
functionality and performance, and both of those usually bring in
complexity :(.

 Python feels like the wrong technology to use for multithreading.  I'm
 curious if, in the future, next time we rewrite RWT (and I do believe that
 there will be such a next time), we could pick a language that allows this
 more naturally.

It is true that Python's multithreading support is not great (it works
as long as you don't need to interact with subprocesses, and as long
as you don't need truly concurrent threads executing python code,
because of the global interpreter lock). However,much of NRWT is built
around a shared-nothing message-passing approach; do you think that
was the wrong architecture, or would you have preferred to use a tool
that had the same architecture but did it more easily (perhaps in
something like Erlang)?

-- Dirk
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev

Re: [webkit-dev] Defaulting --exit-after-n-failures to 0

I have no objection either to increasing the defaults for either of
these numbers or making it possible to have different defaults per
port.

Do you want to suggest different defaults? Should we use ORWT's
(infinite failures and infinite crashes by default)?

-- Dirk

On Fri, Jun 8, 2012 at 12:31 PM, Ryosuke Niwa rn...@webkit.org wrote:
 On Fri, Jun 8, 2012 at 12:12 PM, Ojan Vafai o...@chromium.org wrote:

 On Fri, Jun 8, 2012 at 12:08 PM, Ryosuke Niwa rn...@webkit.org wrote:

 On Fri, Jun 8, 2012 at 12:05 PM, Ojan Vafai o...@chromium.org wrote:

 We'll still default --exit-after-n-crashes-or-timeouts to 20. That seems
 more useful to me since you'd rarely hit this case locally and want to
 continue running tests.


 We should also increase this number. Non-chromium bots hit this limit all
 the time, and it's one of most frequent complaints about
 new-run-webkit-tests.


 I have no objection to upping this number. We picked 20 fairly
 arbitrarily. I think we picked it to match the number that the
 build.webkit.org bots were using at the time. That said, every non-chromium
 bot is passing in 20 via the commandline as best I can tell, so changing the
 default won't change anything there. Is it really true that non-chromium
 bots are hitting this frequently?


 Yes, particularly in debug builds. Jessie can tell you all the glory story
 about how she had to disable lots of tests to make Windows bots work :(

 - Ryosuke

___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev

Re: [webkit-dev] Running layout tests for the chromium port, using the multi-processing architecture and fully sandboxed

On Fri, Jun 8, 2012 at 12:18 PM, Jochen Eisinger joc...@chromium.orgwrote:

 I've implemented initial support for running layout tests using chromium's
 content_shell instead of test_shell as the current DRT implementation does.
 It's still a very experimental, but might already be interesting for some
 of you to try.


This is great! Thanks a lot on working this.

1. Make sure your WebKit is at least at r119852 (see
 http://trac.webkit.org/wiki/Chromium for prerequisites)
 2. Apply the attachment from https://bugs.webkit.org/show_bug.cgi?id=87045
 3. In Source/WebKit/chromium run gclient sync
 4. build webkit as usual

 E.g. for a debug build on Linux, this should give you
 out/Debug/content_shell

 You can now run layout tests like this:

 new-run-webkit-tests --chromium --debug --driver-name=content_shell
 --additional-drt-flag=--dump-render-tree LayoutTests/storage/indexeddb/*

 You'll notice that not all tests are passing yet, mainly because not all
 (or actually, almost none of the) layoutTestController features are
 implemented yet.


Where is layoutTestController implemented? We definitely need to move the
implementation of layoutTestController, eventSender, etc... into WebKit
repository because we often rename functions, etc... in WebKit. It's
unacceptable to require having to modify Chromium code in order to do this
refactoring in the future.

Per my other thread about sharing IDLs between DumpRenderTree and
WebKitTestRunner, I would like to see us sharing IDL with WebKitTestRunner
instead of adding yet another binding code.

Another missing feature is producing pixel results. However, I'm currently
 concentrating on text results, as I think the biggest benefit is the
 ability to run storage tests on the real storage implementation.


That sounds great to me but we definitely need to support pixel results
eventually. I'm more than happy to help you on that but that requires the
codebase to be moved into WebKit repository.

- Ryosuke
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev

Re: [webkit-dev] Defaulting --exit-after-n-failures to 0

On Fri, Jun 8, 2012 at 12:37 PM, Dirk Pranke dpra...@chromium.org wrote:

 I have no objection either to increasing the defaults for either of
 these numbers or making it possible to have different defaults per
 port.

 Do you want to suggest different defaults? Should we use ORWT's
 (infinite failures and infinite crashes by default)?


Unbounded failures  crashes sound good to me although I can definitely see
a value in passing some fixed value on bots so as to reduce the cycle time
when 10,000+ tests start to crash. Something like 100, 300, or 500 would
probably be acceptable.

- Ryosuke
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev

Re: [webkit-dev] Defaulting --exit-after-n-failures to 0

On Fri, Jun 8, 2012 at 12:44 PM, Ryosuke Niwa rn...@webkit.org wrote:
 On Fri, Jun 8, 2012 at 12:37 PM, Dirk Pranke dpra...@chromium.org wrote:

 I have no objection either to increasing the defaults for either of
 these numbers or making it possible to have different defaults per
 port.

 Do you want to suggest different defaults? Should we use ORWT's
 (infinite failures and infinite crashes by default)?


 Unbounded failures  crashes sound good to me although I can definitely see
 a value in passing some fixed value on bots so as to reduce the cycle time
 when 10,000+ tests start to crash. Something like 100, 300, or 500 would
 probably be acceptable.


NRWT defaults to 500 failures, so you're suggesting we just up the
default # of crashes to 500? Anyone have any other numbers jumping out
at them?

-- Dirk
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev

Re: [webkit-dev] can we stop using Skipped files?

On Jun 8, 2012, at 12:31 PM, Dirk Pranke wrote:

 On Fri, Jun 8, 2012 at 10:56 AM, Filip Pizlo fpi...@apple.com wrote:
 
 It's a lot harder to dive into, a lot more cumbersome to improve, and not
 any easier to maintain.
 
 
 I definitely agree that NRWT is more complicated than it seems like it
 should be; it got contorted as we added all the features we needed to
 add, and I have been in a simplify mode over the past few months. I
 would welcome any feedback where you think things are overly complex.

This is a difficult question - it's unfortunately easier to observe that 
something is complex, than it is to pinpoint why it is complex.  But I will try.

1) Code locality.  I can open Tools/Scripts/old-run-webkit-tests and pretty 
rapidly discover (a) how options are parsed, (b) how platform differences are 
handled, (c) how tests are found, and (d) how tests are run.  I can hack all of 
this code because it's all in one place.  I don't have to be a domain expert to 
do it.  Hell, I don't even have to be good a Perl to find my way around.

2) Code size:

[pizlo@wartooth OpenSource] wc Tools/Scripts/old-run-webkit-tests 
2796   10316   98733 Tools/Scripts/old-run-webkit-tests

[pizlo@wartooth OpenSource] wc Tools/Scripts/new-run-webkit-tests 
Tools/Scripts/webkitpy/layout_tests/*.py 
Tools/Scripts/webkitpy/layout_tests/*/*.py
 bunch of stuff
   23197   91897 1049914 total

That's a *HUGE* difference.  Consider that NRWT just adds only one thing that 
most people care about: parallelism.  Is an 8x increase in code size justified?

I know that LoC metrics are evil in most cases.  But this is not most cases.  
This is an order-of-magnitude difference.  That's 8x more code I have to look 
at to find what I want.  That's 8x more code that I potentially have to edit to 
add a feature.  That's 8x more code that could have a bug.  And so on.  Badness!

 
 The fact that it is unittested is part of the problem.  It restricts what
 you can do to the interfaces used internally in the code, which makes larger
 changes much harder to pull off.  It also steepens the learning curve.
 
 This is true. Of course, like all tests, it's a tradeoff. If it's not
 tested, it's very easy for one to change things and break some other
 feature that people need.

But the act of running layout tests is in itself a regression test of the 
layout test harness just as much as it is a regression test of WebKit.

So the unit tests are superfluous.  In particular, if I had to pick between 
only having unit tests or only having regression tests, I might pick unit 
tests.  But if I already have regression tests then I'm unlikely to want to 
incur technical debt to build unit tests, particularly since unit tests 
requiring changing the infrastructure to make the code more testable, which 
then leads to the problems listed above.

 
 Look, the test harness should be a thing that helps us get work done, not an
 end unto itself.  The harness should err on the side of understandability,
 hackability, and, in short, simplicity.
 
 I definitely agree that the harness should be a means and not an end.
 I of course like those ilities, too, but we also  include
 functionality and performance, and both of those usually bring in
 complexity :(.

Right, and we are all for functionality and performance.  That's good stuff.

But to me, one of the primary goals of tooling infrastructure should be that it 
is as simple as possible.  This is of course a huge challenge - doing more with 
less is always harder than allowing the code to bloat ad infinitum.  But the 
need for simplicity is particularly acute in something that as central to our 
understanding of WebKit's conformance and correctness as the test harness.  
This is the part of the system that must not be wrong or flaky, because if it 
is, then it diminishes the value of having tests at all, which then leads to 
horrible bugs not getting caught.

 
 Python feels like the wrong technology to use for multithreading.  I'm
 curious if, in the future, next time we rewrite RWT (and I do believe that
 there will be such a next time), we could pick a language that allows this
 more naturally.
 
 It is true that Python's multithreading support is not great (it works
 as long as you don't need to interact with subprocesses, and as long
 as you don't need truly concurrent threads executing python code,
 because of the global interpreter lock). However,much of NRWT is built
 around a shared-nothing message-passing approach; do you think that
 was the wrong architecture, or would you have preferred to use a tool
 that had the same architecture but did it more easily (perhaps in
 something like Erlang)?

I really like approach of shared-nothing message-passing, but was under the 
(possibly mistaken) impression that NRWT has some threading in it.

I just wish that the message passing could be done with less code.  8x less 
code.

I am intrigued by the notion of using Erlang, but worry that it would reduce

Re: [webkit-dev] Running layout tests for the chromium port, using the multi-processing architecture and fully sandboxed

2012-06-08 Thread Jochen Eisinger

On Fri, Jun 8, 2012 at 9:41 PM, Ryosuke Niwa rn...@webkit.org wrote:

On Fri, Jun 8, 2012 at 12:18 PM, Jochen Eisinger joc...@chromium.orgwrote:

I've implemented initial support for running layout tests using
chromium's content_shell instead of test_shell as the current DRT
implementation does. It's still a very experimental, but might already be
interesting for some of you to try.

This is great! Thanks a lot on working this.

1. Make sure your WebKit is at least at r119852 (see
http://trac.webkit.org/wiki/Chromium for prerequisites)
2. Apply the attachment from
https://bugs.webkit.org/show_bug.cgi?id=87045
3. In Source/WebKit/chromium run gclient sync
4. build webkit as usual

E.g. for a debug build on Linux, this should give you
out/Debug/content_shell

You can now run layout tests like this:

new-run-webkit-tests --chromium --debug --driver-name=content_shell
--additional-drt-flag=--dump-render-tree LayoutTests/storage/indexeddb/*

You'll notice that not all tests are passing yet, mainly because not all
(or actually, almost none of the) layoutTestController features are
implemented yet.

Where is layoutTestController implemented? We definitely need to move the
implementation of layoutTestController, eventSender, etc... into WebKit
repository because we often rename functions, etc... in WebKit. It's
unacceptable to require having to modify Chromium code in order to do this
refactoring in the future.

It's currently here:
http://code.google.com/searchframe#OAMlx_jo-ck/src/content/shell/layout_test_controller.jsexact_package=chromium

Per my other thread about sharing IDLs between DumpRenderTree and
WebKitTestRunner, I would like to see us sharing IDL with WebKitTestRunner
instead of adding yet another binding code.

Another missing feature is producing pixel results. However, I'm currently
concentrating on text results, as I think the biggest benefit is the
ability to run storage tests on the real storage implementation.

That sounds great to me but we definitely need to support pixel results
eventually. I'm more than happy to help you on that but that requires the
codebase to be moved into WebKit repository.

Here's the basic problem: content_shell depends on content, so moving this
on the webkit repository would mean that webkit depended on content.

Another solution would be to formalize the test shell API our current
layout test controller in webkit uses (Tools/DumpRenderTree/chromium), and
add a method to chromium's webkit support library that returns a webview
that supports all the hooks required. The webview could then either be
implemented by test_shell or by content_shell

best
-jochen
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev

Re: [webkit-dev] Running layout tests for the chromium port, using the multi-processing architecture and fully sandboxed

On Fri, Jun 8, 2012 at 12:51 PM, Jochen Eisinger joc...@chromium.orgwrote:

 On Fri, Jun 8, 2012 at 9:41 PM, Ryosuke Niwa rn...@webkit.org wrote:

 Per my other thread about sharing IDLs between DumpRenderTree and
 WebKitTestRunner, I would like to see us sharing IDL with WebKitTestRunner
 instead of adding yet another binding code.

 Another missing feature is producing pixel results. However, I'm
 currently concentrating on text results, as I think the biggest benefit is
 the ability to run storage tests on the real storage implementation.


 That sounds great to me but we definitely need to support pixel results
 eventually. I'm more than happy to help you on that but that requires the
 codebase to be moved into WebKit repository.


 Here's the basic problem: content_shell depends on content, so moving this
 on the webkit repository would mean that webkit depended on content.

 Another solution would be to formalize the test shell API our current
 layout test controller in webkit uses (Tools/DumpRenderTree/chromium), and
 add a method to chromium's webkit support library that returns a webview
 that supports all the hooks required. The webview could then either be
 implemented by test_shell or by content_shell


I don't know any architectural details of content_shell so it's hard for me
to comment on this matter, but I don't see a problem in
WebKitChromiumTestRunner (hypothetical name to be consistent with
WebKitTestRunner) depending on content_shell given that WebKitTestRunner
also depends on WebKit2 API.

- Ryosuke
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev

Re: [webkit-dev] can we stop using Skipped files?

On Fri, Jun 8, 2012 at 12:23 PM, Filip Pizlo fpi...@apple.com wrote:

 On Jun 8, 2012, at 12:19 PM, Dirk Pranke wrote:

 On Fri, Jun 8, 2012 at 8:21 AM, Filip Pizlo fpi...@apple.com wrote:

 On Jun 8, 2012, at 4:38 AM, Balazs Kelemen kbal...@webkit.org wrote:

 On 06/08/2012 09:46 AM, Osztrogonac Csaba wrote:
 Hi,

 Dirk Pranke írta:
 I believe most if not all of the ports have started using either
 TestExpectations files or a combination of TestExpectations files
 (except for the Apple Win port).

 Can we explicitly switch to the TestExpectations files at this point
 and drop support for Skipped files on the other ports (and perhaps
 disable old-run-webkit-tests for all but apple win)?

 Until NRWT can't handle cascaded TestExpectations - 
 https://bugs.webkit.org/show_bug.cgi?id=65834,
 Qt port can't drop supporting Skipped files. We have many tests skipped 
 in qt-5.0, qt-5.0-wk1,
 qt-5.0-wk2, wk2 Skipped lists. We can't migrate all of them to the only 
 one TestExpectations.

 And I disagree with disabling ORWT at all. Qt port still support using 
 ORWT locally.
 It is better for gardening than NRWT. NRWT regularly has problems with 
 generating
 new results for a given platform dir (qt,qt-5.0,qt-5.0-wk1,...), it 
 doesn't support
 the good --skipped=only option . If folks don't want to use it, just not 
 use, but
 disabling for everyone by fiat isn't a friendly thing.

 1. These are real weaknesses of nrwt, we should fix them. If gardening is 
 better with orwt (i doubt that is the case, but I don't do gardening 
 regularly), we should improve nrwt, i.e. reimplement features from orwt.

 I applaud your enthusiasm.

 2. I believe basically everybody agrees that we should drop orwt, except 
 you Ossy. Maybe I'm wrong. So, is there anybody still want to have support 
 for orwt? If so, why?

 I'm with Ossy on this.

 Getting rid of ORWT would be a show stopper for me.

 Can you file bugs or give me examples of why this is true? Are there
 specific features missing, or bugs you're hitting?

 Here's a really nasty one, that has been sitting untouched since November:

 https://bugs.webkit.org/show_bug.cgi?id=71634


That bug had fallen off my radar, if it was ever on it (sorry! looks
like this was filed when I was working on chromium side things).

I've now created a tracking bug -
https://bugs.webkit.org/show_bug.cgi?id=88680 - to track all of the
issues I am aware of that we should fix before deleting ORWT. I've
added this, and the bug ossy mentioned about Qt platform baselines,
and the cascading expectations bug.

I am doing another pass over all the open bugs I've seen that are
NRWT-related, and will add more, but if anyone has any others you'd
like to add, please do so.

There is a separate tracking bug -
https://bugs.webkit.org/show_bug.cgi?id=64491 , polish NRWT until it
shines, that I would prefer to use for any non-show-stopping bugs
(this should not be taken to think those bugs are not important, mind
you).

-- Dirk
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev

Re: [webkit-dev] can we stop using Skipped files?

On Fri, Jun 8, 2012 at 12:50 PM, Filip Pizlo fpi...@apple.com wrote:

 On Jun 8, 2012, at 12:31 PM, Dirk Pranke wrote:
   On Fri, Jun 8, 2012 at 10:56 AM, Filip Pizlo fpi...@apple.com wrote:
 
  It's a lot harder to dive into, a lot more cumbersome to improve, and
 not
  any easier to maintain.
 
 
  I definitely agree that NRWT is more complicated than it seems like it
  should be; it got contorted as we added all the features we needed to
  add, and I have been in a simplify mode over the past few months. I
  would welcome any feedback where you think things are overly complex.

 This is a difficult question - it's unfortunately easier to observe that
 something is complex, than it is to pinpoint why it is complex.  But I will
 try.

 1) Code locality.  I can open Tools/Scripts/old-run-webkit-tests and
 pretty rapidly discover (a) how options are parsed, (b) how platform
 differences are handled, (c) how tests are found, and (d) how tests are
 run.  I can hack all of this code because it's all in one place.  I don't
 have to be a domain expert to do it.  Hell, I don't even have to be good a
 Perl to find my way around.

 2) Code size:

 [pizlo@wartooth OpenSource] wc Tools/Scripts/old-run-webkit-tests
2796   10316   98733 Tools/Scripts/old-run-webkit-tests

 [pizlo@wartooth OpenSource] wc Tools/Scripts/new-run-webkit-tests
 Tools/Scripts/webkitpy/layout_tests/*.py
 Tools/Scripts/webkitpy/layout_tests/*/*.py
  bunch of stuff
   23197 91897 1049914 total

 That's a *HUGE* difference.  Consider that NRWT just adds only one thing
 that most people care about: parallelism.  Is an 8x increase in code size
 justified?

 I know that LoC metrics are evil in most cases.  But this is not most
 cases.  This is an order-of-magnitude difference.  That's 8x more code I
 have to look at to find what I want.  That's 8x more code that I
 potentially have to edit to add a feature.  That's 8x more code that could
 have a bug.  And so on.  Badness!


Part of that comes from the fact layout_tests directory also includes port
classes, which knows various aspects of each port, and they're also used in
webkit-patch and other Python tools.

But the act of running layout tests is in itself a regression test of the
 layout test harness just as much as it is a regression test of WebKit.

 So the unit tests are superfluous.  In particular, if I had to pick
 between only having unit tests or only having regression tests, I might
 pick unit tests.  But if I already have regression tests then I'm unlikely
 to want to incur technical debt to build unit tests, particularly since
 unit tests requiring changing the infrastructure to make the code more
 testable, which then leads to the problems listed above.


I agree that we've accumulated way too many unit tests in webkitpy and some
of our unit test code is hideous but I think it's quite unrealistic not to
have any unit tests. We've had many regressions in the past, and it's hard
to make sure your code works on all platforms without tests. So it's the
balance that's important.

I am intrigued by the notion of using Erlang, but worry that it would
 reduce effective hackability due to there being less Erlang experience in
 the universe.  I also don't want the porting of Erlang's runtime to be a
 gating factor for porting WebKit itself.  Hence, I fear that we should
 stick to broadly accepted languages, like Python, or Perl, or if need be,
 C++.


If I were to write a new test runner, I'd use C++ without a question. It
has solid threading support in almost all operating systems, is statically
typed, and virtually everyone who contributes to WebKit can read/write well.

- Ryosuke
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev

Re: [webkit-dev] can we stop using Skipped files?


On Jun 8, 2012, at 1:04 PM, Ryosuke Niwa wrote:

 On Fri, Jun 8, 2012 at 12:50 PM, Filip Pizlo fpi...@apple.com wrote:
 On Jun 8, 2012, at 12:31 PM, Dirk Pranke wrote:
   On Fri, Jun 8, 2012 at 10:56 AM, Filip Pizlo fpi...@apple.com wrote:
 
  It's a lot harder to dive into, a lot more cumbersome to improve, and not
  any easier to maintain.
 
 
  I definitely agree that NRWT is more complicated than it seems like it
  should be; it got contorted as we added all the features we needed to
  add, and I have been in a simplify mode over the past few months. I
  would welcome any feedback where you think things are overly complex.
 
 This is a difficult question - it's unfortunately easier to observe that 
 something is complex, than it is to pinpoint why it is complex.  But I will 
 try.
 
 1) Code locality.  I can open Tools/Scripts/old-run-webkit-tests and pretty 
 rapidly discover (a) how options are parsed, (b) how platform differences are 
 handled, (c) how tests are found, and (d) how tests are run.  I can hack all 
 of this code because it's all in one place.  I don't have to be a domain 
 expert to do it.  Hell, I don't even have to be good a Perl to find my way 
 around.
 
 2) Code size:
 
 [pizlo@wartooth OpenSource] wc Tools/Scripts/old-run-webkit-tests
2796   10316   98733 Tools/Scripts/old-run-webkit-tests
 
 [pizlo@wartooth OpenSource] wc Tools/Scripts/new-run-webkit-tests 
 Tools/Scripts/webkitpy/layout_tests/*.py 
 Tools/Scripts/webkitpy/layout_tests/*/*.py
  bunch of stuff
   23197 91897 1049914 total
 
 That's a *HUGE* difference.  Consider that NRWT just adds only one thing that 
 most people care about: parallelism.  Is an 8x increase in code size 
 justified?
 
 I know that LoC metrics are evil in most cases.  But this is not most cases.  
 This is an order-of-magnitude difference.  That's 8x more code I have to look 
 at to find what I want.  That's 8x more code that I potentially have to edit 
 to add a feature.  That's 8x more code that could have a bug.  And so on.  
 Badness!
 
 Part of that comes from the fact layout_tests directory also includes port 
 classes, which knows various aspects of each port, and they're also used in 
 webkit-patch and other Python tools.
 
 But the act of running layout tests is in itself a regression test of the 
 layout test harness just as much as it is a regression test of WebKit.
 
 So the unit tests are superfluous.  In particular, if I had to pick between 
 only having unit tests or only having regression tests, I might pick unit 
 tests.  But if I already have regression tests then I'm unlikely to want to 
 incur technical debt to build unit tests, particularly since unit tests 
 requiring changing the infrastructure to make the code more testable, which 
 then leads to the problems listed above.
 
 I agree that we've accumulated way too many unit tests in webkitpy and some 
 of our unit test code is hideous but I think it's quite unrealistic not to 
 have any unit tests. We've had many regressions in the past, and it's hard to 
 make sure your code works on all platforms without tests. So it's the balance 
 that's important.

I think you've had regressions because the code is too complex.

 
 I am intrigued by the notion of using Erlang, but worry that it would reduce 
 effective hackability due to there being less Erlang experience in the 
 universe.  I also don't want the porting of Erlang's runtime to be a gating 
 factor for porting WebKit itself.  Hence, I fear that we should stick to 
 broadly accepted languages, like Python, or Perl, or if need be, C++.
 
 If I were to write a new test runner, I'd use C++ without a question. It has 
 solid threading support in almost all operating systems, is statically typed, 
 and virtually everyone who contributes to WebKit can read/write well.

I would try to avoid using C++ unless we really had to.  The goal here is 
simplicity, and C++ is not the best for that.

 
 - Ryosuke
 

___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev

Re: [webkit-dev] Running layout tests for the chromium port, using the multi-processing architecture and fully sandboxed

2012-06-08 Thread Jochen Eisinger

On Fri, Jun 8, 2012 at 9:55 PM, Ryosuke Niwa rn...@webkit.org wrote:

 On Fri, Jun 8, 2012 at 12:51 PM, Jochen Eisinger joc...@chromium.orgwrote:

  On Fri, Jun 8, 2012 at 9:41 PM, Ryosuke Niwa rn...@webkit.org wrote:

 Per my other thread about sharing IDLs between DumpRenderTree and
 WebKitTestRunner, I would like to see us sharing IDL with WebKitTestRunner
 instead of adding yet another binding code.

 Another missing feature is producing pixel results. However, I'm
 currently concentrating on text results, as I think the biggest benefit is
 the ability to run storage tests on the real storage implementation.


 That sounds great to me but we definitely need to support pixel results
 eventually. I'm more than happy to help you on that but that requires the
 codebase to be moved into WebKit repository.


 Here's the basic problem: content_shell depends on content, so moving
 this on the webkit repository would mean that webkit depended on content.

 Another solution would be to formalize the test shell API our current
 layout test controller in webkit uses (Tools/DumpRenderTree/chromium), and
 add a method to chromium's webkit support library that returns a webview
 that supports all the hooks required. The webview could then either be
 implemented by test_shell or by content_shell


 I don't know any architectural details of content_shell so it's hard for
 me to comment on this matter, but I don't see a problem in
 WebKitChromiumTestRunner (hypothetical name to be consistent with
 WebKitTestRunner) depending on content_shell given that WebKitTestRunner
 also depends on WebKit2 API.


Today, when somebody adds e.g. a new callback to LayoutTestController, they
can just patch DumpRenderTree. If our test runner lived (in large parts) in
chromium's repository, such a change would require a patch to chromium.
This would either put an additional burden on all WebKit developers, or our
test runner constantly gets out of sync.



 - Ryosuke


___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev

Re: [webkit-dev] can we stop using Skipped files?

On Fri, Jun 8, 2012 at 1:06 PM, Filip Pizlo fpi...@apple.com wrote:


 On Jun 8, 2012, at 1:04 PM, Ryosuke Niwa wrote:

 I agree that we've accumulated way too many unit tests in webkitpy and
 some of our unit test code is hideous but I think it's quite unrealistic
 not to have any unit tests. We've had many regressions in the past, and
 it's hard to make sure your code works on all platforms without tests. So
 it's the balance that's important.


 I think you've had regressions because the code is too complex.


I don't think that's true from my experience working on webkitpy so far.
The root of problem is that we support way too many configurations 
platforms, and Chromium port has had a completely different test runner
program called test_shell. We've mostly transitioned (except Mac?) to
DumpRenderTree at this point, but that has added a lot of code complexity
(it probably accounts 20-30% if not 40-50% of all the code in layout_tests).

We also have a lot of abstractions around filesystem, etc... in
webkitpy/common, which I'm not a big fun of. I think some of that is
inherited from the era where we had to use some features that weren't
supported in Python 2.5. Now that we no longer support Python 2.5, we can
probably cleanup some of that mess.

  I am intrigued by the notion of using Erlang, but worry that it would
 reduce effective hackability due to there being less Erlang experience in
 the universe.  I also don't want the porting of Erlang's runtime to be a
 gating factor for porting WebKit itself.  Hence, I fear that we should
 stick to broadly accepted languages, like Python, or Perl, or if need be,
 C++.


 If I were to write a new test runner, I'd use C++ without a question. It
 has solid threading support in almost all operating systems, is statically
 typed, and virtually everyone who contributes to WebKit can read/write well.


 I would try to avoid using C++ unless we really had to.  The goal here is
 simplicity, and C++ is not the best for that.


Can you think of any other programming language that has a solid threading
support and everyone in the WebKit community knows how to read/write?  Like
you implied, I would never ever use Java for example.

- Ryosuke
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev

Re: [webkit-dev] Running layout tests for the chromium port, using the multi-processing architecture and fully sandboxed

On Fri, Jun 8, 2012 at 1:09 PM, Jochen Eisinger joc...@chromium.org wrote:

 On Fri, Jun 8, 2012 at 9:55 PM, Ryosuke Niwa rn...@webkit.org wrote:

 On Fri, Jun 8, 2012 at 12:51 PM, Jochen Eisinger joc...@chromium.orgwrote:

  On Fri, Jun 8, 2012 at 9:41 PM, Ryosuke Niwa rn...@webkit.org wrote:

 Per my other thread about sharing IDLs between DumpRenderTree and
 WebKitTestRunner, I would like to see us sharing IDL with WebKitTestRunner
 instead of adding yet another binding code.

 Another missing feature is producing pixel results. However, I'm
 currently concentrating on text results, as I think the biggest benefit is
 the ability to run storage tests on the real storage implementation.


 That sounds great to me but we definitely need to support pixel results
 eventually. I'm more than happy to help you on that but that requires the
 codebase to be moved into WebKit repository.


 Here's the basic problem: content_shell depends on content, so moving
 this on the webkit repository would mean that webkit depended on content.

 Another solution would be to formalize the test shell API our current
 layout test controller in webkit uses (Tools/DumpRenderTree/chromium), and
 add a method to chromium's webkit support library that returns a webview
 that supports all the hooks required. The webview could then either be
 implemented by test_shell or by content_shell


 I don't know any architectural details of content_shell so it's hard for
 me to comment on this matter, but I don't see a problem in
 WebKitChromiumTestRunner (hypothetical name to be consistent with
 WebKitTestRunner) depending on content_shell given that WebKitTestRunner
 also depends on WebKit2 API.


 Today, when somebody adds e.g. a new callback to LayoutTestController,
 they can just patch DumpRenderTree. If our test runner lived (in large
 parts) in chromium's repository, such a change would require a patch to
 chromium. This would either put an additional burden on all WebKit
 developers, or our test runner constantly gets out of sync.


Yes, that's why I'm suggesting that this new test runner needs to be in the
WebKit repository. It's not acceptable to require WebKit contributors to
modify code in the Chromium repository in order to refactor or add or
remove features to the test runner.

- Ryosuke
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev

Re: [webkit-dev] can we stop using Skipped files?

On Fri, Jun 8, 2012 at 12:50 PM, Filip Pizlo fpi...@apple.com wrote:
 On Jun 8, 2012, at 12:31 PM, Dirk Pranke wrote:

 On Fri, Jun 8, 2012 at 10:56 AM, Filip Pizlo fpi...@apple.com wrote:

 It's a lot harder to dive into, a lot more cumbersome to improve, and not
 any easier to maintain.


 I definitely agree that NRWT is more complicated than it seems like it
 should be; it got contorted as we added all the features we needed to
 add, and I have been in a simplify mode over the past few months. I
 would welcome any feedback where you think things are overly complex.

 This is a difficult question - it's unfortunately easier to observe that 
 something is complex, than it is to pinpoint why it is complex.  But I will 
 try.


Thanks! Believe you me, I've spent a lot of time pondering this with
this codebase myself.

 1) Code locality.  I can open Tools/Scripts/old-run-webkit-tests and pretty 
 rapidly discover (a) how options are parsed, (b) how platform differences are 
 handled, (c) how tests are found, and (d) how tests are run.  I can hack all 
 of this code because it's all in one place.  I don't have to be a domain 
 expert to do it.  Hell, I don't even have to be good a Perl to find my way 
 around.


Yup, that's a downside to having things in multiple files, all right.
More on this below ...

 2) Code size:

 [pizlo@wartooth OpenSource] wc Tools/Scripts/old-run-webkit-tests
    2796   10316   98733 Tools/Scripts/old-run-webkit-tests

 [pizlo@wartooth OpenSource] wc Tools/Scripts/new-run-webkit-tests 
 Tools/Scripts/webkitpy/layout_tests/*.py 
 Tools/Scripts/webkitpy/layout_tests/*/*.py
  bunch of stuff
   23197 91897 1049914 total

 That's a *HUGE* difference.  Consider that NRWT just adds only one thing that 
 most people care about: parallelism.  Is an 8x increase in code size 
 justified?

 I know that LoC metrics are evil in most cases.  But this is not most cases.  
 This is an order-of-magnitude difference.  That's 8x more code I have to look 
 at to find what I want.  That's 8x more code that I potentially have to edit 
 to add a feature.  That's 8x more code that could have a bug.  And so on.  
 Badness!


About half of that code is tests. The actual code to do the
parallelism is about ~500 lines; I have patches posted somewhere that
I haven't gotten around to landing that reduce that to about ~250
lines (or less, I don't recall exactly)

Much of the additional code actually is for other things. There's a
lot more options for logging and gathering statistics -- most of it
probably unneeded -- and there's a *lot* of complexity added for
handling all of the different test expectations options. There's
probably other stuff that can be deleted or simplified. It would be
interesting to break that down, and get the code more structured into
the stuff you need to be aware of and the stuff you can ignore.


 But the act of running layout tests is in itself a regression test of the 
 layout test harness just as much as it is a regression test of WebKit.

 So the unit tests are superfluous.

Running the layout tests is a regression test, but both ORWT and NRWT
have dozens of options. Do you know which ones are used, and in which
combinations? Do you actually run all of them? If not, are you willing
to let the bots and other devs be your testers?

There's actually a single file of integration tests that covers NRWT
pretty well (it basically documented the above). However, others have
found that they want unit tests for the different modules to make
things more individually testable (and in some cases, you need unit
tests for things that aren't easily tested via integration tests), so
that's how we get so many tests.

 I really like approach of shared-nothing message-passing, but was under the 
 (possibly mistaken) impression that NRWT has some threading in it.

It did use threads heavily at one point, and the threading didn't work
right, which led me to rewrite it into the current architecture. We do
actually still use threads in one case, to support the --singly flag
(we used to need threads for this with chromium), but we don't even
need that any more.

-- Dirk
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev

[webkit-dev] Using the WebExposed keyword for web-facing changes

2012-06-08 Thread Peter Beverloo

Hi webkit-dev,

*If you work on web-facing features, or run into another bug which does,
please consider adding the WebExposed keyword to it.*

Many of you will be familiar with my WebKit updates, which are now also
being published on the WebKit blog. Writing these involves reading every
commit that lands in WebKit's repository. Last year, May counted 2,126
revisions. This year there were 3,068. As a result of the steep increase in
volume, it's becoming increasingly hard for all parties to keep up --
Web(Kit) developers, ports and all other interested parties. Not every one
of them has time to read through all changes.

In an effort to increase visibility of Web Platform facing changes, I'd
like to introduce the WebExposed keyword. It is intended to be applied to
any bug which introduces, modifies (including behavioral changes) or
removes features important to Web developers, such as DOM properties and
methods, JavaScript features and CSS properties and values.

https://bugs.webkit.org/buglist.cgi?keywords=WebExposed

Increased visibility of these changes has a number of advantages.

Firstly, Web Developers will have more insight in what's happening in
WebKit. The changes will be visible on the bug list itself, but also
through the RSS feed Bugzilla will curate for it.

Furthermore, it may be used as an overview for ports to keep track of the
web-facing changes which happen during their release cycles, and it will
also come in useful for evangelizing features, writing documentation and
managing outreach. With work being done by many vendors in many areas of
WebCore, I'm hoping the keyword can become a valuable aid in this respect.

If you work on web-facing features, or run into another bug which does,
please consider adding the WebExposed keyword to it. This of course isn't
mandatory, but it will help others who are interested in keeping track.

Thanks,
Peter
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev

Re: [webkit-dev] can we stop using Skipped files?

On Fri, Jun 8, 2012 at 1:16 PM, Ryosuke Niwa rn...@webkit.org wrote:
 I don't think that's true from my experience working on webkitpy so far. The
 root of problem is that we support way too many configurations  platforms,
 and Chromium port has had a completely different test runner program called
 test_shell. We've mostly transitioned (except Mac?) to DumpRenderTree at
 this point, but that has added a lot of code complexity (it probably
 accounts 20-30% if not 40-50% of all the code in layout_tests).

Nope, supporting the old test-shell mode is only about 300 lines of
code :). We support a lot of configurations and platforms, of course,
but so does ORWT.

 We also have a lot of abstractions around filesystem, etc... in
 webkitpy/common, which I'm not a big fun of. I think some of that is
 inherited from the era where we had to use some features that weren't
 supported in Python 2.5. Now that we no longer support Python 2.5, we can
 probably cleanup some of that mess.

Most of these abstractions were either added to make testing easier
(and faster since we didn't have to write to a real filesystem) or to
make the code more portable (e.g., to deal with the differences
between windows filenames and unix filenames). Python 2.4 and 2.5 had
little to do with it. These abstractions are used by the other tools
like webkit-patch as well.

-- Dirk
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev

Re: [webkit-dev] can we stop using Skipped files?

On Fri, Jun 8, 2012 at 1:27 PM, Dirk Pranke dpra...@chromium.org wrote:

 Most of these abstractions were either added to make testing easier
 (and faster since we didn't have to write to a real filesystem)


That sounds like a bad idea.

- Ryosuke
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev

Re: [webkit-dev] can we stop using Skipped files?

On Fri, Jun 8, 2012 at 1:41 PM, Ojan Vafai o...@chromium.org wrote:

 On Fri, Jun 8, 2012 at 1:37 PM, Ryosuke Niwa rn...@webkit.org wrote:

 On Fri, Jun 8, 2012 at 1:27 PM, Dirk Pranke dpra...@chromium.org wrote:

 Most of these abstractions were either added to make testing easier
 (and faster since we didn't have to write to a real filesystem)


 That sounds like a bad idea.


 FWIW, I thought so at first as well, but as I've worked with the
 FileSystem abstraction, I've found it to be very helpful and provide almost
 no extra abstraction burden since it tries to closely match the Python APIs.


It's extra code we have to read. For testing, we can just directly mock the
native filesystem classes  methods Python provides.

In general, I agree with Filip's sentiment that we have way too much code.

- Ryosuke
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev

Re: [webkit-dev] Defaulting --exit-after-n-failures to 0

On Fri, Jun 8, 2012 at 1:38 PM, Ojan Vafai o...@chromium.org wrote:

 On Fri, Jun 8, 2012 at 12:46 PM, Dirk Pranke dpra...@chromium.org wrote:

 On Fri, Jun 8, 2012 at 12:44 PM, Ryosuke Niwa rn...@webkit.org wrote:
  On Fri, Jun 8, 2012 at 12:37 PM, Dirk Pranke dpra...@chromium.org
 wrote:
 
  I have no objection either to increasing the defaults for either of
  these numbers or making it possible to have different defaults per
  port.
 
  Do you want to suggest different defaults? Should we use ORWT's
  (infinite failures and infinite crashes by default)?
 
 
  Unbounded failures  crashes sound good to me although I can definitely
 see
  a value in passing some fixed value on bots so as to reduce the cycle
 time
  when 10,000+ tests start to crash. Something like 100, 300, or 500 would
  probably be acceptable.
 

 NRWT defaults to 500 failures, so you're suggesting we just up the
 default # of crashes to 500? Anyone have any other numbers jumping out
 at them?


 I'm OK with the ORWT defaults of infinite for both. Then we never have to
 have this discussion again. :) It can be on the buildbot maintainers to
 pass in the right flags. All of the non-chromium ports already do anyways
 as best I can tell.


That's an argument I can buy.

- Ryosuke
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev

Re: [webkit-dev] can we stop using Skipped files?

2012-06-08 Thread Zoltan Herczeg

Hi Dirk,

 At any rate, I believe we are definitely open to adding new features;
 feel free to suggest them or work on them!

I am happy to hear that.

https://bugs.webkit.org/show_bug.cgi?id=88680

This is definitely a right step! And it looks like still a lot of things
to do before NRWT reach ORWT level.

We decided to have a meeting in the beginning of the next week where we
discuss the needs of our bots and devices (like parallel layout testing on
multiple ARM devices). We can send you a summary, and decide the best way
to add them to NRWT. We always think RWT as a swiss knife not an end user
software.

Regards,
Zoltan


___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev

[webkit-dev] GTK+ port's help needed to get rid of FAIL test expectation