On Fri, Oct 26, 2012 at 11:38 AM, Ryosuke Niwa wrote:
> On Fri, Oct 26, 2012 at 11:33 AM, Elliott Sprehn
> wrote:
>>
>> On Fri, Oct 26, 2012 at 11:17 AM, Ryosuke Niwa wrote:
>> > ...
>> >
>> > I agree this is a good change but it appears that we should add more
>> > cache/loader tests before cha
up and doing the equivalent (clear caches
> between tests) for the mac and/or gtk ports' DRTs?
>
>
> On Wed, Aug 8, 2012 at 2:35 PM, Dirk Pranke wrote:
>>
>> On Wed, Aug 8, 2012 at 10:47 AM, Ojan Vafai wrote:
>> > See https://bugs.webkit.org/show_bug.cgi?id=9319
eature. I suspect that the zip size from the bots has just
>> ballooned above that cut-off and fixing LayoutTestResults to only
>> include the relevant files would probably make the EWSes start
>> uploading results.zip files again. :)
>>
>> On Wed, Oct 24, 2012 at 12:33
I think there is some general interest in a feature like this.
I need to sync up w/ Adam and Eric and figure out what all might be
required here, so we might do this but I can't say for sure yet ...
-- Dirk
On Wed, Oct 24, 2012 at 8:40 AM, Adam Barth wrote:
> I don't have any current plans to i
On Sun, Oct 21, 2012 at 2:17 PM, Maciej Stachowiak wrote:
>
> Apple did not ship the last release of Safari to SnowLeopard and we have no
> plans to maintain SnowLeopard support on trunk. We haven't actively ripped
> out SL-specific ifdefs because we were under the impression that the Chromium
aph is now finally a tree?? :)
>>
>> https://docs.google.com/drawings/d/1z65SKkWrD4Slm6jobIphHwwRADyUtjOAxwGBVKBY8Kc/edit
>>
>> On Wed, Oct 17, 2012 at 8:15 PM, Dirk Pranke wrote:
>>> All of the Chromium ports will use baselines in their port-specific
>>> directo
All of the Chromium ports will use baselines in their port-specific
directories, then fall back through various paths to platform/chromium
and then to next to the test.
Which means you Apple folks can feel free to break things in
platform/mac to your hearts' content :).
Let me know if you see any
Hi all,
You can largely ignore this if you never modify Chromium baselines or
TestExpectations. But if you do modify them ...
I'm working on getting our new 10.8/MountainLion bot green. It looks
like Apple changed their text rendering a bit in the latest release
and so we need lots more baselines
Makes sense to me ...
On Thu, Oct 4, 2012 at 2:46 PM, Ojan Vafai wrote:
> TL;DR: We should add a NeedsRebaseline keyword to TestExpectations and add
> garden-o-matic tooling for it for the cases where someone commits a
> change/test that they know will need new results for different ports (e.g.
>
On Wed, Oct 3, 2012 at 4:18 PM, Dirk Pranke wrote:
> On Wed, Oct 3, 2012 at 3:39 PM, Darin Adler wrote:
>> On Oct 3, 2012, at 9:10 AM, Ryosuke Niwa wrote:
>>
>>> Could you tell us what your directory structure look like and where you're
>>> executing tha
On Wed, Oct 3, 2012 at 3:39 PM, Darin Adler wrote:
> On Oct 3, 2012, at 9:10 AM, Ryosuke Niwa wrote:
>
>> Could you tell us what your directory structure look like and where you're
>> executing that command? Looks like a path confusion.
>
> My WebKit source tree is in ~/Safari/OpenSource and my
Think Eric and Alexey identified the problem here ...
-- Dirk
On Wed, Oct 3, 2012 at 9:44 AM, Alexey Proskuryakov wrote:
>
> 03.10.2012, в 9:19, Eric Seidel написал(а):
>
>> Perhaps make is missing? or the java directory its trying to build is
>> missing and it's just printing the wrong path?
>
cleanup, but of course I'll be happy
to troubleshoot/answer questions, etc :).
-- Dirk
On Thu, Sep 27, 2012 at 12:24 PM, Dirk Pranke wrote:
> Now landed in http://trac.webkit.org/changeset/129788 . Let me know if
> there are any problems.
>
> -- Dirk
>
> On Thu, Sep 27, 2012 at
Now landed in http://trac.webkit.org/changeset/129788 . Let me know if
there are any problems.
-- Dirk
On Thu, Sep 27, 2012 at 10:15 AM, Dirk Pranke wrote:
> Last call on this ... I will be removing the files later today unless
> something goes awry :).
>
> -- Dirk
>
> On Fri,
Last call on this ... I will be removing the files later today unless
something goes awry :).
-- Dirk
On Fri, Sep 21, 2012 at 3:41 PM, Dirk Pranke wrote:
> Hi all,
>
> Now that we're on the new TestExpectations syntax, I'm planning to
> remove the Skipped files (and supp
On a related note, there is a gradual movement to pass more command
line flags along with each test to DRT during run-webkit-tests (to
toggle between pixel tests and non, change timeout values, etc.), and
being able to ensure we reset the state to the default between each
test is kinda necessary fo
Hi all,
Now that we're on the new TestExpectations syntax, I'm planning to
remove the Skipped files (and support for them from NRWT).
I have a change posted that will add minimal support for the syntax to
old-run-webkit-tests in https://bugs.webkit.org/show_bug.cgi?id=97276
; basically any file o
As of http://trac.webkit.org/changeset/129265 .
(There are still some things to clean up in the code now that the old
syntax is gone, but hardly anyone will care about that).
I am not aware of any open bugs or issues with the new syntax. Speak
now or I'll assume we're all good :).
Cheers,
-- Di
gt;> > - WBR, Alexey Proskuryakov
>> >
>> >
>> > 20.09.2012, в 3:54, Osztrogonac Csaba написал(а):
>> >
>> >> Unfortunately r129047 broke the results.html, see
>> >> https://bugs.webkit.org/show_bug.cgi?id=96845#c9 for details.
>> >&
The URL is already optional. However, I would encourage people to
actually create useful bugs rather than not adding comments to the
TestExpectations/Skipped files.
I'm not a big fan of having to look in two different places to figure
out what's broken, and having bugs for things makes it easier t
Assuming my changes stick, all of the TestExpectations files in the
repo have been converted to the new syntax. The old syntax is still
supported as well, but please don't use it :).
http://trac.webkit.org/wiki/TestExpectations
At the moment I'm chasing down some minor issues but I haven't notice
Apologies for the inconvenience.
-- Dirk
On Wed, Sep 12, 2012 at 4:29 PM, Dirk Pranke wrote:
> Hi all,
>
> The new format of the much-debated TestExpectations syntax will be
> landing soon (hopefully in the next couple days).
>
> For those of who have forgotten / repressed th
On Wed, Sep 19, 2012 at 2:00 PM, Dirk Pranke wrote:
> On Wed, Sep 19, 2012 at 1:54 PM, Allan Sandfeld Jensen
> wrote:
>> On Wednesday 19 September 2012, Dirk Pranke wrote:
>>> After some limited amount of thought, I'm inclined to agree with
>>> Ryosuke and Oss
On Wed, Sep 19, 2012 at 1:54 PM, Allan Sandfeld Jensen
wrote:
> On Wednesday 19 September 2012, Dirk Pranke wrote:
>> After some limited amount of thought, I'm inclined to agree with
>> Ryosuke and Ossy here. If we have tests that don't depend on checking
>>
After some limited amount of thought, I'm inclined to agree with
Ryosuke and Ossy here. If we have tests that don't depend on checking
the metrics, can they just be dumpAsText tests or reftests instead?
I thought the initial motivation for --ignore-metrics was for new
ports to at least confirm tha
On Fri, Sep 14, 2012 at 9:47 AM, Sergio Villar Senin wrote:
> En 14/09/12 18:34, Sergio Villar Senin escribiu:
>> En 13/09/12 01:29, Dirk Pranke escribiu:
>>> Hi all,
>>>
>>> The new format of the much-debated TestExpectations syntax will be
>>> la
On Thu, Sep 13, 2012 at 12:32 PM, Ryosuke Niwa wrote:
> Hi,
>
> This discussion came out of https://bugs.webkit.org/show_bug.cgi?id=96569.
>
> Should we allow URLs other than webkit.org/b/ to be used in
> TestExpectations?
>
> I vote for yes, and in fact, adding new URLs should be easy. What do yo
On Wed, Sep 12, 2012 at 11:00 PM, Maciej Stachowiak wrote:
>
> On Sep 12, 2012, at 10:36 PM, Ojan Vafai wrote:
>
> On Wed, Sep 12, 2012 at 6:40 PM, Maciej Stachowiak wrote:
>>
>> - Is this approach substantially less time and effort than adding a
>> histogram-style metric? I expect you have adde
Also, for anyone wondering, the "check in failing test baselines"
feature will be landing shortly after this sticks. I'll send out
another note with details on that when we get there.
-- Dirk
On Wed, Sep 12, 2012 at 4:29 PM, Dirk Pranke wrote:
> Hi all,
>
> The new fo
Hi all,
The new format of the much-debated TestExpectations syntax will be
landing soon (hopefully in the next couple days).
For those of who have forgotten / repressed the earlier debates, the
new syntax looks something like:
webkit.org/b/12345 [ Mac Vista] fast/html/keygen.html [ ImageOnlyFail
On Tue, Aug 21, 2012 at 4:16 PM, Maciej Stachowiak wrote:
>
> On Aug 21, 2012, at 3:23 PM, Ojan Vafai wrote:
>
> On Mon, Aug 20, 2012 at 6:03 PM, Maciej Stachowiak wrote:
>>
>> Here's how I imagine the workflow when a sheriff or just innocent
>> bystander notices a deterministically failing test
Sorry for the delays in getting back to this ... it's been a busy week.
On Mon, Aug 20, 2012 at 6:03 PM, Maciej Stachowiak wrote:
>
> Sorry, I overlooked these questions earlier.
>
> On Aug 17, 2012, at 7:36 PM, Dirk Pranke wrote:
>
>> I'm not sure if I lik
The Chromium canaries now exit after 5000 failures or 1000 crashes/timeouts.
Ideally we'd hold off on this change until we can get some sort of a
fix or workaround to https://bugs.webkit.org/show_bug.cgi?id=94665
though (and I'm working on this today), or life might be annoyingly
painful for us.
On Sat, Aug 18, 2012 at 8:31 PM, Filip Pizlo wrote:
>
> On Aug 18, 2012, at 5:55 PM, Maciej Stachowiak wrote:
>
>>
>> On Aug 18, 2012, at 5:11 PM, Filip Pizlo wrote:
>>
>>> Maybe at this point we can agree to let Dirk land some variant of this with
>>> whatever half-way sensible name (any of th
I'm not sure if I like this idea or not. A couple of observations/questions ...
1) I wouldn't want to call it '-correct' unless we were sure it was
correct; '-previous' is better in that regard
2) the issue with keeping a '-correct' in the tree is that it's quite
possible for a previous correct e
On Fri, Aug 17, 2012 at 5:43 PM, Filip Pizlo wrote:
> Then I am on board.
>
> We still do need to revisit the handling of flaky tests. The current
> approach is an absolute disaster. (I normally love exaggerating, but in this
> case, I feel no satisfaction in doing so because it is at best an
All non-flaky failures, yes.
Flaky tests would still require entries in the TestExpectations files
at this time; discussion of how to treat them is a separate topic.
-- Dirk
On Fri, Aug 17, 2012 at 5:35 PM, Filip Pizlo wrote:
> +1, contingent upon the following: are we agreeing that all current
On Fri, Aug 17, 2012 at 5:01 PM, Ryosuke Niwa wrote:
> On Fri, Aug 17, 2012 at 4:55 PM, Ojan Vafai wrote:
>>
>> Asserting a test case is 100% correct is nearly impossible for a large
>> percentage of tests. The main advantage it gives us is the ability to have
>> -expected mean "unsure".
>>
>> Le
On Fri, Aug 17, 2012 at 11:29 AM, Ryosuke Niwa wrote:
> On Fri, Aug 17, 2012 at 11:06 AM, Dirk Pranke wrote:
>>
>> > On the other hand, the pixel test output that's correct to one expert
>> > may
>> > not be correct to another expert. For example, I
On Fri, Aug 17, 2012 at 8:07 AM, Ryosuke Niwa wrote:
> On Thu, Aug 16, 2012 at 6:11 PM, Dirk Pranke wrote:
>>
>> On Thu, Aug 16, 2012 at 5:41 PM, Ryosuke Niwa wrote:
>> > On Thu, Aug 16, 2012 at 5:18 PM, Dirk Pranke
>> > wrote:
>> >>
>>
On Thu, Aug 16, 2012 at 5:41 PM, Ryosuke Niwa wrote:
> On Thu, Aug 16, 2012 at 5:18 PM, Dirk Pranke wrote:
>>
>> On Thu, Aug 16, 2012 at 3:50 PM, Stephen Chenney
>> wrote:
>> > I agree with the priorities above, at least. I also agree with the
>> > overall
On Thu, Aug 16, 2012 at 3:50 PM, Stephen Chenney wrote:
> On Thu, Aug 16, 2012 at 6:15 PM, Ryosuke Niwa wrote:
>>
>> On Thu, Aug 16, 2012 at 3:04 PM, Dirk Pranke wrote:
>>>
>>> On Thu, Aug 16, 2012 at 2:23 PM, Ryosuke Niwa wrote:
>>> > Like Filip,
On Thu, Aug 16, 2012 at 2:23 PM, Ryosuke Niwa wrote:
> On Thu, Aug 16, 2012 at 2:05 PM, Dirk Pranke wrote:
>>
>> I think your observations are correct, but at least my experience as a
>> gardener/sheriff leads me to a different conclusion. Namely, when I'm
>> loo
On Wed, Aug 15, 2012 at 6:02 PM, Filip Pizlo wrote:
>
> 2) Possibility of the sheriff getting it wrong.
>
> (2) concerns me most. We're talking about using filenames to serve as a
> kind of unchecked comment. We already know that comments are usually bad
> because there is no protection against
On Wed, Aug 15, 2012 at 5:19 PM, Ryosuke Niwa wrote:
> I have a concern that a lot of people wouldn't know what the "correct"
> output is for a given test.
>
> For a lot of pixel tests, deciding whether a given output is correct or not
> is really hard. e.g. some seemingly insignificant anti-alias
On Wed, Aug 15, 2012 at 5:00 PM, Filip Pizlo wrote:
>
> On Aug 15, 2012, at 4:02 PM, Dirk Pranke wrote:
>
>> On Wed, Aug 15, 2012 at 3:06 PM, Filip Pizlo wrote:
>>> Apparently I was somewhat unclear. Let me restate. We have the following
>>> mechanisms avail
On Wed, Aug 15, 2012 at 3:06 PM, Filip Pizlo wrote:
> Apparently I was somewhat unclear. Let me restate. We have the following
> mechanisms available when a test fails:
>
> 1) Check in a new -expected.* file.
>
> 2) Modify the test.
>
> 3) Modify a TestExpectations file.
>
> 4) Add the test to
On Wed, Aug 15, 2012 at 1:36 PM, Michael Saboff wrote:
> It seems to me that there are two issues here. One is Chromium specific
> about process conformity. It seems to me that should stay a Chromium issue
> without making the mechanism more complex for all ports. The other ports
> seem to m
s or not, and I personally think it's
worth trying. The solution I've described is the least intrusive
mechanism we can try that I've yet come up with.
-- Dirk
On Wed, Aug 15, 2012 at 12:22 PM, Dirk Pranke wrote:
> Hi all,
>
> As many of you know, we normally treat the -e
On Wed, Aug 15, 2012 at 12:27 PM, Filip Pizlo wrote:
> This sounds like it's adding even more complication to an already complicated
> system.
In some ways, yes. In other ways, perhaps it will allow us to simplify
things; e.g., if we are checking in failing tests, there is much less
of a need fo
Hi all,
As many of you know, we normally treat the -expected files as
"regression" tests rather than "correctness" tests; they are intended
to capture the current behavior of the tree. As such, they
historically have not distinguished between a "correct failure" and an
"incorrect failure".
The ch
On Thu, Aug 9, 2012 at 3:40 PM, Xianzhu Wang wrote:
> Hi,
>
> I want to skip several whole directories for chromium-android because of the
> related features are not available. For example, plugin/.
>
> 1. Add the following line in platform/chromium/TestExpectations:
> WONTFIX SKIP ANDROID : plugi
On Wed, Aug 8, 2012 at 10:47 AM, Ojan Vafai wrote:
> See https://bugs.webkit.org/show_bug.cgi?id=93195.
>
> media/W3C/video/networkState/networkState_during_progress.html and
> media/video-poster-blocked-by-willsendrequest.html are flaky on all
> platforms because they behave differently if the lo
On Wed, Aug 1, 2012 at 12:48 PM, Dirk Pranke wrote:
> For anyone bikeshedding along at home, my current plan of record (and
> changes to what has been written earlier in the thread) ...
>
> 1) '--verbose --verbose' to increase verbosity is right out
> 2) we can't us
For anyone bikeshedding along at home, my current plan of record (and
changes to what has been written earlier in the thread) ...
1) '--verbose --verbose' to increase verbosity is right out
2) we can't use '--debug' to get debug-level output, since that is
already used to run the debug build
3) as
On Tue, Jul 31, 2012 at 7:02 PM, Ryosuke Niwa wrote:
> On Tue, Jul 31, 2012 at 6:59 PM, Dirk Pranke wrote:
>>
>> On Tue, Jul 31, 2012 at 6:36 PM, Ryosuke Niwa wrote:
>> > On Tue, Jul 31, 2012 at 6:29 PM, Dirk Pranke
>> > wrote:
>> >>
>&g
On Tue, Jul 31, 2012 at 6:36 PM, Ryosuke Niwa wrote:
> On Tue, Jul 31, 2012 at 6:29 PM, Dirk Pranke wrote:
>>
>> I'm finally getting around to cleaning up the byzantine mass of
>> options in new-run-webkit-tests that controls what gets printed to
>> stderr and std
Hi all,
I'm finally getting around to cleaning up the byzantine mass of
options in new-run-webkit-tests that controls what gets printed to
stderr and stdout during a test run.
The patch is posted in https://bugs.webkit.org/show_bug.cgi?id=92432.
To quote the changelog:
[All of the --print X,Y,Z
On Thu, Jul 26, 2012 at 11:00 AM, Balazs Kelemen wrote:
> Hi webkittens!
>
> I am going to upload a patch to
> https://bugs.webkit.org/show_bug.cgi?id=92398 that will remove the
> --pixel-tests option from test drivers. Don't worry, I don't want to kill
> pixel testing, I want to be able to co
At the top of the garden-o-matic page there is a line like "Latest
revision processed by every bot: 122499 (trunk is at 122524)". I think
that does what you want?
On Thu, Jul 12, 2012 at 3:56 PM, Peter Kasting wrote:
> On Thu, Jul 12, 2012 at 3:17 PM, Ojan Vafai wrote:
>>
>> https://trac.webkit.
On Thu, Jul 12, 2012 at 10:53 AM, Ryosuke Niwa wrote:
> On Thu, Jul 12, 2012 at 10:43 AM, Stephen Chenney
> wrote:
>>
>> As several people have shown, it is quite easy to come up with a formula
>> that shows the cost of maintaining comments is much lower than the cost of
>> living without.
>
>
>
Seems fine now. Leap second?
-- Dirk
On Sun, Jul 1, 2012 at 7:34 AM, William Siegrist wrote:
> There was a network issue that has since been resolved. If you're still
> having trouble, please let me know.
>
> -Bill
>
>
>
> On Jun 30, 2012, at 8:17 PM, Dirk Pranke
Hi all,
It seems like DNS for webkit.org is down ... I can still get to
build.webkit.org, but everything else is timing out?
-- Dirk
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
On Fri, Jun 29, 2012 at 9:00 AM, Balazs Kelemen wrote:
> I think this is just the default, WebKitTestRunner has a --timeout that
> should control this if given. If that's not the case than it seems like a
> bug for me. On the other hand, I don't think run-webkit-tests supports
> setting custom tim
Hi all,
A new keyword 'NRWT' for new-run-webkit-tests-related issues has been
added to bugzilla (thanks Eric!). I've updated all of the open bugs I
have found that are NRWT-related, and will try to keep things up to
date in the future, but if you felt like adding the keywords when
filing new bugs
On Thu, Jun 14, 2012 at 10:37 PM, Ojan Vafai wrote:
>
>
> On Thu, Jun 14, 2012 at 9:20 PM, Maciej Stachowiak wrote:
>>
>>
>> On Jun 14, 2012, at 9:06 PM, Adam Barth wrote:
>>
>> > On Thu, Jun 14, 2012 at 9:02 PM, Ojan Vafai wrote:
>> >>
>> >> Seems like it will be a common error to mark a refte
On Thu, Jun 14, 2012 at 4:22 PM, Maciej Stachowiak wrote:
>
> On Jun 14, 2012, at 1:47 PM, Ryosuke Niwa wrote:
>
>
> On Thu, Jun 14, 2012 at 1:44 PM, Peter Kasting
> wrote:
>>
>> On Thu, Jun 14, 2012 at 1:39 PM, Elliot Poger wrote:
>>>
>>> Can someone please remind me why IMAGE+TEXT even exists
On Thu, Jun 14, 2012 at 4:34 PM, Adam Barth wrote:
> I too would like to see us remove TEXT+IMAGE. It's really confusing
> to non-experts, and it doesn't scale as we introduce new kinds of
> failures (like Audio). Do we really need TEXT+IMAGE+AUDIO,
> TEXT+AUDIO, and IMAGE+AUDIO?
AUDIO tests ca
On Thu, Jun 14, 2012 at 1:44 PM, Peter Kasting wrote:
> On Thu, Jun 14, 2012 at 1:39 PM, Elliot Poger wrote:
>>
>> Can someone please remind me why IMAGE+TEXT even exists?
>>
>> Wouldn't it be simpler to just mark a test as follows?
>>
>> IMAGE : allow image failure; go red if there is a text fai
On Wed, Jun 13, 2012 at 5:42 PM, Maciej Stachowiak wrote:
>
> On Jun 13, 2012, at 3:58 PM, Darin Adler wrote:
>
>> On Jun 13, 2012, at 3:53 PM, Dirk Pranke wrote:
>>
>>> * we use "\" (backslash) as a delimiter instead of ":" and "="
&
On Wed, Jun 13, 2012 at 5:05 PM, Ryosuke Niwa wrote:
> On Wed, Jun 13, 2012 at 4:55 PM, Tom Zakrajsek wrote:
>>
>> As long as we're considering "TitleCase" for the keywords, could we use it
>> to keep all of them as single words?
>>
>> WontFix, SkipCrash, SkipTimeout
>
>
> For skipped tests, it d
tokens because that would
cause an explosion of tokens :).
-- Dirk
On Wed, Jun 13, 2012 at 4:55 PM, Tom Zakrajsek wrote:
> As long as we're considering "TitleCase" for the keywords, could we use it
> to keep all of them as single words?
>
> WontFix, SkipCrash, SkipT
On Wed, Jun 13, 2012 at 4:38 PM, Ryosuke Niwa wrote:
> On Wed, Jun 13, 2012 at 4:32 PM, Dirk Pranke wrote:
>>
>> On Wed, Jun 13, 2012 at 4:26 PM, Ryosuke Niwa wrote:
>> > On Wed, Jun 13, 2012 at 4:12 PM, Benjamin Poulain
>> > wrote:
>> >>
On Wed, Jun 13, 2012 at 4:26 PM, Ryosuke Niwa wrote:
> On Wed, Jun 13, 2012 at 4:12 PM, Benjamin Poulain
> wrote:
>>
>> On Wed, Jun 13, 2012 at 3:53 PM, Dirk Pranke wrote:
>> > webkit.org/12345 WIN MAC DEBUG \
>> > animations/stop-animation-on-sus
On Wed, Jun 13, 2012 at 3:58 PM, Darin Adler wrote:
> On Jun 13, 2012, at 3:53 PM, Dirk Pranke wrote:
>
>> * we use "\" (backslash) as a delimiter instead of ":" and "="
>
> Seems worse to me. When I see a backslash I assume it’s a line continua
Hi all,
Because I have infinite patience for bikeshedding, I thought I would
send out Yet Another note on the proposed changes to the expectation
syntax.
Based on the last thread, I'm planning to change ORWT so that it will
recognize the syntax in the TestExpectations files and treat any
non-PASS
Does this
> particular fuzzer test catch enough bugs to justify its run-time? If yes
> then we should keep it. And if nobody can recall a time when the test saved
> them from making a broken commit, or when it helped a bot watcher identify a
> genuinely broken changeset, then we
On Wed, Jun 13, 2012 at 12:05 PM, Darin Adler wrote:
> On Jun 12, 2012, at 5:17 PM, Ojan Vafai wrote:
>
>> It's great to use a fuzzer in order to find cases where we're broken and
>> then make reduced layout tests from those.
>
>
> Generally we do require a test each time we fix a bug. So it’s a
I agree that the fuzzer should be used to create dedicated layout
tests, but we shouldn't run the fuzzer itself as part of the layout
test regression. I would have no objection to it being a separate test
step.
-- Dirk
On Tue, Jun 12, 2012 at 5:17 PM, Ojan Vafai wrote:
> See https://bugs.webkit.
On Mon, Jun 11, 2012 at 11:45 PM, Ryosuke Niwa wrote:
> On Mon, Jun 11, 2012 at 11:37 PM, Dirk Pranke wrote:
>>
>> On Thu, Jun 7, 2012 at 4:47 PM, Dirk Pranke wrote:
>> > I believe most if not all of the ports have started using either
>> > TestExpec
On Thu, Jun 7, 2012 at 4:47 PM, Dirk Pranke wrote:
> I believe most if not all of the ports have started using either
> TestExpectations files or a combination of TestExpectations files
> (except for the Apple Win port).
>
> Can we explicitly switch to the TestExpectations file
On Mon, Jun 11, 2012 at 5:46 PM, Maciej Stachowiak wrote:
>
> On Jun 10, 2012, at 9:26 AM, Ojan Vafai wrote:
>
> On Sun, Jun 10, 2012 at 4:54 AM, Balazs Kelemen wrote:
>>>
>>> So the unit tests are superfluous. In particular, if I had to pick
>>> between only having unit tests or only having re
On Mon, Jun 11, 2012 at 2:24 PM, Ryosuke Niwa wrote:
> On Mon, Jun 11, 2012 at 2:21 PM, Jacob Goldstein wrote:
>>
>> Can we just create an imported-w3c folder at the same level as
>> LayoutTests?
>
>
> You mean at trunk? I don't think that makes sense, and our testing
> infrastructure doesn't sup
On Fri, Jun 8, 2012 at 8:58 PM, Zoltan Herczeg wrote:
> Hi Dirk,
>
>> At any rate, I believe we are definitely open to adding new features;
>> feel free to suggest them or work on them!
>
> I am happy to hear that.
>
> https://bugs.webkit.org/show_bug.cgi?id=88680
>
> This is definitely a right st
On Fri, Jun 8, 2012 at 1:16 PM, Ryosuke Niwa wrote:
> I don't think that's true from my experience working on webkitpy so far. The
> root of problem is that we support way too many configurations & platforms,
> and Chromium port has had a completely different test runner program called
> test_shel
On Fri, Jun 8, 2012 at 12:50 PM, Filip Pizlo wrote:
> On Jun 8, 2012, at 12:31 PM, Dirk Pranke wrote:
>
>> On Fri, Jun 8, 2012 at 10:56 AM, Filip Pizlo wrote:
>>>
>>> It's a lot harder to dive into, a lot more cumbersome to improve, and not
>>> an
On Fri, Jun 8, 2012 at 12:23 PM, Filip Pizlo wrote:
>
> On Jun 8, 2012, at 12:19 PM, Dirk Pranke wrote:
>
>> On Fri, Jun 8, 2012 at 8:21 AM, Filip Pizlo wrote:
>>>
>>> On Jun 8, 2012, at 4:38 AM, Balazs Kelemen wrote:
>>>
>>>> On
On Fri, Jun 8, 2012 at 12:44 PM, Ryosuke Niwa wrote:
> On Fri, Jun 8, 2012 at 12:37 PM, Dirk Pranke wrote:
>>
>> I have no objection either to increasing the defaults for either of
>> these numbers or making it possible to have different defaults per
>> port.
&
I have no objection either to increasing the defaults for either of
these numbers or making it possible to have different defaults per
port.
Do you want to suggest different defaults? Should we use ORWT's
(infinite failures and infinite crashes by default)?
-- Dirk
On Fri, Jun 8, 2012 at 12:31 P
On Fri, Jun 8, 2012 at 10:56 AM, Filip Pizlo wrote:
>
> It's a lot harder to dive into, a lot more cumbersome to improve, and not
> any easier to maintain.
>
I definitely agree that NRWT is more complicated than it seems like it
should be; it got contorted as we added all the features we needed t
On Fri, Jun 8, 2012 at 10:14 AM, Zoltan Herczeg wrote:
> Hi,
>
>> I don't see why it would make sense to keep two parallel tools for this
>> once all the workflow bugs people have are addressed.
>
> The reason is easy. In the past when people tried to add new features to
> NRWT, they were not allo
On Fri, Jun 8, 2012 at 8:21 AM, Filip Pizlo wrote:
>
> On Jun 8, 2012, at 4:38 AM, Balazs Kelemen wrote:
>
>> On 06/08/2012 09:46 AM, Osztrogonac Csaba wrote:
>>> Hi,
>>>
>>> Dirk Pranke írta:
>>>> I believe most if not all of the ports have
Hi Ossy,
Thanks for your reply ...
On Fri, Jun 8, 2012 at 12:46 AM, Osztrogonac Csaba wrote:
> Hi,
>
> Dirk Pranke írta:
>
>> I believe most if not all of the ports have started using either
>> TestExpectations files or a combination of TestExpectations files
>>
I believe most if not all of the ports have started using either
TestExpectations files or a combination of TestExpectations files
(except for the Apple Win port).
Can we explicitly switch to the TestExpectations files at this point
and drop support for Skipped files on the other ports (and perhap
On Wed, Jun 6, 2012 at 10:36 PM, Peter Kasting wrote:
> On Wed, Jun 6, 2012 at 10:22 PM, Ryosuke Niwa wrote:
>>
>> Now that everyone knows the problem, I propose to rename FAIL to DIFF.
>>
>> FAIL should mean that the test fails, not that it fails with image, text,
>> or image and text failures.
The reason for this (which is debatable) is that CRASH and TIMEOUT are
deemed to be more serious and shouldn't be suppressed as lightly.
MISSING, on the other hand, just indicates that there's something
wrong (tests should never have missing results for very long).
-- Dirk
On Wed, Jun 6, 2012 at
On Wed, May 23, 2012 at 2:59 PM, Jacob Goldstein wrote:
> As a side note to this discussion, there is talk in the W3C community
> regarding their test approval process. At the recent working group
> meetings in Germany the idea was floated to simply approve all tests that
> are currently waiting
On Wed, May 23, 2012 at 2:30 PM, Maciej Stachowiak wrote:
>
> Are you concerned just about the actual pixel results or also about keeping
> render tree dumps up to date?
Both are more maintenance than a text-only test. In my experience,
maintaining pixel tests is more expensive, but I also don't
On Wed, May 23, 2012 at 1:41 PM, Ryosuke Niwa wrote:
>> > The only sane argument I've heard so far to gate pixel tests is that the
>> > correctness of such tests need to be manually inspected, which requires
>> > a
>> > lot of manual labor and is very error prone.
>>
>> I'm assuming the above incl
On Wed, May 23, 2012 at 11:56 AM, Ryosuke Niwa wrote:
> As I have said in the past, we should just import all tests, and treat
> non-text, non-ref tests as pixel tests. If we wanted to reduce the number of
> pixel tests we import, then we should submit those patches to W3C instead of
> directly su
101 - 200 of 395 matches
Mail list logo
