On Thu, Aug 16, 2012 at 6:11 PM, Dirk Pranke dpra...@chromium.org wrote:
On Thu, Aug 16, 2012 at 5:41 PM, Ryosuke Niwa rn...@webkit.org wrote:
On Thu, Aug 16, 2012 at 5:18 PM, Dirk Pranke dpra...@chromium.org
wrote:
On Thu, Aug 16, 2012 at 3:50 PM, Stephen Chenney schen...@chromium.org
On Fri, Aug 17, 2012 at 8:07 AM, Ryosuke Niwa rn...@webkit.org wrote:
On Thu, Aug 16, 2012 at 6:11 PM, Dirk Pranke dpra...@chromium.org wrote:
On Thu, Aug 16, 2012 at 5:41 PM, Ryosuke Niwa rn...@webkit.org wrote:
On Thu, Aug 16, 2012 at 5:18 PM, Dirk Pranke dpra...@chromium.org
wrote:
I don't have Dirk's patience, stamina and determination to explain and
evangelize the advantages of better awareness of current LayoutTests
coverage, I will chime in saying that I gladly support any effort to
improve it. I am especially excited about the ability to distinguish
failing, passing,
On Fri, Aug 17, 2012 at 11:06 AM, Dirk Pranke dpra...@chromium.org wrote:
On the other hand, the pixel test output that's correct to one expert
may
not be correct to another expert. For example, I might think that one
editing test's output is correct because it shows the feature we're
On Fri, Aug 17, 2012 at 11:29 AM, Ryosuke Niwa rn...@webkit.org wrote:
On Fri, Aug 17, 2012 at 11:06 AM, Dirk Pranke dpra...@chromium.org wrote:
On the other hand, the pixel test output that's correct to one expert
may
not be correct to another expert. For example, I might think that one
On Wed, Aug 15, 2012 at 5:19 PM, Ryosuke Niwa rn...@webkit.org wrote:
I have a concern that a lot of people wouldn't know what the correct
output is for a given test.
For a lot of pixel tests, deciding whether a given output is correct or not
is really hard. e.g. some seemingly insignificant
On Wed, Aug 15, 2012 at 6:02 PM, Filip Pizlo fpi...@apple.com wrote:
2) Possibility of the sheriff getting it wrong.
(2) concerns me most. We're talking about using filenames to serve as a
kind of unchecked comment. We already know that comments are usually bad
because there is no
On Thu, Aug 16, 2012 at 2:05 PM, Dirk Pranke dpra...@chromium.org wrote:
I think your observations are correct, but at least my experience as a
gardener/sheriff leads me to a different conclusion. Namely, when I'm
looking at a newly failing test, it is difficult if not impossible for
me to
On Thu, Aug 16, 2012 at 2:32 PM, Filip Pizlo fpi...@apple.com wrote:
On Aug 16, 2012, at 2:13 PM, Dirk Pranke wrote:
On Wed, Aug 15, 2012 at 6:02 PM, Filip Pizlo fpi...@apple.com wrote:
2) Possibility of the sheriff getting it wrong.
(2) concerns me most. We're talking about using
On Thu, Aug 16, 2012 at 2:32 PM, Filip Pizlo fpi...@apple.com wrote:
1) Switching to skipping flaky tests wholesale in all ports would be
great, and then we could get rid of the flakiness support.
Then you don't notice when a flaky tests stops being flaky. The cost of
flakiness support on the
On Thu, Aug 16, 2012 at 3:04 PM, Dirk Pranke dpra...@chromium.org wrote:
On Thu, Aug 16, 2012 at 2:23 PM, Ryosuke Niwa rn...@webkit.org wrote:
Like Filip, I'm extremely concerned about the prospect of us introducing
yet-another-way-of-doing-things, and not be able to get rid of it later.
1. On Thu, Aug 16, 2012 at 5:18 PM, Dirk Pranke dpra...@chromium.orgwrote:
On Thu, Aug 16, 2012 at 3:50 PM, Stephen Chenney schen...@chromium.org
wrote:
I agree with the priorities above, at least. I also agree with the
overall
goal of making our implementation match our philosophy on
On Thu, Aug 16, 2012 at 5:41 PM, Ryosuke Niwa rn...@webkit.org wrote:
On Thu, Aug 16, 2012 at 5:18 PM, Dirk Pranke dpra...@chromium.org wrote:
On Thu, Aug 16, 2012 at 3:50 PM, Stephen Chenney schen...@chromium.org
wrote:
I agree with the priorities above, at least. I also agree with the
This sounds like it's adding even more complication to an already complicated
system.
Given how many tests we currently have, I also don't buy that continuing to run
a test that is already known to fail provides much benefit. If the test covers
two things and one fails while the other
On Wed, Aug 15, 2012 at 12:27 PM, Filip Pizlo fpi...@apple.com wrote:
This sounds like it's adding even more complication to an already complicated
system.
In some ways, yes. In other ways, perhaps it will allow us to simplify
things; e.g., if we are checking in failing tests, there is much
I've received at least one bit of feedback off-list that it might not
have been clear what problems I was trying to solve and whether the
solution would add enough benefit to be worth it. Let me try to
restate things, in case this helps ...
The problems I'm attempting to solve are:
1) Chromium
It seems to me that there are two issues here. One is Chromium specific about
process conformity. It seems to me that should stay a Chromium issue without
making the mechanism more complex for all ports. The other ports seem to make
things work using the existing framework.
The other
I think this is the best compromise between the benefits of the
historically chromium approach (i.e. add to TestExpectations) and the
historically non-chromium approach (either skip the test or check in a
failing result, usually the latter). The only thing Dirk's proposal changes
for these ports
On Aug 15, 2012, at 12:27 PM, Filip Pizlo fpi...@apple.com wrote:
This sounds like it's adding even more complication to an already complicated
system.
Given how many tests we currently have, I also don't buy that continuing to
run a test that is already known to fail provides much
On Wed, Aug 15, 2012 at 1:36 PM, Michael Saboff msab...@apple.com wrote:
It seems to me that there are two issues here. One is Chromium specific
about process conformity. It seems to me that should stay a Chromium issue
without making the mechanism more complex for all ports. The other
On Aug 15, 2012, at 2:16 PM, Maciej Stachowiak m...@apple.com wrote:
On Aug 15, 2012, at 12:27 PM, Filip Pizlo fpi...@apple.com wrote:
This sounds like it's adding even more complication to an already
complicated system.
Given how many tests we currently have, I also don't buy that
Apparently I was somewhat unclear. Let me restate. We have the following
mechanisms available when a test fails:
1) Check in a new -expected.* file.
2) Modify the test.
3) Modify a TestExpectations file.
4) Add the test to a Skipped file.
5) Remove the test entirely.
I have no problem
On Wed, Aug 15, 2012 at 3:06 PM, Filip Pizlo fpi...@apple.com wrote:
Apparently I was somewhat unclear. Let me restate. We have the following
mechanisms available when a test fails:
1) Check in a new -expected.* file.
2) Modify the test.
3) Modify a TestExpectations file.
4) Add the
On Aug 15, 2012, at 4:02 PM, Dirk Pranke dpra...@chromium.org wrote:
On Wed, Aug 15, 2012 at 3:06 PM, Filip Pizlo fpi...@apple.com wrote:
Apparently I was somewhat unclear. Let me restate. We have the following
mechanisms available when a test fails:
1) Check in a new -expected.* file.
I have a concern that a lot of people wouldn't know what the correct
output is for a given test.
For a lot of pixel tests, deciding whether a given output is correct or not
is really hard. e.g. some seemingly insignificant anti-alias different may
turn out be a result of a bug in Skia and other
On Wed, Aug 15, 2012 at 5:00 PM, Filip Pizlo fpi...@apple.com wrote:
On Aug 15, 2012, at 4:02 PM, Dirk Pranke dpra...@chromium.org wrote:
On Wed, Aug 15, 2012 at 3:06 PM, Filip Pizlo fpi...@apple.com wrote:
Apparently I was somewhat unclear. Let me restate. We have the following
On Wed, Aug 15, 2012 at 5:00 PM, Filip Pizlo fpi...@apple.com wrote:
I believe that the cognitive load is greater than any benefit from
catching bugs incidentally by continuing to run a (1-fail) or (3) test, and
continuing to evaluate whether or not the expectation matches some notions
of
The typical approach used in situations that you describe is to rebase, not
skip. This avoids the problem of not knowing when the test started passing.
Hence, I'm not sure what you're implying. Maybe a better example would help.
On Aug 15, 2012, at 5:39 PM, Peter Kasting
On Wed, Aug 15, 2012 at 5:45 PM, Filip Pizlo fpi...@apple.com wrote:
The typical approach used in situations that you describe is to rebase,
not skip.
Which is precisely what Dirk is proposing. Literally the only difference
here is that the rebased test expectation would contain an optional
On Aug 15, 2012, at 5:51 PM, Peter Kasting pkast...@chromium.org wrote:
On Wed, Aug 15, 2012 at 5:45 PM, Filip Pizlo fpi...@apple.com wrote:
The typical approach used in situations that you describe is to rebase, not
skip.
Which is precisely what Dirk is proposing. Literally the only
On Wed, Aug 15, 2012 at 6:02 PM, Filip Pizlo fpi...@apple.com wrote:
2) Possibility of the sheriff getting it wrong.
(2) concerns me most. We're talking about using filenames to serve as a
kind of unchecked comment. We already know that comments are usually bad
because there is no
31 matches
Mail list logo
