[webkit-dev] Please be cautious when rolling out old patches

2012-08-16 Thread Ryosuke Niwa
Hi,

I don't think we should be rolling out patches that are 2-3 months old
without fully understanding the consequences of doing so even if they had
introduced a regression. I'd even argue that rolling out such an old patch
should require a formal review.

Best regards,
Ryosuke Niwa
Software Engineer
Google Inc.
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo/webkit-dev


[webkit-dev] Proposal to enable compile flags only for EWS run

2012-08-16 Thread Bruno Abinader
Hi WebKit :)

As previously discussed, we decided that compile flag only was the
best option for CSS3 Text Decoration feature set (landed in
http://trac.webkit.org/changeset/125716 ). I believe this was a
general decision and got promptly implemented as such (however I still
maintain a runtime flag patch at
https://bugs.webkit.org/show_bug.cgi?id=93966 in case it is ever
needed).

What I want to discuss is an issue that raises with the decision to
only have compile flag: As it comes disabled by default, further
development on this feature will not be able to be checked by EWS or
any other build bot. This affects layout tests regression checking and
compile-time error checking, so I wonder how can we manage to fix
this? I have some proposals below:

1) Propose a special EWS-only build variable to contain compile
flags not enabled by default.
I might not know how difficult it would be to implement this, but
sounds like a clean approach to me.

2) Change compile flag value to become enabled by default.
I believe this is not possible because, as discussed earlier, it would
prematurely expose the feature to the web.

3) Add runtime flag, so compile flag would be enabled by default, but
runtime flag would be disabled.
That was I actually proposed in
http://lists.webkit.org/pipermail/webkit-dev/2012-August/021878.html
(CSS Regions implements like this, with both compile and runtime
flags).

Assuming that 2) and 3) are not possible due to previous discussion,
this makes me wonder if 1) is feasible. I believe other feature flags
might had the same situation before, so how you guys managed to check
for errors? I appreciate your input and feedback!

Best regards,

-- 
Bruno de Oliveira Abinader
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo/webkit-dev


Re: [webkit-dev] Proposal to enable compile flags only for EWS run

2012-08-16 Thread Adam Barth
Currently the EWS bots use the same configuration as the bots on
build.webkit.org.  We do that so they give accurate information about
what effect a given patch is going to have on the state of the tree
when the patch lands.  If we build using different flags on the EWS
than on build.webkit.org, they lose that predictive power.

What we've done to address this issue in the past is to set up a new
bot on build.webkit.org that builds and tests with flag turned on.
For example, during the development of flexbox, we had a flexbox bot
that the folks who worked on flexible cared about and everyone else
ignored.  I believe we did the same thing for grid layout.

I'd recommend setting up a separate bot on build.webkit.org.

Adam


On Thu, Aug 16, 2012 at 6:13 AM, Bruno Abinader brunoabina...@gmail.com wrote:
 Hi WebKit :)

 As previously discussed, we decided that compile flag only was the
 best option for CSS3 Text Decoration feature set (landed in
 http://trac.webkit.org/changeset/125716 ). I believe this was a
 general decision and got promptly implemented as such (however I still
 maintain a runtime flag patch at
 https://bugs.webkit.org/show_bug.cgi?id=93966 in case it is ever
 needed).

 What I want to discuss is an issue that raises with the decision to
 only have compile flag: As it comes disabled by default, further
 development on this feature will not be able to be checked by EWS or
 any other build bot. This affects layout tests regression checking and
 compile-time error checking, so I wonder how can we manage to fix
 this? I have some proposals below:

 1) Propose a special EWS-only build variable to contain compile
 flags not enabled by default.
 I might not know how difficult it would be to implement this, but
 sounds like a clean approach to me.

 2) Change compile flag value to become enabled by default.
 I believe this is not possible because, as discussed earlier, it would
 prematurely expose the feature to the web.

 3) Add runtime flag, so compile flag would be enabled by default, but
 runtime flag would be disabled.
 That was I actually proposed in
 http://lists.webkit.org/pipermail/webkit-dev/2012-August/021878.html
 (CSS Regions implements like this, with both compile and runtime
 flags).

 Assuming that 2) and 3) are not possible due to previous discussion,
 this makes me wonder if 1) is feasible. I believe other feature flags
 might had the same situation before, so how you guys managed to check
 for errors? I appreciate your input and feedback!

 Best regards,

 --
 Bruno de Oliveira Abinader
 ___
 webkit-dev mailing list
 webkit-dev@lists.webkit.org
 http://lists.webkit.org/mailman/listinfo/webkit-dev
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo/webkit-dev


Re: [webkit-dev] Proposal to enable compile flags only for EWS run

2012-08-16 Thread Adam Barth
Why not just build and run the tests locally?  This sounds like a CSS
feature that should more or less work the same for every port.

Adam


On Thu, Aug 16, 2012 at 9:53 AM, Peter Beverloo pe...@chromium.org wrote:
 Depending on how much longer the feature will be in development, it may not
 be worth setting up a new bot. Bruno seems to be mostly interested in
 getting EWS results, whereas results on the waterfall would only show up
 after committing the actual change.

 Something you could consider is to have a patch specifically crafted for
 getting EWS results. To take the Chromium Linux EWS bot as an example, you
 can enable the feature in Chromium's features.gypi and remove the SKIP entry
 from the TestExpectations file (these changes need to be in the patch
 itself), asking the bot to compile everything and run layout tests for the
 feature. As this patch wouldn't be intended to be committed, don't request
 review or commit, and manually click on the Request EWS button on the
 Bugzilla page.

 If you're using webkit-patch, this would work:
 webkit-patch upload --no-review -m Patch for EWS

 Similar changes can be made to enable it for other ports as well, i.e. for
 the Mac port which also runs layout tests. A downside of this is that you
 have to maintain a certain boilerplate of code for enabling your feature,
 and that you have to remember to remove this boilerplate when uploading the
 patch for review and/or commit. I guess you could set up a git workflow to
 make this a lot easier, though.

 Peter

 On Thu, Aug 16, 2012 at 5:38 PM, Adam Barth aba...@webkit.org wrote:

 Currently the EWS bots use the same configuration as the bots on
 build.webkit.org.  We do that so they give accurate information about
 what effect a given patch is going to have on the state of the tree
 when the patch lands.  If we build using different flags on the EWS
 than on build.webkit.org, they lose that predictive power.

 What we've done to address this issue in the past is to set up a new
 bot on build.webkit.org that builds and tests with flag turned on.
 For example, during the development of flexbox, we had a flexbox bot
 that the folks who worked on flexible cared about and everyone else
 ignored.  I believe we did the same thing for grid layout.

 I'd recommend setting up a separate bot on build.webkit.org.

 Adam


 On Thu, Aug 16, 2012 at 6:13 AM, Bruno Abinader brunoabina...@gmail.com
 wrote:
  Hi WebKit :)
 
  As previously discussed, we decided that compile flag only was the
  best option for CSS3 Text Decoration feature set (landed in
  http://trac.webkit.org/changeset/125716 ). I believe this was a
  general decision and got promptly implemented as such (however I still
  maintain a runtime flag patch at
  https://bugs.webkit.org/show_bug.cgi?id=93966 in case it is ever
  needed).
 
  What I want to discuss is an issue that raises with the decision to
  only have compile flag: As it comes disabled by default, further
  development on this feature will not be able to be checked by EWS or
  any other build bot. This affects layout tests regression checking and
  compile-time error checking, so I wonder how can we manage to fix
  this? I have some proposals below:
 
  1) Propose a special EWS-only build variable to contain compile
  flags not enabled by default.
  I might not know how difficult it would be to implement this, but
  sounds like a clean approach to me.
 
  2) Change compile flag value to become enabled by default.
  I believe this is not possible because, as discussed earlier, it would
  prematurely expose the feature to the web.
 
  3) Add runtime flag, so compile flag would be enabled by default, but
  runtime flag would be disabled.
  That was I actually proposed in
  http://lists.webkit.org/pipermail/webkit-dev/2012-August/021878.html
  (CSS Regions implements like this, with both compile and runtime
  flags).
 
  Assuming that 2) and 3) are not possible due to previous discussion,
  this makes me wonder if 1) is feasible. I believe other feature flags
  might had the same situation before, so how you guys managed to check
  for errors? I appreciate your input and feedback!
 
  Best regards,
 
  --
  Bruno de Oliveira Abinader
  ___
  webkit-dev mailing list
  webkit-dev@lists.webkit.org
  http://lists.webkit.org/mailman/listinfo/webkit-dev
 ___
 webkit-dev mailing list
 webkit-dev@lists.webkit.org
 http://lists.webkit.org/mailman/listinfo/webkit-dev


___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo/webkit-dev


Re: [webkit-dev] Proposal to enable compile flags only for EWS run

2012-08-16 Thread Peter Beverloo
It depends on the kind of feature you're working on, indeed.

While I don't know what Bruno's use-case is, In the case of text
decoration, I guess it could involve testing the complex test code paths
which can be different for port/platform combinations.

Peter

On Thu, Aug 16, 2012 at 6:00 PM, Adam Barth aba...@webkit.org wrote:

 Why not just build and run the tests locally?  This sounds like a CSS
 feature that should more or less work the same for every port.

 Adam


 On Thu, Aug 16, 2012 at 9:53 AM, Peter Beverloo pe...@chromium.org
 wrote:
  Depending on how much longer the feature will be in development, it may
 not
  be worth setting up a new bot. Bruno seems to be mostly interested in
  getting EWS results, whereas results on the waterfall would only show up
  after committing the actual change.
 
  Something you could consider is to have a patch specifically crafted for
  getting EWS results. To take the Chromium Linux EWS bot as an example,
 you
  can enable the feature in Chromium's features.gypi and remove the SKIP
 entry
  from the TestExpectations file (these changes need to be in the patch
  itself), asking the bot to compile everything and run layout tests for
 the
  feature. As this patch wouldn't be intended to be committed, don't
 request
  review or commit, and manually click on the Request EWS button on the
  Bugzilla page.
 
  If you're using webkit-patch, this would work:
  webkit-patch upload --no-review -m Patch for EWS
 
  Similar changes can be made to enable it for other ports as well, i.e.
 for
  the Mac port which also runs layout tests. A downside of this is that you
  have to maintain a certain boilerplate of code for enabling your feature,
  and that you have to remember to remove this boilerplate when uploading
 the
  patch for review and/or commit. I guess you could set up a git workflow
 to
  make this a lot easier, though.
 
  Peter
 
  On Thu, Aug 16, 2012 at 5:38 PM, Adam Barth aba...@webkit.org wrote:
 
  Currently the EWS bots use the same configuration as the bots on
  build.webkit.org.  We do that so they give accurate information about
  what effect a given patch is going to have on the state of the tree
  when the patch lands.  If we build using different flags on the EWS
  than on build.webkit.org, they lose that predictive power.
 
  What we've done to address this issue in the past is to set up a new
  bot on build.webkit.org that builds and tests with flag turned on.
  For example, during the development of flexbox, we had a flexbox bot
  that the folks who worked on flexible cared about and everyone else
  ignored.  I believe we did the same thing for grid layout.
 
  I'd recommend setting up a separate bot on build.webkit.org.
 
  Adam
 
 
  On Thu, Aug 16, 2012 at 6:13 AM, Bruno Abinader 
 brunoabina...@gmail.com
  wrote:
   Hi WebKit :)
  
   As previously discussed, we decided that compile flag only was the
   best option for CSS3 Text Decoration feature set (landed in
   http://trac.webkit.org/changeset/125716 ). I believe this was a
   general decision and got promptly implemented as such (however I still
   maintain a runtime flag patch at
   https://bugs.webkit.org/show_bug.cgi?id=93966 in case it is ever
   needed).
  
   What I want to discuss is an issue that raises with the decision to
   only have compile flag: As it comes disabled by default, further
   development on this feature will not be able to be checked by EWS or
   any other build bot. This affects layout tests regression checking and
   compile-time error checking, so I wonder how can we manage to fix
   this? I have some proposals below:
  
   1) Propose a special EWS-only build variable to contain compile
   flags not enabled by default.
   I might not know how difficult it would be to implement this, but
   sounds like a clean approach to me.
  
   2) Change compile flag value to become enabled by default.
   I believe this is not possible because, as discussed earlier, it would
   prematurely expose the feature to the web.
  
   3) Add runtime flag, so compile flag would be enabled by default, but
   runtime flag would be disabled.
   That was I actually proposed in
   http://lists.webkit.org/pipermail/webkit-dev/2012-August/021878.html
   (CSS Regions implements like this, with both compile and runtime
   flags).
  
   Assuming that 2) and 3) are not possible due to previous discussion,
   this makes me wonder if 1) is feasible. I believe other feature flags
   might had the same situation before, so how you guys managed to check
   for errors? I appreciate your input and feedback!
  
   Best regards,
  
   --
   Bruno de Oliveira Abinader
   ___
   webkit-dev mailing list
   webkit-dev@lists.webkit.org
   http://lists.webkit.org/mailman/listinfo/webkit-dev
  ___
  webkit-dev mailing list
  webkit-dev@lists.webkit.org
  http://lists.webkit.org/mailman/listinfo/webkit-dev
 
 


Re: [webkit-dev] Proposal to enable compile flags only for EWS run

2012-08-16 Thread Peter Beverloo
Depending on how much longer the feature will be in development, it may not
be worth setting up a new bot. Bruno seems to be mostly interested in
getting EWS results, whereas results on the waterfall would only show up
after committing the actual change.

Something you could consider is to have a patch specifically crafted for
getting EWS results. To take the Chromium Linux EWS bot as an example, you
can enable the feature in Chromium's features.gypi and remove the SKIP
entry from the TestExpectations file (these changes need to be in the patch
itself), asking the bot to compile everything and run layout tests for the
feature. As this patch wouldn't be intended to be committed, don't request
review or commit, and manually click on the Request EWS button on the
Bugzilla page.

If you're using webkit-patch, this would work:
webkit-patch upload --no-review -m Patch for EWS

Similar changes can be made to enable it for other ports as well, i.e. for
the Mac port which also runs layout tests. A downside of this is that you
have to maintain a certain boilerplate of code for enabling your feature,
and that you have to remember to remove this boilerplate when uploading the
patch for review and/or commit. I guess you could set up a git workflow to
make this a lot easier, though.

Peter

On Thu, Aug 16, 2012 at 5:38 PM, Adam Barth aba...@webkit.org wrote:

 Currently the EWS bots use the same configuration as the bots on
 build.webkit.org.  We do that so they give accurate information about
 what effect a given patch is going to have on the state of the tree
 when the patch lands.  If we build using different flags on the EWS
 than on build.webkit.org, they lose that predictive power.

 What we've done to address this issue in the past is to set up a new
 bot on build.webkit.org that builds and tests with flag turned on.
 For example, during the development of flexbox, we had a flexbox bot
 that the folks who worked on flexible cared about and everyone else
 ignored.  I believe we did the same thing for grid layout.

 I'd recommend setting up a separate bot on build.webkit.org.

 Adam


 On Thu, Aug 16, 2012 at 6:13 AM, Bruno Abinader brunoabina...@gmail.com
 wrote:
  Hi WebKit :)
 
  As previously discussed, we decided that compile flag only was the
  best option for CSS3 Text Decoration feature set (landed in
  http://trac.webkit.org/changeset/125716 ). I believe this was a
  general decision and got promptly implemented as such (however I still
  maintain a runtime flag patch at
  https://bugs.webkit.org/show_bug.cgi?id=93966 in case it is ever
  needed).
 
  What I want to discuss is an issue that raises with the decision to
  only have compile flag: As it comes disabled by default, further
  development on this feature will not be able to be checked by EWS or
  any other build bot. This affects layout tests regression checking and
  compile-time error checking, so I wonder how can we manage to fix
  this? I have some proposals below:
 
  1) Propose a special EWS-only build variable to contain compile
  flags not enabled by default.
  I might not know how difficult it would be to implement this, but
  sounds like a clean approach to me.
 
  2) Change compile flag value to become enabled by default.
  I believe this is not possible because, as discussed earlier, it would
  prematurely expose the feature to the web.
 
  3) Add runtime flag, so compile flag would be enabled by default, but
  runtime flag would be disabled.
  That was I actually proposed in
  http://lists.webkit.org/pipermail/webkit-dev/2012-August/021878.html
  (CSS Regions implements like this, with both compile and runtime
  flags).
 
  Assuming that 2) and 3) are not possible due to previous discussion,
  this makes me wonder if 1) is feasible. I believe other feature flags
  might had the same situation before, so how you guys managed to check
  for errors? I appreciate your input and feedback!
 
  Best regards,
 
  --
  Bruno de Oliveira Abinader
  ___
  webkit-dev mailing list
  webkit-dev@lists.webkit.org
  http://lists.webkit.org/mailman/listinfo/webkit-dev
 ___
 webkit-dev mailing list
 webkit-dev@lists.webkit.org
 http://lists.webkit.org/mailman/listinfo/webkit-dev

___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo/webkit-dev


Re: [webkit-dev] Proposal to enable compile flags only for EWS run

2012-08-16 Thread Adam Barth
In that case, he might want to start with one port, get that working
well, and then expand to the other ports.

Adam


On Thu, Aug 16, 2012 at 10:03 AM, Peter Beverloo pe...@chromium.org wrote:
 It depends on the kind of feature you're working on, indeed.

 While I don't know what Bruno's use-case is, In the case of text decoration,
 I guess it could involve testing the complex test code paths which can be
 different for port/platform combinations.

 Peter


 On Thu, Aug 16, 2012 at 6:00 PM, Adam Barth aba...@webkit.org wrote:

 Why not just build and run the tests locally?  This sounds like a CSS
 feature that should more or less work the same for every port.

 Adam


 On Thu, Aug 16, 2012 at 9:53 AM, Peter Beverloo pe...@chromium.org
 wrote:
  Depending on how much longer the feature will be in development, it may
  not
  be worth setting up a new bot. Bruno seems to be mostly interested in
  getting EWS results, whereas results on the waterfall would only show up
  after committing the actual change.
 
  Something you could consider is to have a patch specifically crafted for
  getting EWS results. To take the Chromium Linux EWS bot as an example,
  you
  can enable the feature in Chromium's features.gypi and remove the SKIP
  entry
  from the TestExpectations file (these changes need to be in the patch
  itself), asking the bot to compile everything and run layout tests for
  the
  feature. As this patch wouldn't be intended to be committed, don't
  request
  review or commit, and manually click on the Request EWS button on the
  Bugzilla page.
 
  If you're using webkit-patch, this would work:
  webkit-patch upload --no-review -m Patch for EWS
 
  Similar changes can be made to enable it for other ports as well, i.e.
  for
  the Mac port which also runs layout tests. A downside of this is that
  you
  have to maintain a certain boilerplate of code for enabling your
  feature,
  and that you have to remember to remove this boilerplate when uploading
  the
  patch for review and/or commit. I guess you could set up a git workflow
  to
  make this a lot easier, though.
 
  Peter
 
  On Thu, Aug 16, 2012 at 5:38 PM, Adam Barth aba...@webkit.org wrote:
 
  Currently the EWS bots use the same configuration as the bots on
  build.webkit.org.  We do that so they give accurate information about
  what effect a given patch is going to have on the state of the tree
  when the patch lands.  If we build using different flags on the EWS
  than on build.webkit.org, they lose that predictive power.
 
  What we've done to address this issue in the past is to set up a new
  bot on build.webkit.org that builds and tests with flag turned on.
  For example, during the development of flexbox, we had a flexbox bot
  that the folks who worked on flexible cared about and everyone else
  ignored.  I believe we did the same thing for grid layout.
 
  I'd recommend setting up a separate bot on build.webkit.org.
 
  Adam
 
 
  On Thu, Aug 16, 2012 at 6:13 AM, Bruno Abinader
  brunoabina...@gmail.com
  wrote:
   Hi WebKit :)
  
   As previously discussed, we decided that compile flag only was the
   best option for CSS3 Text Decoration feature set (landed in
   http://trac.webkit.org/changeset/125716 ). I believe this was a
   general decision and got promptly implemented as such (however I
   still
   maintain a runtime flag patch at
   https://bugs.webkit.org/show_bug.cgi?id=93966 in case it is ever
   needed).
  
   What I want to discuss is an issue that raises with the decision to
   only have compile flag: As it comes disabled by default, further
   development on this feature will not be able to be checked by EWS or
   any other build bot. This affects layout tests regression checking
   and
   compile-time error checking, so I wonder how can we manage to fix
   this? I have some proposals below:
  
   1) Propose a special EWS-only build variable to contain compile
   flags not enabled by default.
   I might not know how difficult it would be to implement this, but
   sounds like a clean approach to me.
  
   2) Change compile flag value to become enabled by default.
   I believe this is not possible because, as discussed earlier, it
   would
   prematurely expose the feature to the web.
  
   3) Add runtime flag, so compile flag would be enabled by default, but
   runtime flag would be disabled.
   That was I actually proposed in
   http://lists.webkit.org/pipermail/webkit-dev/2012-August/021878.html
   (CSS Regions implements like this, with both compile and runtime
   flags).
  
   Assuming that 2) and 3) are not possible due to previous discussion,
   this makes me wonder if 1) is feasible. I believe other feature flags
   might had the same situation before, so how you guys managed to check
   for errors? I appreciate your input and feedback!
  
   Best regards,
  
   --
   Bruno de Oliveira Abinader
   ___
   webkit-dev mailing list
   webkit-dev@lists.webkit.org
   

Re: [webkit-dev] Proposal to enable compile flags only for EWS run

2012-08-16 Thread Bruno Abinader
On Thu, Aug 16, 2012 at 1:28 PM, Adam Barth aba...@webkit.org wrote:
 In that case, he might want to start with one port, get that working
 well, and then expand to the other ports.

 Adam

That's exactly what I am doing :) As I'm mostly familiar with Qt, it
is the chosen platform were I am basing layout test results. I'm
building and testing locally, both with and without the feature flag
enabled, to make sure I'm not breaking any previous behavior. All new
CSS3 text decorations are getting proper layout tests, and I am also
taking care of editing feature, which I found proper spec already (
https://dvcs.w3.org/hg/editing/raw-file/tip/editing.html ) and
studying it. So far I just found basic text decoration support, so
this is pretty well assured by current implementation.



 On Thu, Aug 16, 2012 at 10:03 AM, Peter Beverloo pe...@chromium.org wrote:
 It depends on the kind of feature you're working on, indeed.

 While I don't know what Bruno's use-case is, In the case of text decoration,
 I guess it could involve testing the complex test code paths which can be
 different for port/platform combinations.

Indeed. So far there were no big changes between platforms (except
pixel tests font size/type differences, for instance). But it's always
good to be backed by EWS results to be sure everything's fine. I
enjoyed your solution (to provide a separate patch to make EWS run the
tests) which won't get landed, and that's what I'm going to do until
we find a better solution. Thanks for the input!

-- 
Bruno de Oliveira Abinader
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo/webkit-dev


Re: [webkit-dev] Proposal to enable compile flags only for EWS run

2012-08-16 Thread Ryosuke Niwa
On Thu, Aug 16, 2012 at 11:01 AM, Bruno Abinader brunoabina...@gmail.comwrote:

 On Thu, Aug 16, 2012 at 1:28 PM, Adam Barth aba...@webkit.org wrote:
  In that case, he might want to start with one port, get that working
  well, and then expand to the other ports.
 
  Adam

 That's exactly what I am doing :) As I'm mostly familiar with Qt, it
 is the chosen platform were I am basing layout test results. I'm
 building and testing locally, both with and without the feature flag
 enabled, to make sure I'm not breaking any previous behavior. All new
 CSS3 text decorations are getting proper layout tests, and I am also
 taking care of editing feature, which I found proper spec already (
 https://dvcs.w3.org/hg/editing/raw-file/tip/editing.html ) and
 studying it. So far I just found basic text decoration support, so
 this is pretty well assured by current implementation.

 
 
  On Thu, Aug 16, 2012 at 10:03 AM, Peter Beverloo pe...@chromium.org
 wrote:
  It depends on the kind of feature you're working on, indeed.
 
  While I don't know what Bruno's use-case is, In the case of text
 decoration,
  I guess it could involve testing the complex test code paths which can
 be
  different for port/platform combinations.

 Indeed. So far there were no big changes between platforms (except
 pixel tests font size/type differences, for instance). But it's always
 good to be backed by EWS results to be sure everything's fine. I
 enjoyed your solution (to provide a separate patch to make EWS run the
 tests) which won't get landed, and that's what I'm going to do until
 we find a better solution. Thanks for the input!


FYI, non-Chromium EWS don't run tests.

- Ryosuke
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo/webkit-dev


Re: [webkit-dev] Proposal to enable compile flags only for EWS run

2012-08-16 Thread Peter Beverloo
On Thu, Aug 16, 2012 at 7:25 PM, Ryosuke Niwa rn...@webkit.org wrote:

 On Thu, Aug 16, 2012 at 11:01 AM, Bruno Abinader 
 brunoabina...@gmail.comwrote:

 On Thu, Aug 16, 2012 at 1:28 PM, Adam Barth aba...@webkit.org wrote:
  In that case, he might want to start with one port, get that working
  well, and then expand to the other ports.
 
  Adam

 That's exactly what I am doing :) As I'm mostly familiar with Qt, it
 is the chosen platform were I am basing layout test results. I'm
 building and testing locally, both with and without the feature flag
 enabled, to make sure I'm not breaking any previous behavior. All new
 CSS3 text decorations are getting proper layout tests, and I am also
 taking care of editing feature, which I found proper spec already (
 https://dvcs.w3.org/hg/editing/raw-file/tip/editing.html ) and
 studying it. So far I just found basic text decoration support, so
 this is pretty well assured by current implementation.

 
 
  On Thu, Aug 16, 2012 at 10:03 AM, Peter Beverloo pe...@chromium.org
 wrote:
  It depends on the kind of feature you're working on, indeed.
 
  While I don't know what Bruno's use-case is, In the case of text
 decoration,
  I guess it could involve testing the complex test code paths which can
 be
  different for port/platform combinations.

 Indeed. So far there were no big changes between platforms (except
 pixel tests font size/type differences, for instance). But it's always
 good to be backed by EWS results to be sure everything's fine. I
 enjoyed your solution (to provide a separate patch to make EWS run the
 tests) which won't get landed, and that's what I'm going to do until
 we find a better solution. Thanks for the input!


 FYI, non-Chromium EWS don't run tests.


The Mac EWS used to run tests, until about a week ago. They're disabled
until more hardware is available (see r125058), but hopefully it also means
tests will eventually be ran again :-).

Peter

- Ryosuke


___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo/webkit-dev


Re: [webkit-dev] Proposal and WIP implementation for refactoring cross thread communication

2012-08-16 Thread Sam Weinig
Function is definitely useful in non-cross thread cases.  It is analogous to 
std::function, and its use cases are very similar.  Please don't rename it.

-Sam

On Aug 15, 2012, at 7:35 PM, Kwonjin Jeong g...@company100.net wrote:

 I also thought about what you said. But I'm not sure whether WTF::Function 
 will be used for cross thread communication only.
 
 So, I'd like to know whether WTF::Function won't be used for single threaded 
 case.
 
 On Tue, Aug 14, 2012 at 8:57 AM, KwangYul Seo sk...@company100.net wrote:
 Currently, there is no code using WTF::Function in single-threaded code. It 
 seems WTF::Function is intended to be used for cross thread communication. 
 One evidence is that FunctionImplBase is thread-safe ref-counted, which is 
 not super efficient for single threaded use cases.
 
 If we have no plan to use WTF::Function for single threaded code, why don't 
 we rename WTF::Function to WTF::CrossThreadFunction and add more features 
 such as cross thread copying there?
 
 Regards,
 Kwang Yul Seo
 
 On Mon, Aug 13, 2012 at 9:26 PM, Kwonjin Jeong g...@company100.net wrote:
 Hi, WebKit.
 
 There are several implementations of threads, tasks and cross thread 
 communication. e.g. FileThread, DatabaseThread and StorageThread perform 
 blocking IO asynchronously and invoke callbacks using ScriptExecutionContext. 
 Each of these implementation is pretty similar, but there is no code sharing 
 at all.
 
 So, my team at Company 100 is trying to refactor cross thread communication. 
 We wrote an API proposal and modified some thread code to use our new API. 
 
 As the first step, we added two new abstractions for cross thread 
 communcation:
 
 1) TaskThread is our new abstraction for the task thread. FileThread, 
 DatabaseThread and other task threads reuse the code by inheriting from 
 TaskThread.
 
 2) We also added CrossThreadFunction. It's used to pass a task to the task 
 thread and invoke callbacks. CrossThreadFunction is similar to WTF::Function 
 but it has some additional features. e.g., cross-thread parameter copying and 
 ref counting.
 
 Please refer to the following bug for details:
 
 https://bugs.webkit.org/show_bug.cgi?id=92365
 
 We welcome feedbacks on our work.
 
 Thanks.
 
 -- 
 
 Kwonjin Jeong
 Computer Scientist
 Company 100, Inc.
 Phone: +82-10-9528-2581
 
 
 ___
 webkit-dev mailing list
 webkit-dev@lists.webkit.org
 http://lists.webkit.org/mailman/listinfo/webkit-dev
 
 
 
 
 
 -- 
 
 Kwonjin Jeong
 Computer Scientist
 Company 100, Inc.
 Phone: +82-10-9528-2581
 
 ___
 webkit-dev mailing list
 webkit-dev@lists.webkit.org
 http://lists.webkit.org/mailman/listinfo/webkit-dev

___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo/webkit-dev


Re: [webkit-dev] A proposal for handling failing layout tests and TestExpectations

2012-08-16 Thread Dirk Pranke
On Wed, Aug 15, 2012 at 5:19 PM, Ryosuke Niwa rn...@webkit.org wrote:
 I have a concern that a lot of people wouldn't know what the correct
 output is for a given test.

 For a lot of pixel tests, deciding whether a given output is correct or not
 is really hard. e.g. some seemingly insignificant anti-alias different may
 turn out be a result of a bug in Skia and other graphics library or WebCore
 code that uses it.

 As a result,

 people may check in wrong correct results
 people may add failing results even though new results are more or less
 correct


 This leads me to think that just checking in the current output as
 -expected.png and filing bugs separately is a better approach.


I think your observations are correct, but at least my experience as a
gardener/sheriff leads me to a different conclusion. Namely, when I'm
looking at a newly failing test, it is difficult if not impossible for
me to know if the existing baseline was previously believed to be
correct or not, and thus it's hard for me to tell if the new baseline
should be considered worse, better, or different. In theory I could go
look at the changelog for each test, but I would be skeptical if that
had enough useful information (I would expect most comments to be
along the lines of rebaselining after X with no indication if the
output is correct or not. This is just a theory, though.

This is why I want to test this theory :). It seems like if we got
experience with this on one (or more) ports for a couple of months we
would have a much more well-informed opinion, and I'm not seeing a
huge downside to at least trying this idea out.

-- Dirk
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo/webkit-dev


Re: [webkit-dev] A proposal for handling failing layout tests and TestExpectations

2012-08-16 Thread Dirk Pranke
On Wed, Aug 15, 2012 at 6:02 PM, Filip Pizlo fpi...@apple.com wrote:

 2) Possibility of the sheriff getting it wrong.

 (2) concerns me most.  We're talking about using filenames to serve as a
 kind of unchecked comment.  We already know that comments are usually bad
 because there is no protection against them going stale.


Sheriffs can already get things wrong (and rebaseline when they
shouldn't). I believe that adding passing/failing to expected will
make things better in this regard, not worse.

Another idea/observation is that if we have multiple types of
expectation files, it might be easier to set up watchlists, e.g., let
me know whenever a file gets checked into fast/forms with an -expected
or -failing result. It seems like this might be useful, but I'm not
sure.

 In particular, to further clarify my position, if someone were to argue that
 Dirk's proposal would be a wholesale replacement for TestExpectations, then
 I would be more likely to be on board, since I very much like the idea of
 reducing the number of ways of doing things.  Maybe that's a good way to
 reach compromise.

 Dirk, what value do you see in TestExpectations were your change to be
 landed?  Do scenarios still exist where there would be a test for which (a)
 there is no -fail.* file, (b) the test is not skipped, and (c) it's marked
 with some annotation in TestExpectations?  I'm most interested in the
 question of such scenarios exist, since in my experience, whenever a test is
 not rebased, is not skipped, and is marked as failing in TestExpectations,
 it ends up just causing gardening overhead later.

This is a good question, because it is definitely my intent that this
change replace some existing practices, not add to them.

Currently, the Chromium port uses TestExpectations entries for four
different kinds of things: tests we don't ever plan to fix (WONTFIX),
tests that we skip because not doing so causes other tests to break,
tests that fail (reliably), and tests that are flaky.

Skipped files do not let you distinguish (programmatically) between
the first two categories, and so my plan is to replace Skipped files
with TestExpectations (using the new syntax discussed a month or so
ago) soon (next week or two at the latest).

I would like to replace using TestExpectations for failing tests (at
least for tests that are expected to keep failing indefinitely because
someone isn't working on an active fix) with this new mechanism.

That leaves flaky tests. One can debate what the right thing to do w/
flaky tests is here; I'm inclined to argue that flakiness is at least
as bad as failing, and we should probably be skipping them, but the
Chromium port has not yet actively tried this approach (I think other
ports probably have experience here, though).

Does that help answer your question / sway you at all?

-- Dirk
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo/webkit-dev


Re: [webkit-dev] A proposal for handling failing layout tests and TestExpectations

2012-08-16 Thread Ryosuke Niwa
On Thu, Aug 16, 2012 at 2:05 PM, Dirk Pranke dpra...@chromium.org wrote:

 I think your observations are correct, but at least my experience as a
 gardener/sheriff leads me to a different conclusion. Namely, when I'm
 looking at a newly failing test, it is difficult if not impossible for
 me to know if the existing baseline was previously believed to be
 correct or not, and thus it's hard for me to tell if the new baseline
 should be considered worse, better, or different.


How does the proposal solve this problem? Right now gardeners have two
options:

   - Rebaseline
   - Add a test expectation

Once we implemented the proposal, we have at least three options:

   - Rebaseline correct.png
   - Add/rebaseline expected.png
   - Add/rebaseline failure.png
   - (Optionally) Add a test expectation.

And that's a lot of options to choose from. The more options we have, the
more likely people make mistakes. We're already inconsistent in how
-expected.png is used because some people make mistakes. I'm afraid that
adding another set of expected results result in even more mistakes and a
unrecoverable mess.

This is why I want to test this theory :). It seems like if we got
 experience with this on one (or more) ports for a couple of months we
 would have a much more well-informed opinion, and I'm not seeing a
 huge downside to at least trying this idea out.


Sure. But if we're doing this experiment on trunk, thereby imposing
significant cognitive load on every other port, then I'd like to see us
setting *both* an exact date at which we decide whether this approach is
good or not and criteria by which we decide this *before *the experiment
starts.

Like Filip, I'm *extremely* concerned about the prospect of us introducing
yet-another-way-of-doing-things, and not be able to get rid of it later.

- Ryosuke
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo/webkit-dev


Re: [webkit-dev] A proposal for handling failing layout tests and TestExpectations

2012-08-16 Thread Ryosuke Niwa
On Thu, Aug 16, 2012 at 2:32 PM, Filip Pizlo fpi...@apple.com wrote:


 On Aug 16, 2012, at 2:13 PM, Dirk Pranke wrote:

  On Wed, Aug 15, 2012 at 6:02 PM, Filip Pizlo fpi...@apple.com wrote:
 
  2) Possibility of the sheriff getting it wrong.
 
  (2) concerns me most.  We're talking about using filenames to serve as a
  kind of unchecked comment.  We already know that comments are usually
 bad
  because there is no protection against them going stale.
 
 
  Sheriffs can already get things wrong (and rebaseline when they
  shouldn't). I believe that adding passing/failing to expected will
  make things better in this regard, not worse.

 In what way do things become better?  Because other people will see what
 the sheriff believed about the result?

 Can you articulate some more about what happens when you have both
 -expected and -failing?

 My specific concern is that after someone checks in a fix, we will have
 some sheriff accidentally misjudge the change in behavior to be a
 regression, and check in a -failing file.  And then we end up in a world of
 confusion.

 This is why I think that just having -expected files is better.  It is a
 kind of recognition that we're tracking changes in behavior, rather than
 comparing against some almighty notion of what it means to be correct.

 
  Another idea/observation is that if we have multiple types of
  expectation files, it might be easier to set up watchlists, e.g., let
  me know whenever a file gets checked into fast/forms with an -expected
  or -failing result. It seems like this might be useful, but I'm not
  sure.
 
  In particular, to further clarify my position, if someone were to argue
 that
  Dirk's proposal would be a wholesale replacement for TestExpectations,
 then
  I would be more likely to be on board, since I very much like the idea
 of
  reducing the number of ways of doing things.  Maybe that's a good way to
  reach compromise.
 
  Dirk, what value do you see in TestExpectations were your change to be
  landed?  Do scenarios still exist where there would be a test for which
 (a)
  there is no -fail.* file, (b) the test is not skipped, and (c) it's
 marked
  with some annotation in TestExpectations?  I'm most interested in the
  question of such scenarios exist, since in my experience, whenever a
 test is
  not rebased, is not skipped, and is marked as failing in
 TestExpectations,
  it ends up just causing gardening overhead later.
 
  This is a good question, because it is definitely my intent that this
  change replace some existing practices, not add to them.
 
  Currently, the Chromium port uses TestExpectations entries for four
  different kinds of things: tests we don't ever plan to fix (WONTFIX),
  tests that we skip because not doing so causes other tests to break,
  tests that fail (reliably), and tests that are flaky.
 
  Skipped files do not let you distinguish (programmatically) between
  the first two categories, and so my plan is to replace Skipped files
  with TestExpectations (using the new syntax discussed a month or so
  ago) soon (next week or two at the latest).
 
  I would like to replace using TestExpectations for failing tests (at
  least for tests that are expected to keep failing indefinitely because
  someone isn't working on an active fix) with this new mechanism.
 
  That leaves flaky tests. One can debate what the right thing to do w/
  flaky tests is here; I'm inclined to argue that flakiness is at least
  as bad as failing, and we should probably be skipping them, but the
  Chromium port has not yet actively tried this approach (I think other
  ports probably have experience here, though).
 
  Does that help answer your question / sway you at all?

 Yes, it does - it answers my question, though it perhaps doesn't sway me.
  My concerns are still that:

 1) Switching to skipping flaky tests wholesale in all ports would be
 great, and then we could get rid of the flakiness support.


This is not necessarily helpful in some cases. There are cases some test
make subsequent tests flaky so skipping a flaky test doesn't fix the
problem. Instead, it just makes the next test flaky. The right fix, of
course, to skip/fix the culprit but pinpointing the test that makes
subsequent tests more flaky has proved to be a time consuming and
challenging task.

2) The WONTFIX mode in TestExpectations feels to me more like a statement
 that you're just trying to see if the test doesn't crash.  Correct?  Either
 way, it's confusing.


Yes, I'd argue that they should just be skipped but that's a bikeshedding
for another time.

- Ryosuke
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo/webkit-dev


Re: [webkit-dev] A proposal for handling failing layout tests and TestExpectations

2012-08-16 Thread Ojan Vafai
On Thu, Aug 16, 2012 at 2:32 PM, Filip Pizlo fpi...@apple.com wrote:

 1) Switching to skipping flaky tests wholesale in all ports would be
 great, and then we could get rid of the flakiness support.


Then you don't notice when a flaky tests stops being flaky. The cost of
flakiness support on the project does not seem large to me and it's pretty
frequent that a test will only be flaky for a few hundred runs (e.g. due to
a bad checkin that gets fixed), so we can then remove it from
TestExpectations and run the test as normal.

2) The WONTFIX mode in TestExpectations feels to me more like a statement
 that you're just trying to see if the test doesn't crash.  Correct?  Either
 way, it's confusing.


WONTFIX is for tests that don't make sense to fix for the given port (e.g.
dashboard-specific tests for non-Apple ports). It's a way of distinguishing
tests that we're skipping because of a bug on our part vs. tests that we're
skipping because the test doesn't apply to the port.

3) Your new mechanism feels like it's already covered by our existing use
 of -expected files.  I'm not quite convinced that having -failing in
 addition to -expected files would be all that helpful.


But there are many cases where we *know* the result is incorrect, e.g. it
doesn't match a spec. Sure, there are also many cases where it's not clear
what the correct behavior is.
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo/webkit-dev


Re: [webkit-dev] A proposal for handling failing layout tests and TestExpectations

2012-08-16 Thread Ryosuke Niwa
On Thu, Aug 16, 2012 at 3:04 PM, Dirk Pranke dpra...@chromium.org wrote:

 On Thu, Aug 16, 2012 at 2:23 PM, Ryosuke Niwa rn...@webkit.org wrote:
  Like Filip, I'm extremely concerned about the prospect of us introducing
  yet-another-way-of-doing-things, and not be able to get rid of it later.

 Presumably the only way we'd be not able to get rid of it would be if
 some port actually liked it, in which case, why would we get rid of it?


Because we already have too many ways to deal with test failures.

Why don't we at least merge Skipped and TestExpectations first.

- Ryosuke
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo/webkit-dev


Re: [webkit-dev] A proposal for handling failing layout tests and TestExpectations

2012-08-16 Thread Ryosuke Niwa
   1. On Thu, Aug 16, 2012 at 5:18 PM, Dirk Pranke dpra...@chromium.orgwrote:

On Thu, Aug 16, 2012 at 3:50 PM, Stephen Chenney schen...@chromium.org
 wrote:
  I agree with the priorities above, at least. I also agree with the
 overall
  goal of making our implementation match our philosophy on testing.
 
  Ryosuke has raised a very valid point: it is not clear what a Gardener
  should do with a test that has changed behavior. Personally, the area I
 deal
  with is very susceptible to the single pixel differences matter issue,
 and
  I would prefer gardeners to flag updates in some way so that I can look
 at
  it myself (or one of the very few other experts can look at it). Maybe
 I'm
  just a control freak.
 
  In the current situation, gardeners are not likely to roll out a patch
 short
  of build failures, obvious test or perf regressions, or security issues.
 For
  the bulk of cases where a test result has changed, the gardener will
 either
  add it to expectations or go ahead and update the result.
 
  The result is a mixture of incorrect expected results and files listed in
  TestExpectations that may or may not be correct. The way I deal with this
  right now is to maintain a snapshot of TestExpectations and occasionally
 go
  through and look for newly added results, and check if they are in fact
  correct. And every now and then go look at older entries to see if they
 have
  been updated. Or I get lucky and notice that a new expected result has
 been
  checked in that is incorrect (I can mostly catch that by following
 checkins
  in the code).
 
  The proposed change does not alter this fundamental dynamic. Depending on
  what policy gardeners adopt, they might now rename the result as failing
 and
  remove the expected, or maybe they'll just update the expected. I'm
 still in
  a situation where I don't know the exact status of any given result, and
  where I have no easy way of knowing if a gardener has correctly updated
 an
  expected or failing image.

 So, if we had -passing/-failing, then people who knew what the results
 were supposed to be in a given directory (call them owners) could
 rename the existing -expected files into one category or another, and
 then a gardener could add newly-failing tests as -expected; it would
 then be trivial for the owner to look for new -expected files in that
 directory.

 Right now, an owner would have to periodically scan the directory for
 files changed since the last time they scanned the directory, and
 would have no way of knowing whether an updated file was correct or
 not (perhaps you could filter out by who committed the change, but
 it's certainly harder than ls *-expected).

 So, I would contend that my proposal would make it easier for us to
 have a process by which gardeners could punt on changes they're unsure
 about and for owners to subsequently review them. I think you could
 have a similar model if we just checked in *all* new results into
 -expected, but it would be harder to implement (perhaps not
 impossible, though, I haven't given it much thought yet).


That seems like a much better model. Right now, one of the biggest problem
is that people rebaseline tests or add test expectations without
understanding what the correct output is. It happens for almost all ports.

Having -correct.png doesn't necessarily help that because even a seemingly
innocent one-pixel difference could be a bug (a patch may only add a
baseline to one platform), and it's really hard to tell what the expected
result is unless you have worked on the rendering code for a long time.

On the other hand, if you follow the chromium model of just
 suppressing the file, then we have no idea if the failure you
 currently see has any relationship to the failure that was originally
 seen, or if the file that is currently checked in has any relationship
 to what correct might now need to be (we know that the file was
 correct at some point but is now stale). So, in the chromium model, at
 least, you can do a grep through TestExpectations to look for failing
 tests, but you don't have the history you would get by looking at a
 history of changes to -expected/-passing/-failing.

 In a world where all failures are triaged promptly by qualified
 owners, all of this mechanism is unnecessary. Unfortunately, we don't
 live in that world at the moment.


I'd argue that this is a bigger problem worth solving. Differentiating
expected failure and correct results isn't going to help if gardeners were
to keep making mistakes.

- Ryosuke
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo/webkit-dev


Re: [webkit-dev] A proposal for handling failing layout tests and TestExpectations

2012-08-16 Thread Dirk Pranke
On Thu, Aug 16, 2012 at 5:41 PM, Ryosuke Niwa rn...@webkit.org wrote:
 On Thu, Aug 16, 2012 at 5:18 PM, Dirk Pranke dpra...@chromium.org wrote:

 On Thu, Aug 16, 2012 at 3:50 PM, Stephen Chenney schen...@chromium.org
 wrote:
  I agree with the priorities above, at least. I also agree with the
  overall
  goal of making our implementation match our philosophy on testing.
 
  Ryosuke has raised a very valid point: it is not clear what a Gardener
  should do with a test that has changed behavior. Personally, the area I
  deal
  with is very susceptible to the single pixel differences matter issue,
  and
  I would prefer gardeners to flag updates in some way so that I can look
  at
  it myself (or one of the very few other experts can look at it). Maybe
  I'm
  just a control freak.
 
  In the current situation, gardeners are not likely to roll out a patch
  short
  of build failures, obvious test or perf regressions, or security issues.
  For
  the bulk of cases where a test result has changed, the gardener will
  either
  add it to expectations or go ahead and update the result.
 
  The result is a mixture of incorrect expected results and files listed
  in
  TestExpectations that may or may not be correct. The way I deal with
  this
  right now is to maintain a snapshot of TestExpectations and occasionally
  go
  through and look for newly added results, and check if they are in fact
  correct. And every now and then go look at older entries to see if they
  have
  been updated. Or I get lucky and notice that a new expected result has
  been
  checked in that is incorrect (I can mostly catch that by following
  checkins
  in the code).
 
  The proposed change does not alter this fundamental dynamic. Depending
  on
  what policy gardeners adopt, they might now rename the result as failing
  and
  remove the expected, or maybe they'll just update the expected. I'm
  still in
  a situation where I don't know the exact status of any given result, and
  where I have no easy way of knowing if a gardener has correctly updated
  an
  expected or failing image.

 So, if we had -passing/-failing, then people who knew what the results
 were supposed to be in a given directory (call them owners) could
 rename the existing -expected files into one category or another, and
 then a gardener could add newly-failing tests as -expected; it would
 then be trivial for the owner to look for new -expected files in that
 directory.

 Right now, an owner would have to periodically scan the directory for
 files changed since the last time they scanned the directory, and
 would have no way of knowing whether an updated file was correct or
 not (perhaps you could filter out by who committed the change, but
 it's certainly harder than ls *-expected).

 So, I would contend that my proposal would make it easier for us to
 have a process by which gardeners could punt on changes they're unsure
 about and for owners to subsequently review them. I think you could
 have a similar model if we just checked in *all* new results into
 -expected, but it would be harder to implement (perhaps not
 impossible, though, I haven't given it much thought yet).


 That seems like a much better model.

To clarify: what does That refer to? My idea for gardeners to check
in -expected and owners to move to -passing/failing, or the idea
that we check in new results into -expected and have some other way
of reviewing new baselines? (or something else)?


 Right now, one of the biggest problem
 is that people rebaseline tests or add test expectations without
 understanding what the correct output is. It happens for almost all ports.

 Having -correct.png doesn't necessarily help that because even a seemingly
 innocent one-pixel difference could be a bug (a patch may only add a
 baseline to one platform), and it's really hard to tell what the expected
 result is unless you have worked on the rendering code for a long time.


Yes. I hate gardening/sheriffing because I feel like I usually don't
know whether to rebaseline or not :) (Although I suspect having
-correct would be better than not having it, at least).

 In a world where all failures are triaged promptly by qualified
 owners, all of this mechanism is unnecessary. Unfortunately, we don't
 live in that world at the moment.


 I'd argue that this is a bigger problem worth solving. Differentiating
 expected failure and correct results isn't going to help if gardeners were
 to keep making mistakes.


It is a bigger problem worth solving, but it is also much harder to
solve. As long as we have the combination of

1) changes can cause lots of tests to change expectations and need new baselines
2) it's acceptable to land changes without the accompanying updated baselines
3) there is pressure to keep the bots green (not only for its own
sake, but so that we can more easily identify *new* failures)
4) insufficient resources watching the tree (which is really true for
every port today, even chromium)

i think we're 

[webkit-dev] WebKitQt

2012-08-16 Thread blake fiddler
Hey!

I have problem with WebKit  Qt port build.
Why you disable Qt4.x ??? As I can understand now WebkitQt works only with
5.0 version?
But Qt5 isn't stable version - it's beta.
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo/webkit-dev


Re: [webkit-dev] WebKitQt

2012-08-16 Thread Jarred Nicholls
On Thu, Aug 16, 2012 at 11:42 PM, blake fiddler blake.fidd...@gmail.comwrote:

 Hey!

 I have problem with WebKit  Qt port build.


Please address Qt related questions to the webkit-qt mailing list in the
future.


 Why you disable Qt4.x ??? As I can understand now WebkitQt works only with
 5.0 version?


You understand correct, sir.  I recommend subscribing to webkit-qt as well,
as this change was discussed.


 But Qt5 isn't stable version - it's beta.


Building WebKit from trunk is the antithesis of stable, so I hardly think
that matters.  I wouldn't dare speak for the QtWebKit team, but I'd imagine
that supporting both major versions became too cumbersome.



 ___
 webkit-dev mailing list
 webkit-dev@lists.webkit.org
 http://lists.webkit.org/mailman/listinfo/webkit-dev

___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo/webkit-dev