We agreed on IRC to mark all the http layout tests as PASS CRASH TIMEOUT on the mac. This way we'll still catch tests that start failing. Unfortunately, I went to make this change and couldn't because we have some http tests in the ignore file and you can't have tests listed in both the ignore and fixable lists.
This all got me thinking about how noone ever understands the concept of ignore versus skip and that really it's kind of silly that we have two different files. What do people think of merging the two files and creating a new identifier that's more clear than ignore (e.g. NEVER_FIX)? Then we can make it so that you can't list duplicate expectations for a given test, that would leave us with the following: // This is ok. LayoutTests/http = CRASH FAIL NEVER_FIX : LayoutTests/http/othertest.html = FAIL // This throws a duplicate expectations exception.. LayoutTests/http/othertest.html = CRASH NEVER_FIX : LayoutTests/http/othertest.html = FAIL Ojan On Fri, Mar 13, 2009 at 10:52 AM, Ojan Vafai <[email protected]> wrote: > Being deputy this week, I've seen a ton of flaky tests on the mac. The vast > majority of these tests have been the http tests. A good chunk of them > timeout and a smaller chunk crash. > The mac build is red almost every run because of flaky tests. I did an > experiment this week of adding every flaky test I saw to the test lists, but > more just kept coming. It seems to me that we should just mark all the http > tests as flaky until someone has time to seriously look into what the source > of the problem is. > Any objections? > Ojan --~--~---------~--~----~------------~-------~--~----~ Chromium Developers mailing list: [email protected] View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev -~----------~----~----~----~------~----~------~--~---
