-chromium-reviews +chromium-dev Don't we currently pass a large percentage of the LayoutTests/media tests? Won't we lose coverage of these tests by doing this? Maybe it's worth the savings maintenance cost until we have a complete solution for them?
Ojan On Thu, Aug 27, 2009 at 11:46 AM, Dimitri Glazkov <[email protected]>wrote: > > BTW, about to filter out all LayoutTests/media out for now, skipping > them. We can't have codecs for most of these tests, so we (Alpha and > myself) decided it's easier to skip them for now. Permanent solution > coming in Q4. > > :DG< > > On Thu, Aug 27, 2009 at 11:42 AM, Ojan Vafai<[email protected]> wrote: > > I see what you're saying. I think the best solution here is to just have > one > > line with two bugs listed. This will also help whoever is going to fix > the > > test as they'll be able to easily see all the reason's it's failing. > > BUG20376 BUG13907 WIN : LayoutTests/media/video-src-remove.html = TIMEOUT > > FAIL > > That OK? > > On Thu, Aug 27, 2009 at 11:02 AM, Jeremy Orlow <[email protected]> > wrote: > >> > >> Yes, but it's not flaky now. If you'd like, I can move the commented > out > >> one down to right after the new version of it. I think what'd be even > >> better is if multiple entries for one layout test didn't cause the > script to > >> parse. Or it'd at least be nice if we could list 2 bugs for one test. > >> The commented out one is NOT dead....it's another type of failure that > is > >> currently overshadowed by the new type of failure. > >> > >> On Thu, Aug 27, 2009 at 10:57 AM, Ojan Vafai <[email protected]> wrote: > >>> > >>> On Thu, Aug 27, 2009 at 10:34 AM, Jeremy Orlow <[email protected]> > >>> wrote: > >>>> > >>>> On Thu, Aug 27, 2009 at 10:31 AM, <[email protected]> wrote: > >>>>> > >>>>> http://codereview.chromium.org/173555/diff/1/2 > >>>>> File webkit/tools/layout_tests/test_expectations.txt (right): > >>>>> > >>>>> http://codereview.chromium.org/173555/diff/1/2#newcode745 > >>>>> Line 745: //BUG13907 WIN : LayoutTests/media/video-src-remove.html = > >>>>> FAIL > >>>>> Why leave this in commented out instead of just removing it? > >>>> > >>>> Well, there's 2 problems with that test. 1) it's flaky and 2) it's > >>>> failing. Someone fixing 2 doesn't necessarily fix 1. So they'll > probably > >>>> want to add it back into that list when they're finished. It's also > >>>> documentation. I don't really see any downside. > >>> > >>> We have a process for dealing with flaky tests. If it passes sometimes > >>> and fails sometimes, mark it PASS FAIL. Why is this case an exception? > I see > >>> two downsides: > >>> > >>> Now when it fails, it will turn the tree red. > >>> If we make a practice of leaving in commented out tests this file > becomes > >>> even more bloated than it is now. > > > > > --~--~---------~--~----~------------~-------~--~----~ Chromium Developers mailing list: [email protected] View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev -~----------~----~----~----~------~----~------~--~---
