The media tests are highly platform specific at the moment due to differences in codec support. Keeping them lumped together under one bug works for me. Andrew
On Thu, Aug 27, 2009 at 1:55 PM, Dimitri Glazkov <[email protected]>wrote: > > Tests only pass if they are actually not loading any media. Which is a > number of tests, yes. As an alternative maybe we could just agree > among all WebKit gardeners to dump the new failures to bug 16779 and > not add them to the end of test_expectations? > > :DG< > > On Thu, Aug 27, 2009 at 1:25 PM, Ojan Vafai<[email protected]> wrote: > > -chromium-reviews > > +chromium-dev > > Don't we currently pass a large percentage of the LayoutTests/media > tests? > > Won't we lose coverage of these tests by doing this? > > Maybe it's worth the savings maintenance cost until we have a complete > > solution for them? > > > > Ojan > > On Thu, Aug 27, 2009 at 11:46 AM, Dimitri Glazkov <[email protected] > > > > wrote: > >> > >> BTW, about to filter out all LayoutTests/media out for now, skipping > >> them. We can't have codecs for most of these tests, so we (Alpha and > >> myself) decided it's easier to skip them for now. Permanent solution > >> coming in Q4. > >> > >> :DG< > >> > >> On Thu, Aug 27, 2009 at 11:42 AM, Ojan Vafai<[email protected]> wrote: > >> > I see what you're saying. I think the best solution here is to just > have > >> > one > >> > line with two bugs listed. This will also help whoever is going to fix > >> > the > >> > test as they'll be able to easily see all the reason's it's failing. > >> > BUG20376 BUG13907 WIN : LayoutTests/media/video-src-remove.html = > >> > TIMEOUT > >> > FAIL > >> > That OK? > >> > On Thu, Aug 27, 2009 at 11:02 AM, Jeremy Orlow <[email protected]> > >> > wrote: > >> >> > >> >> Yes, but it's not flaky now. If you'd like, I can move the commented > >> >> out > >> >> one down to right after the new version of it. I think what'd be > even > >> >> better is if multiple entries for one layout test didn't cause the > >> >> script to > >> >> parse. Or it'd at least be nice if we could list 2 bugs for one > test. > >> >> The commented out one is NOT dead....it's another type of failure > that > >> >> is > >> >> currently overshadowed by the new type of failure. > >> >> > >> >> On Thu, Aug 27, 2009 at 10:57 AM, Ojan Vafai <[email protected]> > wrote: > >> >>> > >> >>> On Thu, Aug 27, 2009 at 10:34 AM, Jeremy Orlow <[email protected] > > > >> >>> wrote: > >> >>>> > >> >>>> On Thu, Aug 27, 2009 at 10:31 AM, <[email protected]> wrote: > >> >>>>> > >> >>>>> http://codereview.chromium.org/173555/diff/1/2 > >> >>>>> File webkit/tools/layout_tests/test_expectations.txt (right): > >> >>>>> > >> >>>>> http://codereview.chromium.org/173555/diff/1/2#newcode745 > >> >>>>> Line 745: //BUG13907 WIN : LayoutTests/media/video-src-remove.html > = > >> >>>>> FAIL > >> >>>>> Why leave this in commented out instead of just removing it? > >> >>>> > >> >>>> Well, there's 2 problems with that test. 1) it's flaky and 2) it's > >> >>>> failing. Someone fixing 2 doesn't necessarily fix 1. So they'll > >> >>>> probably > >> >>>> want to add it back into that list when they're finished. It's > also > >> >>>> documentation. I don't really see any downside. > >> >>> > >> >>> We have a process for dealing with flaky tests. If it passes > sometimes > >> >>> and fails sometimes, mark it PASS FAIL. Why is this case an > exception? > >> >>> I see > >> >>> two downsides: > >> >>> > >> >>> Now when it fails, it will turn the tree red. > >> >>> If we make a practice of leaving in commented out tests this file > >> >>> becomes > >> >>> even more bloated than it is now. > >> > > >> > > > > > > > > > --~--~---------~--~----~------------~-------~--~----~ Chromium Developers mailing list: [email protected] View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev -~----------~----~----~----~------~----~------~--~---
