In practice, people tend to only test on one platform and create baselines for one platform. Normally, it's sufficient to add a comment to the tests_fixable file that the test is fixed on platform X and probably needs to be rebaselined on platform Y and Z.
Anyway, I think the fix is to change run_webkit_tests.py --new-baseline to check to see if it's a pixel test or not and if it's not a pixel test, it should automatically copy the results into chromium-win and chromium-mac. I'll look into doing this today. tony On Thu, 29 Jan 2009, Dean McNamee wrote: > > I guess Mads's point here was that he works on V8, and when he wants > to fix something for V8 (rebaseline), it's not clear to him where it > should go. Should he copy it into all 3 places? The idea was maybe > there should be a chromium-common (which is not chromium-win), where > we can stick fallbacks where we know all platforms should match. > > It gets difficult to manage expectations across 3 platforms, > especially when you think they should be the same. We've had that a > lot now, someone stumbles over a broken test on Linux, and finds out > that it was rebaselined on Windows already, etc. It's just confusing > / a lot of work for someone like Mads's on the V8 team to know how to > handle all 3 platforms differently... > > On Thu, Jan 29, 2009 at 4:50 PM, Thomas Van Lenten > <[email protected]> wrote: > > > > On Thu, Jan 29, 2009 at 10:44 AM, Dean McNamee <[email protected]> wrote: > >> > >> On Thu, Jan 29, 2009 at 4:43 PM, Mark Mentovai <[email protected]> wrote: > >> > On the Mac, I think we want to match Apple WebKit baselines. I don't > >> > know if there are any baselines currently in chromium-win that we > >> > should share. > >> > >> All of the V8 differences, for example. > > > > We copy those into chromium-mac as needed. But the majority of the expected > > files come down to fonts and the windows files wouldn't be of any use > > there. In the pixel dumps, again, font and controls pretty much make using > > the windows ones pointless. > > > > TVL > > > >> > >> > > >> > Mark > >> > > >> > Dean wrote: > >> >> I talked to Mads a bit, basically: > >> >> > >> >> 1) I think the Mac expected result fallback is currently wrong, it > >> >> doesn't seem to look in chromium-win correctly. This is probably > >> >> causing a lot of failures. > >> >> > >> >> 2) We should move chromium-win to chromium (or chromium-common), and > >> >> then chromium-win should not be a fallback. This might be more > >> >> confusing to manage, but it's also less confusing to understand that > >> >> everything should / can fallback to the Windows expectations. > >> >> > >> >> On Thu, Jan 29, 2009 at 2:49 PM, Mads Sig Ager <[email protected]> > >> >> wrote: > >> >>> > >> >>> It seems that when running layout tests on linux, if there are no > >> >>> special expected results for linux in chromium-linux, we fallback to > >> >>> the special expected results for windows in chromium-win. This is not > >> >>> the case on mac if there are no results in chromium-mac, we take the > >> >>> expectations that are next to the test even if there are other > >> >>> expectations in chromium-win. Is that on purpose? > >> >>> > >> >>> A related question: what is the intention with our custom expected > >> >>> resulsts? If we need to change the expectation for all three > >> >>> platforms, should we only add the new expectations in chromium-win? > >> >>> That sounds confusing to me. Maybe we should have a chromium-common? > >> >>> > >> >>> Cheers, -- Mads > >> >>> > >> >>> >>> > >> >>> > >> >> > >> > > >> > >> >> > > > > > > > --~--~---------~--~----~------------~-------~--~----~ Chromium Developers mailing list: [email protected] View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev -~----------~----~----~----~------~----~------~--~---
