The end result here sounds good to me. Just a few side comments: * If we ever set up a directory for baselines common to all Chromium platforms, it should be called 'chromium'. This matches the WebKit side, where they have mac-tiger, mac-leopard, and mac. In fact, the layout-test script is already set up to look in 'chromium' (after chromium-<this-platform> and before chromium-win or mac, IIRC).
* We initially decided to avoid such a directory because it makes it harder to figure out what to do when you make a change. For example, when one platform diverges it's not clear that you have to move a file from chromium into each other platform's directory. The kjs/v8/common split got terribly jumbled after a while. Deliberate duplication is inconvenient, but at least it's straightforward. * If you run_webkit_tests with the --sources option, it will tell you the full path to the expected result file it's using for each comparison (text and image). - Pam On Thu, Jan 29, 2009 at 10:52 PM, Mads Sig Ager <[email protected]> wrote: > > Thanks Tony, that would make it easier to work with. > > I still think that it is pretty confusing that chromium-linux does a > fallback to chromium-win when chromium-mac doesn't. But maybe there > is no way to avoid confusion here. > > -- Mads > > On Thu, Jan 29, 2009 at 6:17 PM, <[email protected]> wrote: > > In practice, people tend to only test on one platform and create > > baselines for one platform. Normally, it's sufficient to add a comment > > to the tests_fixable file that the test is fixed on platform X and > > probably needs to be rebaselined on platform Y and Z. > > > > Anyway, I think the fix is to change run_webkit_tests.py --new-baseline > > to check to see if it's a pixel test or not and if it's not a pixel test, > > it should automatically copy the results into chromium-win and > > chromium-mac. > > > > I'll look into doing this today. > > > > tony > > > > On Thu, 29 Jan 2009, Dean McNamee wrote: > > > >> > >> I guess Mads's point here was that he works on V8, and when he wants > >> to fix something for V8 (rebaseline), it's not clear to him where it > >> should go. Should he copy it into all 3 places? The idea was maybe > >> there should be a chromium-common (which is not chromium-win), where > >> we can stick fallbacks where we know all platforms should match. > >> > >> It gets difficult to manage expectations across 3 platforms, > >> especially when you think they should be the same. We've had that a > >> lot now, someone stumbles over a broken test on Linux, and finds out > >> that it was rebaselined on Windows already, etc. It's just confusing > >> / a lot of work for someone like Mads's on the V8 team to know how to > >> handle all 3 platforms differently... > >> > >> On Thu, Jan 29, 2009 at 4:50 PM, Thomas Van Lenten > >> <[email protected]> wrote: > >> > > >> > On Thu, Jan 29, 2009 at 10:44 AM, Dean McNamee <[email protected]> > wrote: > >> >> > >> >> On Thu, Jan 29, 2009 at 4:43 PM, Mark Mentovai <[email protected]> > wrote: > >> >> > On the Mac, I think we want to match Apple WebKit baselines. I > don't > >> >> > know if there are any baselines currently in chromium-win that we > >> >> > should share. > >> >> > >> >> All of the V8 differences, for example. > >> > > >> > We copy those into chromium-mac as needed. But the majority of the > expected > >> > files come down to fonts and the windows files wouldn't be of any use > >> > there. In the pixel dumps, again, font and controls pretty much make > using > >> > the windows ones pointless. > >> > > >> > TVL > >> > > >> >> > >> >> > > >> >> > Mark > >> >> > > >> >> > Dean wrote: > >> >> >> I talked to Mads a bit, basically: > >> >> >> > >> >> >> 1) I think the Mac expected result fallback is currently wrong, it > >> >> >> doesn't seem to look in chromium-win correctly. This is probably > >> >> >> causing a lot of failures. > >> >> >> > >> >> >> 2) We should move chromium-win to chromium (or chromium-common), > and > >> >> >> then chromium-win should not be a fallback. This might be more > >> >> >> confusing to manage, but it's also less confusing to understand > that > >> >> >> everything should / can fallback to the Windows expectations. > >> >> >> > >> >> >> On Thu, Jan 29, 2009 at 2:49 PM, Mads Sig Ager <[email protected] > > > >> >> >> wrote: > >> >> >>> > >> >> >>> It seems that when running layout tests on linux, if there are no > >> >> >>> special expected results for linux in chromium-linux, we fallback > to > >> >> >>> the special expected results for windows in chromium-win. This > is not > >> >> >>> the case on mac if there are no results in chromium-mac, we take > the > >> >> >>> expectations that are next to the test even if there are other > >> >> >>> expectations in chromium-win. Is that on purpose? > >> >> >>> > >> >> >>> A related question: what is the intention with our custom > expected > >> >> >>> resulsts? If we need to change the expectation for all three > >> >> >>> platforms, should we only add the new expectations in > chromium-win? > >> >> >>> That sounds confusing to me. Maybe we should have a > chromium-common? > >> >> >>> > >> >> >>> Cheers, -- Mads > >> >> >>> > >> >> >>> >>> > >> >> >>> > >> >> >> > >> >> > > >> >> > >> >> >> > >> > > >> > > >> > >> >> > > > > > > --~--~---------~--~----~------------~-------~--~----~ Chromium Developers mailing list: [email protected] View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev -~----------~----~----~----~------~----~------~--~---
