[chromium-dev] Re: Layout test expectations and fallbacks
The end result here sounds good to me. Just a few side comments: * If we ever set up a directory for baselines common to all Chromium platforms, it should be called 'chromium'. This matches the WebKit side, where they have mac-tiger, mac-leopard, and mac. In fact, the layout-test script is already set up to look in 'chromium' (after chromium-this-platform and before chromium-win or mac, IIRC). * We initially decided to avoid such a directory because it makes it harder to figure out what to do when you make a change. For example, when one platform diverges it's not clear that you have to move a file from chromium into each other platform's directory. The kjs/v8/common split got terribly jumbled after a while. Deliberate duplication is inconvenient, but at least it's straightforward. * If you run_webkit_tests with the --sources option, it will tell you the full path to the expected result file it's using for each comparison (text and image). - Pam On Thu, Jan 29, 2009 at 10:52 PM, Mads Sig Ager a...@chromium.org wrote: Thanks Tony, that would make it easier to work with. I still think that it is pretty confusing that chromium-linux does a fallback to chromium-win when chromium-mac doesn't. But maybe there is no way to avoid confusion here. -- Mads On Thu, Jan 29, 2009 at 6:17 PM, t...@chromium.org wrote: In practice, people tend to only test on one platform and create baselines for one platform. Normally, it's sufficient to add a comment to the tests_fixable file that the test is fixed on platform X and probably needs to be rebaselined on platform Y and Z. Anyway, I think the fix is to change run_webkit_tests.py --new-baseline to check to see if it's a pixel test or not and if it's not a pixel test, it should automatically copy the results into chromium-win and chromium-mac. I'll look into doing this today. tony On Thu, 29 Jan 2009, Dean McNamee wrote: I guess Mads's point here was that he works on V8, and when he wants to fix something for V8 (rebaseline), it's not clear to him where it should go. Should he copy it into all 3 places? The idea was maybe there should be a chromium-common (which is not chromium-win), where we can stick fallbacks where we know all platforms should match. It gets difficult to manage expectations across 3 platforms, especially when you think they should be the same. We've had that a lot now, someone stumbles over a broken test on Linux, and finds out that it was rebaselined on Windows already, etc. It's just confusing / a lot of work for someone like Mads's on the V8 team to know how to handle all 3 platforms differently... On Thu, Jan 29, 2009 at 4:50 PM, Thomas Van Lenten thoma...@chromium.org wrote: On Thu, Jan 29, 2009 at 10:44 AM, Dean McNamee de...@chromium.org wrote: On Thu, Jan 29, 2009 at 4:43 PM, Mark Mentovai m...@chromium.org wrote: On the Mac, I think we want to match Apple WebKit baselines. I don't know if there are any baselines currently in chromium-win that we should share. All of the V8 differences, for example. We copy those into chromium-mac as needed. But the majority of the expected files come down to fonts and the windows files wouldn't be of any use there. In the pixel dumps, again, font and controls pretty much make using the windows ones pointless. TVL Mark Dean wrote: I talked to Mads a bit, basically: 1) I think the Mac expected result fallback is currently wrong, it doesn't seem to look in chromium-win correctly. This is probably causing a lot of failures. 2) We should move chromium-win to chromium (or chromium-common), and then chromium-win should not be a fallback. This might be more confusing to manage, but it's also less confusing to understand that everything should / can fallback to the Windows expectations. On Thu, Jan 29, 2009 at 2:49 PM, Mads Sig Ager a...@chromium.org wrote: It seems that when running layout tests on linux, if there are no special expected results for linux in chromium-linux, we fallback to the special expected results for windows in chromium-win. This is not the case on mac if there are no results in chromium-mac, we take the expectations that are next to the test even if there are other expectations in chromium-win. Is that on purpose? A related question: what is the intention with our custom expected resulsts? If we need to change the expectation for all three platforms, should we only add the new expectations in chromium-win? That sounds confusing to me. Maybe we should have a chromium-common? Cheers,-- Mads --~--~-~--~~~---~--~~ Chromium Developers mailing list: chromium-dev@googlegroups.com View archives, change email options, or unsubscribe:
[chromium-dev] Re: Layout test expectations and fallbacks
I talked to Mads a bit, basically: 1) I think the Mac expected result fallback is currently wrong, it doesn't seem to look in chromium-win correctly. This is probably causing a lot of failures. 2) We should move chromium-win to chromium (or chromium-common), and then chromium-win should not be a fallback. This might be more confusing to manage, but it's also less confusing to understand that everything should / can fallback to the Windows expectations. On Thu, Jan 29, 2009 at 2:49 PM, Mads Sig Ager a...@chromium.org wrote: It seems that when running layout tests on linux, if there are no special expected results for linux in chromium-linux, we fallback to the special expected results for windows in chromium-win. This is not the case on mac if there are no results in chromium-mac, we take the expectations that are next to the test even if there are other expectations in chromium-win. Is that on purpose? A related question: what is the intention with our custom expected resulsts? If we need to change the expectation for all three platforms, should we only add the new expectations in chromium-win? That sounds confusing to me. Maybe we should have a chromium-common? Cheers,-- Mads --~--~-~--~~~---~--~~ Chromium Developers mailing list: chromium-dev@googlegroups.com View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev -~--~~~~--~~--~--~---
[chromium-dev] Re: Layout test expectations and fallbacks
On the Mac, I think we want to match Apple WebKit baselines. I don't know if there are any baselines currently in chromium-win that we should share. Mark Dean wrote: I talked to Mads a bit, basically: 1) I think the Mac expected result fallback is currently wrong, it doesn't seem to look in chromium-win correctly. This is probably causing a lot of failures. 2) We should move chromium-win to chromium (or chromium-common), and then chromium-win should not be a fallback. This might be more confusing to manage, but it's also less confusing to understand that everything should / can fallback to the Windows expectations. On Thu, Jan 29, 2009 at 2:49 PM, Mads Sig Ager a...@chromium.org wrote: It seems that when running layout tests on linux, if there are no special expected results for linux in chromium-linux, we fallback to the special expected results for windows in chromium-win. This is not the case on mac if there are no results in chromium-mac, we take the expectations that are next to the test even if there are other expectations in chromium-win. Is that on purpose? A related question: what is the intention with our custom expected resulsts? If we need to change the expectation for all three platforms, should we only add the new expectations in chromium-win? That sounds confusing to me. Maybe we should have a chromium-common? Cheers,-- Mads --~--~-~--~~~---~--~~ Chromium Developers mailing list: chromium-dev@googlegroups.com View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev -~--~~~~--~~--~--~---
[chromium-dev] Re: Layout test expectations and fallbacks
On Thu, Jan 29, 2009 at 4:43 PM, Mark Mentovai m...@chromium.org wrote: On the Mac, I think we want to match Apple WebKit baselines. I don't know if there are any baselines currently in chromium-win that we should share. All of the V8 differences, for example. Mark Dean wrote: I talked to Mads a bit, basically: 1) I think the Mac expected result fallback is currently wrong, it doesn't seem to look in chromium-win correctly. This is probably causing a lot of failures. 2) We should move chromium-win to chromium (or chromium-common), and then chromium-win should not be a fallback. This might be more confusing to manage, but it's also less confusing to understand that everything should / can fallback to the Windows expectations. On Thu, Jan 29, 2009 at 2:49 PM, Mads Sig Ager a...@chromium.org wrote: It seems that when running layout tests on linux, if there are no special expected results for linux in chromium-linux, we fallback to the special expected results for windows in chromium-win. This is not the case on mac if there are no results in chromium-mac, we take the expectations that are next to the test even if there are other expectations in chromium-win. Is that on purpose? A related question: what is the intention with our custom expected resulsts? If we need to change the expectation for all three platforms, should we only add the new expectations in chromium-win? That sounds confusing to me. Maybe we should have a chromium-common? Cheers,-- Mads --~--~-~--~~~---~--~~ Chromium Developers mailing list: chromium-dev@googlegroups.com View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev -~--~~~~--~~--~--~---
[chromium-dev] Re: Layout test expectations and fallbacks
On Thu, Jan 29, 2009 at 10:44 AM, Dean McNamee de...@chromium.org wrote: On Thu, Jan 29, 2009 at 4:43 PM, Mark Mentovai m...@chromium.org wrote: On the Mac, I think we want to match Apple WebKit baselines. I don't know if there are any baselines currently in chromium-win that we should share. All of the V8 differences, for example. We copy those into chromium-mac as needed. But the majority of the expected files come down to fonts and the windows files wouldn't be of any use there. In the pixel dumps, again, font and controls pretty much make using the windows ones pointless. TVL Mark Dean wrote: I talked to Mads a bit, basically: 1) I think the Mac expected result fallback is currently wrong, it doesn't seem to look in chromium-win correctly. This is probably causing a lot of failures. 2) We should move chromium-win to chromium (or chromium-common), and then chromium-win should not be a fallback. This might be more confusing to manage, but it's also less confusing to understand that everything should / can fallback to the Windows expectations. On Thu, Jan 29, 2009 at 2:49 PM, Mads Sig Ager a...@chromium.org wrote: It seems that when running layout tests on linux, if there are no special expected results for linux in chromium-linux, we fallback to the special expected results for windows in chromium-win. This is not the case on mac if there are no results in chromium-mac, we take the expectations that are next to the test even if there are other expectations in chromium-win. Is that on purpose? A related question: what is the intention with our custom expected resulsts? If we need to change the expectation for all three platforms, should we only add the new expectations in chromium-win? That sounds confusing to me. Maybe we should have a chromium-common? Cheers,-- Mads --~--~-~--~~~---~--~~ Chromium Developers mailing list: chromium-dev@googlegroups.com View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev -~--~~~~--~~--~--~---
[chromium-dev] Re: Layout test expectations and fallbacks
I guess Mads's point here was that he works on V8, and when he wants to fix something for V8 (rebaseline), it's not clear to him where it should go. Should he copy it into all 3 places? The idea was maybe there should be a chromium-common (which is not chromium-win), where we can stick fallbacks where we know all platforms should match. It gets difficult to manage expectations across 3 platforms, especially when you think they should be the same. We've had that a lot now, someone stumbles over a broken test on Linux, and finds out that it was rebaselined on Windows already, etc. It's just confusing / a lot of work for someone like Mads's on the V8 team to know how to handle all 3 platforms differently... On Thu, Jan 29, 2009 at 4:50 PM, Thomas Van Lenten thoma...@chromium.org wrote: On Thu, Jan 29, 2009 at 10:44 AM, Dean McNamee de...@chromium.org wrote: On Thu, Jan 29, 2009 at 4:43 PM, Mark Mentovai m...@chromium.org wrote: On the Mac, I think we want to match Apple WebKit baselines. I don't know if there are any baselines currently in chromium-win that we should share. All of the V8 differences, for example. We copy those into chromium-mac as needed. But the majority of the expected files come down to fonts and the windows files wouldn't be of any use there. In the pixel dumps, again, font and controls pretty much make using the windows ones pointless. TVL Mark Dean wrote: I talked to Mads a bit, basically: 1) I think the Mac expected result fallback is currently wrong, it doesn't seem to look in chromium-win correctly. This is probably causing a lot of failures. 2) We should move chromium-win to chromium (or chromium-common), and then chromium-win should not be a fallback. This might be more confusing to manage, but it's also less confusing to understand that everything should / can fallback to the Windows expectations. On Thu, Jan 29, 2009 at 2:49 PM, Mads Sig Ager a...@chromium.org wrote: It seems that when running layout tests on linux, if there are no special expected results for linux in chromium-linux, we fallback to the special expected results for windows in chromium-win. This is not the case on mac if there are no results in chromium-mac, we take the expectations that are next to the test even if there are other expectations in chromium-win. Is that on purpose? A related question: what is the intention with our custom expected resulsts? If we need to change the expectation for all three platforms, should we only add the new expectations in chromium-win? That sounds confusing to me. Maybe we should have a chromium-common? Cheers,-- Mads --~--~-~--~~~---~--~~ Chromium Developers mailing list: chromium-dev@googlegroups.com View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev -~--~~~~--~~--~--~---
[chromium-dev] Re: Layout test expectations and fallbacks
V8 differences from JSCore are really the only area where some extra effort is needed. As Mark said, for the vast majority of tests we want to use the WebCore from webkit.org, not chromium-win. Yes, it sucks we have to put stuff in two places (chromium-win and chromium-mac) but I don't see any other way around it. Also, the test results should be more or less identical (at least they have been in the cases I've seen), so it really is just copying the same file to two places in the tree. The V8 differences are far far fewer in number than the general pixel results. What else would you propose? On Thu, Jan 29, 2009 at 10:53 AM, Dean McNamee de...@chromium.org wrote: I guess Mads's point here was that he works on V8, and when he wants to fix something for V8 (rebaseline), it's not clear to him where it should go. Should he copy it into all 3 places? The idea was maybe there should be a chromium-common (which is not chromium-win), where we can stick fallbacks where we know all platforms should match. It gets difficult to manage expectations across 3 platforms, especially when you think they should be the same. We've had that a lot now, someone stumbles over a broken test on Linux, and finds out that it was rebaselined on Windows already, etc. It's just confusing / a lot of work for someone like Mads's on the V8 team to know how to handle all 3 platforms differently... On Thu, Jan 29, 2009 at 4:50 PM, Thomas Van Lenten thoma...@chromium.org wrote: On Thu, Jan 29, 2009 at 10:44 AM, Dean McNamee de...@chromium.org wrote: On Thu, Jan 29, 2009 at 4:43 PM, Mark Mentovai m...@chromium.org wrote: On the Mac, I think we want to match Apple WebKit baselines. I don't know if there are any baselines currently in chromium-win that we should share. All of the V8 differences, for example. We copy those into chromium-mac as needed. But the majority of the expected files come down to fonts and the windows files wouldn't be of any use there. In the pixel dumps, again, font and controls pretty much make using the windows ones pointless. TVL Mark Dean wrote: I talked to Mads a bit, basically: 1) I think the Mac expected result fallback is currently wrong, it doesn't seem to look in chromium-win correctly. This is probably causing a lot of failures. 2) We should move chromium-win to chromium (or chromium-common), and then chromium-win should not be a fallback. This might be more confusing to manage, but it's also less confusing to understand that everything should / can fallback to the Windows expectations. On Thu, Jan 29, 2009 at 2:49 PM, Mads Sig Ager a...@chromium.org wrote: It seems that when running layout tests on linux, if there are no special expected results for linux in chromium-linux, we fallback to the special expected results for windows in chromium-win. This is not the case on mac if there are no results in chromium-mac, we take the expectations that are next to the test even if there are other expectations in chromium-win. Is that on purpose? A related question: what is the intention with our custom expected resulsts? If we need to change the expectation for all three platforms, should we only add the new expectations in chromium-win? That sounds confusing to me. Maybe we should have a chromium-common? Cheers,-- Mads -- Mike Pinkerton Mac Weenie pinker...@google.com --~--~-~--~~~---~--~~ Chromium Developers mailing list: chromium-dev@googlegroups.com View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev -~--~~~~--~~--~--~---
[chromium-dev] Re: Layout test expectations and fallbacks
In the past, when we were tracking v8 and kjs baselines, I believe we
did have a common output dir for tests that matched. The problem
then becomes there are a lot of different places to look for a given
layout test output. I use a little shell script already so it
wouldn't bother me much. *shrug*
% ../test-results.sh LayoutTests/http/tests/media/video-play-stall-seek*
webkit/data/layout_tests/platform/chromium-linux/LayoutTests/http/tests/media/video-play-stall-seek-expected.checksum
webkit/data/layout_tests/platform/chromium-linux/LayoutTests/http/tests/media/video-play-stall-seek-expected.png
webkit/data/layout_tests/platform/chromium-win/LayoutTests/http/tests/media/video-play-stall-seek-expected.checksum
webkit/data/layout_tests/platform/chromium-win/LayoutTests/http/tests/media/video-play-stall-seek-expected.png
% cat ../test-results.sh
#!/bin/bash
target=$1
webkit_style_target=${target#LayoutTests/}
base=webkit/data/layout_tests
chrome_platform=$base/platform
webkit_platform=$base/LayoutTests/platform
for platform in $chrome_platform/* $webkit_platform/*; do
for file in $platform/$target $platform/$webkit_style_target; do
[ -e $file ] echo $file
done
done
exit 0
On Thu, Jan 29, 2009 at 8:17 AM, Mike Pinkerton pinker...@chromium.org wrote:
V8 differences from JSCore are really the only area where some extra
effort is needed. As Mark said, for the vast majority of tests we want
to use the WebCore from webkit.org, not chromium-win.
Yes, it sucks we have to put stuff in two places (chromium-win and
chromium-mac) but I don't see any other way around it. Also, the test
results should be more or less identical (at least they have been in
the cases I've seen), so it really is just copying the same file to
two places in the tree. The V8 differences are far far fewer in number
than the general pixel results.
What else would you propose?
On Thu, Jan 29, 2009 at 10:53 AM, Dean McNamee de...@chromium.org wrote:
I guess Mads's point here was that he works on V8, and when he wants
to fix something for V8 (rebaseline), it's not clear to him where it
should go. Should he copy it into all 3 places? The idea was maybe
there should be a chromium-common (which is not chromium-win), where
we can stick fallbacks where we know all platforms should match.
It gets difficult to manage expectations across 3 platforms,
especially when you think they should be the same. We've had that a
lot now, someone stumbles over a broken test on Linux, and finds out
that it was rebaselined on Windows already, etc. It's just confusing
/ a lot of work for someone like Mads's on the V8 team to know how to
handle all 3 platforms differently...
On Thu, Jan 29, 2009 at 4:50 PM, Thomas Van Lenten
thoma...@chromium.org wrote:
On Thu, Jan 29, 2009 at 10:44 AM, Dean McNamee de...@chromium.org wrote:
On Thu, Jan 29, 2009 at 4:43 PM, Mark Mentovai m...@chromium.org wrote:
On the Mac, I think we want to match Apple WebKit baselines. I don't
know if there are any baselines currently in chromium-win that we
should share.
All of the V8 differences, for example.
We copy those into chromium-mac as needed. But the majority of the expected
files come down to fonts and the windows files wouldn't be of any use
there. In the pixel dumps, again, font and controls pretty much make using
the windows ones pointless.
TVL
Mark
Dean wrote:
I talked to Mads a bit, basically:
1) I think the Mac expected result fallback is currently wrong, it
doesn't seem to look in chromium-win correctly. This is probably
causing a lot of failures.
2) We should move chromium-win to chromium (or chromium-common), and
then chromium-win should not be a fallback. This might be more
confusing to manage, but it's also less confusing to understand that
everything should / can fallback to the Windows expectations.
On Thu, Jan 29, 2009 at 2:49 PM, Mads Sig Ager a...@chromium.org
wrote:
It seems that when running layout tests on linux, if there are no
special expected results for linux in chromium-linux, we fallback to
the special expected results for windows in chromium-win. This is not
the case on mac if there are no results in chromium-mac, we take the
expectations that are next to the test even if there are other
expectations in chromium-win. Is that on purpose?
A related question: what is the intention with our custom expected
resulsts? If we need to change the expectation for all three
platforms, should we only add the new expectations in chromium-win?
That sounds confusing to me. Maybe we should have a chromium-common?
Cheers,-- Mads
--
Mike Pinkerton
Mac Weenie
pinker...@google.com
--~--~-~--~~~---~--~~
Chromium Developers mailing list: chromium-dev@googlegroups.com
View archives, change email options, or unsubscribe:
http://groups.google.com/group/chromium-dev
[chromium-dev] Re: Layout test expectations and fallbacks
In practice, people tend to only test on one platform and create baselines for one platform. Normally, it's sufficient to add a comment to the tests_fixable file that the test is fixed on platform X and probably needs to be rebaselined on platform Y and Z. Anyway, I think the fix is to change run_webkit_tests.py --new-baseline to check to see if it's a pixel test or not and if it's not a pixel test, it should automatically copy the results into chromium-win and chromium-mac. I'll look into doing this today. tony On Thu, 29 Jan 2009, Dean McNamee wrote: I guess Mads's point here was that he works on V8, and when he wants to fix something for V8 (rebaseline), it's not clear to him where it should go. Should he copy it into all 3 places? The idea was maybe there should be a chromium-common (which is not chromium-win), where we can stick fallbacks where we know all platforms should match. It gets difficult to manage expectations across 3 platforms, especially when you think they should be the same. We've had that a lot now, someone stumbles over a broken test on Linux, and finds out that it was rebaselined on Windows already, etc. It's just confusing / a lot of work for someone like Mads's on the V8 team to know how to handle all 3 platforms differently... On Thu, Jan 29, 2009 at 4:50 PM, Thomas Van Lenten thoma...@chromium.org wrote: On Thu, Jan 29, 2009 at 10:44 AM, Dean McNamee de...@chromium.org wrote: On Thu, Jan 29, 2009 at 4:43 PM, Mark Mentovai m...@chromium.org wrote: On the Mac, I think we want to match Apple WebKit baselines. I don't know if there are any baselines currently in chromium-win that we should share. All of the V8 differences, for example. We copy those into chromium-mac as needed. But the majority of the expected files come down to fonts and the windows files wouldn't be of any use there. In the pixel dumps, again, font and controls pretty much make using the windows ones pointless. TVL Mark Dean wrote: I talked to Mads a bit, basically: 1) I think the Mac expected result fallback is currently wrong, it doesn't seem to look in chromium-win correctly. This is probably causing a lot of failures. 2) We should move chromium-win to chromium (or chromium-common), and then chromium-win should not be a fallback. This might be more confusing to manage, but it's also less confusing to understand that everything should / can fallback to the Windows expectations. On Thu, Jan 29, 2009 at 2:49 PM, Mads Sig Ager a...@chromium.org wrote: It seems that when running layout tests on linux, if there are no special expected results for linux in chromium-linux, we fallback to the special expected results for windows in chromium-win. This is not the case on mac if there are no results in chromium-mac, we take the expectations that are next to the test even if there are other expectations in chromium-win. Is that on purpose? A related question: what is the intention with our custom expected resulsts? If we need to change the expectation for all three platforms, should we only add the new expectations in chromium-win? That sounds confusing to me. Maybe we should have a chromium-common? Cheers,-- Mads --~--~-~--~~~---~--~~ Chromium Developers mailing list: chromium-dev@googlegroups.com View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev -~--~~~~--~~--~--~---
[chromium-dev] Re: Layout test expectations and fallbacks
Thanks Tony, that would make it easier to work with. I still think that it is pretty confusing that chromium-linux does a fallback to chromium-win when chromium-mac doesn't. But maybe there is no way to avoid confusion here. -- Mads On Thu, Jan 29, 2009 at 6:17 PM, t...@chromium.org wrote: In practice, people tend to only test on one platform and create baselines for one platform. Normally, it's sufficient to add a comment to the tests_fixable file that the test is fixed on platform X and probably needs to be rebaselined on platform Y and Z. Anyway, I think the fix is to change run_webkit_tests.py --new-baseline to check to see if it's a pixel test or not and if it's not a pixel test, it should automatically copy the results into chromium-win and chromium-mac. I'll look into doing this today. tony On Thu, 29 Jan 2009, Dean McNamee wrote: I guess Mads's point here was that he works on V8, and when he wants to fix something for V8 (rebaseline), it's not clear to him where it should go. Should he copy it into all 3 places? The idea was maybe there should be a chromium-common (which is not chromium-win), where we can stick fallbacks where we know all platforms should match. It gets difficult to manage expectations across 3 platforms, especially when you think they should be the same. We've had that a lot now, someone stumbles over a broken test on Linux, and finds out that it was rebaselined on Windows already, etc. It's just confusing / a lot of work for someone like Mads's on the V8 team to know how to handle all 3 platforms differently... On Thu, Jan 29, 2009 at 4:50 PM, Thomas Van Lenten thoma...@chromium.org wrote: On Thu, Jan 29, 2009 at 10:44 AM, Dean McNamee de...@chromium.org wrote: On Thu, Jan 29, 2009 at 4:43 PM, Mark Mentovai m...@chromium.org wrote: On the Mac, I think we want to match Apple WebKit baselines. I don't know if there are any baselines currently in chromium-win that we should share. All of the V8 differences, for example. We copy those into chromium-mac as needed. But the majority of the expected files come down to fonts and the windows files wouldn't be of any use there. In the pixel dumps, again, font and controls pretty much make using the windows ones pointless. TVL Mark Dean wrote: I talked to Mads a bit, basically: 1) I think the Mac expected result fallback is currently wrong, it doesn't seem to look in chromium-win correctly. This is probably causing a lot of failures. 2) We should move chromium-win to chromium (or chromium-common), and then chromium-win should not be a fallback. This might be more confusing to manage, but it's also less confusing to understand that everything should / can fallback to the Windows expectations. On Thu, Jan 29, 2009 at 2:49 PM, Mads Sig Ager a...@chromium.org wrote: It seems that when running layout tests on linux, if there are no special expected results for linux in chromium-linux, we fallback to the special expected results for windows in chromium-win. This is not the case on mac if there are no results in chromium-mac, we take the expectations that are next to the test even if there are other expectations in chromium-win. Is that on purpose? A related question: what is the intention with our custom expected resulsts? If we need to change the expectation for all three platforms, should we only add the new expectations in chromium-win? That sounds confusing to me. Maybe we should have a chromium-common? Cheers,-- Mads --~--~-~--~~~---~--~~ Chromium Developers mailing list: chromium-dev@googlegroups.com View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev -~--~~~~--~~--~--~---
