[chromium-dev] Re: how to judge layouttests' running result?
Well , I reviewed my layout-tests' output again, and found some errors like: 090629 14:28:30 __init__.py:1032 ERROR LayoutTests/http/tests/xmlhttprequest/xml-encoding.html failed: Text diff mismatch Simplified text diff mismatch /cygdrive/e/mychromesrc/src/third_party/cygwin/bin/wdiff: /tmp/t101c.0: No such file or directory I think I missed something, right? --~--~-~--~~~---~--~~ Chromium Developers mailing list: chromium-dev@googlegroups.com View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev -~--~~~~--~~--~--~---
[chromium-dev] Re: how to judge layouttests' running result?
This is not a pass. Specifically Regressions: Unexpected failures lists the failures. Other times, you'll see Regressions: Unexpected * where * is crash, timeout, failures, etc. Also, it should have popped up a window with the results just for the failing tests. Ojan 2009/6/25 David Jones ds...@163.com I have ran layouttests on Windows XP, and got a result as: Expected to fail, but passed (5): LayoutTests/css2.1/t0805-c5519-brdr-r-01-e.html LayoutTests/css2.1/t0905-c5525-fltblck-00-d-ag.html LayoutTests/css2.1/t0905-c5525-flthw-00-c-g.html LayoutTests/css2.1/t0905-c5526-flthw-00-c-g.html LayoutTests/fast/encoding/invalid-UTF-8.html Expected to timeout, but passed (1): LayoutTests/http/tests/security/credentials-in-referer.html Regressions: Unexpected failures (8): LayoutTests/fast/css/css2-system-fonts.html = FAIL LayoutTests/fast/dom/anchor-toString.html = FAIL LayoutTests/fast/dom/java-applet-calls.html = FAIL LayoutTests/fast/dom/object-embed-plugin-scripting.html = FAIL LayoutTests/http/tests/security/ cross-frame-access-protocol-explicit-domain.ht ml = FAIL LayoutTests/http/tests/security/cross-frame-access-protocol.html = FAIL LayoutTests/platform/win/fast/text/uniscribe-missing-glyph.html = FAIL LayoutTests/plugins/embed-attributes-setting.html = FAIL Regressions: Unexpected timeouts (1): LayoutTests/http/tests/security/originHeader/origin-header-for-https.html = TI MEOUT -- = Tests to be fixed for the current release (786): 86 test cases (10.9%) Passed 364 test cases (46.3%) Skipped 289 test cases (36.8%) Text diff mismatch 191 test cases (24.3%) Simplified text diff mismatch 160 test cases (20.4%) Image mismatch 11 test cases (1.4%) Test timed out 6 test cases (0.8%) Test shell crashed 2 test cases (0.3%) No expected image found = Tests we want to pass for the current release (8984): 8271 test cases (92.1%) Passed 364 test cases (4.1%) Skipped 301 test cases (3.4%) Text diff mismatch 201 test cases (2.2%) Simplified text diff mismatch 164 test cases (1.8%) Image mismatch 12 test cases (0.1%) Test timed out 6 test cases (0.1%) Test shell crashed 2 test cases (0.0%) No expected image found = Tests to be fixed for a future release (0): = All tests (10884): 8753 test cases (80.4%) Passed 2130 test cases (19.6%) Skipped 411 test cases (3.8%) Text diff mismatch 302 test cases (2.8%) Simplified text diff mismatch 183 test cases (1.7%) Image mismatch 13 test cases (0.1%) Test timed out 6 test cases (0.1%) Test shell crashed 2 test cases (0.0%) No expected image found I don't know what it tells, am I passed the layouttests? There's also a layouttests in webkit, what's the relation between chrome's layouttest and webkit's layouttest? I think webkit's layouttest only run on Leopard. -- 200万种商品,最低价格,疯狂诱惑你http://count.mail.163.com/redirect/footer.htm?f=http://gouwu.youdao.com --~--~-~--~~~---~--~~ Chromium Developers mailing list: chromium-dev@googlegroups.com View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev -~--~~~~--~~--~--~---
[chromium-dev] Re: how to judge layouttests' running result?
http://dev.chromium.org/developers/testing/webkit-layout-tests says: ./run_webkit_tests.sh LayoutTests/path/to/your/test.html --full-results-html will produce a page including links to the expected result, actual result, and diff. Does that help? 2009/6/25 David Jones ds...@163.com: I have ran layouttests on Windows XP, and got a result as: Expected to fail, but passed (5): LayoutTests/css2.1/t0805-c5519-brdr-r-01-e.html LayoutTests/css2.1/t0905-c5525-fltblck-00-d-ag.html LayoutTests/css2.1/t0905-c5525-flthw-00-c-g.html LayoutTests/css2.1/t0905-c5526-flthw-00-c-g.html LayoutTests/fast/encoding/invalid-UTF-8.html Expected to timeout, but passed (1): LayoutTests/http/tests/security/credentials-in-referer.html Regressions: Unexpected failures (8): LayoutTests/fast/css/css2-system-fonts.html = FAIL LayoutTests/fast/dom/anchor-toString.html = FAIL LayoutTests/fast/dom/java-applet-calls.html = FAIL LayoutTests/fast/dom/object-embed-plugin-scripting.html = FAIL LayoutTests/http/tests/security/cross-frame-access-protocol-explicit-domain.ht ml = FAIL LayoutTests/http/tests/security/cross-frame-access-protocol.html = FAIL LayoutTests/platform/win/fast/text/uniscribe-missing-glyph.html = FAIL LayoutTests/plugins/embed-attributes-setting.html = FAIL Regressions: Unexpected timeouts (1): LayoutTests/http/tests/security/originHeader/origin-header-for-https.html = TI MEOUT -- = Tests to be fixed for the current release (786): 86 test cases (10.9%) Passed 364 test cases (46.3%) Skipped 289 test cases (36.8%) Text diff mismatch 191 test cases (24.3%) Simplified text diff mismatch 160 test cases (20.4%) Image mismatch 11 test cases (1.4%) Test timed out 6 test cases (0.8%) Test shell crashed 2 test cases (0.3%) No expected image found = Tests we want to pass for the current release (8984): 8271 test cases (92.1%) Passed 364 test cases (4.1%) Skipped 301 test cases (3.4%) Text diff mismatch 201 test cases (2.2%) Simplified text diff mismatch 164 test cases (1.8%) Image mismatch 12 test cases (0.1%) Test timed out 6 test cases (0.1%) Test shell crashed 2 test cases (0.0%) No expected image found = Tests to be fixed for a future release (0): = All tests (10884): 8753 test cases (80.4%) Passed 2130 test cases (19.6%) Skipped 411 test cases (3.8%) Text diff mismatch 302 test cases (2.8%) Simplified text diff mismatch 183 test cases (1.7%) Image mismatch 13 test cases (0.1%) Test timed out 6 test cases (0.1%) Test shell crashed 2 test cases (0.0%) No expected image found I don't know what it tells, am I passed the layouttests? There's also a layouttests in webkit, what's the relation between chrome's layouttest and webkit's layouttest? I think webkit's layouttest only run on Leopard. 200万种商品,最低价格,疯狂诱惑你 --~--~-~--~~~---~--~~ Chromium Developers mailing list: chromium-dev@googlegroups.com View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev -~--~~~~--~~--~--~---
[chromium-dev] Re: how to judge layouttests' running result?
I usually try to execute the regressing tests manually (most of the time by just open the html file in test shell, sometimes you have to start some local web server). If they pass, I tell myself that the test is just flaky. If they still fail, I start debugging. 2009/6/25 David Jones ds...@163.com: I have ran layouttests on Windows XP, and got a result as: Expected to fail, but passed (5): LayoutTests/css2.1/t0805-c5519-brdr-r-01-e.html LayoutTests/css2.1/t0905-c5525-fltblck-00-d-ag.html LayoutTests/css2.1/t0905-c5525-flthw-00-c-g.html LayoutTests/css2.1/t0905-c5526-flthw-00-c-g.html LayoutTests/fast/encoding/invalid-UTF-8.html Expected to timeout, but passed (1): LayoutTests/http/tests/security/credentials-in-referer.html Regressions: Unexpected failures (8): LayoutTests/fast/css/css2-system-fonts.html = FAIL LayoutTests/fast/dom/anchor-toString.html = FAIL LayoutTests/fast/dom/java-applet-calls.html = FAIL LayoutTests/fast/dom/object-embed-plugin-scripting.html = FAIL LayoutTests/http/tests/security/cross-frame-access-protocol-explicit-domain.ht ml = FAIL LayoutTests/http/tests/security/cross-frame-access-protocol.html = FAIL LayoutTests/platform/win/fast/text/uniscribe-missing-glyph.html = FAIL LayoutTests/plugins/embed-attributes-setting.html = FAIL Regressions: Unexpected timeouts (1): LayoutTests/http/tests/security/originHeader/origin-header-for-https.html = TI MEOUT -- = Tests to be fixed for the current release (786): 86 test cases (10.9%) Passed 364 test cases (46.3%) Skipped 289 test cases (36.8%) Text diff mismatch 191 test cases (24.3%) Simplified text diff mismatch 160 test cases (20.4%) Image mismatch 11 test cases (1.4%) Test timed out 6 test cases (0.8%) Test shell crashed 2 test cases (0.3%) No expected image found = Tests we want to pass for the current release (8984): 8271 test cases (92.1%) Passed 364 test cases (4.1%) Skipped 301 test cases (3.4%) Text diff mismatch 201 test cases (2.2%) Simplified text diff mismatch 164 test cases (1.8%) Image mismatch 12 test cases (0.1%) Test timed out 6 test cases (0.1%) Test shell crashed 2 test cases (0.0%) No expected image found = Tests to be fixed for a future release (0): = All tests (10884): 8753 test cases (80.4%) Passed 2130 test cases (19.6%) Skipped 411 test cases (3.8%) Text diff mismatch 302 test cases (2.8%) Simplified text diff mismatch 183 test cases (1.7%) Image mismatch 13 test cases (0.1%) Test timed out 6 test cases (0.1%) Test shell crashed 2 test cases (0.0%) No expected image found I don't know what it tells, am I passed the layouttests? There's also a layouttests in webkit, what's the relation between chrome's layouttest and webkit's layouttest? I think webkit's layouttest only run on Leopard. 200万种商品,最低价格,疯狂诱惑你 --~--~-~--~~~---~--~~ Chromium Developers mailing list: chromium-dev@googlegroups.com View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev -~--~~~~--~~--~--~---
