The errors are odd. They seem to be randomly distributed.
Do you use Mac Ports? I have seen strange errors before when Mac Ports
override system libraries.
Benjamin
On 9/3/13 4:11 PM, Jeff wrote:
Thank you Benjamin, Alexey
This time, "I use run-webkit-tests -2" and following are my test this
time. There are not so much fail, but is it normal?
From results.html, I can find:
Tests that failed text/pixel/audio diff (5)
Tests that had stderr output (5)
Flaky tests (failed the first run and passed on retry) (5)
Tests expected to fail but passed (20)
Maybe I just mis-understanding the results.html?
BEGIN OF TEST RESULT
=========================
Zheng-no-Mac-Pro:WebKit_git zhengxu$ Tools/Scripts/run-webkit-tests -2
Using port 'mac-mountainlion-wk2'
Test configuration: <mountainlion, x86_64, debug>
Placing test results in
/Users/zhengxu/03_workspace/webkit/WebKit_git/WebKitBuild/Debug/layout-test-results
Baseline search path: mac-wk2 -> wk2 -> mac -> generic
Using Debug build
Pixel tests disabled
Regular timeout: 80000, slow test timeout: 400000
Command line:
/Users/zhengxu/03_workspace/webkit/WebKit_git/WebKitBuild/Debug/WebKitTestRunner
-
--lint-test-files warnings:
LayoutTests/platform/mac/TestExpectations:1236 More specific entry for
fast/block/float/overhanging-tall-block.html on line
LayoutTests/platform/mac/TestExpectations:1236 overrides line
LayoutTests/platform/mac/TestExpectations:1102.
fast/block/float/overhanging-tall-block.html
Found 34997 tests; running 32001, skipping 2996.
Running 18 WebKitTestRunners in parallel.
[19/32001] compositing/backgrounds/fixed-background-on-descendant.html
passed unexpectedly
[20/32001] compositing/background-color/background-color-alpha.html
passed unexpectedly
compositing/backgrounds/fixed-backgrounds.html -> ref test hashes
didn't match but diff passed
[47/32001] compositing/backgrounds/fixed-backgrounds.html passed
unexpectedly
compositing/background-color/background-color-change-to-text.html ->
ref test hashes didn't match but diff passed
[47/32001]
compositing/background-color/background-color-change-to-text.html
passed unexpectedly
[68/32001]
compositing/background-color/background-color-composite.html passed
unexpectedly
[79/32001]
compositing/background-color/background-color-container.html passed
unexpectedly
[106/32001]
compositing/background-color/background-color-content-clip.html passed
unexpectedly
[138/32001]
compositing/background-color/background-color-padding-clip.html passed
unexpectedly
[165/32001] compositing/background-color/background-color-simple.html
passed unexpectedly
[186/32001]
compositing/background-color/background-color-text-change.html passed
unexpectedly
[276/32001] compositing/images/positioned-image-content-rect.html
passed unexpectedly
[594/32001]
compositing/geometry/fixed-position-composited-page-scale-smaller-than-viewport.html
passed unexpectedly
[694/32001]
http/tests/cache/willsendrequest-returns-null-for-memory-cache-load.html
passed unexpectedly
css3/filters/custom/custom-filter-blend-modes.html -> ref test
hashes didn't match but diff passed
css3/unicode-bidi-isolate-aharon-failing.html -> ref test hashes
didn't match but diff passed
[938/32001] css3/unicode-bidi-isolate-aharon-failing.html passed
unexpectedly
[1081/32001] compositing/video/video-object-fit.html failed
unexpectedly (text diff)
css3/filters/custom/composited/custom-filter-blend-modes.html -> ref
test hashes didn't match but diff passed
[2770/32001]
css3/filters/custom/effect-custom-transform-parameters.html passed
unexpectedly
[5468/32001] editing/style/make-text-writing-direction-inline-mac.html
failed unexpectedly (text diff)
[5577/32001] editing/style/make-text-writing-direction-inline-win.html
failed unexpectedly (text diff)
fast/css/sticky/sticky-left-percentage.html -> ref test hashes
didn't match but diff passed
[9087/32001] http/tests/inspector/inspect-element.html failed
unexpectedly (text diff)
[9694/32001] fast/dom/Range/range-expand.html failed unexpectedly
(text diff)
[12142/32001]
fast/layers/no-clipping-overflow-hidden-added-after-transition.html
passed unexpectedly
[14235/32001] fast/events/scrollbar-double-click.html failed
unexpectedly (text diff)
[15010/32001] fast/repaint/region-painting-in-composited-view.html
passed unexpectedly
fast/regions/fixed-inside-named-flow-zIndex.html -> ref test hashes
didn't match but diff passed
fast/regions/fixed-pos-elem-in-region.html -> ref test hashes didn't
match but diff passed
[15560/32001] fast/regions/fixed-pos-elem-in-region.html passed
unexpectedly
[15619/32001] fast/regions/fixed-pos-region-in-nested-flow.html failed
unexpectedly (reference mismatch)
[15671/32001] fast/text/international/cjk-segmentation.html failed
unexpectedly (text diff)
[19221/32001] inspector/storage-panel-dom-storage-update.html failed
unexpectedly (text diff)
[20326/32001] jquery/manipulation.html failed unexpectedly (text diff)
svg/repaint/buffered-rendering-static-image.html -> ref test hashes
didn't match but diff passed
[27261/32001] svg/repaint/buffered-rendering-static-image.html passed
unexpectedly
[29599/32001] http/tests/navigation/back-twice-without-commit.html
passed unexpectedly
Retrying 10 unexpected failure(s) ...
Running 1 WebKitTestRunner.
[3/10] editing/style/make-text-writing-direction-inline-mac.html
failed unexpectedly (text diff)
[4/10] editing/style/make-text-writing-direction-inline-win.html
failed unexpectedly (text diff)
[5/10] fast/dom/Range/range-expand.html failed unexpectedly (text diff)
[7/10] fast/regions/fixed-pos-region-in-nested-flow.html failed
unexpectedly (reference mismatch)
[8/10] fast/text/international/cjk-segmentation.html failed
unexpectedly (text diff)
31971 tests ran as expected, 30 didn't:
Expected to timeout, but passed: (1)
http/tests/navigation/back-twice-without-commit.html
Expected to fail, but passed: (20)
compositing/background-color/background-color-alpha.html
compositing/background-color/background-color-change-to-text.html
compositing/background-color/background-color-composite.html
compositing/background-color/background-color-container.html
compositing/background-color/background-color-content-clip.html
compositing/background-color/background-color-padding-clip.html
compositing/background-color/background-color-simple.html
compositing/background-color/background-color-text-change.html
compositing/backgrounds/fixed-background-on-descendant.html
compositing/backgrounds/fixed-backgrounds.html
compositing/geometry/fixed-position-composited-page-scale-smaller-than-viewport.html
compositing/images/positioned-image-content-rect.html
css3/filters/custom/effect-custom-transform-parameters.html
css3/unicode-bidi-isolate-aharon-failing.html
fast/layers/no-clipping-overflow-hidden-added-after-transition.html
fast/regions/fixed-pos-elem-in-region.html
fast/repaint/region-painting-in-composited-view.html
http/tests/cache/willsendrequest-returns-null-for-memory-cache-load.html
svg/repaint/buffered-rendering-static-image.html
tables/mozilla_expected_failures/bugs/bug178855.xml
Unexpected flakiness: text-only failures (5)
compositing/video/video-object-fit.html [ Failure Pass ]
fast/events/scrollbar-double-click.html [ Failure Pass ]
http/tests/inspector/inspect-element.html [ Failure Pass ]
inspector/storage-panel-dom-storage-update.html [ Failure Pass ]
jquery/manipulation.html [ Failure Pass ]
Regressions: Unexpected text-only failures (4)
editing/style/make-text-writing-direction-inline-mac.html [ Failure ]
editing/style/make-text-writing-direction-inline-win.html [ Failure ]
fast/dom/Range/range-expand.html [ Failure ]
fast/text/international/cjk-segmentation.html [ Failure ]
Regressions: Unexpected image-only failures (1)
fast/regions/fixed-pos-region-in-nested-flow.html [ ImageOnlyFailure ]
Failed to execute Tools/Scripts/new-run-webkit-tests at
Tools/Scripts/run-webkit-tests line 100.
=========================
END OF TEST RESULT
(2013/09/04 7:08), Benjamin Poulain wrote:
On 9/3/13 6:43 AM, 徐征 wrote:
I am not familiar with layout test of webkit. I created some patch
but hope it is easy to find any degression via layout test.
Everytime, I create some patch, I just run
|Tools/Script/run-webkit-tests and all the time there are many
fail(more than 100).|
||I think maybe I just use wrong option for this script.
|It there anyway to pass all the test in run-webkit-tests?
Or, if it is normal that not all test are passed, is there any way
to confirm degression efficient?
I am using OSX10.8.4.
And I find in http://www.webkit.org/quality/testing.html, it is
written that "Regression tests must be run on Leopard", should I run
test on Leopard?|
The testing webpage is wrong, I'll fix it.
The common ways to run the tests are "run-webkit-tests -2" and
"run-webkit-test".
OS X 10.8.4 is the best platform to run the tests, you should be able
to run them without problems.
What kind of errors do you get?
Benjamin
_______________________________________________
webkit-help mailing list
webkit-help@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-help
