- Revision
- 191380
- Author
- youenn.fab...@crf.canon.fr
- Date
- 2015-10-21 00:53:37 -0700 (Wed, 21 Oct 2015)
Log Message
W3C test importer should not import reference files as regular tests
https://bugs.webkit.org/show_bug.cgi?id=150331
Reviewed by Darin Adler.
LayoutTests/imported/w3c:
* resources/ImportExpectations: Removed some Skip expectations as test importer no longer will handle these files as tests, but as reference files.
Tools:
Updated test_parser to return information that a file can be a referencefile or a manual test.
Updated test importer to skip those files.
Updated/Added test parser unit tests accordingly.
Updated test importer to import all not-test files, as they are usually used by tests and lead to test failing or timing out.
Updated test importer to log these 'resource' files that are not in a resources folder, and suggest to import script user
to mark these files as Skip, since they are not direct tests.
* Scripts/webkitpy/w3c/test_importer.py:
(TestImporter.__init__):
(TestImporter.find_importable_tests):
(TestImporter.import_tests):
* Scripts/webkitpy/w3c/test_parser.py:
(TestParser.analyze_test):
* Scripts/webkitpy/w3c/test_parser_unittest.py:
(test_analyze_non_html_file):
(test_reference_test):
LayoutTests:
* TestExpectations: Adding Skip expectations for web-platform-test resource files (currently commented out as not all are yet imported).
Modified Paths
Diff
Modified: trunk/LayoutTests/ChangeLog (191379 => 191380)
--- trunk/LayoutTests/ChangeLog 2015-10-21 07:44:33 UTC (rev 191379)
+++ trunk/LayoutTests/ChangeLog 2015-10-21 07:53:37 UTC (rev 191380)
@@ -1,5 +1,14 @@
2015-10-21 Youenn Fablet <youenn.fab...@crf.canon.fr>
+ W3C test importer should not import reference files as regular tests
+ https://bugs.webkit.org/show_bug.cgi?id=150331
+
+ Reviewed by Darin Adler.
+
+ * TestExpectations: Adding Skip expectations for web-platform-test resource files (currently commented out as not all are yet imported).
+
+2015-10-21 Youenn Fablet <youenn.fab...@crf.canon.fr>
+
Update web-platform-tests tests to the same revision as tools
https://bugs.webkit.org/show_bug.cgi?id=149656
Modified: trunk/LayoutTests/TestExpectations (191379 => 191380)
--- trunk/LayoutTests/TestExpectations 2015-10-21 07:44:33 UTC (rev 191379)
+++ trunk/LayoutTests/TestExpectations 2015-10-21 07:53:37 UTC (rev 191380)
@@ -266,6 +266,68 @@
imported/w3c/web-platform-tests/html/semantics/grouping-content/the-ol-element/reversed-1c.html [ Skip ]
imported/w3c/web-platform-tests/html/semantics/selectors/pseudo-classes/focus-iframe.html [ Skip ]
+# Resource files not in resources folder
+#imported/w3c/web-platform-tests/common/blank.html [ Skip ]
+#imported/w3c/web-platform-tests/dom/nodes/Document-createElement-namespace-tests/bare_mathml.html [ Skip ]
+#imported/w3c/web-platform-tests/dom/nodes/Document-createElement-namespace-tests/bare_mathml.xhtml [ Skip ]
+#imported/w3c/web-platform-tests/dom/nodes/Document-createElement-namespace-tests/bare_mathml.xml [ Skip ]
+#imported/w3c/web-platform-tests/dom/nodes/Document-createElement-namespace-tests/bare_svg.html [ Skip ]
+#imported/w3c/web-platform-tests/dom/nodes/Document-createElement-namespace-tests/bare_svg.xhtml [ Skip ]
+#imported/w3c/web-platform-tests/dom/nodes/Document-createElement-namespace-tests/bare_svg.xml [ Skip ]
+#imported/w3c/web-platform-tests/dom/nodes/Document-createElement-namespace-tests/bare_xhtml.html [ Skip ]
+#imported/w3c/web-platform-tests/dom/nodes/Document-createElement-namespace-tests/bare_xhtml.xhtml [ Skip ]
+#imported/w3c/web-platform-tests/dom/nodes/Document-createElement-namespace-tests/bare_xhtml.xml [ Skip ]
+#imported/w3c/web-platform-tests/dom/nodes/Document-createElement-namespace-tests/empty.html [ Skip ]
+#imported/w3c/web-platform-tests/dom/nodes/Document-createElement-namespace-tests/empty.xhtml [ Skip ]
+#imported/w3c/web-platform-tests/dom/nodes/Document-createElement-namespace-tests/empty.xml [ Skip ]
+#imported/w3c/web-platform-tests/dom/nodes/Document-createElement-namespace-tests/mathml.html [ Skip ]
+#imported/w3c/web-platform-tests/dom/nodes/Document-createElement-namespace-tests/mathml.xhtml [ Skip ]
+#imported/w3c/web-platform-tests/dom/nodes/Document-createElement-namespace-tests/mathml.xml [ Skip ]
+#imported/w3c/web-platform-tests/dom/nodes/Document-createElement-namespace-tests/minimal_html.html [ Skip ]
+#imported/w3c/web-platform-tests/dom/nodes/Document-createElement-namespace-tests/minimal_html.xhtml [ Skip ]
+#imported/w3c/web-platform-tests/dom/nodes/Document-createElement-namespace-tests/minimal_html.xml [ Skip ]
+#imported/w3c/web-platform-tests/dom/nodes/Document-createElement-namespace-tests/svg.html [ Skip ]
+#imported/w3c/web-platform-tests/dom/nodes/Document-createElement-namespace-tests/svg.xhtml [ Skip ]
+#imported/w3c/web-platform-tests/dom/nodes/Document-createElement-namespace-tests/svg.xml [ Skip ]
+#imported/w3c/web-platform-tests/dom/nodes/Document-createElement-namespace-tests/xhtml.html [ Skip ]
+#imported/w3c/web-platform-tests/dom/nodes/Document-createElement-namespace-tests/xhtml.xhtml [ Skip ]
+#imported/w3c/web-platform-tests/dom/nodes/Document-createElement-namespace-tests/xhtml.xml [ Skip ]
+#imported/w3c/web-platform-tests/dom/nodes/Document-createElement-namespace-tests/xhtml_ns_changed.html [ Skip ]
+#imported/w3c/web-platform-tests/dom/nodes/Document-createElement-namespace-tests/xhtml_ns_changed.xhtml [ Skip ]
+#imported/w3c/web-platform-tests/dom/nodes/Document-createElement-namespace-tests/xhtml_ns_changed.xml [ Skip ]
+#imported/w3c/web-platform-tests/dom/nodes/Document-createElement-namespace-tests/xhtml_ns_removed.html [ Skip ]
+#imported/w3c/web-platform-tests/dom/nodes/Document-createElement-namespace-tests/xhtml_ns_removed.xhtml [ Skip ]
+#imported/w3c/web-platform-tests/dom/nodes/Document-createElement-namespace-tests/xhtml_ns_removed.xml [ Skip ]
+#imported/w3c/web-platform-tests/dom/nodes/Node-parentNode-iframe.html [ Skip ]
+#imported/w3c/web-platform-tests/dom/nodes/ParentNode-querySelector-All-content.html [ Skip ]
+#imported/w3c/web-platform-tests/dom/nodes/ParentNode-querySelector-All-content.xht [ Skip ]
+#imported/w3c/web-platform-tests/dom/nodes/getElementsByClassNameFrame.htm [ Skip ]
+#imported/w3c/web-platform-tests/dom/ranges/Range-test-iframe.html [ Skip ]
+#imported/w3c/web-platform-tests/html/dom/dynamic-markup-insertion/document-write/nested-document-write-1.html [ Skip ]
+#imported/w3c/web-platform-tests/html/dom/dynamic-markup-insertion/document-write/nested-document-write-2.html [ Skip ]
+#imported/w3c/web-platform-tests/html/dom/dynamic-markup-insertion/opening-the-input-stream/004-1.html [ Skip ]
+#imported/w3c/web-platform-tests/html/dom/dynamic-markup-insertion/opening-the-input-stream/010-1.html [ Skip ]
+#imported/w3c/web-platform-tests/html/dom/dynamic-markup-insertion/opening-the-input-stream/010-2.html [ Skip ]
+#imported/w3c/web-platform-tests/html/dom/dynamic-markup-insertion/opening-the-input-stream/011-1.html [ Skip ]
+#imported/w3c/web-platform-tests/html/dom/dynamic-markup-insertion/opening-the-input-stream/012-1.html [ Skip ]
+#imported/w3c/web-platform-tests/html/dom/dynamic-markup-insertion/opening-the-input-stream/013-1.html [ Skip ]
+#imported/w3c/web-platform-tests/html/dom/dynamic-markup-insertion/opening-the-input-stream/014-1.html [ Skip ]
+#imported/w3c/web-platform-tests/html/dom/dynamic-markup-insertion/opening-the-input-stream/015-1.html [ Skip ]
+#imported/w3c/web-platform-tests/html/dom/dynamic-markup-insertion/opening-the-input-stream/016-1.html [ Skip ]
+#imported/w3c/web-platform-tests/html/dom/dynamic-markup-insertion/opening-the-input-stream/document.open-03-frame.html [ Skip ]
+#imported/w3c/web-platform-tests/html/semantics/document-metadata/the-base-element/example.html [ Skip ]
+#imported/w3c/web-platform-tests/html/semantics/document-metadata/the-base-element/example2.html [ Skip ]
+#imported/w3c/web-platform-tests/html/semantics/embedded-content/the-img-element/environment-changes/iframed.sub.html [ Skip ]
+#imported/w3c/web-platform-tests/html/semantics/embedded-content/the-img-element/sizes/sizes-iframed.sub.html [ Skip ]
+#imported/w3c/web-platform-tests/html/semantics/embedded-content/the-object-element/test0.html [ Skip ]
+#imported/w3c/web-platform-tests/html/semantics/embedded-content/the-object-element/test1.html [ Skip ]
+#imported/w3c/web-platform-tests/html/semantics/embedded-content/the-object-element/test2.html [ Skip ]
+#imported/w3c/web-platform-tests/html/semantics/forms/attributes-common-to-form-controls/dirname-ltr-iframe.html [ Skip ]
+#imported/w3c/web-platform-tests/html/semantics/grouping-content/the-ol-element/reversed-1a.html [ Skip ]
+#imported/w3c/web-platform-tests/html/semantics/grouping-content/the-ol-element/reversed-1b.html [ Skip ]
+#imported/w3c/web-platform-tests/html/semantics/grouping-content/the-ol-element/reversed-1c.html [ Skip ]
+#imported/w3c/web-platform-tests/html/semantics/selectors/pseudo-classes/focus-iframe.html [ Skip ]
+
# Skip W3C tests that are too slow in debug builds.
webkit.org/b/148546 [ Debug ] imported/w3c/web-platform-tests/dom/interfaces.html [ Skip ]
webkit.org/b/148546 [ Debug ] imported/w3c/web-platform-tests/dom/ranges/Range-compareBoundaryPoints.html [ Skip ]
Modified: trunk/LayoutTests/imported/w3c/ChangeLog (191379 => 191380)
--- trunk/LayoutTests/imported/w3c/ChangeLog 2015-10-21 07:44:33 UTC (rev 191379)
+++ trunk/LayoutTests/imported/w3c/ChangeLog 2015-10-21 07:53:37 UTC (rev 191380)
@@ -1,5 +1,14 @@
2015-10-21 Youenn Fablet <youenn.fab...@crf.canon.fr>
+ W3C test importer should not import reference files as regular tests
+ https://bugs.webkit.org/show_bug.cgi?id=150331
+
+ Reviewed by Darin Adler.
+
+ * resources/ImportExpectations: Removed some Skip expectations as test importer no longer will handle these files as tests, but as reference files.
+
+2015-10-21 Youenn Fablet <youenn.fab...@crf.canon.fr>
+
Update web-platform-tests tests to the same revision as tools
https://bugs.webkit.org/show_bug.cgi?id=149656
Modified: trunk/LayoutTests/imported/w3c/resources/ImportExpectations (191379 => 191380)
--- trunk/LayoutTests/imported/w3c/resources/ImportExpectations 2015-10-21 07:44:33 UTC (rev 191379)
+++ trunk/LayoutTests/imported/w3c/resources/ImportExpectations 2015-10-21 07:53:37 UTC (rev 191380)
@@ -61,7 +61,6 @@
web-platform-tests/custom-elements [ Skip ]
#web-platform-tests/dom [ Pass ]
web-platform-tests/dom/events/Event-dispatch-propagation-stopped.html [ Skip ]
-web-platform-tests/dom/nodes/Document-createElement-namespace-tests [ Skip ]
web-platform-tests/dom/nodes/Document-createProcessingInstruction.xhtml [ Skip ]
web-platform-tests/dom/nodes/Document-getElementsByTagName.xhtml [ Skip ]
web-platform-tests/dom/nodes/Element-childElement-null.svg [ Skip ]
@@ -102,99 +101,13 @@
#web-platform-tests/html [ Pass ]
web-platform-tests/html/browsers [ Skip ]
web-platform-tests/html/dom/documents/dom-tree-accessors/Document.currentScript.sub.html [ Skip ]
-web-platform-tests/html/dom/dynamic-markup-insertion/opening-the-input-stream/010.html [ Skip ]
-web-platform-tests/html/dom/elements/global-attributes/dir_auto-EN-L-ref-expected.html [ Skip ]
-web-platform-tests/html/dom/elements/global-attributes/dir_auto-EN-L-ref.html [ Skip ]
-web-platform-tests/html/dom/elements/global-attributes/dir_auto-EN-R-ref-expected.html [ Skip ]
-web-platform-tests/html/dom/elements/global-attributes/dir_auto-EN-R-ref.html [ Skip ]
-web-platform-tests/html/dom/elements/global-attributes/dir_auto-L-ref-expected.html [ Skip ]
-web-platform-tests/html/dom/elements/global-attributes/dir_auto-L-ref.html [ Skip ]
-web-platform-tests/html/dom/elements/global-attributes/dir_auto-N-EN-L-ref-expected.html [ Skip ]
-web-platform-tests/html/dom/elements/global-attributes/dir_auto-N-EN-L-ref.html [ Skip ]
-web-platform-tests/html/dom/elements/global-attributes/dir_auto-N-EN-R-ref-expected.html [ Skip ]
-web-platform-tests/html/dom/elements/global-attributes/dir_auto-N-EN-R-ref.html [ Skip ]
-web-platform-tests/html/dom/elements/global-attributes/dir_auto-N-EN-ref-expected.html [ Skip ]
-web-platform-tests/html/dom/elements/global-attributes/dir_auto-N-EN-ref.html [ Skip ]
-web-platform-tests/html/dom/elements/global-attributes/dir_auto-N-L-ref-expected.html [ Skip ]
-web-platform-tests/html/dom/elements/global-attributes/dir_auto-N-L-ref.html [ Skip ]
-web-platform-tests/html/dom/elements/global-attributes/dir_auto-N-R-ref-expected.html [ Skip ]
-web-platform-tests/html/dom/elements/global-attributes/dir_auto-N-R-ref.html [ Skip ]
-web-platform-tests/html/dom/elements/global-attributes/dir_auto-R-ref-expected.html [ Skip ]
-web-platform-tests/html/dom/elements/global-attributes/dir_auto-R-ref.html [ Skip ]
-web-platform-tests/html/dom/elements/global-attributes/dir_auto-contained-L-ref-expected.html [ Skip ]
-web-platform-tests/html/dom/elements/global-attributes/dir_auto-contained-L-ref.html [ Skip ]
-web-platform-tests/html/dom/elements/global-attributes/dir_auto-contained-R-ref-expected.html [ Skip ]
-web-platform-tests/html/dom/elements/global-attributes/dir_auto-contained-R-ref.html [ Skip ]
-web-platform-tests/html/dom/elements/global-attributes/dir_auto-contained-bdi-L-ref-expected.html [ Skip ]
-web-platform-tests/html/dom/elements/global-attributes/dir_auto-contained-bdi-L-ref.html [ Skip ]
-web-platform-tests/html/dom/elements/global-attributes/dir_auto-contained-bdi-R-ref-expected.html [ Skip ]
-web-platform-tests/html/dom/elements/global-attributes/dir_auto-contained-bdi-R-ref.html [ Skip ]
-web-platform-tests/html/dom/elements/global-attributes/dir_auto-contained-dir-L-ref-expected.html [ Skip ]
-web-platform-tests/html/dom/elements/global-attributes/dir_auto-contained-dir-L-ref.html [ Skip ]
-web-platform-tests/html/dom/elements/global-attributes/dir_auto-contained-dir-R-ref-expected.html [ Skip ]
-web-platform-tests/html/dom/elements/global-attributes/dir_auto-contained-dir-R-ref.html [ Skip ]
-web-platform-tests/html/dom/elements/global-attributes/dir_auto-contained-dir_auto-L-ref-expected.html [ Skip ]
-web-platform-tests/html/dom/elements/global-attributes/dir_auto-contained-dir_auto-L-ref.html [ Skip ]
-web-platform-tests/html/dom/elements/global-attributes/dir_auto-contained-dir_auto-R-ref-expected.html [ Skip ]
-web-platform-tests/html/dom/elements/global-attributes/dir_auto-contained-dir_auto-R-ref.html [ Skip ]
-web-platform-tests/html/dom/elements/global-attributes/dir_auto-contained-script-L-ref-expected.html [ Skip ]
-web-platform-tests/html/dom/elements/global-attributes/dir_auto-contained-script-L-ref.html [ Skip ]
-web-platform-tests/html/dom/elements/global-attributes/dir_auto-contained-script-R-ref-expected.html [ Skip ]
-web-platform-tests/html/dom/elements/global-attributes/dir_auto-contained-script-R-ref.html [ Skip ]
-web-platform-tests/html/dom/elements/global-attributes/dir_auto-contained-style-L-ref-expected.html [ Skip ]
-web-platform-tests/html/dom/elements/global-attributes/dir_auto-contained-style-L-ref.html [ Skip ]
-web-platform-tests/html/dom/elements/global-attributes/dir_auto-contained-style-R-ref-expected.html [ Skip ]
-web-platform-tests/html/dom/elements/global-attributes/dir_auto-contained-style-R-ref.html [ Skip ]
-web-platform-tests/html/dom/elements/global-attributes/dir_auto-contained-textarea-L-ref-expected.html [ Skip ]
-web-platform-tests/html/dom/elements/global-attributes/dir_auto-contained-textarea-L-ref.html [ Skip ]
-web-platform-tests/html/dom/elements/global-attributes/dir_auto-contained-textarea-R-ref-expected.html [ Skip ]
-web-platform-tests/html/dom/elements/global-attributes/dir_auto-contained-textarea-R-ref.html [ Skip ]
-web-platform-tests/html/dom/elements/global-attributes/dir_auto-input-EN-L-ref-expected.html [ Skip ]
-web-platform-tests/html/dom/elements/global-attributes/dir_auto-input-EN-L-ref.html [ Skip ]
-web-platform-tests/html/dom/elements/global-attributes/dir_auto-input-EN-R-ref-expected.html [ Skip ]
-web-platform-tests/html/dom/elements/global-attributes/dir_auto-input-EN-R-ref.html [ Skip ]
-web-platform-tests/html/dom/elements/global-attributes/dir_auto-input-L-ref-expected.html [ Skip ]
-web-platform-tests/html/dom/elements/global-attributes/dir_auto-input-L-ref.html [ Skip ]
-web-platform-tests/html/dom/elements/global-attributes/dir_auto-input-N-EN-L-ref-expected.html [ Skip ]
-web-platform-tests/html/dom/elements/global-attributes/dir_auto-input-N-EN-L-ref.html [ Skip ]
-web-platform-tests/html/dom/elements/global-attributes/dir_auto-input-N-EN-R-ref-expected.html [ Skip ]
-web-platform-tests/html/dom/elements/global-attributes/dir_auto-input-N-EN-R-ref.html [ Skip ]
-web-platform-tests/html/dom/elements/global-attributes/dir_auto-input-N-EN-ref-expected.html [ Skip ]
-web-platform-tests/html/dom/elements/global-attributes/dir_auto-input-N-EN-ref.html [ Skip ]
-web-platform-tests/html/dom/elements/global-attributes/dir_auto-input-N-L-ref-expected.html [ Skip ]
-web-platform-tests/html/dom/elements/global-attributes/dir_auto-input-N-L-ref.html [ Skip ]
-web-platform-tests/html/dom/elements/global-attributes/dir_auto-input-N-R-ref-expected.html [ Skip ]
-web-platform-tests/html/dom/elements/global-attributes/dir_auto-input-N-R-ref.html [ Skip ]
-web-platform-tests/html/dom/elements/global-attributes/dir_auto-input-R-ref-expected.html [ Skip ]
-web-platform-tests/html/dom/elements/global-attributes/dir_auto-input-R-ref.html [ Skip ]
-web-platform-tests/html/dom/elements/global-attributes/dir_auto-input-script-EN-L-ref-expected.html [ Skip ]
-web-platform-tests/html/dom/elements/global-attributes/dir_auto-input-script-EN-L-ref.html [ Skip ]
-web-platform-tests/html/dom/elements/global-attributes/dir_auto-input-script-EN-R-ref-expected.html [ Skip ]
-web-platform-tests/html/dom/elements/global-attributes/dir_auto-input-script-EN-R-ref.html [ Skip ]
-web-platform-tests/html/dom/elements/global-attributes/dir_auto-input-script-L-ref-expected.html [ Skip ]
-web-platform-tests/html/dom/elements/global-attributes/dir_auto-input-script-L-ref.html [ Skip ]
-web-platform-tests/html/dom/elements/global-attributes/dir_auto-input-script-N-EN-L-ref-expected.html [ Skip ]
-web-platform-tests/html/dom/elements/global-attributes/dir_auto-input-script-N-EN-L-ref.html [ Skip ]
-web-platform-tests/html/dom/elements/global-attributes/dir_auto-input-script-N-EN-R-ref-expected.html [ Skip ]
-web-platform-tests/html/dom/elements/global-attributes/dir_auto-input-script-N-EN-R-ref.html [ Skip ]
-web-platform-tests/html/dom/elements/global-attributes/dir_auto-input-script-N-EN-ref-expected.html [ Skip ]
-web-platform-tests/html/dom/elements/global-attributes/dir_auto-input-script-N-EN-ref.html [ Skip ]
-web-platform-tests/html/dom/elements/global-attributes/dir_auto-input-script-N-L-ref-expected.html [ Skip ]
-web-platform-tests/html/dom/elements/global-attributes/dir_auto-input-script-N-L-ref.html [ Skip ]
-web-platform-tests/html/dom/elements/global-attributes/dir_auto-input-script-N-R-ref-expected.html [ Skip ]
-web-platform-tests/html/dom/elements/global-attributes/dir_auto-input-script-N-R-ref.html [ Skip ]
-web-platform-tests/html/dom/elements/global-attributes/dir_auto-input-script-R-ref-expected.html [ Skip ]
-web-platform-tests/html/dom/elements/global-attributes/dir_auto-input-script-R-ref.html [ Skip ]
-web-platform-tests/html/dom/elements/global-attributes/dir_auto-isolate-ref-expected.html [ Skip ]
-web-platform-tests/html/dom/elements/global-attributes/dir_auto-isolate-ref.html [ Skip ]
-web-platform-tests/html/dom/elements/global-attributes/dir_auto-textarea-script-N-between-Rs-ref-expected.html [ Skip ]
-web-platform-tests/html/dom/elements/global-attributes/dir_auto-textarea-script-N-between-Rs-ref.html [ Skip ]
web-platform-tests/html/dom/reflection-embedded.html [ Skip ]
web-platform-tests/html/dom/reflection-forms.html [ Skip ]
web-platform-tests/html/dom/reflection-grouping.html [ Skip ]
web-platform-tests/html/dom/reflection-metadata.html [ Skip ]
web-platform-tests/html/dom/reflection-misc.html [ Skip ]
web-platform-tests/html/dom/reflection-obsolete.html [ Skip ]
+web-platform-tests/html/dom/reflection-original.html [ Skip ]
web-platform-tests/html/dom/reflection-sections.html [ Skip ]
web-platform-tests/html/dom/reflection-tabular.html [ Skip ]
web-platform-tests/html/dom/reflection-text.html [ Skip ]
@@ -203,50 +116,15 @@
web-platform-tests/html/infrastructure [ Skip ]
web-platform-tests/html/obsolete [ Skip ]
web-platform-tests/html/rendering [ Skip ]
+
web-platform-tests/html/semantics/document-metadata/styling/LinkStyle.html [ Skip ]
+web-platform-tests/html/semantics/embedded-content/media-elements/error-codes [ Skip ]
+web-platform-tests/html/semantics/embedded-content/media-elements/loading-the-media-resource [ Skip ]
+web-platform-tests/html/semantics/embedded-content/the-iframe-element [ Skip ]
+web-platform-tests/html/semantics/embedded-content/media-elements/track/track-element/cors [ Skip ]
+web-platform-tests/html/semantics/embedded-content/the-embed-element/embed-represent-nothing-04.html [ Skip ]
web-platform-tests/html/semantics/embedded-content/media-elements/audio_loop_base.html [ Skip ]
web-platform-tests/html/semantics/embedded-content/media-elements/error-codes [ Skip ]
-web-platform-tests/html/semantics/embedded-content/media-elements/loading-the-media-resource/autoplay-overrides-preload.html [ Skip ]
-web-platform-tests/html/semantics/embedded-content/media-elements/loading-the-media-resource/load-events-networkState.html [ Skip ]
-web-platform-tests/html/semantics/embedded-content/media-elements/loading-the-media-resource/load-removes-queued-error-event.html [ Skip ]
-web-platform-tests/html/semantics/embedded-content/media-elements/loading-the-media-resource/resource-selection-candidate-insert-before.html [ Skip ]
-web-platform-tests/html/semantics/embedded-content/media-elements/loading-the-media-resource/resource-selection-candidate-moved.html [ Skip ]
-web-platform-tests/html/semantics/embedded-content/media-elements/loading-the-media-resource/resource-selection-candidate-remove-addEventListener.html [ Skip ]
-web-platform-tests/html/semantics/embedded-content/media-elements/loading-the-media-resource/resource-selection-candidate-remove-no-listener.html [ Skip ]
-web-platform-tests/html/semantics/embedded-content/media-elements/loading-the-media-resource/resource-selection-candidate-remove-onerror.html [ Skip ]
-web-platform-tests/html/semantics/embedded-content/media-elements/loading-the-media-resource/resource-selection-invoke-audio-constructor-no-src.html [ Skip ]
-web-platform-tests/html/semantics/embedded-content/media-elements/loading-the-media-resource/resource-selection-invoke-audio-constructor.html [ Skip ]
-web-platform-tests/html/semantics/embedded-content/media-elements/loading-the-media-resource/resource-selection-invoke-in-sync-event.html [ Skip ]
-web-platform-tests/html/semantics/embedded-content/media-elements/loading-the-media-resource/resource-selection-invoke-insert-fragment-into-document.html [ Skip ]
-web-platform-tests/html/semantics/embedded-content/media-elements/loading-the-media-resource/resource-selection-invoke-insert-into-document.html [ Skip ]
-web-platform-tests/html/semantics/embedded-content/media-elements/loading-the-media-resource/resource-selection-invoke-insert-into-iframe.html [ Skip ]
-web-platform-tests/html/semantics/embedded-content/media-elements/loading-the-media-resource/resource-selection-invoke-insert-parent-into-document.html [ Skip ]
-web-platform-tests/html/semantics/embedded-content/media-elements/loading-the-media-resource/resource-selection-invoke-insert-source-in-div.html [ Skip ]
-web-platform-tests/html/semantics/embedded-content/media-elements/loading-the-media-resource/resource-selection-invoke-insert-source-in-namespace.html [ Skip ]
-web-platform-tests/html/semantics/embedded-content/media-elements/loading-the-media-resource/resource-selection-invoke-insert-source-networkState.html [ Skip ]
-web-platform-tests/html/semantics/embedded-content/media-elements/loading-the-media-resource/resource-selection-invoke-insert-source-not-in-document.html [ Skip ]
-web-platform-tests/html/semantics/embedded-content/media-elements/loading-the-media-resource/resource-selection-invoke-insert-source.html [ Skip ]
-web-platform-tests/html/semantics/embedded-content/media-elements/loading-the-media-resource/resource-selection-invoke-load.html [ Skip ]
-web-platform-tests/html/semantics/embedded-content/media-elements/loading-the-media-resource/resource-selection-invoke-pause-networkState.html [ Skip ]
-web-platform-tests/html/semantics/embedded-content/media-elements/loading-the-media-resource/resource-selection-invoke-pause.html [ Skip ]
-web-platform-tests/html/semantics/embedded-content/media-elements/loading-the-media-resource/resource-selection-invoke-play.html [ Skip ]
-web-platform-tests/html/semantics/embedded-content/media-elements/loading-the-media-resource/resource-selection-invoke-remove-from-document-networkState.html [ Skip ]
-web-platform-tests/html/semantics/embedded-content/media-elements/loading-the-media-resource/resource-selection-invoke-remove-from-document.html [ Skip ]
-web-platform-tests/html/semantics/embedded-content/media-elements/loading-the-media-resource/resource-selection-invoke-remove-src.html [ Skip ]
-web-platform-tests/html/semantics/embedded-content/media-elements/loading-the-media-resource/resource-selection-invoke-set-src-in-namespace.html [ Skip ]
-web-platform-tests/html/semantics/embedded-content/media-elements/loading-the-media-resource/resource-selection-invoke-set-src-networkState.html [ Skip ]
-web-platform-tests/html/semantics/embedded-content/media-elements/loading-the-media-resource/resource-selection-invoke-set-src-not-in-document.html [ Skip ]
-web-platform-tests/html/semantics/embedded-content/media-elements/loading-the-media-resource/resource-selection-invoke-set-src.html [ Skip ]
-web-platform-tests/html/semantics/embedded-content/media-elements/loading-the-media-resource/resource-selection-pointer-control.html [ Skip ]
-web-platform-tests/html/semantics/embedded-content/media-elements/loading-the-media-resource/resource-selection-pointer-insert-br.html [ Skip ]
-web-platform-tests/html/semantics/embedded-content/media-elements/loading-the-media-resource/resource-selection-pointer-insert-source.html [ Skip ]
-web-platform-tests/html/semantics/embedded-content/media-elements/loading-the-media-resource/resource-selection-pointer-insert-text.html [ Skip ]
-web-platform-tests/html/semantics/embedded-content/media-elements/loading-the-media-resource/resource-selection-pointer-remove-source-after.html [ Skip ]
-web-platform-tests/html/semantics/embedded-content/media-elements/loading-the-media-resource/resource-selection-pointer-remove-source.html [ Skip ]
-web-platform-tests/html/semantics/embedded-content/media-elements/loading-the-media-resource/resource-selection-pointer-remove-text.html [ Skip ]
-web-platform-tests/html/semantics/embedded-content/media-elements/loading-the-media-resource/resource-selection-remove-source.html [ Skip ]
-web-platform-tests/html/semantics/embedded-content/media-elements/loading-the-media-resource/resource-selection-remove-src.html [ Skip ]
-web-platform-tests/html/semantics/embedded-content/media-elements/loading-the-media-resource/resource-selection-source-media.html [ Skip ]
web-platform-tests/html/semantics/embedded-content/media-elements/location-of-the-media-resource/currentSrc.html [ Skip ]
web-platform-tests/html/semantics/embedded-content/media-elements/networkState_during_progress.html [ Skip ]
web-platform-tests/html/semantics/embedded-content/media-elements/offsets-into-the-media-resource/currentTime.html [ Skip ]
@@ -260,8 +138,8 @@
web-platform-tests/html/semantics/embedded-content/media-elements/track/track-element/cloneNode.html [ Skip ]
web-platform-tests/html/semantics/embedded-content/media-elements/track/track-element/cors [ Skip ]
web-platform-tests/html/semantics/embedded-content/media-elements/video_loop_base.html [ Skip ]
-web-platform-tests/html/semantics/embedded-content/the-embed-element/embed-represent-nothing-04-expected.html [ Skip ]
-web-platform-tests/html/semantics/embedded-content/the-embed-element/embed-represent-nothing-04.html [ Skip ]
+web-platform-tests/html/semantics/embedded-content/the-area-element/area_nohref.xhtml [ Skip ]
+web-platform-tests/html/semantics/embedded-content/the-area-element/test-area.xhtml [ Skip ]
web-platform-tests/html/semantics/embedded-content/the-img-element/relevant-mutations.html [ Skip ]
web-platform-tests/html/semantics/embedded-content/the-object-element/object-events.html [ Skip ]
web-platform-tests/html/semantics/forms/textfieldselection/select-event.html [ Skip ]
@@ -330,3 +208,5 @@
web-platform-tests/webstorage [ Skip ]
web-platform-tests/webvtt [ Skip ]
web-platform-tests/workers [ Skip ]
+
+web-platform-tests/testharness_runner.html [ Skip ]
Modified: trunk/Tools/ChangeLog (191379 => 191380)
--- trunk/Tools/ChangeLog 2015-10-21 07:44:33 UTC (rev 191379)
+++ trunk/Tools/ChangeLog 2015-10-21 07:53:37 UTC (rev 191380)
@@ -1,3 +1,28 @@
+2015-10-21 Youenn Fablet <youenn.fab...@crf.canon.fr>
+
+ W3C test importer should not import reference files as regular tests
+ https://bugs.webkit.org/show_bug.cgi?id=150331
+
+ Reviewed by Darin Adler.
+
+ Updated test_parser to return information that a file can be a referencefile or a manual test.
+ Updated test importer to skip those files.
+ Updated/Added test parser unit tests accordingly.
+
+ Updated test importer to import all not-test files, as they are usually used by tests and lead to test failing or timing out.
+ Updated test importer to log these 'resource' files that are not in a resources folder, and suggest to import script user
+ to mark these files as Skip, since they are not direct tests.
+
+ * Scripts/webkitpy/w3c/test_importer.py:
+ (TestImporter.__init__):
+ (TestImporter.find_importable_tests):
+ (TestImporter.import_tests):
+ * Scripts/webkitpy/w3c/test_parser.py:
+ (TestParser.analyze_test):
+ * Scripts/webkitpy/w3c/test_parser_unittest.py:
+ (test_analyze_non_html_file):
+ (test_reference_test):
+
2015-10-20 Aakash Jain <aakash_j...@apple.com>
run-webkit-tests does not copy all crash logs for layout test failures on iOS
Modified: trunk/Tools/Scripts/webkitpy/w3c/test_importer.py (191379 => 191380)
--- trunk/Tools/Scripts/webkitpy/w3c/test_importer.py 2015-10-21 07:44:33 UTC (rev 191379)
+++ trunk/Tools/Scripts/webkitpy/w3c/test_importer.py 2015-10-21 07:53:37 UTC (rev 191380)
@@ -166,11 +166,14 @@
self._webkit_root = webkit_finder.webkit_base()
self.destination_directory = webkit_finder.path_from_webkit_base("LayoutTests", options.destination)
- self.layout_tests_w3c_path = webkit_finder.path_from_webkit_base('LayoutTests', 'imported', 'w3c')
+ self.tests_w3c_relative_path = self.filesystem.join('imported', 'w3c')
+ self.layout_tests_w3c_path = webkit_finder.path_from_webkit_base('LayoutTests', self.tests_w3c_relative_path)
self.tests_download_path = webkit_finder.path_from_webkit_base('WebKitBuild', 'w3c-tests')
self._test_downloader = None
+ self._potential_test_resource_files = []
+
self.import_list = []
self._importing_downloaded_tests = source_directory is None
@@ -253,11 +256,19 @@
test_parser = TestParser(vars(self.options), filename=fullpath, host=self.host)
test_info = test_parser.analyze_test()
if test_info is None:
- # If html file is in a "resources" folder, it should be copied anyway
- if self.filesystem.basename(self.filesystem.dirname(fullpath)) == "resources":
- copy_list.append({'src': fullpath, 'dest': filename})
+ # This is probably a resource file.
+ if self.filesystem.basename(self.filesystem.dirname(fullpath)) != "resources":
+ self._potential_test_resource_files.append({'src': fullpath, 'dest': filename})
+ copy_list.append({'src': fullpath, 'dest': filename})
continue
+ if 'manualtest' in test_info.keys():
+ continue
+
+ if 'referencefile' in test_info.keys():
+ # Skip it since, the corresponding reference test should have a link to this file
+ continue
+
if 'reference' in test_info.keys():
reftests += 1
total_tests += 1
@@ -413,6 +424,11 @@
for prefixed_value in sorted(total_prefixed_property_values, key=lambda p: total_prefixed_property_values[p]):
_log.info(' %s: %s', prefixed_value, total_prefixed_property_values[prefixed_value])
+ if self._potential_test_resource_files:
+ _log.info('The following files may be resource files and should be marked as skipped in the TestExpectations:')
+ for filename in sorted([test['src'] for test in self._potential_test_resource_files]):
+ _log.info(filename.replace(self.source_directory, self.tests_w3c_relative_path) + ' [ Skip ]')
+
def remove_deleted_files(self, import_directory, new_file_list):
""" Reads an import log in |import_directory|, compares it to the |new_file_list|, and removes files not in the new list."""
Modified: trunk/Tools/Scripts/webkitpy/w3c/test_parser.py (191379 => 191380)
--- trunk/Tools/Scripts/webkitpy/w3c/test_parser.py 2015-10-21 07:44:33 UTC (rev 191379)
+++ trunk/Tools/Scripts/webkitpy/w3c/test_parser.py 2015-10-21 07:53:37 UTC (rev 191380)
@@ -95,6 +95,9 @@
_log.error('%s has a reference link but is missing the "href"', self.filesystem)
return None
+ if (ref_file == self.filename):
+ return {'referencefile': self.filename}
+
if self.ref_doc is None:
self.load_file(ref_file, True)
@@ -114,10 +117,12 @@
test_info['reference_support_info'] = {}
# we check for wpt manual test before checking for jstest, as some WPT manual tests can be classified as CSS JS tests
elif self.is_wpt_manualtest():
- test_info = None
+ test_info = {'test': self.filename, 'manualtest': True}
elif self.is_jstest():
test_info = {'test': self.filename, 'jstest': True}
- elif self.options['all'] is True and not('-ref' in self.filename) and not('reference' in self.filename):
+ elif '-ref' in self.filename or 'reference' in self.filename:
+ test_info = {'referencefile': self.filename}
+ elif self.options['all'] is True:
test_info = {'test': self.filename}
return test_info
Modified: trunk/Tools/Scripts/webkitpy/w3c/test_parser_unittest.py (191379 => 191380)
--- trunk/Tools/Scripts/webkitpy/w3c/test_parser_unittest.py 2015-10-21 07:44:33 UTC (rev 191379)
+++ trunk/Tools/Scripts/webkitpy/w3c/test_parser_unittest.py 2015-10-21 07:53:37 UTC (rev 191380)
@@ -166,7 +166,7 @@
parser = TestParser(options, os.path.join(test_path, 'somefile-manual.html'))
test_info = parser.analyze_test(test_contents=test_html)
- self.assertEqual(test_info, None, 'test_info is None')
+ self.assertTrue(test_info['manualtest'], 'test_info is None')
def test_analyze_pixel_test_all_true(self):
""" Tests analyze_test() using a test that is neither a reftest or jstest with all=False """
@@ -228,3 +228,26 @@
parser = TestParser(options, os.path.join(os.path.dirname(__file__), 'test_parser.py'))
test_info = parser.analyze_test()
self.assertEqual(test_info, None, 'no tests should have been found in this file')
+
+ def test_reference_test(self):
+ """ Tests analyze_test() using a test that is a reference file having a <link rel="match"> tag"""
+
+ test_html = """<html>
+<head>
+<title>CSS Test: DESCRIPTION OF TEST</title>
+<link rel="match" href="" />
+<link rel="author" title="NAME_OF_AUTHOR" />
+<style type="text/css"><![CDATA[
+CSS FOR TEST
+]]></style>
+</head>
+<body>
+CONTENT OF TEST
+</body>
+</html>
+"""
+ test_path = os.path.join(os.path.sep, 'some', 'madeup', 'path')
+ parser = TestParser(options, os.path.join(test_path, 'test-ref.html'))
+ test_info = parser.analyze_test(test_contents=test_html)
+
+ self.assertTrue('referencefile' in test_info, 'test should be detected as reference file')
