[webkit-dev] Importing W3C tests for editing

2014-02-05 Thread Lukasz Bialek 
Hi all,

 

Some time ago there was a conversation about importing W3C tests into
LayoutTests for WebKit
(https://lists.webkit.org/pipermail/webkit-dev/2013-November/025876.html).
This conversation was paused in November with conclusion, that it may be a
good idea.

 

A brief research showed, that number of editing tests available in
https://github.com/w3c/web-platform-tests is very small. However, there are
a lot of editing tests placed directly in W3C HTML Editing APIs
specification
(https://dvcs.w3.org/hg/editing/raw-file/tip/editing.html#conformance-tests)
. It may be a good idea to import those tests instead.

 

There are 90412 tests covering all aspects of W3C specification. Those tests
can be divided into test cases containing several tests with structure
similar to this:

 

PASS [[stylewithcss,true]] foo[bar]baz: execCommand(stylewithcss,
false, true) return value 

PASS [[stylewithcss,true]] foo[bar]baz checks for modifications to
non-editable content 

PASS [[stylewithcss,true]] foo[bar]baz compare innerHTML 

PASS [[stylewithcss,true]] foo[bar]baz
queryCommandIndeterm(stylewithcss) before 

PASS [[stylewithcss,true]] foo[bar]baz
queryCommandState(stylewithcss) before 

FAIL [[stylewithcss,true]] foo[bar]baz
queryCommandValue(stylewithcss) before assert_equals: Wrong result
returned expected  but got false

PASS [[stylewithcss,true]] foo[bar]baz
queryCommandIndeterm(stylewithcss) after 

PASS [[stylewithcss,true]] foo[bar]baz
queryCommandState(stylewithcss) after 

FAIL [[stylewithcss,true]] foo[bar]baz
queryCommandValue(stylewithcss) after assert_equals: Wrong result returned
expected  but got true

 

As a result, we would have 5536 new test files divided into 36 categories:

- backcolor: 70

- bold: 210

- createlink: 48

- defaultparagraphseparator: 11

- delete: 494

- fontname: 141

- fontsize: 156

- forecolor: 147

- formatblock: 362

- forwarddelete: 471

- hilitecolor: 82

- indent: 143

- inserthorizontalrule: 117

- inserthtml: 104

- insertimage: 70

- insertlinebreak: 166

- insertorderedlist: 135

- insertparagraph: 320

- inserttext: 238

- insertunorderedlist: 148

- italic: 143

- justifycenter: 294

- justifyfull: 233

- justifyleft: 149

- justifyright: 233

- outdent: 175

- quasit: 1

- removeformat: 143

- selectall: 1

- strikethrough: 145

- stylewithcss: 17

- subscript: 85

- superscript: 87

- underline: 145

- unlink: 35

- usecss: 17

 

Furthermore, newly created tests has been run and results showed, that
1120/5536 tests run as expected (all sub-tests returned PASS) and 4416/5536
did not. There are 4 WebProcess crushes; 12 tests have some stderr output.
Rest of failures are because of one or more sub-tests failing. Running all
5536 tests takes 32 min 51.978 sec.

 

What do you think about it? Is it worth paying some time for importing those
tests and then aligning WebKit with W3C specification?

 

Best regards,

Lukasz Bialek

___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev

Re: [webkit-dev] Importing W3C tests for editing

2014-02-05 Thread Robin Berjon 

On 05/02/2014 13:16 , Lukasz Bialek wrote:

What do you think about it? Is it worth paying some time for importing
those tests and then aligning WebKit with W3C specification?


Depending on the amount of energy you're willing to dedicate to this, 
the ideal option could well be to import those tests into the 
web-platform-tests repo and then into WebKit.


Note that I've heard reports in the past that some browsers tried to 
align with the spec but couldn't. The problem is that many of the 
discrepancies are difficult to feature-detect so that an awful lot of 
rich text editing code out there branches off UA identification. As a 
result, fixing bugs that bring browsers more in line with one another 
breaks the Web.


I'd love to hear indication that this is less of a SNAFU than previously 
reported. If not we might be stuck and in need of a more radical solution.


--
Robin Berjon - http://berjon.com/ - @robinberjon
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev

Re: [webkit-dev] Importing W3C tests for editing

2014-02-05 Thread youenn fablet 
We are still far from importing into WebKit the whole test set from
web-platform-tests repo.
Improving the situation here would already be a great step forward.

It would also be good to regularly run the whole web-platform-tests repo
test set so as to get statistics in terms of feature alignment and test
import completeness/synchronization. See
https://bugs.webkit.org/show_bug.cgi?id=127095#c9 for rough notes on that.
I hope to work on that once the web platform test server will be integrated
into WebKit.
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev