Hi,
partly quoting myself from 
https://github.com/w3c/web-platform-tests/pull/1272 :

Nearly all tests in the XHR test suite have (potentially outdated) meta data 
linking them to specific assertations in the spec. (Technically this is a set 
of space-separated XPath expressions for each link @rel=help element which, 
combined with the section linked to from the HREF, identifies a given 
assertation).

I've written some new code to actually make use of this data (I used to have 
some, but it was an Opera UserJS script using the localStorage available to 
such scripts - a bit too browser-specific, and I lost it when leaving Opera 
anyway). 

In this pull request, I add one Python script to extract the meta data into a 
single JSON file, and one javascript to iterate over the resulting JSON data 
and annotate the spec.

To test the outcome, load http://www.w3.org/TR/XMLHttpRequest/ and run this 
from the browser's dev tools' console:

document.body.appendChild(document.createElement('script')).src='http://hallvord.com/temp/xhr/annotate_spec.js'

You may have to disable mixed content blocking in your browser if the spec 
loads over https.

To be clear: the expected effect of running the script is

1) Every assertation that have at least one associated test, get a light green 
background
2) One or more link(s) to relevant test(s) are added after each assertation.

As I said, some of the meta data is subtly outdated - this is especially 
evident in the "open method" section, where most links are off by one LI 
(clearly, we spec authors added a LI since the meta data was edited). I will 
find some time to review and fix such issues. Also, sections that are in fact 
being tested may lack annotations.

In any case, hopefully the net outcome of this experiment is that we can state 
with a lot of confidence that we have a high test coverage for our spec, and 
that we can push coverage further by identifying corners that remain un-tested.

Please test and comment :)

-Hallvord

Reply via email to