I'm about to issue a CfC on publishing Selectors as a CR, independent of getting the test suite done. Because it has taken a long time not to get it done, and the result is no CR.

We will need to agree on a Test Suite, and on exit criteria. So this message is to see if there is any disagreement.

A possible question is whether it counts to have a JS implementation or similar. I think it would be reasonable, so long as it is a completely independent implementation, although I would rather have it *in addition to* native implementations in shipping products.

On Wed, 24 Jun 2009 15:56:11 +0200, Lachlan Hunt <[email protected]> wrote:

Charles McCathieNevile wrote:
Actually, based on feedback on the list (thanks Maciej and Robin), and
talking to Lachy, we are thinking that we should seperate out the tests
that *require* CSS 3 selectors, to make the test suite check
implementation of the API, and then require at least two 100% complete
and completely interoperable implementations.

I believe Lachy will be following up on this about now - both for the
list and the test suite.

Here is the revised proposal for the exit criteria.

* Tested implementations are required to have support for:
   - Selectors API
   - Selectors defined in CSS 2.1.
   - HTML

* Tested implementaions may optionally support:
   - Selectors introduced in Selectors Level 3
   - XHTML
   - SVG

At least two implementations must pass 100% of the baseline testsuite and should pass additional tests, dependent on the following conditions:

* The baseline testsuite comprises tests that check for conformance to
   all requirements in the API using only HTML and Selectors defined in
   CSS 2.1.

* Tests using Selectors introduced in Selectors Level 3, or XHTML+SVG,
   are considered to be additional tests.

I wonder if we need to make these additional tests rather than baseline, for the purposes of demonstrating that browsers get this spec right.

* An additional test may be marked as N/A for an implementation if:
   - The test uses a selector that the implementation does not support
   - The test uses XHTML+SVG that the implementation does not support

* Implementations are not required to pass all additional tests,
   however no failures must be caused by an incorrect implementation of
   the API itself. Failures of additional tests caused only by an
   incorrect implementation of Selectors do not count.


This implies that the testsuite should be split into several files:

I think that at most we should be designating tests as baseline or additional, rather than trying to classify the various kinds of additional files.

1. Baseline containing tests using only HTML and CSS 2.1
2. Additional tests using XHTML+SVG and CSS2.1 (equivalent to the
    previous test, but with the addition of SVG-related tests)

Maybe, maybe not, as per above.

3. Additional tests using HTML and Selectors 3
4. Additional tests using XHTML and Selectors 3

cheers

Chaals

--
Charles McCathieNevile  Opera Software, Standards Group
    je parle français -- hablo español -- jeg lærer norsk
http://my.opera.com/chaals       Try Opera: http://www.opera.com

Reply via email to