On 9/20/14, 5:03 AM, Karl Dubost wrote:
The biggest issue I have with this is exiting CR without anything resembling a
comprehensive enough test suite
* What is a comprehensive enough test suite?
Ideally, one that has a test for every normative requirement in the
specification. This means at least one test per sentence of normative
text, basically.
In practice, this is a very high bar, because that includes testing
various interactions between features, which can get pretty hairy.
A good start, though, would be direct testing of at least all the
obvious conformance requirements explicitly listed in the specification,
if not of their non-obvious interactions.
* How far the current test suite is from the comprehensive test suite you would
have wished.
I haven't looked into this in detail, honestly.
Given that I know there are parts of the specification that don't match
browsers and that no one has brought up, clearly "somewhat"....
* Does Mozilla has a comprehensive test suite on the same set of features?
Probably not.
to ensure anything like interop on some of the core hard pieces (they left out
the navigation algorithm, smart, but still have the bogus WindowProxy spec in
this supposed PR, for example).
s/they/we/
The first rule of a group in which we (Mozilla) participate is to include
yourself in the discussion. It helps a lot to change the attitude with regards
to the issues.
I don't think Mozilla meaningfully participates in this working group.
We've tried, but the environment was hostile, and our participation
seemed generally unwelcome, so we gave up for all but process purposes.
If a group explicitly chooses to exclude me from the discussion, I feel
no particular need to consider myself part of that group, so I am
sticking by my "they". Nor do I feel any particular responsibility for
their actions, for what it's worth.
My second biggest issue is that I don't have a concrete proposal for addressing
this the first issue.
The test suite?
Yes. I have no concrete proposal for scrounging up the resources to
evaluate which aspects of the test suite are lacking, much less for
writing tests to remedy that lack.
My biggest issue with HTML5 spec is that it is too big to be meaningfully
implementable and/or testable.
We have a slight problem, don't we? It's not like the plan is to lose
any of these features, and browsers _are_ expected to implement them in
non-buggy ways.
It is not necessary solvable for this round, but that could teach us on how to
improve how to develop the future of features for the Web with more testing
upfront and more modular approach.
A more modular approach doesn't necessarily help, since you have to test
interactions between the modules (though it sure makes it easier to
ignore this need). In the end, whatever amount of interacting stuff you
have will require testing en-masse.
Now maybe a modular approach will mean that there won't be interactions.
Or maybe it'll mean the interactions are less obvious and easier to
overlook and get wrong. We'll see.
100% agreed on more testing up front.
Basically we can learn from our mistakes. Not everything is lost ^_^
Again, agreed.
It's here where I have a disconnect with the first comment. Be whatwg spec or
w3c spec if we dim that a comprehensive test suite is important then there
should be one whatever the stamp on the text.
Yes, agreed. I should have been clearer.
The important part to me about implementations is that implementations
shouldn't follow the known-bogus parts of the HTML5 REC once said
bogosity if fixed in the WHATWG spec and HTML5.1 (with the former more
likely to happen sooner).
-Boris
_______________________________________________
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform
