On Sun, Sep 21, 2014 at 4:00 PM, James Graham <ja...@hoppipolla.co.uk> wrote: > On 20/09/14 03:46, Boris Zbarsky wrote: >> On 9/19/14, 8:23 PM, L. David Baron wrote: >>> W3C recently published the following proposed recommendation (the >>> stage before W3C's final stage, Recommendation): >> >> The biggest issue I have with this is exiting CR without anything >> resembling a comprehensive enough test suite to ensure anything like >> interop on some of the core hard pieces (they left out the navigation >> algorithm, smart, but still have the bogus WindowProxy spec in this >> supposed PR, for example). > > Obviously I agree that good testing of implementations is key to > interoperability.
Talking about testing in the context of the snapshot reads like pretending that we should treat the snapshot as something other than a lawyer point of reference for the purpose of the PP. Of course, if something in the spec is wrong and we implement something else, maybe the PP isn't that strong on that point, but it still seems worthwhile to get PP commitment for the stuff that happens to be right or close enough to right for lawyer purposes. > I also agree that we should encourage vendors to > create and run shared tests for the web technologies that we implement. > I am substantially less convinced that tying these tests to the spec > lifecycle makes sense. The W3C Process encourages people to think of > interoperability as a binary condition; either implementations are > interoperable or they are not. This leads to ideas like the CSS WG's > rule that two implementations must pass every test. On the face of it > this may appear eminently sensible. However the incentives for testing > under such a regime are not well aligned with the goal of finding bugs > in implementations; in essence people are encouraged to write as many > tests as are needed to satisfy the letter of the rules but to make them > all as shallow and unlikely to find bugs as possible to avoid causing > unwanted holdups in the specification process. I would much prefer to > have a testsuite written by people making a genuine effort to find > errors in implementations even if the result is that no one passes every > single test. These effects of tying tests to snapshots are clearly harmful. We should track EDs for testing and implementation instead of targeting known-stale snapshots. > My concrete suggestion is that we, as an organisation, work to achieve > parity between the tests we require to ship our own code and those we > release in ways that can be used to support a spec and, more > importantly, those that can be used verify that different > implementations match up. In practice this means not writing tests in > Mozilla-specific formats, and making sure that we have a way to upstream > tests that we've written. Making this process as low-overhead as > possible is something that I'm working on. Yes. Furthermore, the test damage from snapshots lessens if the people who do the work to write the tests contribute tests written to EDs instead of snapshots. Our tip of the tree of code should follow the tip of the tree for specs and the tests we contribute should make sense for our tip of the tree. > However if we as an organisation really care about testing > core platform features which already have an implementation in gecko, > one way to achieve that would be to give someone working on Servo the > explicit job of creating testsuites for the big-ticket features they > implement as they implement them. I agree. >> Maybe it all doesn't matter too much as long as implementors keep >> reading the whatwg spec instead. > > In terms of HTML at the W3C it's pretty clear that they've dropped the > ball, and haven't even realised it yet. There was a thread started five > days ago about the future of HTML after this release and so far it has > gained exactly no responses from implementors. The WG turned into an > unproductive, bureaucratic, nightmare and succeeded in driving out their > core constituency Sadly, yes. > leaving the remaining participants to debate topics of > little consequence. FWIW, this bit is being spun into Twitter propaganda about Mozilla not caring about accessibility, so it might be worthwhile to be careful with the phrasing. For the people tweeting: It's not like a11y was the only thing left. Let's not forget Polyglot. See https://twitter.com/mattur/status/510818045749899264 and https://twitter.com/mattur/status/461240899503407105 if you still think Polyglot is a worthy WG deliverable. And to the extent debates that have remained fit under the topic of a11y--and in that light what James said may seem offensive--it doesn't do a11y any favors to hold the permathreads about longdesc as a symbol of pro-a11y activity. (Note that e.g. http://lists.w3.org/Archives/Public/public-html-admin/2014Aug/0028.html is not an anti-a11y or not-caring-about-a11y statement!) But both Polyglot and the longdesc stuff are in extension specs, so while they are relevant to the WG bureaucracy and unproductivity, they are less relevant to the HTML5 snapshot at hand. Even if one believes all of http://www.w3.org/wiki/HTML/W3C-WHATWG-Differences to be actual a11y improvements over the WHATWG version that we should implement and it would be wrong for James' characterization to extend to them, making those improvements through a framework that drove browser implementors away from public-html isn't really a success worth bragging about (or, from the Mozilla perspective on the whole a good tradeoff). Also, if our a11y team is implementing the W3C spec on those points, it would be more productive to say so here than to fuel the flames on Twitter. > This idea of shipping on a date-based schedule isn't actually > obviously bad, as long as you set the expectations correctly, which is > something the W3C will get wrong. I think our comments to the W3C or otherwise should not support the meme that snapshots are something that browser developers, Web developers, ebook platforms or government procurements should target. I think we should not object to the snapshot publication though, since if our objection was successful (it wouldn't be), we'd fail to get a PP snapshot out of all the trouble that trying to work with the HTML WG has caused. To get the snapshot for the purpose of the PP but to pre-empt the use of Mozilla's "support" for promoting the snapshot for purposes where referring to stale specs is harmful, maybe choosing the option to explicitly abstain is the way to achieve that. > It would indeed be nice if W3C would > embrace this fully and move to a WHATWG-style model with no eratta but a > spec kept continually up-to-date in the areas where there is a need for > change, and absolute rigidity in the areas where reality dictates that > change is impossible. However moving them there is a longer term goal > that seems to be met with extreme resistance from people who haven't > fully grasped how shipping a web browser works. Yes. -- Henri Sivonen hsivo...@hsivonen.fi https://hsivonen.fi/ _______________________________________________ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
