Hi, Marcos-
Marcos Caceres wrote (on 1/19/10 10:49 AM):
Hi all,
A draft of "A Method for Writing Testable Conformance Clauses and its
Applications" in now available for review online [1]. For those that
have not seen it, it basically just documents how we are standardizing
the Widget specs and some basic QA processes:
http://dev.w3.org/2008/dev-ind-testing/extracting-test-assertions-pub.html
Please consider this a working draft, as it likely contains typos, and a
couple of half-baked ideas, etc. Comments are, of course, welcomed. It
is expected that this document will be published as a working group note
at some point in the future.
This is an interesting doc and case study, with useful best-practices
that mirror some of my own experience in creating specs and test suites.
I plan on integrating this into specs I edit, where possible.
I think you could emphasize algorithms a bit more, which is
traditionally a weak point in W3C specs.
It may or may not be appropriate for this document to discuss document
readability more; W3C specs traditionally serve multiple audiences, such
as implementers, tutorial/book writers, and content creators (authors),
and while there is a move to make specs more implementer-focused and
separate out these concerns, this decreases the value of the specs as
community artifacts, makes tutorials more likely to be available only
much later and to be error-prone, since they are not written by the same
group of people who wrote the prose of the spec.
I really like the goal and execution of this practice. Some of it
reminds me of my experimental Speki [1] project, which was an
exploration of a wiki toolbar to mark up specs for easier spec
production. One aspect of this was to create special stylesheets for
making the conformance criteria more obvious (follow the link, and
select the "implementers" presentation on the sidebar for a
demonstration). I applied some of this markup to DOM3 Events, but not
consistently enough. This was part of my Project Tinker [2]
architecture proposal (which I've neglected for a year or two).
Overall, I am very supportive of this kind of detailed analysis, and I
appreciate the time and care you took to develop and write it up. This
kind of document stands to make the production and quality of specs much
better. My chief suggestion is that, having developed your workflow and
tools, you should make them available to other groups; I am interested
in reusing your tools for DOM3 Events and SVG, specifically the markup
and implementation report tools, and with some modification, the test
suite markup and tools. SVG has its own tools and processes for some
parts of this, but I strongly believe that the benefit increases the
more specs use the same toolchain.
Here are some specific comments, mostly typo corrections or
clarification suggestions:
1. Common mistakes
* For each of the commons mistakes, include corrections for the faulty
prose.
2. The method
* "stable identifier" may be a little vague and abstract; if you mean to
add an @id, say, "give them a stable identifier, (such as) an 'id'
attribute value that is consistent between drafts". If you mean
something else, describe it.
* Typo: "(see )"
* Typo: "learnt" -> "learned"
2.1 Relationship to the standardization process
* "tangible objects": it might be better to use the term "deliverables",
since that is the wording of the Process Document and charters; at the
least, associate the 2 terms... "tangible objects (also called the
"deliverables" of the group) ..."
* Include "issue tracking software" in the list of tools provided by the
W3C, which can be used as part of the test/spec feedback loop.
3. Structural components of a conformance clause
* "Product" might be better stated as a "Conformance Category", which
includes authoring tools and authors. While tests against authored
content are not in the scope for this document, editors need to be aware
of the various conformance categories that they need to keep in mind
when writing, of which "user agents" are only one category. DOM3 Events
describes the following conformance categories: Web browsers and other
dynamic or interactive user agents; Authoring tools; Authors and
content; and Specifications and host languages.
4. Conventions for marking-up conformance clauses
* Typo: "anylising" -> "analyzing"
* Typo: "Prerequisites: <"If"" -> "Prerequisites: "If""
* Typo: "Working Groups is" -> "Working Group is"
* "Hyperlink to the appropriate terms." Show, don't tell. Give us an
example here, to contextualize what you mean by "the appropriate terms",
even though the full example is down below: "Hyperlink to the
appropriate terms. For example, "encounters a <a class="term"
href="#file">file</a> matching""
5. Extracting conformance clauses
* "in favor of using JavaScript" -> "in favor of using a custom
JavaScript library called _____"... make it easy to discuss and share
* Typo: "with am XML document" -> "with an XML document"
* Typo: "it allows to quickly assess" -> "it allows quick assessment"
* Typo: "informant ion" -> "information"
* Typo: "describe individual results" -> "describes individual results"
* Typo: "attrite" -> "attribute"
* Grammaro: This is a sentence fragment: "And where the test ran, but it
was not possible to determine if the test actually passed or failed
(labeled as "incomplete")." -> "And where the test ran, but it was not
possible to determine if the test actually passed or failed, the verdict
was indicated as "incomplete"."
* Typo: "three separate steps (, marking it up the specification," ->
"three separate steps (marking it up in the specification," or "three
separate steps (marking up the specification,"
* Typo: "and in practice to work best as an iterative process." -> "and
in practice work best as an iterative process."
[1] http://www.schepers.cc/speki/index.php?title=MonkeyML
[2] http://www.schepers.cc/tinker/
Regards-
-Doug Schepers
W3C Team Contact, SVG and WebApps WGs
