Hi Dom,
Dominique Hazael-Massieux wrote:
Hi Marcos,
I saw that the test suite for TWI was discussed on the WebApps call
today:
http://www.w3.org/2009/11/12-wam-minutes.html#item05
Since the discussion didn’t allude at all to my mail below about
generated test cases, I thought I would point you to it explicitly in
case you had missed it.
We had not missed it, but we are not up to the point where we can use
automated tests. For the purpose of standardization, the creation of
*manual* test cases forces the Working Group (in this case, me!) to
verify that every testable assertion is correctly written and testable
against a product that can claim conformance to a specification.
We believe that an important side effect is that, through the *manual*
creation of test cases, the test suite can reduce the variability of the
specification. Reducing the variability makes the specification more
precise, which in turn can make the specification easier to interpret
and implement (to paraphrase my testing mentor - you ;)).
On the other hand, automated test generation can generate a large number
of test cases and is less prone to human errors. But, at the same time,
it cannot test some things that are written in the prose. For example, a
AU must not fire Storage events when first populating the preferences
attribute. This is impossible to express in IDL.
Kind regards,
Marcos
