Hi,
the reason why the Wiki page speaks of mandatory tests I have mentioned
in a previous mail:
Jörg Jahnke schrieb:
The problem with such tests not being mandatory is that, sooner or
later, some tests would break. That again would lead to a state where
the user of the tests could not be sure whether a broken test-case means
that he introduced a bug or whether he just encountered an old problem
that broke the test-cases before. He would have to start a tedious
search to find out the cause of the problem - just like the testers have
to do nowadays. And then people would simply not use the tests because
the efforts are too high...
Ause just informed me about another solution that might remove the need
to have the test run on every CWS i.e. we wouldn't need to have the
tests mandatory. His idea is to run the tests on the Master Workspace
prior to announcing the CWS as "ready for CWS use". If a test fails then
this would result in a P1 issue that has to be fixed before the MWS can
be used by everyone. Very similar to how we handle it for the Smoketest
on the MWS nowadays.
Additionally the list of tests to run would be checked in to CVS, so
that we could disable a tests for every user on a given milestone if a
fix cannot be done in time.
That way a developer could get an _optional_ means at hand of doing
regression tests, with no obligation to always run these tests. If the
developer feels that he should run the tests, then he could do so and
invest the (machine) time. If he thinks that the tests will be no
additional help, he just does not run them.
Of course the question then is how often such a regression happens. If
we have to expect to have half a dozen P1 bugs each milestone due to the
mass of regressions, then the "mandatory for every CWS" seems the better
solution to me. But if we expect to have such a P1 bug from the
automatic tests only once every 2 or 3 milestones (or hopefully even
less often), then this seems an acceptable way to me.
Does that make sense?
Jörg
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
