Hi Andre,
sorry for the late reply. :-(
The idea of a sanity check is very good. In the Sun QA team we discussed
the sanity checks some times ago. The OOo QA and we have to manage so
many releases, that it isn't possible to test all functionality at the
last builds again and again.
But as you said, it is very difficult to find the right number of tests.
We decided to let the most functionality tested by ~40 automated test
cases. Only some visibility tests are on our list for a sanity check by
hand on the last builds.
In my opinion the intention of a sanity check is, to check that the
global functionality isn't broken in the last builds. These test can be
done by automated testing with TestTool (udpate tests). When starts with
sanity tests, the general testing has to be finished. Only showstopper
are interesting, general tasks should be fixed in the next update.
I would strike the list of test cases for a sanity check to 10-15 cases.
The needed time for the check shouldn't be more than 15-30 minutes for a
build (inclusive the installation). Additionally the automated tests
with TestTool should be done for each build (some hours) - but this is
time for machines and not working hours for a person.
Again, which tests should be integrated into a sanity check are very
difficult. ;-)
Regards,
Thorsten
Andre Schnabel wrote On 06/30/06 13:25,:
Hi,
I'v created a new scenario in TCM, called "OOo release sanity". I'd like
to use this as the absolut minimum of tests, that a build needs to pass
to be approved for release.
This should help in situations like we have with 2.0.3:
- a series of RC's has already been tested
- a small amount of stoppers needs to be fixed
- running the full tests again would take a lot of time
- running no tests at all would break our QA policy
I'm not shure, what test cases should be put to the scenario. The
current set is my first attemt .. and as always, at the one hand it
seems not enough to be shure we have no bugs .. at the other it seem to
be to many tests to be able to go through the scenario quickly.
Comments are welcome (no problem, if they would take some time, I know,
native lang teams are still busy with 2.0.3 testing).
andré
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
