Hi Mathias,
Mathias Bauer wrote:
[...]
You are right, there were not found all regressions in Master by the
automated tests. But some of them were found, when some more tests are
mandatory. In the past only 2 smaller tests are mandatory for approving
a CWS. Many testers run more than these tests, but not all. Therefore
some regressions went into the Master, which could be identified by
the test cases.
I didn't talk about tests on a CWS (I know that we only have a few
mandatory tests), I was talking about the regressions that haven't been
detected by the "release testing" on the master.
We will never find all regressions with TestTool or any other tooling
or human testing. This has nothing to do with 'release testing' or so.
We do not have test cases which identify problems in displaying
documents or something similar. Perhaps intensive usage of ConvWatch
could help a little bit. But to make this mandatory is another story!
When we talked about the last big regressions with CWS aw024 or the
CWS with AutomaticStyles in Writer. I do not have any solution until
now, how such regressions could be minimized in the near future. Our
test mechanisms do not find such regressions. Only a mass of human
testers could help here. So we are trying to bring more QA members of
the community to the relevant master builds for intensive testing of the
new feature.
So I do not think, that it make sense to discuss only the 'release
testing' mode. In the past the regressions were integrated before the
QA started with switching in this mode.
We should try to identify tests that will be able to detect regressions
in code where we know that it is prone to regressions. I don't want to
make tests mandatory if it tests code that most probably will not create
a single regression in 5 years or so.
This isn't the case.
[...]
and then deciding how to deal with them. If you think that the 45 test
cases identified by the QA team are a proper selection we should have a
closer look on them and identify which code they test.
When I read all your mails, I think you know, which code bring in the
regressions ;-)
As I wrote some mails ago my suggestion is, to bring only a small set of
mandatory tests. But give the solution to select testing areas. Then you
can run dedicated tests on your implementation. And you will not run
toolbar tests on your bugfix for automated styles.
Thorsten
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
