Hi Ingrid,
Ingrid Halama schrieb:
Mathias Bauer wrote:
The problem is that the usual test runs obviously don't find the bugs
That is not obvious to me. Too often the mandatory tests haven't been
run. And if tests do not find an important problem, hey then the tests
should be improved.
See my post at d...@qa (
http://qa.openoffice.org/servlets/ReadMsg?list=dev&msgNo=11964 ).
Our toolset for automated tests even reports "all ok" if the testttool
correctly identified a stopper bug. Many other bugs cannot be found by
the testtool (e.g. "visual problems" like issue 99662 ).
I think, the benefits of automated testing are overstimated (or the
costs are underestimated).
that now bite us, most of them have been found by users or testers
*working* with the program. Adding more CWS test runs and so shortening
the time for real-life testing will not help us but make things worse.
I don't agree. Preventing the integration of bugs earlier in the
production phase especially before the integration into the master
trunk would give us much more freedom. Now we always need to react on
show stoppers and react and react and uh then the release time line is
on risk. All that, because the bugs are already in the product. If you
instead detect the bugs before they are integrated into the product
you can keep cool, refuse the bad CWS and thus not the release is on
risk but only the single bad CWS.
The point is *if* you detect the bugs in the CWS. At the moment we
obviuosly do not identify enough critical issues while CWS testing (even
if the mandatory tests are done).
....
I am missing a stimulation for good behaviour in this plans. There are
people who do the feature design, who do the developing work, who do
the testing, who create the automatic test, who do the documetnation
and after all these people have done their work and lets assume they
have done it good and without show stoppers, after all this there
comes someone else and says, oh no, I do not think that I want to have
this for this release, there are other things that I want to have more
and in the sum I guess that it might be to much for the next release?
Where is the stimulation for good behaviour here?
The problem is, that we first need to prove that "good behaviour" is
really helpfull and prevents critical bugs in the master. I'm all for
promoting good behaviour - and in most cases I would like to see more
people who follow the rules (like publishing specs correctly at the
specs website).
But we must be allowed to review our processes and identify the parts
that are not so helpfull.
There is none, instead it is a stimulation to push in the changes
quickly into the product and skip careful testing.
If this is some stimulation: thanks for all the chart specs that are
correctly linked at the specs website. These are very helpfull to speed
up testing, get an idea about the new functionality and in many cases
arethe only resource to get our translations correct.
André
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
