Thorsten Ziehm wrote: > We will never find all regressions with TestTool or any other tooling > or human testing. This has nothing to do with 'release testing' or so. > We do not have test cases which identify problems in displaying > documents or something similar. Perhaps intensive usage of ConvWatch > could help a little bit. But to make this mandatory is another story! > > When we talked about the last big regressions with CWS aw024 or the > CWS with AutomaticStyles in Writer. I do not have any solution until > now, how such regressions could be minimized in the near future. Our > test mechanisms do not find such regressions. Only a mass of human > testers could help here. So we are trying to bring more QA members of > the community to the relevant master builds for intensive testing of the > new feature. > > So I do not think, that it make sense to discuss only the 'release > testing' mode. In the past the regressions were integrated before the > QA started with switching in this mode.
I'm not sure if you understood my concern. Let me put it simple: what makes us think that the current tests we are talking about that AFAIK have been used in QA for testing the master for quite some time will help to find regressions that currently stay unnoticed? > When I read all your mails, I think you know, which code bring in the > regressions ;-) I know some places, yes. Of course not all of them. But I don't know which tests we could use to find them. But many of the regressions I remember couldn't be found by automated GUI testing as they manifested themselves by showing some more or less subtile formatting differences. > As I wrote some mails ago my suggestion is, to bring only a small set of > mandatory tests. But give the solution to select testing areas. Then you > can run dedicated tests on your implementation. And you will not run > toolbar tests on your bugfix for automated styles. Anything else would be insane. I took that for granted. But I also want to believe that running several hours of tests for e.g. automatic styles would be worth the effort. This is a good example where I suspect that possible regressions would stay unnoticed by automatic GUI testing. But of course that's open for debate. And exactly this debate is what I want to see happening. So let's wait until the proposed test cases are published and until we have verified that they run reliably. Then the QA and development engineers of the different teams can investigate them and decide if they make sense or if we can create other tests that serve the desited purpose better. *Then* we can decide whether we wanted to run the tests more frequently or even make them mandatory. Ciao, Mathias -- Mathias Bauer (mba) - Project Lead OpenOffice.org Writer OpenOffice.org Engineering at Sun: http://blogs.sun.com/GullFOSS Please don't reply to "[EMAIL PROTECTED]". I use it for the OOo lists and only rarely read other mails sent to it. --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
