Martin Hollmichel wrote: > Do we have some statistics in which areas we have what amount of > regressions ? > > For example I would think that regression caused by broken resources > doesn't occur that much any more, are also easy to find by broad > testing. On the other hand I could image that regressions in document > layout do occur much more often and would be reported much more later > than broken resources ?
Exactly this is one of my concerns. The tests that Jogi mentioned are surely useful to prevent huge disasters but I'm not sure if they are qualified for testing for the regressions we usually have. Regression tests IMHO should tackle the areas where we know that regressions happen more frequently. I hope it's undisputed that we can only execute a very limited set of tests and so it should be our interest to use those test that are able to catch the biggest fish possible. >From the Writer's perspective interesting areas could be text formatting and layout, undo/redo, redlining, numbering and some more. Regressions in resource files or the complete inability to start an application rarely happen and the latter IMHO is already covered by the smoketest. BTW: extending the smoketest would be better than adding another tool for testing. I'm also not concerned about regressions in the "main features" of the applications as usually regressions in these areas are discoverd pretty fast. I'm concerned about regressions that are not so obvious and usually are found too late. My impression was that this was what created the idea to have more regression testing, so we should put our focus on them. These regressions are not only crashes, often they are more or less subtile formatting or functionality differences where I wonder how the provided tests could discover them. Tests are software and we have learned that for good software one needs to understand the requirements first and then select the best design to implement them. I know that we are not perfect in following that but IMHO we should at least try. As my requirement would be to find as much regressions as possible with as less effort as possible I would like to : - identify the areas of regressions (we should have some data about it) - think about what kind of tests are best suited to find them earlier - try to implement such tests so that they are reliable and as easy and fast to execute as possible - help developers to find out fast and easily which test should be applied when As obviously noone else is interested in a content oriented discussion I think I will quit and wait what we can test with the provided tests and if this is what we need. Until then my impression is that continuing this discussion will be less valuable then the waste air our computers create while we are writing our mails. :-) 'nuff said, Mathias -- Mathias Bauer (mba) - Project Lead OpenOffice.org Writer OpenOffice.org Engineering at Sun: http://blogs.sun.com/GullFOSS Please don't reply to "[EMAIL PROTECTED]". I use it for the OOo lists and only rarely read other mails sent to it. --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
