Hi Christian, don't have something (in this very mail) about the "what to do with the bug pile" topic, but a few other items cried for my response ...
>> To improve this we need people who write Testtool testscripts. > > I disagree. The problem is not finding the bugs. Not sure. Looking at the amount of stoppers which came in in 3.1's release phase, and the amount of stoppers already raised for 3.1.1 (and most often for good reasons), I think that *finding* bugs *is* a problem. Of course, or at least in my opinion: Automatically finding bugs is just *one* line of defense. There are others, and personally, I think another line of defense must be with the Devs, by not *introducing* the bugs. IMO, we take way too little measures to prevent bugs, we always concentrate on finding and fixing them. > And honestly, I'm /very/ disappointed of the automatic testing, > admittedly not the testtool, but smoketestoo_native. > Smoketestoo_native failed to > * Detect a regression where the installation would not run /at all/ > because of wrong permissions > * Detect the PDF-export breaker The first of those should of course have been detected. If it hasn't, this means somebody didn't follow the process - and this is a problem you cannot solve at all, except by education. You can have the perfect plans to perfectly manage OOo's quality - if people don't follow those plans, they're pointless. (And yes, the current process requires to run the smoketest before passing a CWS to QA.) The second just means: The smoketest cannot test everything. In fact, I think the smoketest is just a very minimal set, there should be other (automatic) tests for that kind of problem. > Those two simple cases alone describe that testscripts by itself won't > really help in all cases. Agreed. But they *can* help tremendously, if used properly. Admittedly, in my very personal opinion again, the usability of the test tool is so bad, that it completely fails to attract people to write new (or even maintain existing) test tool scripts. In short, if we do not have tools for effectively creating automated test, we won't get those tests (in sufficient number). If we won't get them them, they can't help assuring quality. >> If a developer get's some sort of cook book >> to reproduce an issue the better he/she will be able to find a solution to >> fix this issue. > > This is true, and without that step you cannot complain about an issue > not being fixed. I completely agree. +1 That's something I'd wish to see more often: Bring the issues into a good shape. Add a step-by-step description. Add a sample document. Change the summary to something meaningful, so it can easier be digested. With this, all further work with the issue is much easier, and less time-consuming. >> - verifying issues of just integrated child workspaces within the current >> master build (to verify there are no integration related issues regarding >> this CWS) > > Jup, nice janitorial task that takes off some workload of the other > QA/Devs that can focus on other tasks > +1 >From your experience: Is this worth it? I mean, we probably all agree that in the current modus operandi, there's more work for both QA and Dev than they can accomplish (that's why changing this modus is one of the topics of this thread, as I understand it.). An no, that's not an excuse, that's a matter of fact which we need to deal with. So, in how many cases did checking a VERIFIED FIXED issue in the MWS really detect a problem? And in how many cases was it just moving the issue to CLOSED? I'm asking because my gut feeling is that the latter takes too much time. And seeing that all issues have already been VERIFIED in the CWS, and assuming that *breaking* a fix by merely integrating the CWS is unlikely (though surely possible), I wonder whether auto-CLOSING issues would free previous QA resources. Ciao Frank -- - Frank Schönheit, Software Engineer [email protected] - - Sun Microsystems http://www.sun.com/staroffice - - OpenOffice.org Base http://dba.openoffice.org - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
