On 14 August 2013 19:36, Edwin Sharp <[email protected]> wrote: > Dear Rob > The 4.0 release was too ambitious - we should advance in smaller steps. > Nothing compares to general public testing - betas and release candidates > should not be avoided. > TestLink cases should be less comprehesive (in terms of feature coverage) > and more stress testing oriented. > Regards, > Edwin > > On Wed, Aug 14, 2013, at 19:59, Rob Weir wrote: > > We're working now on AOO 4.0.1, to fix defects in AOO 4.0.0. The fact > > that we're doing this, and their are no arguments against it, shows > > that we value quality. I'd like to take this a step further, and see > > what we can learn from the defects in AOO 4.0.0 and what we can do > > going forward to improve. > > > > Quality, in the end, is a process, not a state of grace. We improve > > by working smarter, not working harder. The goal should be to learn > > and improve, as individuals and as a community. > > > > Every regression that made it into 4.0.0 was added there by a > > programmer. And the defect went undetected by testers. This is not > > to blame. It just means that we're all human. We know that. We all > > make mistakes. I make mistakes. A quality process is not about > > becoming perfect, but about acknowledging that we make mistakes and > > that certain formal and informal practices are needed to prevent and > > detect these mistakes. > > > > But enough about generalities. I'm hoping you'll join with me in > > examining the 32 confirmed 4.0.0 regression defects and answering a > > few questions: > > > > 1) What caused the bug? What was the "root cause"? Note: > > "programmer error" is not really a cause. We should ask what caused > > the error. > > > > 2) What can we do to prevent bugs like this from being checked in? > > > > 3) Why wasn't the bug found during testing? Was it not covered by any > > existing test case? Was a test case run but the defect was not > > recognized? Was the defect introduced into the software after the > > tests had already been executed? > > > > 4) What can we do to ensure that bugs like this are caught during > testing? > > > > So 2 basic questions -- what went wrong and how can we prevent it in > > the future, looked at from perspective of programmers and testers. If > > we can keep these questions in mind, and try to answer them, we may be > > able to find some patterns that can lead to some process changes for > > AOO 4.1. > > > > You can find the 4.0.0 regressions in Bugzilla here: > > > > > https://issues.apache.org/ooo/buglist.cgi?cmdtype=dorem&remaction=run&namedcmd=400_regressions&sharer_id=248521&list_id=80834 > > > > > > Regards, > > > > -Rob >
I strongly believe that one of the things that went wrong is our limited possibility to retest (due to resources), when I look at our current manual testcases, a lot of those could be automated, e.g. with a simple UI macro, that would enable us to run these test cases with every build. It may sound like a dream but where I come from, we did that every night, and it caught a lot of regression bugs and sideeffects. A simple start, if to request that every bug fix, is issued with at least one test case (automated or manual). rgds jan I. > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: [email protected] > > For additional commands, e-mail: [email protected] > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > >
