>From: Thorsten Ziehm <[EMAIL PROTECTED]> >Sent: Mar 16, 2007 6:51 AM >To: [email protected] >Subject: Re: [qa-dev] Nearly 1000 unconfirmed defects are open! > >Hi Mike, > >thank you for your feedback. > >I see the problems you have with the Quality Assurance of OOo. But I do >not see, that is a problem of the QA processes.
Actually, we have fallen way behind on QA. As soon as a milestone is released, we should be testing it for regressions and improper functioning of the program. We also should be testing any fixed problems to insure that they are indeed fixed. This is on top of the mounting pile of new reports. Some are indeed duplicates as the exact same processes are used. However, some are not. > >The job of the Quality Assurance is to assure the quality of the >software. We have to identify the defects before integration or the >regressions or new bugs, when the integration was done. This isn't a >productive part of a feature implementation. It is destructive work. >The QA has to show the development, what they have done wrong. So it >isn't easy for the QA team to find positive aspects in their work, >because we have to show, where the product works wrong. But without >it, the quality will not increase, so the work has to done. > Correct. We should find problems before any user has the chance to look at the program. That way there are NO surprises and work is already underway to fix any malfunctions. It is NOT our position to determine the merits of Enhancements (that is up to the development team). One of our key jobs is to look at any bug reports and determine if this will stop a user from incorporating OpenOffice.org into their daily lives. These are indeed showstopper problems. Nit-picky bugs are not what we need to get involved in. Sorry, but that is the way it is. >How could the QA get positive aspects from their work. You are right, >it is only positive, when your bug which you found or which you worked >on is fixed quickly and will be integrated soon. But there are thousands >of opened issues. OOo is so big, that it isn't possible to fix all these >bugs. The resources are limited - until now. Most of the defects are >fixed by Sun developers. The community development want often only fixed >their own problem or want to bring in some new features. Bug fixing >isn't very interesting, new code is better and everyone in the world can >see, what cool new feature is integrated. > I don't live to satisfy developers. I live to satisfy USERS. If I find a 'bug' then I know that no user will be disturbed if they find it. If a user finds a problem, then I have failed in the QA mission. That is my opinion of what I should be doing. >With the new structure of the QA sites on OOo, we want to collect the >experts in QA team. We want to bring them nearer to the Sun team and >perhaps we will found someone from the community who is willing to be a >Team Lead for an application or an testing area. We want also to bring >new members to a higher level of Quality Assurance. The Sun QA team will >bring test case specifications on the sites to show how they work on new >features. This can be taken to learn the way how to identify problems >and find the root causes quicker. That is very fine and is a good goal. QA folks can always learn from each other. However, I'm not here to satisfy Sun either. The bottom line is if we do not find bugs before the users do, we have failed. When I worked on my degree in Business Management, we were told the best sales method is word of mouth. If someone bad-mouths your product, you have lost not just that one customer, but hundreds. (The average disappointed person will advise 100 others). If you satisfy one user, you have gained a handful of new users. You have to WOW them with: 1. Performance. You can have the best program in the world, but if it is not responsive, no one will use it. 2. Perfection. Your program can have NO unknown problems. This is why it takes years to upgrade Operating Systems and Major software packages. OpenOffice.org should have maybe one to two major releases a year, no more. This allows QA to completely test the program and all of its functionality. 3. Price. You can have the best software and pricepoint it out of the intended audiences value. Of course, we are 'free' but does that mean that technical support will not be there or will my problem take forever to fix? If you cannot provide a solution that users can use, you will fail in your goal to provide a software product that others will use and respect. Microsoft did not get to be the #1 desktop software provider overnight and it will take a long time for OpenOffice.org to work on their dominance. It is up to QA to insure that the entire organization will meet/beat this goal. >Also with the direct contact it will be more possible to identify >critical issues easier and quicker to the development team. But this >does not mean, that each issue will be fixed. The Sun QA team has the >same problem with fixing their bugs as each other on OOo. > Identification of which fix needs to be done now is not easy. That is why we have a voting system. The more affected users, the more votes and thus this is a problem, no matter how small, that needs to be fixed quickly. >The new QA sites are not complete ready and some information will be >updated in the very next days. But take a look at it, and if you are >interested in, please contact the Team Leads. Teh biggest problem that I have encountered time and time again is the complexity of Issue Tracker. I went to submit an issue on the web site in regards to Vista support and could not find and entry for that category. We need to K.I.S.S. the web site. That means, keep it simple and stupid. We should not even imagine that our average user is a high school graduate. Sure we will upset some folks, but then we can explain that we have users that are not as 'smart' as they are. Take a look at the IssueZilla site for Firefox as a good example. This is where we should be. What is broken, what did you do to cause the breakage and some simple system questions. That is all. No search for an existing problem, no maze to go through. You would be amazed at what happens. Are we going to get repeats? Sure, but that also allows us to increase the vote count as QA. We could also build in functionality to allow quick and easy searches based upon the limits we placed so users could look up existing issues. Remember, K.I.S.S. Sure it is going to take effort, but it is very rewarding for both QA and user in the end. It also makes the developer's job easy. > >The Sun QA team is a part of the community. We try to do our best, but >it isn't possible to do all that work for OOo. Mathias Bauer explained >very often on the dev list on OOo, that we need the help to categorize >bugs and features. I beg to differ. Sun's QA folks have a goal, make money. That is done by insuring that no money loosing bugs make it to a production version. That is not true of OpenOffice.org. > >Perhaps I haven't answered all your questions. But as I wrote above, >I do not think it is a problem of the QA process. It is a problem of >the big project (OOo) and the limited resources. > All projects have limited resources. It is the application of those resources to get the 'biggest bang for the buck' is where it counts. If we have an issue that will take a long time to fix and have a minimum of impact, it should sit. If we have a very nagging issue that will add thousands of users and is quick to implement, 'Get-R-Done'. The problem is the issues in the middle. Been around for a while, been worked on and abandoned, or just plain difficult to figure if they have a return that are the one's we need to work on. Sun is also limited to supporting: 1. Solaris 2. Windows 3. Red Hat type Linuxes Nothing else matters, even if the 'smarts' are there and the fix could be done in minutes rather than years. This does not make good business sense (you would be amazed at the number of servers Dell sold when they figured out a fix for Compaq desktops) and should be looked at closely. It also does not make very good sense to fix one build platform's problems and break a non- supported platforms build process. However, the bottom line is: QA IS TO BE THE ULTIMATE USER. We are to test the program and use it in ways that the developers never imagined. BTW, I don't just do this for OpenOffice.org. I get paid to be a QA analyst in my real job. James McKenzie --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
