Hi James,
James Mckenzie schrieb:
...
Given that I have a long time and experience testing software, these are my
thoughts on the QA process:
1. All milestone builds should be run through the complete QA process.
Smoketests are not enough to stop the regression process. Automated tests are
not enough to find functionality and localization issues.
2. Code freeze means that. No new functionalities will be incorporated unless:
a. An issue exists for the code to be added that addresses a functional
flaw in the product.
b. The inclusion of the new code will not cause regression issues.
3. L10N releases are to address language issues. This is not true. L10N
releases should be reasonably free from defects and represent the final release
without full localization. Localization should be the PRIMARY purpose of these
releases, but not the only one.
4. Release candidates MUST be put through FULL functional testing. This means
using the product as it would be in a full production environment. If a new
use for the product is found or a possible misuse case is found, it must be
fully documented and added to both automated testing (if applicable) and to
manual TCM testing. An issue in the Issue Tracker WILL BE created to track the
problem and to document its existence. This issue can be immediately closed if
it is a case of possible misuse if the product successfully rejects improper
input, but if it does not, the issue must be worked within the existing issue
system.
5. NO FINAL RELEASE should have unaddressed issues! If this means delaying
the release by one or two weeks, that should be how it is. Remember, we are a
VOLUNTEER effort with limited support from Sun Microsystems. There are many
releases that are produced outside of Sun's control. These releases need to be
run through the QA process as well as Sun's products as they may still have
unresolved problems or problems that are/were not detected in Sun's limited
test environment.
All of this is in the interest of giving our users, regardless of their native language, the best product we can give them.
Good and valid points. But:
- who should do this job?
- what is FULL functional testing
- what means "no unadressed issues"? (we actually have > 8500 bugs open)
I hope we will get more time for QA when we switch to a 6 months feature
release cycle .. but this would by far not be enough to get all the
thinks done you (and me as well) would like to see. So we need to find a
compromise.
André
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
