Hi,
James Mckenzie wrote:
> Given that I have a long time and experience testing software, these are my
thoughts on the QA process:
>
> 1. All milestone builds should be run through the complete QA process.
Smoketests are not enough to stop the regression process. Automated tests are not
enough to find functionality and localization issues.
> 2. Code freeze means that. No new functionalities will be incorporated
unless:
> a. An issue exists for the code to be added that addresses a functional
flaw in the product.
> b. The inclusion of the new code will not cause regression issues.
> 3. L10N releases are to address language issues. This is not true. L10N
releases should be reasonably free from defects and represent the final release
without full localization. Localization should be the PRIMARY purpose of these
releases, but not the only one.
> 4. Release candidates MUST be put through FULL functional testing. This
means using the product as it would be in a full production environment. If a new
use for the product is found or a possible misuse case is found, it must be fully
documented and added to both automated testing (if applicable) and to manual TCM
testing. An issue in the Issue Tracker WILL BE created to track the problem and
to document its existence. This issue can be immediately closed if it is a case
of possible misuse if the product successfully rejects improper input, but if it
does not, the issue must be worked within the existing issue system.
> 5. NO FINAL RELEASE should have unaddressed issues! If this means delaying
the release by one or two weeks, that should be how it is. Remember, we are a
VOLUNTEER effort with limited support from Sun Microsystems. There are many
releases that are produced outside of Sun's control. These releases need to be
run through the QA process as well as Sun's products as they may still have
unresolved problems or problems that are/were not detected in Sun's limited test
environment.
>
> All of this is in the interest of giving our users, regardless of their
native language, the best product we can give them.
I agree with your thoughts on what we should do for releasing local builds.
For when we should do what, there might be some ideas:
(Type A) Comprehensive test at release test;
Feedback from the release test could address to the next release.
=========== development, translation, integration, ...
========= TCM testing
========= release test on en_US RCs
==================== release test on local RCs
If we find some problems on our local RCs at the time the en_US build
has been already released to the market, what can we do for this release?
It would be to write down information on the problems as additional release
notes in our local web page beside links for download. The valuable
information could be addressed for the next release. We can not rewind
the time to prior to en_US public release without a time machine. We have
no chance to modify source code for this release.
Delay for fixing problems would be three months. Many local users have
to suffer from the problems until the next release.
(Type B) Quick test at release test;
Comprehensive test should be done before the release test.
=========== development, translation, integration, ...
========= TCM testing
========= release test on en_US RCs
========= release test on local RCs
Do the language dependent tests on the earlier builds than release
candidate builds. At the release test, simply conduct a test on basic
functionalities such as installation, typing texts, font rendering,
saving, loading, exporting, printing, copying-and-pasting, and
uninstallation.
If we find some language dependent problems during earlier tests,
we can report them to the developers who can fix them earlier time
of development process.
Delay for fixing problems would be several weeks. And, fortunately,
no casual users would face the problems.
(Type C) More ideas will be welcome.
Further discussion might be about availability of local builds during
development if we would like to conduct the language dependent,
comprehensive test in the earlier time.
Localization for additional texts would have not be done at the time
of development, but existing texts, 99.9% of the texts, would have been
already translated. Therefore, we could have localized builds during a
period of development.
It that true? If so, it might help local communities test new features
at the same time of development even though their additional texts might
not be translated yet. So what? There might be tens order of
not-yet-translated texts.
Think about timing. At better timing we do what we can do under limited
resource, we can have better results at less resource.
Best regards,
Tora
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
