As I was typing up an email about writing more automated tests, I was
getting tripped up by terminology issues. I thought it might be time to
air those out. ;)
In particular, we use the term "unit test" to mean both "tests owned by
developers that run as part of the build process" as well as "tests that
cover an isolated module". Right now, those two are *not* equivalent --
most of the tests run as part of the build process are not strictly unit
tests.
Would it be useful to have a separate term for "automated tests owned by
developers used to determine whether or not the build is acceptable and
not broken"? We've already taken "acceptance tests" to mean "manual
tests that supplement all automated tests used to bless a milestone or
release".
Aparna has a list of test types defined here (and repeated below in case
you don't want to follow the link):
http://wiki.osafoundation.org/bin/view/Projects/ChandlerQAProcess#Types_of_Test_Definitions
* PreCheckInTests -- tests that developers should run before checking in
changes
* UnitTests -- automated tests that are run by the build system.
Developers are responsible for adding unit tests for the components they
develop. For every component in the system there should be corresponding
unit tests that can be run just to validate the functionality of that
component.
* FunctionalTests -- complete set of manual and automated tests to test
the functionality and performance of each feature in the release. These
testcases will match one on one with the testcases in the test
specification. Currently we have very limited number of automated tests
for Chandler and any hellp from the user community will be greatly
appreciated. If you have any expertise in writing automated tests for
desktop UI applications and would like to contribute to Chandler test
development, please contact us.
* Performance tests -- performance testing may be conducted as part of
functional tests to test the application startup time, response times,
memory leak, CPU utilization, etc. The performance criteria will be
developed by the Product/Design team for each release.
* Integration tests -- test cases cases that test the application
completely from end to end, after all functional components are code
complete. This includes test cases with more complex scenarios than
functional tests.
* Regression tests -- subset of automated functional tests that will be
run nightly during the development cycle to ensure no existing
functionality was broken because of new feature development.
* AcceptanceTests -- tests that anyone can run in order to "bless" a
milestone/release. This is a more extensive list of manual tests that
are conducted at the end of each milestone/release.
Can we agree to use this terminology when talking about tests? What
about "PreCheckInTests"? Thoughts?
Cheers,
Katie
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
Open Source Applications Foundation "Dev" mailing list
http://lists.osafoundation.org/mailman/listinfo/dev
