Hi all, During the last QA meeting, it was decided to check if test types are "correct" in Meego test suites (action: tahvo, yann_a, cris_c). Indeed, as initially stated by Jari (on 31st of March, subject:"Definitions of test types in wiki.meego.com/Quality - to be or not to be?" http://lists.meego.com/pipermail/meego-qa/2011-March/001295.html), test types defined in the wiki really differ from test types used in test xmls (http://testplans.meego.com/)
As a reminder, few weeks ago, the test type definitions have been reworked to be in accordance with the standards (ISO9126/ISTQB) and it was decided all together to keep only the main test types of functional and non-functional quality characteristics. As of now, the main test types defined in the wiki are (link<http://wiki.meego.com/Quality/Test_areas_and_types>): - Functional positive (previously functional) - Functional negative (new test type) - Security (new test type) - Performance (covers several sub test types: Framerate, Latency, Response, Benchmark, Memory, Throughput and Load) - Reliability (covers several sub test types: Low-resource, Recovery, Iterative and Long-lasting) - User Experience (already existing) - Maintainability (new test type) - Portability (new test type) Hereafter are listed the test type status for each Meego test suite (mcts, handset UX, mnts and SDK) and a proposal is added when a mismatch with the wiki is found. 1- MCTS The following test types are currently used in Core test plan (2344 test cases, see the feature coverage matrix here http://testplans.meego.com/mcts/): - Non Functional - only 1 test case - PROPOSAL: should be "moved" to the most appropriate one of the non-functional quality characteristics: Performance, Reliability, User experience, Maintainability or Portability. - Benchmark - 56 test cases - PROPOSAL: should be "replaced" with Performance. - Certification - 2 test cases - PROPOSAL: should be "replaced" with the most appropriate of the 8 main test types defined in the wiki. - FUTE - 17 test cases - PROPOSAL: should be "changed" to Functional positive or Functional negative. - Framerate - 10 test cases - PROPOSAL: should be "replaced" with Performance. - Functional - 1746 test cases - PROPOSAL: should be "changed" to Functional positive or Functional negative. - ITER - 2 test cases - PROPOSAL: should be "replaced" with Reliability. - Iterative - 136 test cases - PROPOSAL: should be "replaced" with Reliability. - LATE - 4 test cases - PROPOSAL: should be "replaced" with Performance. - Latency - 48 test cases - PROPOSAL: should be "replaced" with Performance. - Long-lasting - 20 test cases - PROPOSAL: should be "replaced" with Reliability. - Low-resource - only 1 test case - PROPOSAL: should "be replaced" with Reliability. - N/A - 251 test cases - PROPOSAL: should be "updated" with the most appropriate of the 8 main test types defined in the wiki. - Throughput - 50 test cases - PROPOSAL: should be "replaced" with Performance. 2- Handset UX The following test types are currently used in Handset UX test plan (718 test cases, see the feature coverage matrix here http://testplans.meego.com/handset-ux-tests/): - Functional - 493 test cases - PROPOSAL: should be "changed" to Functional positive or Functional negative. - I18N - 4 test cases - PROPOSAL: internationalization is not a test type, so one of the 8 main test types should "apply" to I18N testing. - L10N - 2 test cases - PROPOSAL: localization is not a test type, so one of the 8 main test types should "apply" to L10N testing. - N/A - 185 test cases - PROPOSAL: should be "updated" with the most appropriate of the 8 main test types defined in the wiki. - Non Functional - 30 test cases - PROPOSAL: should be "moved" to one of the non-functional quality characteristics: Performance, Reliability, User experience, Maintainability or Portability. - System integration - only 1 test case - PROPOSAL: according to the ISTQB definition "System integration" is not a test type but a test level, so should be "replaced" and "updated" with the most appropriate of the 8 main test types defined in the wiki - User experience - 3 test cases - PROPOSAL: test type already existing, so nothing to be changed. 3- MNTS In Netbook UX test suite, all the test types are set to N/A (1857 test cases, see the feature coverage matrix here http://testplans.meego.com/mnts/): PROPOSAL: All the test cases should be "updated" the most appropriate of the 8 main test types defined in the wiki. 4- SDK The following test types are currently used in Handset UX test plan (72 test cases, see the feature coverage matrix here http://testplans.meego.com/meego-sdk-tests/ ): - Functional - 71 test cases - PROPOSAL: should be "changed" to Functional positive or Functional negative. - N/A - only 1 test case - PROPOSAL: should be "updated" with the most appropriate of the 8 main test types defined in the wiki. Your feedback and review of proposal are very welcome, Christophe & Yann --------------------------------------------------------------------- Intel Corporation SAS (French simplified joint stock company) Registered headquarters: "Les Montalets"- 2, rue de Paris, 92196 Meudon Cedex, France Registration Number: 302 456 199 R.C.S. NANTERRE Capital: 4,572,000 Euros This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies.
_______________________________________________ MeeGo-qa mailing list MeeGo-qa@lists.meego.com http://lists.meego.com/listinfo/meego-qa
