Ville, looks testplans.meego.com is not in QA tool list currently, what's its development plan? Currently, for single tester, he does can use testplanner and testrun_ui to complete his job. But for a test group, we are expecting a central test execution plan tool which can help test leader to create overall test plan, assign tester accordingly. I will submit a FEA request to elaborate it.
WangJing -----Original Message----- From: [email protected] [mailto:[email protected]] On Behalf Of Ville Ilvonen Sent: Tuesday, December 14, 2010 3:20 AM To: [email protected] Subject: Re: [Meego-qa] QA tool proposal Sorry for delay but here's comments to this proposal from tools development perspective. Let's discuss this tomorrow in MeeGo Quality weekly meeting. On 11/30/2010 10:52 AM, ext Yu, Ling L wrote: > > Attachment is the proposal about new QA tool discussed on Dublin QA > Workshop (un-conference day). > Thanks. I checked it through - here's some comments. Good news is that many parts are being implemented during MeeGo 1.1 and 1.2 cadence. Page 4: Feature Test Traceability (Mapping between Tests and Features) - http://bugs.meego.com/show_bug.cgi?id=9767 Feature Verification View - there's not feature request for qa-reports.meego.com yet - #9767 enables this ^ Accumulative test result view - ? Feature test status (link to latest test results) - see comments below, needs to be planned together with MeeGo bugzilla masters. Page 5: "Inconsistent of test case template and tests.xml" - http://bugs.meego.com/show_bug.cgi?id=11253 "Testplanner is a single function tool to edit tests.xml for single user, not convenient for query and statistics" - Every testplanner can use their own copy of testplanner on their own computer. You can use http://178.79.147.53/ (to be testplans.meego.com) and gitorious.org repositories for queries and statistics "Create test execution plan for UX and Core with flexible filters for test cases" - this has been implemented in testrunner UI - see http://www.youtube.com/watch?annotation_id=annotation_64395&v=sfQcCHTmvyo&feature=iv for demo of it. (Thanks Kyösti for great video) Publish test execution plans to community to engage contributors for test execution - Testplans have been made visibile since Dublin - http://178.79.147.53/ - I believe key part of publishing test execution plans is to make test management meetings in open (e.g. IRC) so that interested parties can join and agreements on who executes what, who decides what is being executed and so on can be done together the MeeGo way. For my knowledge this part of test management is not open currently? "Lack of a tool to support inputting test result by multiple testers involved in a same test session and merge the results into one test report" - This has been supported since the beginning of qa-reports.meego.com Test publisher who uses qa-reports.meego.com can add several results.xmls which are then merged into one report. Jarno promised to help with this by improving user documentation of qa-reports.meego.com. "Need a central place to deliver quality metrics from two views: the accumulate test result view, feature verification view" - quick solution to this was possibility to add attachments to test reports at qa-reports.meego.com. Jarno reported this was implemented and test publishers can see this at for example http://qa-reports.meego.com/1.2/Handset/Acceptance/N900/249/edit (Add attachment) Page 6-7 - proposal: We will implement mappings (see above for details) and enhance QA report tool with other tools. In practice - gitorious.org plays database for our tests. Datafields of test plan XMLs will be enhanced over time. Pages 8-11: These are unfortunately not in priority order. If everything is priority one (or two) - everything is equally important. In practice we have priority order in bugs.meego.com > Development Tools - when you search and select all tests tools and finally select results in priority order you will see current priority order of QA tools development. > Key Messages: > > Expect a unified, systematic and user-friendly MeeGo test management > tool to support test design, execution plan, test report, and > traceability of tests and features. > That's good expectation. We are moving towards that with good speed but there's still some development to do. > > Current tools only cover part of segments of whole QA process. > Yes, but good news is that we are getting testing done. Results are published every day so people make this process work. > > Expect a QA management tool for the mapping between tests and > features: 1. tests -> feature; 2: feature -> tests, at the same time, > provide query and statistics > This is being implemented - [FEA] Feature-mapping between test plan and tests visible in reports http://bugs.meego.com/show_bug.cgi?id=9767 > > Expect a QA tool to support test execution planning; it should be a > user friendly online tool with following functions: > Testplanner and testrunner support test planning and execution. We aim to make them user-friendly and improve them. Online support is being provided with access to check out test plans from git repositories and push changes back there. meego.gitorious.org we use has been online very well. > > - Create test execution plan for UX and Core with flexible > > - Create test execution plan according to features to be tested. > > - Publish test execution plans online, and any contributors can > be easy to pick test execution plan for test execution > This is being provided with gitorious access. Anyone can pick them for execution with testrunner. Test publisher needs to collect all results.xmls from different people and publish them to qa-reports.meego.com to create compiled test report. Publishing test plans is currently in http://gitorious.org/qa-tools/mcts and http://gitorious.org/qa-tools/mnts We are also making those human-friendly views from those plans under http://178.79.147.53/ (DNS via testplans.meego.com to be requested) > > Link the test reports on QA report tool to test execution plan > We need to discuss this with error manager(s). I'll contact Eric Le-Roux. We discussed this also with Jarno - bugzilla should support possibility to update link to latest results pretty easily. This will be enabled when #9767 is implemented. > > In featurezilla or somewhere, provide the feature verification status > and link feature to latest test result. > > Our proposal is to enhance the current QA report tool to support above > functionality considering QA report tool is web online tool using > database. > MeeGo works with contributions. Gitorious.org repositories are kind of databases but with version control. > > We are gradually submitting our proposals to bugzilla as feature > requests for QA tools. > Please do. Please submit also your contributions as patches and merge requests - tools maintainers will review those for sure. Regards, -Ville > Thanks > > -Ling > > > _______________________________________________ > MeeGo-qa mailing list > [email protected] > http://lists.meego.com/listinfo/meego-qa > _______________________________________________ MeeGo-qa mailing list [email protected] http://lists.meego.com/listinfo/meego-qa _______________________________________________ MeeGo-qa mailing list [email protected] http://lists.meego.com/listinfo/meego-qa
