Hi, +1 to this "volunteers to contribute" model.
Do we already have "result combiner" kind of tool for the test lead to use prior submitting to qa-reports? And do we have "plan splitter" for splitting tasks (xml-files) to smaller ones? Br, Jari -----Original Message----- From: [email protected] [mailto:[email protected]] On Behalf Of Ilvonen Ville.P (Nokia-MS/Tampere) Sent: 14 December 2010 15:52 To: ext Wang, Jing J Cc: [email protected] Subject: Re: [Meego-qa] QA tool proposal On 12/14/10 5:48 AM, ext Wang, Jing J wrote: > Ville, looks testplans.meego.com is not in QA tool list currently, > what's its development plan? It's development plan was to make human-readable versions of XMLs. Vesa Poikajärvi did great job with it. Simple validation and style transformation. And he was very fast to provide it. Cheers for him. > Currently, for single tester, he does > can use testplanner and testrun_ui to complete his job. The idea is that every single tester in a test group can use their own testplanner and testrunner and then send results to test lead who can combine all results files of test group and publish them at once. In practice test lead is expected to work for example as follows: "We are going to execute the following set of test cases as follows: John Doe will execute this set of tests. Jane Doe will execute that set of tests, Matti Meikäläinen those tests and Mou Mou will execute those tests. Target for publishing results is today by nn:nn hours so please deliver me the results by that so I can publish them. In addition we have these optional tests which could be tested by anyone. Any volunteers to contribute?" That could be done in public (e.g. IRC or email to this list) and that way people in community would be able to participate so they choose to. This is as of today - let's see what we come up with in future. > But for a > test group, we are expecting a central test execution plan tool which > can help test leader to create overall test plan, assign tester > accordingly. I will submit a FEA request to elaborate it. That sounds good expectation but I haven't got any requests from other test leads to develop such work management system. It would be quite some effort from development perspective and we're getting testing done without it which is the key focus while progressing towards 1.2 Anyone for this idea? -Ville > WangJing -----Original Message----- From: > [email protected] > [mailto:[email protected]] On Behalf Of Ville Ilvonen > Sent: Tuesday, December 14, 2010 3:20 AM To: > [email protected] Subject: Re: [Meego-qa] QA tool proposal > > Sorry for delay but here's comments to this proposal from tools > development perspective. Let's discuss this tomorrow in MeeGo Quality > weekly meeting. > > On 11/30/2010 10:52 AM, ext Yu, Ling L wrote: >> >> Attachment is the proposal about new QA tool discussed on Dublin >> QA Workshop (un-conference day). >> > Thanks. I checked it through - here's some comments. Good news is > that many parts are being implemented during MeeGo 1.1 and 1.2 > cadence. > > Page 4: Feature Test Traceability (Mapping between Tests and > Features) - http://bugs.meego.com/show_bug.cgi?id=9767 Feature > Verification View - there's not feature request for > qa-reports.meego.com yet - #9767 enables this ^ Accumulative test > result view - ? Feature test status (link to latest test results) - > see comments below, needs to be planned together with MeeGo bugzilla > masters. > > Page 5: > > "Inconsistent of test case template and tests.xml" - > http://bugs.meego.com/show_bug.cgi?id=11253 > > "Testplanner is a single function tool to edit tests.xml for single > user, not convenient for query and statistics" - Every testplanner > can use their own copy of testplanner on their own computer. You can > use http://178.79.147.53/ (to be testplans.meego.com) and > gitorious.org repositories for queries and statistics > > "Create test execution plan for UX and Core with flexible filters > for test cases" - this has been implemented in testrunner UI - see > http://www.youtube.com/watch?annotation_id=annotation_64395&v=sfQcCHTmvyo&feature=iv > > for demo of it. (Thanks Kyösti for great video) > > Publish test execution plans to community to engage contributors for > test execution - Testplans have been made visibile since Dublin - > http://178.79.147.53/ - I believe key part of publishing test > execution plans is to make test management meetings in open (e.g. > IRC) so that interested parties can join and agreements on who > executes what, who decides what is being executed and so on can be > done together the MeeGo way. For my knowledge this part of test > management is not open currently? > > "Lack of a tool to support inputting test result by multiple testers > involved in a same test session and merge the results into one test > report" - This has been supported since the beginning of > qa-reports.meego.com Test publisher who uses qa-reports.meego.com > can add several results.xmls which are then merged into one report. > Jarno promised to help with this by improving user documentation of > qa-reports.meego.com. > > "Need a central place to deliver quality metrics from two views: the > accumulate test result view, feature verification view" - quick > solution to this was possibility to add attachments to test reports > at qa-reports.meego.com. Jarno reported this was implemented and > test publishers can see this at for example > http://qa-reports.meego.com/1.2/Handset/Acceptance/N900/249/edit > (Add attachment) > > Page 6-7 - proposal: > > We will implement mappings (see above for details) and enhance QA > report tool with other tools. In practice - gitorious.org plays > database for our tests. Datafields of test plan XMLs will be enhanced > over time. > > Pages 8-11: > > These are unfortunately not in priority order. If everything is > priority one (or two) - everything is equally important. In practice > we have priority order in bugs.meego.com> Development Tools - when > you search and select all tests tools and finally select results in > priority order you will see current priority order of QA tools > development. > >> Key Messages: >> >> Expect a unified, systematic and user-friendly MeeGo test >> management tool to support test design, execution plan, test >> report, and traceability of tests and features. >> > That's good expectation. We are moving towards that with good speed > but there's still some development to do. >> >> Current tools only cover part of segments of whole QA process. >> > Yes, but good news is that we are getting testing done. Results are > published every day so people make this process work. >> >> Expect a QA management tool for the mapping between tests and >> features: 1. tests -> feature; 2: feature -> tests, at the same >> time, provide query and statistics >> > This is being implemented - [FEA] Feature-mapping between test plan > and tests visible in reports > http://bugs.meego.com/show_bug.cgi?id=9767 >> >> Expect a QA tool to support test execution planning; it should be >> a user friendly online tool with following functions: >> > Testplanner and testrunner support test planning and execution. We > aim to make them user-friendly and improve them. Online support is > being provided with access to check out test plans from git > repositories and push changes back there. meego.gitorious.org we use > has been online very well. >> >> - Create test execution plan for UX and Core with flexible >> >> - Create test execution plan according to features to be tested. >> >> - Publish test execution plans online, and any contributors >> can be easy to pick test execution plan for test execution >> > This is being provided with gitorious access. Anyone can pick them > for execution with testrunner. Test publisher needs to collect all > results.xmls from different people and publish them to > qa-reports.meego.com to create compiled test report. > > Publishing test plans is currently in > http://gitorious.org/qa-tools/mcts and > http://gitorious.org/qa-tools/mnts We are also making those > human-friendly views from those plans under http://178.79.147.53/ > (DNS via testplans.meego.com to be requested) >> >> Link the test reports on QA report tool to test execution plan >> > We need to discuss this with error manager(s). I'll contact Eric > Le-Roux. We discussed this also with Jarno - bugzilla should support > possibility to update link to latest results pretty easily. This will > be enabled when #9767 is implemented. >> >> In featurezilla or somewhere, provide the feature verification >> status and link feature to latest test result. >> >> Our proposal is to enhance the current QA report tool to support >> above functionality considering QA report tool is web online tool >> using database. >> > MeeGo works with contributions. Gitorious.org repositories are kind > of databases but with version control. >> >> We are gradually submitting our proposals to bugzilla as feature >> requests for QA tools. >> > Please do. Please submit also your contributions as patches and > merge requests - tools maintainers will review those for sure. > > Regards, > > -Ville > >> Thanks >> >> -Ling >> >> >> _______________________________________________ MeeGo-qa mailing >> list [email protected] >> http://lists.meego.com/listinfo/meego-qa >> > _______________________________________________ MeeGo-qa mailing > list [email protected] > http://lists.meego.com/listinfo/meego-qa _______________________________________________ MeeGo-qa mailing list [email protected] http://lists.meego.com/listinfo/meego-qa _______________________________________________ MeeGo-qa mailing list [email protected] http://lists.meego.com/listinfo/meego-qa
