Let's hope that we will get here also opinions widely from community members and different QA functions. I anyway assume that this grading will be used harmonized way in every verticals (e.g. Core OS, IVI UX, Netbook UX, Handset UX...)
Br, Petri From: ext Tang, Shaofeng [mailto:[email protected]] Sent: 28. joulukuuta 2010 10:22 To: Goswami Mukesh.Ch (Nokia-MS/Bangalore); Mu, Qin; Koski Petri (Nokia-MS/Tampere); [email protected] Subject: RE: [Meego-qa] The grading and comments are available for each feature on qa-reports now Indeed, it will be better if the weight of test cases could be taken into account when auto-calculating the grading. For time being, the issue is the weight of test cases isn't available from the result file yet. Maybe we should consider to add a new column in the CSV format result file for the weight of test cases. Best Regards Shao-Feng From: [email protected] [mailto:[email protected]] Sent: Tuesday, December 28, 2010 3:10 PM To: Mu, Qin; Tang, Shaofeng; [email protected]; [email protected] Subject: RE: [Meego-qa] The grading and comments are available for each feature on qa-reports now Also in my opinion we have to take into account the weight of the bugs(BCM) that fails those test cases as well while grading.. Regards Mukesh Goswami From: [email protected] [mailto:[email protected]] On Behalf Of ext Mu, Qin Sent: Tuesday, December 28, 2010 12:37 PM To: Tang, Shaofeng; Koski Petri (Nokia-MS/Tampere); [email protected] Subject: Re: [Meego-qa] The grading and comments are available for each feature on qa-reports now It might better to have a defined calculation formula for grading score, and take into account in formula the weight of pass-rate in terms of priority of TC. Once the formula is consented, the quality grading can be automatically generated, instead of manual editing. The grading judgement could otherwise be inconsistent between different features/components. Qin, Mu From: [email protected] [mailto:[email protected]] On Behalf Of Tang, Shaofeng Sent: Tuesday, December 28, 2010 2:33 PM To: [email protected]; [email protected] Subject: Re: [Meego-qa] The grading and comments are available for each feature on qa-reports now Hello Petri, Thanks to your feedback. This is only an example to show the 2 new features. It isn't a formal report. In the example, the gradings have been edited for testing the feature itself, that is, the auto-calculated result has been changed by me. And yes, you are right the grading for 12/17 is yellow, when the test-result is submitted. But we change it to test if the grading is editable. In some cases, the pass rate is not the only factor of the grading. Sometimes, we have to take the weight of TC into account. For instance, if we test a component with 5 basic feature test-cases, 3 functional test cases, 1 negative test cases, and 1 i18n test cases. Only the negative test cases and the i18n one are failed. The auto-calculated grading will be yellow. In this situation, we will need to edit the grading manually to green, since the negative test cases and the i18n one is light-weight Regarding the boundary values, for time being, they are only explained in my mail. What is your suggestion for it? Explain them in the reported? Or maybe included in the on-line help of qa-reports? Thanks & Best Regards Shao-Feng From: [email protected] [mailto:[email protected]] Sent: Tuesday, December 28, 2010 1:57 PM To: Tang, Shaofeng; [email protected] Subject: RE: [Meego-qa] The grading and comments are available for each feature on qa-reports now Hello, I also took a deeper look of test report having boundary values as example. I noticed that after opening link http://qa-reports.meego.com/1.2/Handset/Key%20basic%20feature/Ncdk/435 and checking Test Results by Feature from CoreUX side that it has 12 TC's pass out from 17 TC's. In practice TC Pass Rate seems to be 12/17 = 70.6%, grading for this is green. If I understood correctly grading should be green after Pass Rate is over 90%, right? Br, Petri From: [email protected] [mailto:[email protected]] On Behalf Of Koski Petri (Nokia-MS/Tampere) Sent: 28. joulukuuta 2010 07:48 To: [email protected]; [email protected] Subject: Re: [Meego-qa] The grading and comments are available for each feature on qa-reports now Hello, Is this grading itself and its boundary values explained somewhere in the test report itself? "Here is a example of the 2 features. http://qa-reports.meego.com/1.2/Handset/Key%20basic%20feature/Ncdk/435 " Br, Petri From: [email protected] [mailto:[email protected]] On Behalf Of ext Tang, Shaofeng Sent: 28. joulukuuta 2010 03:32 To: [email protected] Subject: [Meego-qa] The grading and comments are available for each feature on qa-reports now Hi, On http://qa-reports.meego.com, 2 new features "Feature Grading" and "Feature comments" are available now. For helping readers get a clear status from feature summary, a brief comment could be added on each feature/test-set to briefly describe its' status. And also a grading will express the status in color. If you commit a new report, the grading will be calculated automatically according to below rules Grading Pass Rate of Total < 40% > 40%, < 90% > 90% And also the grading is editable for some high-weight fail/pass after the report is committed. Here is a example of the 2 features. http://qa-reports.meego.com/1.2/Handset/Key%20basic%20feature/Ncdk/435 Hope the two features are useful to you. Best Regards Shao-Feng
_______________________________________________ MeeGo-qa mailing list [email protected] http://lists.meego.com/listinfo/meego-qa
