Hi Petri, Thanks to your feedback. Again, that is only an instance to explain how the weight of test cases impacts the grading. And no, it isn't in practice. >From now on, our QA reporter (Daniel, and Cathy) will fill both comments and >grading when committing the reports. For the real report with grading, I suppose you just need to wait for a while. Maybe tomorrow.
Regarding the auto-calculated grading, actually we just hope to provide it as a default value to the owner of the reports when he committed the result. And let the owner of the reports to decide what the grading of each feature is via providing the feature "editing grading". Since both the grading of feature and the weight of TC are decided by the tester, I suppose it make sense letting the owner of the test reports decide the grading if the weight of test cases are not available for time being. Regarding the description of grading, if the grading has been updated by the owner of test-reporter, I am not sure the rule will still work correctly. Maybe the comments of each feature could be used to describe the grading as well. Best Regards Shao-Feng From: [email protected] [mailto:[email protected]] Sent: Tuesday, December 28, 2010 2:54 PM To: Tang, Shaofeng; [email protected] Subject: RE: [Meego-qa] The grading and comments are available for each feature on qa-reports now Hello, Ok, thanks for helping me understand here this more better. "For instance, if we test a component with 5 basic feature test-cases, 3 functional test cases, 1 negative test cases, and 1 i18n test cases. Only the negative test cases and the i18n one are failed. The auto-calculated grading will be yellow. In this situation, we will need to edit the grading manually to green, since the negative test cases and the i18n one is light-weight" Situation mentioned above is really interesting. Is it in practice that that we would have application and system test cases mixed in same report? Also it would be really interesting to see how weighting is explained clearly in QA reports grading part, if different individual TC's are having different impact (weight) to grading itself. At least to my ears this sounds little bit complex and there might be clear possibility that people reading QA reports would somehow misunderstood this grading part since it is not self evident or is it? "Regarding the boundary values, for time being, they are only explained in my mail. What is your suggestion for it? Explain them in the reported? Or maybe included in the on-line help of qa-reports?" Well, I would explain these in report somewhere just to make it sure that people using these reports would understand this grading and where it is coming from. Br, Petri From: ext Tang, Shaofeng [mailto:[email protected]] Sent: 28. joulukuuta 2010 08:33 To: Koski Petri (Nokia-MS/Tampere); [email protected] Subject: RE: [Meego-qa] The grading and comments are available for each feature on qa-reports now Hello Petri, Thanks to your feedback. This is only an example to show the 2 new features. It isn't a formal report. In the example, the gradings have been edited for testing the feature itself, that is, the auto-calculated result has been changed by me. And yes, you are right the grading for 12/17 is yellow, when the test-result is submitted. But we change it to test if the grading is editable. In some cases, the pass rate is not the only factor of the grading. Sometimes, we have to take the weight of TC into account. For instance, if we test a component with 5 basic feature test-cases, 3 functional test cases, 1 negative test cases, and 1 i18n test cases. Only the negative test cases and the i18n one are failed. The auto-calculated grading will be yellow. In this situation, we will need to edit the grading manually to green, since the negative test cases and the i18n one is light-weight Regarding the boundary values, for time being, they are only explained in my mail. What is your suggestion for it? Explain them in the reported? Or maybe included in the on-line help of qa-reports? Thanks & Best Regards Shao-Feng From: [email protected] [mailto:[email protected]] Sent: Tuesday, December 28, 2010 1:57 PM To: Tang, Shaofeng; [email protected] Subject: RE: [Meego-qa] The grading and comments are available for each feature on qa-reports now Hello, I also took a deeper look of test report having boundary values as example. I noticed that after opening link http://qa-reports.meego.com/1.2/Handset/Key%20basic%20feature/Ncdk/435 and checking Test Results by Feature from CoreUX side that it has 12 TC's pass out from 17 TC's. In practice TC Pass Rate seems to be 12/17 = 70.6%, grading for this is green. If I understood correctly grading should be green after Pass Rate is over 90%, right? Br, Petri From: [email protected] [mailto:[email protected]] On Behalf Of Koski Petri (Nokia-MS/Tampere) Sent: 28. joulukuuta 2010 07:48 To: [email protected]; [email protected] Subject: Re: [Meego-qa] The grading and comments are available for each feature on qa-reports now Hello, Is this grading itself and its boundary values explained somewhere in the test report itself? "Here is a example of the 2 features. http://qa-reports.meego.com/1.2/Handset/Key%20basic%20feature/Ncdk/435 " Br, Petri From: [email protected] [mailto:[email protected]] On Behalf Of ext Tang, Shaofeng Sent: 28. joulukuuta 2010 03:32 To: [email protected] Subject: [Meego-qa] The grading and comments are available for each feature on qa-reports now Hi, On http://qa-reports.meego.com, 2 new features "Feature Grading" and "Feature comments" are available now. For helping readers get a clear status from feature summary, a brief comment could be added on each feature/test-set to briefly describe its' status. And also a grading will express the status in color. If you commit a new report, the grading will be calculated automatically according to below rules Grading Pass Rate of Total < 40% > 40%, < 90% > 90% And also the grading is editable for some high-weight fail/pass after the report is committed. Here is a example of the 2 features. http://qa-reports.meego.com/1.2/Handset/Key%20basic%20feature/Ncdk/435 Hope the two features are useful to you. Best Regards Shao-Feng
_______________________________________________ MeeGo-qa mailing list [email protected] http://lists.meego.com/listinfo/meego-qa
