Hi SungKyu, Thank you for sharing the QA result from Samsung side.
I think the result looks good to go forward with the release. Hi Pat, Jawid Could you share the Intel QA result? Regards June June Yong Young OIC Open Sourece WG Project Planning & Requirement TG Chair IoTivity Release Function Lead Samsung Electronics Co.,Ltd. Software R&D Center, IoT Solution Lab. | Web & Convergence Team Principal Engineer T: +82-31-301-6107, M: +82-10-9530-6107 E-mail :juney at samsung.com From: ??? [mailto:[email protected]] Sent: Friday, May 29, 2015 6:11 PM To: ???; iotivity-dev at lists.iotivity.org Subject: Samsung QA report for IoTivity 0.9.1 release Dear Mr. June This is Samsung QA report for IoTivity 0.9.1 release. 1. QA Summary - Total Pass Rate : 96.2% (3019/3137) - Active Defects : 15 (0/5/10, Critial/Major/Minor) - Test results satisfies release exit criteria * Pass rate 96.2% (more than 95%) * There are 5 Major defects but they are out of scope of 0.9.1 release. There is no Critical/Major defect in release feature. 2. Test results API & Sample App Test Summary (Pass Rate 96.2 %) No. Component ? Total Ubuntu Tizen Android Arduino Ubuntu (by ver) ? All Ver 2.3 5.0 (Lollipop) AT Mega 2560 12.04(32bit) 12.04(64bit) 14.04(32bit) 14.04(64bit) ? Pass Fail Total Pass % Pass Fail Total Pass Fail Total Pass Fail Total Pass Fail Total Pass Fail Total Pass Fail Total Pass Fail Total Pass Fail Total 1 PPM ? 412 46 458 90.0% 328 36 364 84 10 94 0 0 0 0 0 0 83 9 92 83 9 92 81 9 90 81 9 90 2 SSM ? 491 2 493 99.6% 322 2 324 86 0 86 83 0 83 0 0 0 91 1 92 91 1 92 70 0 70 70 0 70 3 TM ? 922 30 952 96.8% 618 28 646 160 1 161 144 1 145 0 0 0 185 12 197 185 12 197 124 2 126 124 2 126 4 NM ? 228 15 243 93.8% 146 12 158 36 3 39 46 0 46 0 0 0 48 3 51 48 3 51 25 3 28 25 3 28 5 CA ? 966 25 991 97.5% 508 16 524 128 4 132 115 4 119 215 1 216 149 4 153 149 4 153 105 4 109 105 4 109 Total ? 3019 118 3137 96.2% 1922 94 2016 494 18 512 388 5 393 215 1 216 556 29 585 556 29 585 405 18 423 405 18 423 API Test No. Component ? Total Ubuntu Tizen Android (5.0) Arduino Ubuntu (by ver) ? All Ver 2.3 5.0 (Lollipop) AT Mega 2560 12.04(32bit) 12.04(64bit) 14.04(32bit) 14.04(64bit) ? Pass Fail Total Pass % Pass Fail Total Pass Fail Total Pass Fail Total Pass Fail Total Pass Fail Total Pass Fail Total Pass Fail Total Pass Fail Total 1 PPM 405 45 450 90.0% 324 36 360 81 9 90 NA NA 0 NA NA 0 81 9 90 81 9 90 81 9 90 81 9 90 2 SSM 418 0 418 100% 280 0 280 70 0 70 68 0 68 NA NA 0 70 0 70 70 0 70 70 0 70 70 0 70 3 TM 720 10 730 98.6% 496 8 504 120 1 121 104 1 105 NA NA 0 124 2 126 124 2 126 124 2 126 124 2 126 4 NM 141 15 156 90.4% 100 12 112 19 3 22 22 0 22 NA NA 0 25 3 28 25 3 28 25 3 28 25 3 28 5 CA 764 25 789 96.8% 420 16 436 99 4 103 71 4 75 174 1 175 105 4 109 105 4 109 105 4 109 105 4 109 Total ? 2448 95 2543 96.3% 1620 72 1692 389 17 406 265 5 270 174 1 175 405 18 423 405 18 423 405 18 423 405 18 423 Sample App Test No. Component Total Ubuntu Tizen Android Arduino Ubuntu (by ver) All Ver 2.3 5.0 (Lollipop) AT Mega 2560 12.04(32bit) 12.04(64bit) Pass Fail Total Pass % Pass Fail Total Pass Fail Total Pass Fail Total Pass Fail Total Pass Fail Total Pass Fail Total 1 PPM 7 1 8 88% 4 0 4 3 1 4 NA NA 0 NA NA 0 2 0 2 2 0 2 2 SSM 73 2 75 97% 42 2 44 16 0 16 15 0 15 NA NA 0 21 1 22 21 1 22 3 TM 202 20 222 91.0% 122 20 142 40 0 40 40 0 40 NA NA 0 61 10 71 61 10 71 4 NM 87 0 87 100.0% 46 0 46 17 0 17 24 0 24 NA NA 0 23 0 23 23 0 23 5 CA ? 202 0 202 100.0% 88 0 88 29 0 29 44 0 44 41 0 41 44 0 44 44 0 44 Total ? 571 23 594 96.1% 302 22 324 105 1 106 123 0 123 41 0 41 151 11 162 151 11 162 3. Known issues - All active defects are registered in IoTivity Jira [PPM][Android] Stopping Plugin with Empty String <https://jira.iotivity.org/browse/IOT-538> https://jira.iotivity.org/browse/IOT-538 [PPM][Android] PluginManager can not start or stop Plugin with the Key ?Url? <https://jira.iotivity.org/browse/IOT-536> https://jira.iotivity.org/browse/IOT-536 [PPM][Android] After Stopping Plugin, PluginManager can not find that Plugin <https://jira.iotivity.org/browse/IOT-535> https://jira.iotivity.org/browse/IOT-535 [PPM] PPM Tizen sample app is not working properly <https://jira.iotivity.org/browse/IOT-507> https://jira.iotivity.org/browse/IOT-507 [PPM]If push the START PLUGIN button about Hue Plug, Nothing is happened(no showing successful log) <https://jira.iotivity.org/browse/IOT-464> https://jira.iotivity.org/browse/IOT-464 [SSM][Linux] Callback is not coming when SampleApp is run before running sensor apps. <https://jira.iotivity.org/browse/IOT-530> https://jira.iotivity.org/browse/IOT-530 [TM][Linux] JoinMusicPlayer & JoinSpeaker doesn't work on group App and crashes the group app <https://jira.iotivity.org/browse/IOT-534> https://jira.iotivity.org/browse/IOT-534 [TM][Linux][Tizen][Android] When SubscribeCollectionForPresence() API was called to subscribe a collection created in the same process, it returned error code OC_STACK_NO_RESOURCE through callback(expected errorcode: OC_STACK_OK) <https://jira.iotivity.org/browse/IOT-532> https://jira.iotivity.org/browse/IOT-532 [NM][Linux][Tizen]OCStop() does not return OC_STACK_OK <https://jira.iotivity.org/browse/IOT-531> https://jira.iotivity.org/browse/IOT-531 [CA][Linux][Tizen][Ardunio]CA APIs is returning CA_STATUS_OK without CAInitialize <https://jira.iotivity.org/browse/IOT-539> https://jira.iotivity.org/browse/IOT-539 [CA][Linux][Tizen]Multiple CATerminate call makes program hanged <https://jira.iotivity.org/browse/IOT-537> https://jira.iotivity.org/browse/IOT-537 [CA][Tizen] Tizen DTLS is not working for IPv4 network <https://jira.iotivity.org/browse/IOT-533> https://jira.iotivity.org/browse/IOT-533 [CA]CA can not transmit data through LE network in Tizen <https://jira.iotivity.org/browse/IOT-472> https://jira.iotivity.org/browse/IOT-472 [CA] CASendResponse can send message with iinvalid CAResponseInfo_t.result as -1 <https://jira.iotivity.org/browse/IOT-386> https://jira.iotivity.org/browse/IOT-386 [CA] CASendNotification can send message with invalid CAResponseInfo_t.result as -1 <https://jira.iotivity.org/browse/IOT-385> https://jira.iotivity.org/browse/IOT-385 4. Excluded features of QA and 0.9.1 Release - PPM Android - Tizen BT/BLE - Tizen DTLS - NM between Android devices -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.iotivity.org/pipermail/iotivity-dev/attachments/20150529/812ad0b1/attachment.html> -------------- next part -------------- A non-text attachment was scrubbed... Name: image001.gif Type: image/gif Size: 13168 bytes Desc: not available URL: <http://lists.iotivity.org/pipermail/iotivity-dev/attachments/20150529/812ad0b1/attachment.gif>
