Test Objective: Capture the key regression of daily build by automated nightly testing.
Summary: : Two major issues found after comparing recent result, webapi passrate are decreasing compared with 14.43.343.0. Aging downgrade issue(since 13.42.319.0): XWALK-3794<https://crosswalk-project.org/jira/browse/XWALK-3794> Aging downgrade issue(since 14.43.336.0): XWALK-4017<https://crosswalk-project.org/jira/browse/XWALK-4017> XWALK-4019<https://crosswalk-project.org/jira/browse/XWALK-4019> XWALK-4031<https://crosswalk-project.org/jira/browse/XWALK-4031> XWALK-4036<https://crosswalk-project.org/jira/browse/XWALK-4036> XWALK-4040<https://crosswalk-project.org/jira/browse/XWALK-4040> XWALK-4044<https://crosswalk-project.org/jira/browse/XWALK-4044> Downgraded Components: Platform Test component TC Num Pass% Diff w/ 14.43.343.0 Comments Android x86/arm Supplementary APIs/SIMD 1637 98% -1% XWALK-4147<https://crosswalk-project.org/jira/browse/XWALK-4147> [REG] Fail to convert float32x4 to int32x4 when running "webapi-simd-nonw3c-tests" XWALK-4148<https://crosswalk-project.org/jira/browse/XWALK-4148> [REG]FromFloat32x4Bits method of int32x4 and fromInt32x4Bits method of float32x4 are invalid when running "webapi-simd-nonw3c-tests". Overall Test Results: Release Segment Test Set Hardware WebAPIs Runtime Features Embedding APIs (Java) Use Cases Diff w/ 14.43.343.0 Total TCs Pass% Trend Total TCs Pass% Trend Total TCs Pass% Trend Total TCs Pass% Trend Crosswalk on Android Canary Embedded Crosswalk Full Functional Test ASUS MeMO Pad 8 (4.4.2) 11939 90% -- 597 11% -- 196 95% -- 55 93% -1% UseCases passrate is decreasing, because case adjust. Google Nexus 4(4.2.2) 11939 89% -1% 597 11% -- 196 95% -- 55 93% -1% Release Segment Test Set Hardware Sample Apps AppTools Diff w/ 14.43.343.0 Total TCs Pass% Trend Total TCs Pass% Trend Crosswalk on Android Canary Embedded Crosswalk Full Functional Test ASUS MeMO Pad 8 (4.4.2) 47 100% -- 93 82% -4% Due to case adjust,AppTools passrate is decreasing. Google Nexus 4(4.2.2) 47 100% -- 93 82% -4% Release Segment Test Set Hardware WebAPIs (13.42.319.10) Runtime Features (13.42.319.10) Embedding APIs (Java) (13.42.319.10) Use Cases (13.42.319.10) Diff w/ 13.42.319.9 Total TCs Pass% Trend Total TCs Pass% Trend Total TCs Pass% Trend Total TCs Pass% Trend Crosswalk on Android Beta Embedded Crosswalk Full Functional Test ASUS MeMO Pad 8 (4.4.2) 11838 89% -- 610 100% -- 200 92% -- 35 94% -- Google Nexus 4(4.2.2) 11838 89% -- 610 100% -- 200 92% -- 35 94% -- Crosswalk WebView Plugin for Cordova Functional Test ASUS MeMO Pad 8 (4.4.2) 11838 91% +1% 610 100% -- 200 92% -- 35 94% -- Google Nexus 4(4.2.2) 11838 91% +1% 610 100% -- 200 92% -- 35 94% -- Segment Test Set Hardware Sample Apps (13.42.319.10) AppTools (13.42.319.10) Cordova Features (13.42.319.10) Diff w/ 13.42.319.9 Total TCs Pass% Trend Total TCs Pass% Trend Total TCs Pass% Trend Embedded Crosswalk Full Functional Test ASUS MeMO Pad 8 (4.4.2) 47 100% -- 79 71% -21% N/A N/A N/A AppTools passrate is decreasing ,because of case is adjusting. Google Nexus 4(4.2.2) 47 100% -- 79 71% -21% N/A N/A N/A Crosswalk WebView Plugin for Cordova Functional Test ASUS MeMO Pad 8 (4.4.2) 47 100% -- N/A N/A N/A 19 100% -- Google Nexus 4(4.2.2) 47 100% -- N/A N/A N/A 19 100% -- Test Coverage: Full Auto TCs of Web APIs, Crosswalk WRT, Embedding APIs, Use Cases, Sample Apps Test Environment: Crosswalk build: https://download.01.org/crosswalk/releases/crosswalk/ Github Test Suite Links: https://github.com/crosswalk-project/crosswalk-test-suite/ Thanks, Crosswalk QA team
_______________________________________________ Crosswalk-dev mailing list [email protected] https://lists.crosswalk-project.org/mailman/listinfo/crosswalk-dev
