Test Objective: Capture the key regression of daily build by automated nightly testing.
Summary: : One major issue found after comparing recent result, WebAPIs and Sample Apps passrate are decreasing compared with 15.43.360.0. Aging downgrade issue(since 13.42.319.0): XWALK-3794<https://crosswalk-project.org/jira/browse/XWALK-3794> Aging downgrade issue(since 14.43.336.0): XWALK-4017<https://crosswalk-project.org/jira/browse/XWALK-4017> XWALK-4019<https://crosswalk-project.org/jira/browse/XWALK-4019> XWALK-4031<https://crosswalk-project.org/jira/browse/XWALK-4031> XWALK-4036<https://crosswalk-project.org/jira/browse/XWALK-4036> XWALK-4040<https://crosswalk-project.org/jira/browse/XWALK-4040> XWALK-4044<https://crosswalk-project.org/jira/browse/XWALK-4044> Aging downgrade issue(since 15.43.348.0): XWALK-4251<https://crosswalk-project.org/jira/browse/XWALK-4251> Downgraded Components: Platform Test Suites TC Num Pass% Diff w/ 15.43.360.0 Comments Android x86/arm WebAPI/Device/Battery Status API 49 76% -24% XWALK-4382<https://crosswalk-project.org/jira/browse/XWALK-4382> [Reg]Attributes "onchargingchange/onchargingtimechange/ondischargingtimechange/onlevelchange " of Battery Status API did not treat noncallable as null Overall Test Results: Canary - Crosswalk Project for Android Segment Test Set Hardware WebAPIs Embedding APIs (Java) Runtime Features Use Cases Sample Apps Diff w/ 15.43.360.0 Total TCs Pass% Trend Total TCs Pass% Trend Total TCs Pass% Trend Total TCs Pass% Trend Total TCs Pass% Trend Embedded Crosswalk Full Functional Test ASUS MeMO Pad 8(4.4.2) 12008 76% -14% 202 78% -- 597 11% -- 55 96% -- 46 98% -2% WebAPIs passrate decreasing cause: 1.Due to update rebase,so webdriver bdd case needs to adjust. 2.XWALK-4207<https://crosswalk-project.org/jira/browse/XWALK-4207>(M44: Enable SIMD.js in v8-crosswalk) Sample Apps passrate decreasing cause: XWALK-4207<https://crosswalk-project.org/jira/browse/XWALK-4207>(M44: Enable SIMD.js in v8-crosswalk) Google Nexus 4(4.2.2) 12008 77% -14% 202 78% -- 597 11% -- 55 96% -- 46 98% -2% WebAPIs passrate decreasing cause: 1.Due to update rebase,so webdriver bdd case needs to adjust. 2.XWALK-4207<https://crosswalk-project.org/jira/browse/XWALK-4207>(M44: Enable SIMD.js in v8-crosswalk) Sample Apps passrate decreasing cause: XWALK-4207<https://crosswalk-project.org/jira/browse/XWALK-4207>(M44: Enable SIMD.js in v8-crosswalk) Beta - Crosswalk Project for Android (14.43.343.12) Segment Test Set Hardware WebAPIs Embedding APIs (Java) Runtime Features Use Cases Sample Apps Diff w/ 14.43.343.10 Total TCs Pass% Trend Total TCs Pass% Trend Total TCs Pass% Trend Total TCs Pass% Trend Total TCs Pass% Trend Embedded Crosswalk Full(Weekly) Functional Test ASUS MeMO Pad 8(4.4.2) 11976 90% -- 202 78% -- 597 99% -- 55 95% -- 47 100% -- Google Nexus 4(4.2.2) 11976 90% -- 202 78% -- 597 99% -- 55 95% -- 47 100% -- Crosswalk WebView Plugin for Cordova(Weekly) Functional Test ASUS MeMO Pad 8(4.4.2) 11870 92% NEW N/A N/A N/A N/A N/A N/A 27 70% NEW N/A N/A N/A Google Nexus 4(4.2.2) 11870 92% NEW N/A N/A N/A N/A N/A N/A 27 70% NEW N/A N/A N/A Test Coverage: Full Auto TCs of Web APIs, Crosswalk WRT, Embedding APIs, Use Cases, Sample Apps Test Environment: Crosswalk build: https://download.01.org/crosswalk/releases/crosswalk/ Github Test Suite Links: https://github.com/crosswalk-project/crosswalk-test-suite/ Thanks, Crosswalk QA team
_______________________________________________ Crosswalk-dev mailing list [email protected] https://lists.crosswalk-project.org/mailman/listinfo/crosswalk-dev
