Test Objective: Capture the key regression of daily build by automated nightly testing.
Summary: :Three major issues found after comparing recent result, WebAPIs and Sample Apps passrate is decreasing compared with 16.44.406.0. Aging downgrade issue(since 13.42.319.0): XWALK-3794<https://crosswalk-project.org/jira/browse/XWALK-3794> Aging downgrade issue(since 14.43.336.0): XWALK-4017<https://crosswalk-project.org/jira/browse/XWALK-4017> XWALK-4019<https://crosswalk-project.org/jira/browse/XWALK-4019> XWALK-4031<https://crosswalk-project.org/jira/browse/XWALK-4031> XWALK-4036<https://crosswalk-project.org/jira/browse/XWALK-4036> XWALK-4040<https://crosswalk-project.org/jira/browse/XWALK-4040> XWALK-4044<https://crosswalk-project.org/jira/browse/XWALK-4044> Aging downgrade issue(since 15.44.361.0): XWALK-4382<https://crosswalk-project.org/jira/browse/XWALK-4382> Downgraded Components: Platform Test component TC Num Pass% Diff w/ 16.44.406.0 Comments Android x86/arm Supplementary APIs/WebGL - Khronos (Partial) 1597 99% 0% XWALK-5086 [REG]Fail to throw "INVALID_OPERATION" when an attempt to bind a buffer object to the other target Android x86/arm Supplementary APIs/SIMD 1669 87% -10% XWALK-5087 [REG]The function Float32Array is undefined when running 'webapi-simd-nonw3c-tests' component Android x86/arm W3C_HTML5 APIs/Communication/HTML5 Web Messaging 120 92% -3% XWALK-4934 [REG]Get exception test result when running "embedding-api-android-tests". Overall Test Results: Canary - Crosswalk Project for Android Segment Test Set Hardware WebAPIs Embedding APIs (Java) Runtime Features Use Cases Sample Apps Diff w/ 16.44.406.0 Total TCs Pass% Trend Total TCs Pass% Trend Total TCs Pass% Trend Total TCs Pass% Trend Total TCs Pass% Trend Embedded Crosswalk Full Functional Test ASUS MeMO Pad 8(4.4.2) 17452 86% -2% 231 97% +1% 559 100% -- 74 93% +4% 72 74% -7% WebAPIs and Sample Apps passrate decreasing: Chrome version update from M44 to M45 and a part of case is adjusting. Google Nexus 4(4.2.2) 17452 86% -2% 231 97% +1% 559 99% -- 74 81% +4% 72 69% -8% WebAPIs and Sample Apps passrate decreasing: Chrome version update from M44 to M45 and a part of case is adjusting. Beta - Crosswalk Project for Android (15.44.384.9) Segment Test Set Hardware WebAPIs Embedding APIs (Java) Runtime Features Use Cases Sample Apps Diff w/ 15.44.384.5 Total TCs Pass% Trend Total TCs Pass% Trend Total TCs Pass% Trend Total TCs Pass% Trend Total TCs Pass% Trend Embedded Crosswalk Full(Weekly) Functional Test ASUS MeMO Pad 8(4.4.2) 17441 88% +1% 213 96% -- 525 100% -- 54 93% -- 70 74% -25% Sample Apps passrate decreasing due to case adjust. Crosswalk WebView Plugin for Cordova(Weekly) Functional Test Google Nexus 4(4.2.2) 17435 88% +1% N/A N/A N/A N/A N/A N/A 32 84% +84% N/A N/A N/A Beta - Crosswalk Project for Android (15.44.384.9) Segment Test Set Hardware WebAPIs Embedding APIs (Java) Runtime Features Use Cases Sample Apps comment Total TCs Pass% Trend Total TCs Pass% Trend Total TCs Pass% Trend Total TCs Pass% Trend Total TCs Pass% Trend Shared Crosswalk Full(Weekly) Functional Test ASUS MeMO Pad 8(4.4.2) 17441 88% NEW 213 96% NEW N/A N/A N/A N/A N/A N/A 70 74% NEW Test Coverage: Full Auto TCs of Web APIs, Crosswalk WRT, Embedding APIs, Use Cases, Sample Apps Test Environment: Crosswalk build: https://download.01.org/crosswalk/releases/crosswalk/ Github Test Suite Links: https://github.com/crosswalk-project/crosswalk-test-suite/ Thanks, Crosswalk QA team
_______________________________________________ Crosswalk-dev mailing list [email protected] https://lists.crosswalk-project.org/mailman/listinfo/crosswalk-dev
