Test Objective: Capture the key regression of daily build by automated nightly testing.
Summary: : One major issue found after comparing recent result, AppTools and Embedding APIs pass rate are decreasing compared with 14.43.341.0. Aging downgrade issue(since 13.42.319.0): XWALK-3794<https://crosswalk-project.org/jira/browse/XWALK-3794> Aging downgrade issue(since 14.43.336.0): XWALK-4017<https://crosswalk-project.org/jira/browse/XWALK-4017> XWALK-4019<https://crosswalk-project.org/jira/browse/XWALK-4019> XWALK-4031<https://crosswalk-project.org/jira/browse/XWALK-4031> XWALK-4036<https://crosswalk-project.org/jira/browse/XWALK-4036> XWALK-4040<https://crosswalk-project.org/jira/browse/XWALK-4040> XWALK-4044<https://crosswalk-project.org/jira/browse/XWALK-4044> Downgraded Components: Platform Test component TC Num Pass% Diff w/ 14.43.341.0 Comments Android x86/arm Crosswalk App Tools/CLI 91 82% -6% XWALK-4054<https://crosswalk-project.org/jira/browse/XWALK-4054> [app-tools] Incorrect output info when update crosswalk with channel or without arguments Overall Test Results: Release Segment Test Set Hardware WebAPIs Runtime Features Embedding APIs (Java) Use Cases Diff w/ 14.43.341.0 Total TCs Pass% Trend Total TCs Pass% Trend Total TCs Pass% Trend Total TCs Pass% Trend Crosswalk on Android Canary Embedded Crosswalk Full Functional Test ASUS MeMO Pad 8 (4.4.2) 11840 89% -- 597 11% -- 200 91% -1% 37 92% -- EmbeddingAPI passrate is decreasing ,becase Adjust cases according to the change of onCreateWindowRequested feature ZTE Geek V975 (4.2.2) 11482 90% -- 597 11% -- 200 91% -1% 37 92% -- Toshiba Excite Go AT7-C8(4.4.2) 11482 90% -- 597 11% -- 200 91% -1% 37 92% -- Google Nexus 4(4.2.2) 11840 89% -- 597 11% -- 200 91% -1% 37 92% -- Google Nexus 7 (4.4.4) 11482 90% -- 597 11% -- 200 91% -1% 37 92% -- Release Segment Test Set Hardware Sample Apps AppTools Diff w/ 14.43.341.0 Total TCs Pass% Trend Total TCs Pass% Trend Crosswalk on Android Canary Embedded Crosswalk Full Functional Test ASUS MeMO Pad 8 (4.4.2) 47 100% -- 91 82% -6% AppTools passrate is decreasing,becase of bug XWALK-4054 ZTE Geek V975 (4.2.2) 42 100% -- 91 82% -6% Toshiba Excite Go AT7-C8(4.4.2) 42 100% -- 91 82% -6% Google Nexus 4(4.2.2) 47 100% -- 91 82% -6% Google Nexus 7 (4.4.4) 42 100% -- 91 82% -6% Test Coverage: Full Auto TCs of Web APIs, Crosswalk WRT, Embedding APIs, Use Cases, Sample Apps Test Environment: Crosswalk build: https://download.01.org/crosswalk/releases/crosswalk/ Github Test Suite Links: https://github.com/crosswalk-project/crosswalk-test-suite/ Thanks, Crosswalk QA team
_______________________________________________ Crosswalk-dev mailing list [email protected] https://lists.crosswalk-project.org/mailman/listinfo/crosswalk-dev
