Test Objective: Capture the key regression of weekly build by automated nightly testing.
Summary: : Three major regression issues found in this build, the total pass rate is decreasing compared with 12.41.296.5. Downgraded Components: Platform Test component TC Num Pass% Diff w/ 12.41.296.5 Comments Android x86 ASUS T00E Padfone Mini W3C_HTML5 APIs/Device/DeviceOrientation Event Specification (Partial) 40 0% -85% XWALK-3954<https://crosswalk-project.org/jira/browse/XWALK-3954> Android arm Google Nexus 3 W3C_HTML5 APIs/UI/Web Notifications (Partial) 15 0% -100% XWALK-3966<https://crosswalk-project.org/jira/browse/XWALK-3966> Android arm Google Nexus 3 W3C_HTML5 APIs/Media/Web Audio API (Partial) 389 0% -100% XWALK-3967<https://crosswalk-project.org/jira/browse/XWALK-3967> Overall Test Results: Platform Hardware Crosswalk Beta Embeded WebAPI (13.42.319.2) Crosswalk Beta Cordova WebAPI (13.42.319.2) Beta Embedding APIs (13.42.319.2) Beta SampleApp (13.42.319.2) Runtime Features (13.42.319.2) Diff w/ 12.41.296.5 Total TCs Pass% Trend Total TCs Pass% Trend Total TCs Pass% Trend Total TCs Pass% Trend Total TCs Pass% Trend Android arm (Reference device) Coolpad 8297W (4.2.2) 11493 76% -- 11493 77% -1% 200 92% +2% 42 100% -- 610 100% NEW Add more tests this time and cause the score change. Additionally, three regression issues found: deviceorientation issue(XWALK-3954), web Notification issue(XWALK-3966), web audio issue(XWALK-3967) ZTE_Grand_Memo_U5(4.1.2) 11493 90% -- 11493 91% -1% 200 92% +2% 42 100% -- 610 100% NEW HuaWei Honor 3X (4.2.2) 11493 75% -1% 11493 77% -1% 200 92% +2% 42 100% -- 610 100% NEW Good Nexus 3(4.0.4) 11493 73% NEW 11493 74% NEW 200 92% NEW 42 100% NEW 610 100% NEW Kindle Fire HD (Fire OS 3.2) 11493 76% -- 11493 78% -- 200 92% +2% 42 100% -- 610 100% NEW Android x86 (Reference device) ASUS_T00G_Zenfone_6 (4.3) 11493 90% -- 11493 92% -- 200 92% +2% 42 100% -- 610 100% NEW Dell_Venue_8 (4.3) 11493 90% -- 11493 92% -- 200 92% +2% 42 100% -- 610 100% NEW ASUS T00E Padfone Mini (4.3) 11493 89% -- 11493 90% -- 200 92% +2% 42 100% -- 610 100% NEW Ramos i10 (4.2.2) 11493 90% -- 11493 90% -2% 200 92% +2% 42 100% -- 610 100% NEW Samsung Galaxy Tab 3 (4.2.2) 11493 90% -1% 11493 92% -- 200 92% +2% 42 100% -- 610 100% NEW Test Coverage: Full Auto TCs of Web APIs, Crosswalk WRT, Cordova WebAPIs, Embedding APIs, SampleApp Test Environment: Crosswalk build: https://download.01.org/crosswalk/releases/crosswalk/ Github Test Suite Links: https://github.com/crosswalk-project/crosswalk-test-suite/ Thanks, Crosswalk QA team
_______________________________________________ Crosswalk-dev mailing list [email protected] https://lists.crosswalk-project.org/mailman/listinfo/crosswalk-dev
