Quality Summary ------------------------------------------- QA complete Crosswalk App Tools Testing based on Commit ID ef96046c23eeac8c1f01171a1ccbd19c1696bdf1. Glad to see that Extension hooks feature is ready (fixed XWALK-6606<https://crosswalk-project.org/jira/browse/XWALK-6606>), Android apk size is ~20M built on windows (fixed XWALK-6470<https://crosswalk-project.org/jira/browse/XWALK-6470>), RS extensions now can be packaged by apptools which installed by npm (fixed XWALK-6465<https://crosswalk-project.org/jira/browse/XWALK-6465>), "d3dcompiler_47.dll" file is existed in installing folder and application can be launched successfully (fixed XWALK-6575<https://crosswalk-project.org/jira/browse/XWALK-6575>), Windows project can be built out with a crosswalk directory (fixed XWALK-6596<https://crosswalk-project.org/jira/browse/XWALK-6596>), Project can be built out successfully when missing an icon (fixed XWALK-5174<https://crosswalk-project.org/jira/browse/XWALK-5174>), Windows project can be built out using crosswalk-pkg (fixed XWALK-5207<https://crosswalk-project.org/jira/browse/XWALK-5207>), IPA application can be installed on IOS devices (fixed XWALK-6224<https://crosswalk-project.org/jira/browse/XWALK-6224>), The description of error message had been improved (fixed XWALK-4891<https://crosswalk-project.org/jira/browse/XWALK-4891>), Deb project can be built out when "display" section of manifest.json is "minimal-ui" (fixed XWALK-5028<https://crosswalk-project.org/jira/browse/XWALK-5028>), Only 3 permissions enabled default when building project (fixed XWALK-5016<https://crosswalk-project.org/jira/browse/XWALK-5016>), Project can be built using crosswalk-pkg when icon is missing (fixed XWALK-5234<https://crosswalk-project.org/jira/browse/XWALK-5234>), Platforms automatic generated in manifest is correct when using crosswalk-pkg (fixed XWALK-5448<https://crosswalk-project.org/jira/browse/XWALK-5448>), Project can be built when icons smaller than 36px (fixed XWALK-5079<https://crosswalk-project.org/jira/browse/XWALK-5079>), Only check setup for target platforms (fixed XWALK-5606<https://crosswalk-project.org/jira/browse/XWALK-5606>), Parameter -a can be worked now (fixed XWALK-5682<https://crosswalk-project.org/jira/browse/XWALK-5682>). However, 1 REGRESSION issue found: Message "Mismatch of CPU Architecture" pops up after launching embedded app which apk was built by the newest crosswalk-pkg tool (XWALK-6725<https://crosswalk-project.org/jira/browse/XWALK-6725>), and another 4 new issues found: Failed to build msi when there is a "VERSION" file in the crosswalk application project directory (XWALK-6683<https://crosswalk-project.org/jira/browse/XWALK-6683>), Fail to build project without -p android/--platforms=android for Android platform (XWALK-6698<https://crosswalk-project.org/jira/browse/XWALK-6698>), "start_url" is unable to work after modifying it in manifest.json file for iOS platform (XWALK-6715<https://crosswalk-project.org/jira/browse/XWALK-6715>), "standalone" display is not working after modifying it in manifest.json file for iOS platform (XWALK-6716<https://crosswalk-project.org/jira/browse/XWALK-6716>). The overall test results:
New Issues: ----------------------- XWALK-6725<https://crosswalk-project.org/jira/browse/XWALK-6725> P1 [REG][app-tool]Message "Mismatch of CPU Architecture" pops up after launching embedded app which apk was built by the newest crosswalk-pkg tool XWALK-6683<https://crosswalk-project.org/jira/browse/XWALK-6683> P2 Failed to build msi when there is a "VERSION" file in the crosswalk application project directory XWALK-6698<https://crosswalk-project.org/jira/browse/XWALK-6698> P2 [Apptools] Fail to build project without -p android/--platforms=android for Android platform. XWALK-6715<https://crosswalk-project.org/jira/browse/XWALK-6715> P2 [Apptools] "start_url" is unable to work after modifying it in manifest.json file for IOS platform. XWALK-6716<https://crosswalk-project.org/jira/browse/XWALK-6716> P2 [Apptools] "standalone" display is not working after modifying it in manifest.json file for IOS platform. Verified Issues: ----------------------- XWALK-6606<https://crosswalk-project.org/jira/browse/XWALK-6606> WiX XML file should be accessible to extension hooks XWALK-6470<https://crosswalk-project.org/jira/browse/XWALK-6470> P1 crosswalk-pkg adds ~40MB to apk size (not ~20MB) XWALK-6465<https://crosswalk-project.org/jira/browse/XWALK-6465> P1 Crosswalk-app-tool doesn't support package RS extensions installed by npm XWALK-5174<https://crosswalk-project.org/jira/browse/XWALK-5174> P1 crosswalk-package command fails if project is missing an icon XWALK-5207<https://crosswalk-project.org/jira/browse/XWALK-5207> P1 [app-tools] crosswalk-package command fails for windows platforms XWALK-6224<https://crosswalk-project.org/jira/browse/XWALK-6224> P1 IPA app built by apptools is unable to be installed on IOS devices. XWALK-6575<https://crosswalk-project.org/jira/browse/XWALK-6575> P2 [app-tools] missing d3dcompiler_47.dll when packaging Crosswalk for Windows XWALK-6596<https://crosswalk-project.org/jira/browse/XWALK-6596> P2 Windows auto-built releases do not work with as directory XWALK-4891<https://crosswalk-project.org/jira/browse/XWALK-4891> P2 [app-tools]The description of error message need to be improved XWALK-5028<https://crosswalk-project.org/jira/browse/XWALK-5028> P2 [apptools] Unable to build deb when "display" section of manifest.json is "minimal-ui" XWALK-5016<https://crosswalk-project.org/jira/browse/XWALK-5016> P2 Crosswalk app template defines too many permissions XWALK-5234<https://crosswalk-project.org/jira/browse/XWALK-5234> P2 crosswalk-pkg fails when icon is missing XWALK-5448<https://crosswalk-project.org/jira/browse/XWALK-5448> P2 [app-tools] The platforms in manifest by crosswalk-pkg command automatic generated is incorrect for windows platforms XWALK-5079<https://crosswalk-project.org/jira/browse/XWALK-5079> P2 Icons smaller than 36px fail XWALK-5606<https://crosswalk-project.org/jira/browse/XWALK-5606> P2 Only check setup for target platforms XWALK-5682<https://crosswalk-project.org/jira/browse/XWALK-5682> P2 [app-tools] Parameter -a does not work Test Scope Windows 8.1 Windows 10 OS X Deepin Android Usecase Test Comment Total TCs Pass Rate Diff W/last commit ID(*) Total TCs Pass Rate Diff W/last commit ID(*) Total TCs Pass Rate Diff W/last commit ID(*) Total TCs Pass Rate Diff W/last commit ID(*) Total TCs Pass Rate Diff W/last commit ID(*) App Tools Android 32-bit Tests(Embedded) N/A N/A N/A 273 95% -- 273 95% -- N/A N/A N/A 39 92% -- Bugs Exist:XWALK-4056<https://crosswalk-project.org/jira/browse/XWALK-4056>, XWALK-5661<https://crosswalk-project.org/jira/browse/XWALK-5661>, XWALK-5254<https://crosswalk-project.org/jira/browse/XWALK-5254> XWALK-6018<https://crosswalk-project.org/jira/browse/XWALK-6018>, XWALK-6020<https://crosswalk-project.org/jira/browse/XWALK-6020>, XWALK-6012<https://crosswalk-project.org/jira/browse/XWALK-6012>, XWALK-6698<https://crosswalk-project.org/jira/browse/XWALK-6698> App Tools Android 32-bit Tests(Shared) N/A N/A N/A 273 95% -- 273 95% -- N/A N/A N/A 39 92% -- Bugs Exist:XWALK-4056<https://crosswalk-project.org/jira/browse/XWALK-4056>, XWALK-5661<https://crosswalk-project.org/jira/browse/XWALK-5661>, XWALK-5254<https://crosswalk-project.org/jira/browse/XWALK-5254> XWALK-6018<https://crosswalk-project.org/jira/browse/XWALK-6018>, XWALK-6020<https://crosswalk-project.org/jira/browse/XWALK-6020>, XWALK-6012<https://crosswalk-project.org/jira/browse/XWALK-6012>, XWALK-6698<https://crosswalk-project.org/jira/browse/XWALK-6698> App Tools Android 64-bit Tests(Embedded) N/A N/A N/A 273 95% -- 273 95% -- N/A N/A N/A 39 92% -- Bugs Exist:XWALK-4056<https://crosswalk-project.org/jira/browse/XWALK-4056>, XWALK-5661<https://crosswalk-project.org/jira/browse/XWALK-5661>, XWALK-5254<https://crosswalk-project.org/jira/browse/XWALK-5254> XWALK-6018<https://crosswalk-project.org/jira/browse/XWALK-6018>, XWALK-6020<https://crosswalk-project.org/jira/browse/XWALK-6020>, XWALK-6012<https://crosswalk-project.org/jira/browse/XWALK-6012>, XWALK-6698<https://crosswalk-project.org/jira/browse/XWALK-6698> App Tools Android 64-bit Tests (Shared) N/A N/A N/A 273 95% -- 273 95% -- N/A N/A N/A 39 92% -- Bugs Exist:XWALK-4056<https://crosswalk-project.org/jira/browse/XWALK-4056>, XWALK-5661<https://crosswalk-project.org/jira/browse/XWALK-5661>, XWALK-5254<https://crosswalk-project.org/jira/browse/XWALK-5254> XWALK-6018<https://crosswalk-project.org/jira/browse/XWALK-6018>, XWALK-6020<https://crosswalk-project.org/jira/browse/XWALK-6020>, XWALK-6012<https://crosswalk-project.org/jira/browse/XWALK-6012>, XWALK-6698<https://crosswalk-project.org/jira/browse/XWALK-6698> App Tools SampleApp Tests N/A N/A N/A 10 100% -- 10 100% -- 2 100% -- N/A N/A N/A App Tools UseCase Tests (windows/ios/deepin) N/A N/A N/A 23 96% -- 8 63% +63% 7 71% -- N/A N/A N/A Bugs exist: XWALK-6018<https://crosswalk-project.org/jira/browse/XWALK-6018>, XWALK-6020<https://crosswalk-project.org/jira/browse/XWALK-6020>, XWALK-6012<https://crosswalk-project.org/jira/browse/XWALK-6012>, XWALK-6529<https://crosswalk-project.org/jira/browse/XWALK-6529>, XWALK-6715<https://crosswalk-project.org/jira/browse/XWALK-6715>, XWALK-6716<https://crosswalk-project.org/jira/browse/XWALK-6716> Got improvement on IOS due to XWALK-6224<https://crosswalk-project.org/jira/browse/XWALK-6224> fixed. App Tools Tests 144 83% -- 144 83% -- N/A N/A N/A N/A N/A N/A N/A N/A N/A Bugs exist: XWALK-5661<https://crosswalk-project.org/jira/browse/XWALK-5661>, XWALK-6012<https://crosswalk-project.org/jira/browse/XWALK-6012>, XWALK-6527<https://crosswalk-project.org/jira/browse/XWALK-6527>. Total 144 83% -- 1269 94% -- 1110 95% +1% 9 78% -- 156 92% -- Got improvement on IOS due to XWALK-6224<https://crosswalk-project.org/jira/browse/XWALK-6224> fixed. (*) instead of the previous test commit id: cf1a2d70385d50f1abf387a6e7ad1ade15748239 Crosswalk-app-tools Repositories: ----------------------- https://github.com/crosswalk-project/crosswalk-app-tools Release Tag or Commit ID: ef96046c23eeac8c1f01171a1ccbd19c1696bdf1 Debian Backend: https://github.com/crosswalk-project/crosswalk-app-tools-deb Release Tag or Commit ID: c5212e631dc37807be394dd2c86fa0dad9c42767 iOS Backend: https://github.com/crosswalk-project/crosswalk-app-tools-ios Release Tag or Commit ID: abd046da6a127142b01a87e2343f52f0c79bd215 Test Suite Repositories: ----------------------- Command-line Interface: https://github.com/crosswalk-project/crosswalk-test-suite/tree/master/apptools/apptools-android-tests https://github.com/crosswalk-project/crosswalk-test-suite/tree/master/apptools/apptools-ios-tests https://github.com/crosswalk-project/crosswalk-test-suite/tree/master/apptools/apptools-linux-tests https://github.com/crosswalk-project/crosswalk-test-suite/tree/master/apptools/apptools-sampleapp-tests https://github.com/crosswalk-project/crosswalk-test-suite/blob/master/usecase/usecase-apptools-tests/ Common module: https://github.com/crosswalk-project/crosswalk-app-tools/tree/master/test Crosswalk Binary on Android: https://download.01.org/crosswalk/releases/crosswalk/android/beta/18.48.477.11/ Crosswalk Binary on Windows: https://download.01.org/crosswalk/releases/crosswalk/windows/canary/19.49.508.0/[no beta builds for Windows] Test Environment: ----------------------- Android host: Ivy Bridge Desktop, Ubuntu 14, Windows 8.1,Windows10 Android device: ASUS Memo Pad 8, ECS2-8A ,Nexus 5, Nexus 9 iOS host: MacBook Pro 10.11 with XCode 7.3 iOS device: iPad mini 9.3 iOS backend: https://github.com/crosswalk-project/crosswalk-app-tools-ios Linux/Deepin desktop: Deepin 2015 x86_64 with Chinese Linux/Deepin backend: https://github.com/crosswalk-project/crosswalk-app-tools-deb Thanks, Crosswalk QA team
_______________________________________________ Crosswalk-dev mailing list [email protected] https://lists.crosswalk-project.org/mailman/listinfo/crosswalk-dev
