Hi, I have now prepared PR #2043 (https://github.com/apache/cordova-android/pull/2043), which runs the instrumentation tests in a separate workflow, as suggested. All 16 tests passed on API 24 and API 36, and the workflow uploads test reports and logcat for diagnosing failures.
One question that came up is which WebView versions we should test. The selected emulator images currently bundle WebView 69 on API 24 and WebView 133 on API 36. It exists PR #2036 (https://github.com/apache/cordova-android/pull/2036) which fixes a startup race on Android 7.1.1 using WebView version 55. This startup race does not happen on the instrumentation test on WebView 69 used on Android 7.0 and also not on WebView 133 and 150. That does not establish an exact affected-version range. Do we have a minimum supported WebView/Chromium version for the default SystemWebView engine, independently of the minimum Android version? Should we keep testing the bundled versions, or explicitly select a baseline and a recent compatible WebView version? I would appreciate your thoughts on how to proceed with the WebView coverage. Regards, Manuel > Am 31.10.2026 um 16:51 schrieb Manuel Beck via dev <[email protected]>: > >> Maybe the instrumented tests can be its own workflow so that it's >> independent from the quicker unit tests? That way if it ends up >> failing/timing out it can be reran independently as well. > > Sounds good. I will try to create a solution. > >> Am 30.10.2026 um 19:19 schrieb Norman Breau via dev <[email protected]>: >> >> I find instrumented test on GH actions is finicky in my own personal >> experience, but still it probably would be ideal to include it. Tbh I >> thought it was already included but alas it seems to only run the unit tests. >> >> Maybe the instrumented tests can be its own workflow so that it's >> independent from the quicker unit tests? That way if it ends up >> failing/timing out it can be reran independently as well. >> >> On 2026-10-29 2:27 p.m., Manuel Beck wrote: >>> Hi Cordova community, >>> >>> while fixing a double back button event on Android 16 >>> (https://github.com/apache/cordova-android/pull/2029), I noticed that >>> instrumentation tests exists, but that the GitHub Actions workflows do not >>> build or execute them, just JavaScript and Java unit tests. Also the >>> instrumentation tests were outdated and had several build and runtime >>> failures which I fixed in some PRs. >>> >>> On our plugin side, automated tests are run on Android emulators in GitHub >>> Actions, using android-emulator-runner together with Cordova Paramedic. >>> >>> Could we add a GitHub Actions workflow using android-emulator-runner, as >>> our plugin repositories already do, and execute cordova-android’s >>> instrumentation suite with Gradle’s connectedDebugAndroidTest task? >>> Paramedic would not be required for these tests. >>> >>> Do we want to excecute the instrumentation tests at all? I would say, that >>> it is also good for cordova-android to run tests on an emulator. >>> >>> Regards, >>> >>> Manuel >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: [email protected] >> For additional commands, e-mail: [email protected] >> > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
