Re: [DISCUSS] Moving forward with the WebView on AndroidX and cordova-android 10

[Norman Breau]

I think I agree with jcesarmobile on the migration part...

But I understand the problem of maintaining data in web storage containers,
I've gone through this path when trying to migrate away from crosswalk.

Personally, I've moved away from relying on web storage containers to
store any data that can't be easily reconstructed to avoid issues like 
this in the future.


Because I've had somewhat recent experience in actual migration on 
Android (and iOS),
I'll chip in my knowledge about migration strategies, and then followup 
with what I

think Apache Cordova should do.

A strategy would need to be able to migrate and use the new
web storage containers immediately, otherwise you'll run into
an issue where you have two storage containers and you don't
know if you should overwrite it or not for the migration. This
means we cannot be migrating data while it is possible for
the app user to continue writing new data to the old storage containers.

Like someone mentioned before, I have a migration plugin[1] that
was originally written for coming from Crosswalk webview to
native android webview (file protocol). I have a fork to change it
so it can migrate to the ionic webview (on using `https://localhost` 
origin).


The migration paths are hard-coded, but it should be relatively easy
to add code to make it dynamic based on the chosen URL scheme.

The plugin works by doing the following:

1. Checks if the data container exists in the target path. If so, then 
do not continue.
2. Check if the data container exists in the old path. If not, then do 
not continue.

3. Move/Copy the data from the old container to the new container path

This way, when users upgrade your app, it will run the migration code -- 
if needed.
The second time they run the app, the plugin would detect that it's 
already migrated.
Eventually, once you like all your users have been upgraded to a version 
that contains the migration

plugin and actually ran the app, then you can remove the plugin.

Afaik, this plugin only does local storage, so it will have to be 
modified to include

other web containers like IndexedDB.

There is one thing that this plugin does not handle either, and must be 
updated should someone make
a plugin or fork mine. Chrome webview in Chrome 78 has switched to a new 
storage mechanism.
Pre 78 used a sqlite databases, while >= 78 Chrome uses something called 
levelDB.
Chrome 78 does **not** have any migration from sqlite to leveldb (which 
was the source
of their huge fiasco in data loss). Chrome 79 or later will 
automatically migrate

sqlite databases to the levelDB database.

So the migration process needs to be aware that there are two possible 
source directories

and needs to be smart enough to pick one.

I pretty much agree with jcesar that Cordova should be pretty much 
hands-off when it
comes to the actual data migration. Ionic doesn't take charge in data 
migration neither

on initial install or in URL scheme changes.

Instead, Cordova can inform app developers by producing a guide (or 
providing

related resource links) which should document:
- How to create backups so that they can restore the old test database 
so that they
  can test their application rigorously. E.g: You can use ADB to pull 
the old data files
  and push them back, and `adb shell/run-as` to explore the filesystem 
as your

  application user.
- How the Android webview data system works, documenting file paths, how 
origins works

  in relation to these file paths, etc.
- Explain why Cordova took a hands-off approach; e.g: Cordova feels that
  if developers took charge in their own migration, they will be opted 
to be

  more careful about potential data losses.

[1] https://github.com/totalpave/cordova-plugin-crosswalk-data-migration

On 2021-05-04 9:01 a.m., Niklas Merz wrote:

While working on this PR I got an idea for a migration strategy I just
wanted to share here.

Someone could build a plugin that handles the migration of web storage
(Localstorage, IndexedDB etc). The app developer needs to set the new
AndroidInsecureFileModeEnabled preference to true. The app starts on the
file protocol like it used to. Now the plugin can copy all web data via
JS into the native layer. Then it switches to the new origin
https://localhost for example and copies the native stuff from the
plugins memory back into the web storage.

When data is migrated the app can start using the https protocol from
now on and the app developer could disable the preference in a future
update once all devices are migrated.

Migrations are hard but maybe this helps somebody.

On May 3, 2021, niklas merz  wrote:

Hi again,

Thank you all for your feedback.


A lot of Cordova users will never enable this feature if it is an

opt-
in feature.
This is a very good reason for me as well.

I created a PR which introduces a preference to make the
WebViewAssetLoader opt-out: https://github.com/apache/cordova-
android/pull/1222. Please leave your review.

Is this a way to 

Re: [DISCUSS] Moving forward with the WebView on AndroidX and cordova-android 10

[Niklas Merz]

While working on this PR I got an idea for a migration strategy I just
wanted to share here.

Someone could build a plugin that handles the migration of web storage
(Localstorage, IndexedDB etc). The app developer needs to set the new
AndroidInsecureFileModeEnabled preference to true. The app starts on the
file protocol like it used to. Now the plugin can copy all web data via
JS into the native layer. Then it switches to the new origin
https://localhost for example and copies the native stuff from the
plugins memory back into the web storage. 

When data is migrated the app can start using the https protocol from
now on and the app developer could disable the preference in a future
update once all devices are migrated.

Migrations are hard but maybe this helps somebody.

On May 3, 2021, niklas merz  wrote:
> Hi again,
>
> Thank you all for your feedback.
>
> > A lot of Cordova users will never enable this feature if it is an
> opt-
> in feature.
> This is a very good reason for me as well.
>
> I created a PR which introduces a preference to make the
> WebViewAssetLoader opt-out: https://github.com/apache/cordova-
> android/pull/1222. Please leave your review.
>
> Is this a way to proceed for cordova-android 10?
>
> Regards 
> Niklas
>
> On April 29, 2021, Pieter Van Poyer
> 
> wrote:
> > Hey
> >
> > I have to agree with Bryan and Julio.
> > Make it opt-out with a next major release. That's my prefered
> option.
> >
> > A lot of Cordova users will never enable this feature if it is an
> opt-
> > in feature.
> > And yes it would be great if someone did already have some
> guidelines
> > about data migration.
> >
> > Kind regards
> > Pieter Van Poyer
> >
> >
> > -Oorspronkelijk bericht-
> > Van: Niklas Apache 
> > Verzonden: woensdag 28 april 2021 12:14
> > Aan: dev@cordova.apache.org
> > Onderwerp: Re: [DISCUSS] Moving forward with the WebView on AndroidX
> > and cordova-android 10
> >
> > I would agree with Bryan and Julio that we should use the
> opportunity
> > of this breaking release to move the defaults to current Android
> best
> > practices and remove deprecated settings.
> >
> > I just started a PR [1] that introduces a preference to allow the
> use
> > file URLs with this release. This means the deprecated settings
> [2][3]
> > are only set with this settings as well. Darryl is right that
> probably
> > many still need to run their app on the file protocol and they
> should
> > have an easy option to do this with the new release. We can now push
> > new users to avoid this and current users need to check the change
> log
> > and enable this if they really need to.
> >
> > I agree we shouldn't take the responsibilty of a data migration.
> >
> > If we don't find consensus to make the WebViewAssetLoader opt-out we
> > could make it opt-in for now. This would make the upgrade more safe
> > but potentially many apps less secure because developers don't need
> to
> > think about this change.
> >
> > [1]
> >
> https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fcordova-
> >
> android%2Fpull%2F1222data=04%7C01%7CPieter.VanPoyer%40portofantwerp.com%7C3874f4566f504fbf6fca08d90a2e5037%7C2337dcc63a214d95bf72da5551a9b048%7C0%7C0%7C637552016304717784%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000sdata=yaRNsgmiLk6XqWC3NgWiceVCCF4RiEcwDzFuYzVVU%2Fg%3Dreserved=0
> > [2]
> >
> https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdeveloper.android.com%2Freference%2Fandroid%2Fwebkit%2FWebSettings%23setAllowUniversalAccessFromFileURLsdata=04%7C01%7CPieter.VanPoyer%40portofantwerp.com%7C3874f4566f504fbf6fca08d90a2e5037%7C2337dcc63a214d95bf72da5551a9b048%7C0%7C0%7C637552016304717784%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000sdata=ZbClSYsggi0ckZgdsSOHJ1Tq6zermY2krIB4zWjngcw%3Dreserved=0(boolean)
> >
> [3]https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdeveloper.android.com%2Freference%2Fandroid%2Fwebkit%2FWebSettings%23setAllowFileAccessdata=04%7C01%7CPieter.VanPoyer%40portofantwerp.com%7C3874f4566f504fbf6fca08d90a2e5037%7C2337dcc63a214d95bf72da5551a9b048%7C0%7C0%7C637552016304717784%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000sdata=e%2FP3sN894F9SIYkkBSE9Q%2BFz%2FowbKUu4r1le8TgIcf4%3Dreserved=0(boolean)
> >
> > On April 28, 2021, julio cesar sanchez 
> wrote:
> > > My understanding is that file urls can technically continue
> working,
> > > but the defaults changed and some settings need to be enabled, and
> > > some of them are now deprecated and will stop working at some
> point
> > in
> > > the future.
> > >
> > > I think defaulting to the old behavior is ok when doing minor
> > releases
> > > (not just on, but enforced because otherwise it's a breaking
> > change),
> > > but whenever we make a major release, we should always use the new
> > > behavior and make the cordova user 

Re: [DISCUSS] Moving forward with the WebView on AndroidX and cordova-android 10

[Niklas Merz]

Hi again,

Thank you all for your feedback.

> A lot of Cordova users will never enable this feature if it is an opt-
in feature.
This is a very good reason for me as well.

I created a PR which introduces a preference to make the
WebViewAssetLoader opt-out: https://github.com/apache/cordova-
android/pull/1222. Please leave your review.

Is this a way to proceed for cordova-android 10?

Regards 
Niklas

On April 29, 2021, Pieter Van Poyer 
wrote:
> Hey
>
> I have to agree with Bryan and Julio.
> Make it opt-out with a next major release. That's my prefered option.
>
> A lot of Cordova users will never enable this feature if it is an opt-
> in feature.
> And yes it would be great if someone did already have some guidelines
> about data migration.
>
> Kind regards
> Pieter Van Poyer
>
>
> -Oorspronkelijk bericht-
> Van: Niklas Apache 
> Verzonden: woensdag 28 april 2021 12:14
> Aan: dev@cordova.apache.org
> Onderwerp: Re: [DISCUSS] Moving forward with the WebView on AndroidX
> and cordova-android 10
>
> I would agree with Bryan and Julio that we should use the opportunity
> of this breaking release to move the defaults to current Android best
> practices and remove deprecated settings.
>
> I just started a PR [1] that introduces a preference to allow the use
> file URLs with this release. This means the deprecated settings [2][3]
> are only set with this settings as well. Darryl is right that probably
> many still need to run their app on the file protocol and they should
> have an easy option to do this with the new release. We can now push
> new users to avoid this and current users need to check the change log
> and enable this if they really need to.
>
> I agree we shouldn't take the responsibilty of a data migration.
>
> If we don't find consensus to make the WebViewAssetLoader opt-out we
> could make it opt-in for now. This would make the upgrade more safe
> but potentially many apps less secure because developers don't need to
> think about this change.
>
> [1]
> https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fcordova-
> android%2Fpull%2F1222data=04%7C01%7CPieter.VanPoyer%40portofantwerp.com%7C3874f4566f504fbf6fca08d90a2e5037%7C2337dcc63a214d95bf72da5551a9b048%7C0%7C0%7C637552016304717784%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000sdata=yaRNsgmiLk6XqWC3NgWiceVCCF4RiEcwDzFuYzVVU%2Fg%3Dreserved=0
> [2]
> https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdeveloper.android.com%2Freference%2Fandroid%2Fwebkit%2FWebSettings%23setAllowUniversalAccessFromFileURLsdata=04%7C01%7CPieter.VanPoyer%40portofantwerp.com%7C3874f4566f504fbf6fca08d90a2e5037%7C2337dcc63a214d95bf72da5551a9b048%7C0%7C0%7C637552016304717784%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000sdata=ZbClSYsggi0ckZgdsSOHJ1Tq6zermY2krIB4zWjngcw%3Dreserved=0(boolean)
> [3]https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdeveloper.android.com%2Freference%2Fandroid%2Fwebkit%2FWebSettings%23setAllowFileAccessdata=04%7C01%7CPieter.VanPoyer%40portofantwerp.com%7C3874f4566f504fbf6fca08d90a2e5037%7C2337dcc63a214d95bf72da5551a9b048%7C0%7C0%7C637552016304717784%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000sdata=e%2FP3sN894F9SIYkkBSE9Q%2BFz%2FowbKUu4r1le8TgIcf4%3Dreserved=0(boolean)
>
> On April 28, 2021, julio cesar sanchez  wrote:
> > My understanding is that file urls can technically continue working,
> > but the defaults changed and some settings need to be enabled, and
> > some of them are now deprecated and will stop working at some point
> in
> > the future.
> >
> > I think defaulting to the old behavior is ok when doing minor
> releases
> > (not just on, but enforced because otherwise it's a breaking
> change),
> > but whenever we make a major release, we should always use the new
> > behavior and make the cordova user responsibility to enable the old
> > behavior if they need it, because to not "break" existing apps we
> will
> > make all new apps to face the same problem in the future.
> >
> > About the data loss, the data is not really lost, the data is still
> > there, but since the scheme and hostname change it's not accesible
> > anymore, so if the cordova users can enable the old behavior they'll
> > get the data back and should be their responsibility to do so if
> > needed and do it before releasing a new version. We should of course
> > warn about it so they will be aware if they read the release blog
> post
> > or change log.
> >
> > If cordova does the data migration is when we can screw up and do it
> > wrong and cause the massive data loss for all users, I prefer to not
> > put that responsibility into the project. You have not tried to do
> it
> > in your own apps because it's risky, would you be confident to do it
> > for all users?
> >
> > 2021-04-27 21:15 GMT+02:00, Darryl Pogue :
> > > To counter a 

RE: [DISCUSS] Moving forward with the WebView on AndroidX and cordova-android 10

[Pieter Van Poyer]

Hey

I have to agree with Bryan and Julio.
Make it opt-out with a next major release. That's my prefered option.

A lot of Cordova users will never enable this feature if it is an opt-in 
feature.
And yes it would be great if someone did already have some guidelines about 
data migration.

Kind regards
Pieter Van Poyer


-Oorspronkelijk bericht-
Van: Niklas Apache 
Verzonden: woensdag 28 april 2021 12:14
Aan: dev@cordova.apache.org
Onderwerp: Re: [DISCUSS] Moving forward with the WebView on AndroidX and 
cordova-android 10

I would agree with Bryan and Julio that we should use the opportunity of this 
breaking release to move the defaults to current Android best practices and 
remove deprecated settings.

I just started a PR [1] that introduces a preference to allow the use file URLs 
with this release. This means the deprecated settings [2][3] are only set with 
this settings as well. Darryl is right that probably many still need to run 
their app on the file protocol and they should have an easy option to do this 
with the new release. We can now push new users to avoid this and current users 
need to check the change log and enable this if they really need to.

I agree we shouldn't take the responsibilty of a data migration.

If we don't find consensus to make the WebViewAssetLoader opt-out we could make 
it opt-in for now. This would make the upgrade more safe but potentially many 
apps less secure because developers don't need to think about this change.

[1] 
https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fcordova-android%2Fpull%2F1222data=04%7C01%7CPieter.VanPoyer%40portofantwerp.com%7C3874f4566f504fbf6fca08d90a2e5037%7C2337dcc63a214d95bf72da5551a9b048%7C0%7C0%7C637552016304717784%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000sdata=yaRNsgmiLk6XqWC3NgWiceVCCF4RiEcwDzFuYzVVU%2Fg%3Dreserved=0
[2]
https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdeveloper.android.com%2Freference%2Fandroid%2Fwebkit%2FWebSettings%23setAllowUniversalAccessFromFileURLsdata=04%7C01%7CPieter.VanPoyer%40portofantwerp.com%7C3874f4566f504fbf6fca08d90a2e5037%7C2337dcc63a214d95bf72da5551a9b048%7C0%7C0%7C637552016304717784%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000sdata=ZbClSYsggi0ckZgdsSOHJ1Tq6zermY2krIB4zWjngcw%3Dreserved=0(boolean)
[3]https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdeveloper.android.com%2Freference%2Fandroid%2Fwebkit%2FWebSettings%23setAllowFileAccessdata=04%7C01%7CPieter.VanPoyer%40portofantwerp.com%7C3874f4566f504fbf6fca08d90a2e5037%7C2337dcc63a214d95bf72da5551a9b048%7C0%7C0%7C637552016304717784%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000sdata=e%2FP3sN894F9SIYkkBSE9Q%2BFz%2FowbKUu4r1le8TgIcf4%3Dreserved=0(boolean)

On April 28, 2021, julio cesar sanchez  wrote:
> My understanding is that file urls can technically continue working,
> but the defaults changed and some settings need to be enabled, and
> some of them are now deprecated and will stop working at some point in
> the future.
>
> I think defaulting to the old behavior is ok when doing minor releases
> (not just on, but enforced because otherwise it's a breaking change),
> but whenever we make a major release, we should always use the new
> behavior and make the cordova user responsibility to enable the old
> behavior if they need it, because to not "break" existing apps we will
> make all new apps to face the same problem in the future.
>
> About the data loss, the data is not really lost, the data is still
> there, but since the scheme and hostname change it's not accesible
> anymore, so if the cordova users can enable the old behavior they'll
> get the data back and should be their responsibility to do so if
> needed and do it before releasing a new version. We should of course
> warn about it so they will be aware if they read the release blog post
> or change log.
>
> If cordova does the data migration is when we can screw up and do it
> wrong and cause the massive data loss for all users, I prefer to not
> put that responsibility into the project. You have not tried to do it
> in your own apps because it's risky, would you be confident to do it
> for all users?
>
> 2021-04-27 21:15 GMT+02:00, Darryl Pogue :
> > To counter a bit, all of my apps are using the standard Cordova
> > Android WebView, and store all their data in the browser's
> indexedDB.
> > I've had no issues with file URLs (although I expect that will
> change
> > with API 30 enforcement).
> >
> > Losing data in an app update is unacceptable, and for many apps it
> > would be catastrophic (see comments from when Google did a bad
> > indexedDB migration and people lost data[1]). On iOS with WKWebView,
> > I'm stuck in the position of continuing to use file URLs to keep
> > existing data working because there's no supported path for data
> > 

Re: [DISCUSS] Moving forward with the WebView on AndroidX and cordova-android 10

[Niklas Apache]

I would agree with Bryan and Julio that we should use the opportunity of
this breaking release to move the defaults to current Android best
practices and remove deprecated settings.

I just started a PR [1] that introduces a preference to allow the use
file URLs with this release. This means the deprecated settings [2][3]
are only set with this settings as well. Darryl is right that probably
many still need to run their app on the file protocol and they should
have an easy option to do this with the new release. We can now push new
users to avoid this and current users need to check the change log and
enable this if they really need to.

I agree we shouldn't take the responsibilty of a data migration. 

If we don't find consensus to make the WebViewAssetLoader opt-out we
could make it opt-in for now. This would make the upgrade more safe but
potentially many apps less secure because developers don't need to think
about this change.

[1] https://github.com/apache/cordova-android/pull/1222
[2]
https://developer.android.com/reference/android/webkit/WebSettings#setAllowUniversalAccessFromFileURLs(boolean)
[3]https://developer.android.com/reference/android/webkit/WebSettings#setAllowFileAccess(boolean)

On April 28, 2021, julio cesar sanchez  wrote:
> My understanding is that file urls can technically continue working,
> but the defaults changed and some settings need to be enabled, and
> some of them are now deprecated and will stop working at some point in
> the future.
>
> I think defaulting to the old behavior is ok when doing minor releases
> (not just on, but enforced because otherwise it’s a breaking change),
> but whenever we make a major release, we should always use the new
> behavior and make the cordova user responsibility to enable the old
> behavior if they need it, because to not “break” existing apps we will
> make all new apps to face the same problem in the future.
>
> About the data loss, the data is not really lost, the data is still
> there, but since the scheme and hostname change it’s not accesible
> anymore, so if the cordova users can enable the old behavior they’ll
> get the data back and should be their responsibility to do so if
> needed and do it before releasing a new version. We should of course
> warn about it so they will be aware if they read the release blog post
> or change log.
>
> If cordova does the data migration is when we can screw up and do it
> wrong and cause the massive data loss for all users, I prefer to not
> put that responsibility into the project. You have not tried to do it
> in your own apps because it’s risky, would you be confident to do it
> for all users?
>
> 2021-04-27 21:15 GMT+02:00, Darryl Pogue :
> > To counter a bit, all of my apps are using the standard Cordova
> > Android WebView, and store all their data in the browser's
> indexedDB.
> > I've had no issues with file URLs (although I expect that will
> change
> > with API 30 enforcement).
> >
> > Losing data in an app update is unacceptable, and for many apps it
> > would be catastrophic (see comments from when Google did a bad
> > indexedDB migration and people lost data[1]). On iOS with WKWebView,
> > I'm stuck in the position of continuing to use file URLs to keep
> > existing data working because there's no supported path for data
> > migration to the new scheme.
> >
> > We made the decision with Cordova iOS to use file URLs by default to
> > ensure that we didn't unexpectedly break existing apps.
> >
> > We either need to default to using file URLs on Android (which is
> > risky considering the API 30 enforcement is restricting what works
> > there), or provide an officially supported update path that
> preserves
> > and migrates all the relevant data. (If we opt for the migration
> path,
> > it would be nice to have it support iOS as well.)
> >
> > If we release a Cordova update that causes a bunch of existing
> Android
> > apps to lose all their data, it will very likely be a public image
> > disaster from which we will never recover.
> >
> > [1] https://bugs.chromium.org/p/chromium/issues/detail?id=1033655
> >
> > On Tue, Apr 27, 2021 at 8:48 AM Bryan Ellis 
> wrote:
> >>
> >> I also agree.
> >>
> >> I think we should move forward with these changes and use the
> >> WebViewAssetLoader by default.
> >>
> >> If must, we could write a blog post explaining how to use Norman's
> plugin
> >> for data migration. But I do not know if the plugin is complete to
> cover
> >> all data sources and fits this case.
> >>
> >> I believe the change though is necessary.
> >>
> >> Starting from API 30, Google has disabled file access to the
> WebView
> >> because it introduces possible security risks.
> >>
> >> > Apps should not open file:// URLs from any external source in
> WebView,
> >> don't enable this if your app accepts arbitrary URLs from external
> >> sources.
> >>
> >> In our recent release of Cordova-Android, we explicitly set the
> >> `setAllowFileAccess` to `true` to get around the change that 

Re: [DISCUSS] Moving forward with the WebView on AndroidX and cordova-android 10

[julio cesar sanchez]

My understanding is that file urls can technically continue working,
but the defaults changed and some settings need to be enabled, and
some of them are now deprecated and will stop working at some point in
the future.

I think defaulting to the old behavior is ok when doing minor releases
(not just on, but enforced because otherwise it’s a breaking change),
but whenever we make a major release, we should always use the new
behavior and make the cordova user responsibility to enable the old
behavior if they need it, because to not “break” existing apps we will
make all new apps to face the same problem in the future.

About the data loss, the data is not really lost, the data is still
there, but since the scheme and hostname change it’s not accesible
anymore, so if the cordova users can enable the old behavior they’ll
get the data back and should be their responsibility to do so if
needed and do it before releasing a new version. We should of course
warn about it so they will be aware if they read the release blog post
or change log.

If cordova does the data migration is when we can screw up and do it
wrong and cause the massive data loss for all users, I prefer to not
put that responsibility into the project. You have not tried to do it
in your own apps because it’s risky, would you be confident to do it
for all users?

2021-04-27 21:15 GMT+02:00, Darryl Pogue :
> To counter a bit, all of my apps are using the standard Cordova
> Android WebView, and store all their data in the browser's indexedDB.
> I've had no issues with file URLs (although I expect that will change
> with API 30 enforcement).
>
> Losing data in an app update is unacceptable, and for many apps it
> would be catastrophic (see comments from when Google did a bad
> indexedDB migration and people lost data[1]). On iOS with WKWebView,
> I'm stuck in the position of continuing to use file URLs to keep
> existing data working because there's no supported path for data
> migration to the new scheme.
>
> We made the decision with Cordova iOS to use file URLs by default to
> ensure that we didn't unexpectedly break existing apps.
>
> We either need to default to using file URLs on Android (which is
> risky considering the API 30 enforcement is restricting what works
> there), or provide an officially supported update path that preserves
> and migrates all the relevant data. (If we opt for the migration path,
> it would be nice to have it support iOS as well.)
>
> If we release a Cordova update that causes a bunch of existing Android
> apps to lose all their data, it will very likely be a public image
> disaster from which we will never recover.
>
> [1] https://bugs.chromium.org/p/chromium/issues/detail?id=1033655
>
> On Tue, Apr 27, 2021 at 8:48 AM Bryan Ellis  wrote:
>>
>> I also agree.
>>
>> I think we should move forward with these changes and use the
>> WebViewAssetLoader by default.
>>
>> If must, we could write a blog post explaining how to use Norman's plugin
>> for data migration. But I do not know if the plugin is complete to cover
>> all data sources and fits this case.
>>
>> I believe the change though is necessary.
>>
>> Starting from API 30, Google has disabled file access to the WebView
>> because it introduces possible security risks.
>>
>>  > Apps should not open file:// URLs from any external source in WebView,
>> don't enable this if your app accepts arbitrary URLs from external
>> sources.
>>
>> In our recent release of Cordova-Android, we explicitly set the
>> `setAllowFileAccess` to `true` to get around the change that came in the
>> API 30 release. This allowed apps to continue working temporarily, while
>> we would introduce a proper solution in this coming major release,
>> preferably following a secure implementation. I believe we should not
>> default to something that has been publicly announced and known to lead to
>> potential security risks.
>>
>> If we want to support the file scheme to allow users to avoid data loss, I
>> think a config.xml flag can be introduced that users can manually set if
>> they are willing to accept the potential security risks that exist with
>> it. And it could allow them to move over whenever they decide.
>>
>>
>>
>> > On Apr 27, 2021, at 9:07 PM, julio cesar sanchez
>> >  wrote:
>> >
>> > I would vote for defaulting to WebViewAssetLoader but still allow using
>> > file:// from a config.xml preference for the people that are not ready
>> > to
>> > move on.
>> > But on cordova-ios 6 I think we ended up defaulting to file:// and use
>> > the
>> > schemes only as opt-in.
>> >
>> > About migrating data, I don't think that's our job, but we can point
>> > users
>> > to plugins if you know some.
>> >
>> > El mar, 27 abr 2021 a las 8:03, Niklas Apache ()
>> > escribió:
>> >
>> >> Hey folks,
>> >>
>> >> we recently merged a PR [1] which significantly changes how cordova-
>> >> android loads web content in the webview and now need to decide how to
>> >> move proceed.
>> >>
>> >> Google 

Re: [DISCUSS] Moving forward with the WebView on AndroidX and cordova-android 10

[Darryl Pogue]

To counter a bit, all of my apps are using the standard Cordova
Android WebView, and store all their data in the browser's indexedDB.
I've had no issues with file URLs (although I expect that will change
with API 30 enforcement).

Losing data in an app update is unacceptable, and for many apps it
would be catastrophic (see comments from when Google did a bad
indexedDB migration and people lost data[1]). On iOS with WKWebView,
I'm stuck in the position of continuing to use file URLs to keep
existing data working because there's no supported path for data
migration to the new scheme.

We made the decision with Cordova iOS to use file URLs by default to
ensure that we didn't unexpectedly break existing apps.

We either need to default to using file URLs on Android (which is
risky considering the API 30 enforcement is restricting what works
there), or provide an officially supported update path that preserves
and migrates all the relevant data. (If we opt for the migration path,
it would be nice to have it support iOS as well.)

If we release a Cordova update that causes a bunch of existing Android
apps to lose all their data, it will very likely be a public image
disaster from which we will never recover.

[1] https://bugs.chromium.org/p/chromium/issues/detail?id=1033655

On Tue, Apr 27, 2021 at 8:48 AM Bryan Ellis  wrote:
>
> I also agree.
>
> I think we should move forward with these changes and use the 
> WebViewAssetLoader by default.
>
> If must, we could write a blog post explaining how to use Norman's plugin for 
> data migration. But I do not know if the plugin is complete to cover all data 
> sources and fits this case.
>
> I believe the change though is necessary.
>
> Starting from API 30, Google has disabled file access to the WebView because 
> it introduces possible security risks.
>
>  > Apps should not open file:// URLs from any external source in WebView, 
> don't enable this if your app accepts arbitrary URLs from external sources.
>
> In our recent release of Cordova-Android, we explicitly set the 
> `setAllowFileAccess` to `true` to get around the change that came in the API 
> 30 release. This allowed apps to continue working temporarily, while we would 
> introduce a proper solution in this coming major release, preferably 
> following a secure implementation. I believe we should not default to 
> something that has been publicly announced and known to lead to potential 
> security risks.
>
> If we want to support the file scheme to allow users to avoid data loss, I 
> think a config.xml flag can be introduced that users can manually set if they 
> are willing to accept the potential security risks that exist with it. And it 
> could allow them to move over whenever they decide.
>
>
>
> > On Apr 27, 2021, at 9:07 PM, julio cesar sanchez  
> > wrote:
> >
> > I would vote for defaulting to WebViewAssetLoader but still allow using
> > file:// from a config.xml preference for the people that are not ready to
> > move on.
> > But on cordova-ios 6 I think we ended up defaulting to file:// and use the
> > schemes only as opt-in.
> >
> > About migrating data, I don't think that's our job, but we can point users
> > to plugins if you know some.
> >
> > El mar, 27 abr 2021 a las 8:03, Niklas Apache ()
> > escribió:
> >
> >> Hey folks,
> >>
> >> we recently merged a PR [1] which significantly changes how cordova-
> >> android loads web content in the webview and now need to decide how to
> >> move proceed.
> >>
> >> Google introduced the WebViewAssetLoader to make it possible to use web
> >> content from a standard http(s) scheme instead of file:. This was done
> >> to remove security risks [2] and some apps with routing frameworks like
> >> React and Angular need this for proper routing.
> >>
> >> Because cordova-android 10 now uses AndroidX we could implement the
> >> WebViewAssetLoader and remove some deprecated or security related
> >> WebSettings and move the platform forward to current Android standards.
> >>
> >> This change may break some apps now because the origin changes if the
> >> app now runs on https://localhost for example instead of file://.
> >> Changing the origin means losing access to web storage like
> >> localstorage, indexedb etc. First and foremost we need to announce that
> >> change with the release for developers to act but additionally we could
> >> do:
> >>
> >> 1.) Default back to file:// and make the WebViewAssetLoader opt-in via
> >> config.xml. This exposes apps to the security risk:
> >>
> >>> Note: Apps should not open file:// URLs from any external source in
> >> WebView, don't enable this if your app accepts arbitrary URLs from
> >> external sources. It's recommended to always use
> >> androidx.webkit.WebViewAssetLoader
> >> <
> >> https://developer.android.com/reference/androidx/webkit/WebViewAssetLoader
> >>>
> >> to access files including assets and resources over http(s):// schemes,
> >> instead of file:// URLs. To prevent possible security issues targeting

Re: [DISCUSS] Moving forward with the WebView on AndroidX and cordova-android 10

[Bryan Ellis]

I also agree.

I think we should move forward with these changes and use the 
WebViewAssetLoader by default.

If must, we could write a blog post explaining how to use Norman's plugin for 
data migration. But I do not know if the plugin is complete to cover all data 
sources and fits this case.

I believe the change though is necessary.

Starting from API 30, Google has disabled file access to the WebView because it 
introduces possible security risks.

 > Apps should not open file:// URLs from any external source in WebView, don't 
 > enable this if your app accepts arbitrary URLs from external sources.

In our recent release of Cordova-Android, we explicitly set the 
`setAllowFileAccess` to `true` to get around the change that came in the API 30 
release. This allowed apps to continue working temporarily, while we would 
introduce a proper solution in this coming major release, preferably following 
a secure implementation. I believe we should not default to something that has 
been publicly announced and known to lead to potential security risks.

If we want to support the file scheme to allow users to avoid data loss, I 
think a config.xml flag can be introduced that users can manually set if they 
are willing to accept the potential security risks that exist with it. And it 
could allow them to move over whenever they decide.



> On Apr 27, 2021, at 9:07 PM, julio cesar sanchez  
> wrote:
> 
> I would vote for defaulting to WebViewAssetLoader but still allow using
> file:// from a config.xml preference for the people that are not ready to
> move on.
> But on cordova-ios 6 I think we ended up defaulting to file:// and use the
> schemes only as opt-in.
> 
> About migrating data, I don't think that's our job, but we can point users
> to plugins if you know some.
> 
> El mar, 27 abr 2021 a las 8:03, Niklas Apache ()
> escribió:
> 
>> Hey folks,
>> 
>> we recently merged a PR [1] which significantly changes how cordova-
>> android loads web content in the webview and now need to decide how to
>> move proceed.
>> 
>> Google introduced the WebViewAssetLoader to make it possible to use web
>> content from a standard http(s) scheme instead of file:. This was done
>> to remove security risks [2] and some apps with routing frameworks like
>> React and Angular need this for proper routing.
>> 
>> Because cordova-android 10 now uses AndroidX we could implement the
>> WebViewAssetLoader and remove some deprecated or security related
>> WebSettings and move the platform forward to current Android standards.
>> 
>> This change may break some apps now because the origin changes if the
>> app now runs on https://localhost for example instead of file://.
>> Changing the origin means losing access to web storage like
>> localstorage, indexedb etc. First and foremost we need to announce that
>> change with the release for developers to act but additionally we could
>> do:
>> 
>> 1.) Default back to file:// and make the WebViewAssetLoader opt-in via
>> config.xml. This exposes apps to the security risk:
>> 
>>> Note: Apps should not open file:// URLs from any external source in
>> WebView, don't enable this if your app accepts arbitrary URLs from
>> external sources. It's recommended to always use
>> androidx.webkit.WebViewAssetLoader
>> <
>> https://developer.android.com/reference/androidx/webkit/WebViewAssetLoader
>>> 
>> to access files including assets and resources over http(s):// schemes,
>> instead of file:// URLs. To prevent possible security issues targeting
>> Build.VERSION_CODES.Q
>> 
>> and earlier, you should explicitly set this value to false.
>> 
>> 2.) Add a migration for localstorage etc. to the platform to provide a
>> smoother transition
>> 
>> 3.) Use the WebViewAssetLoader only and don't migrate in the platform
>> but point users to a plugin that helps them to manage their migration
>> 
>> Personally I would favor to move to WebViewAssetLoader by default in
>> this breaking release to get apps up to date and adapt to Androids
>> changes. I don't know how many apps would be affected because I suspect
>> many apps are using native storage solutions (SQLite etc.) or are
>> running Ionics WebView with the https scheme already. I am doing both
>> for my apps because of the many localstorage and non https scheme issues
>> we had in the past and I suspect many did as well.
>> 
>> Cordova Android 10 needs to be released rather sooner than later so
>> please leave your feedback.
>> 
>> Thank you very much and kind regards
>> Niklas
>> 
>> [1] https://github.com/apache/cordova-android/pull/1137
>> [2]
>> 
>> https://developer.android.com/reference/android/webkit/WebSettings#setAllowFileAccess(boolean)
>> 

Re: [DISCUSS] Moving forward with the WebView on AndroidX and cordova-android 10

[julio cesar sanchez]

I would vote for defaulting to WebViewAssetLoader but still allow using
file:// from a config.xml preference for the people that are not ready to
move on.
But on cordova-ios 6 I think we ended up defaulting to file:// and use the
schemes only as opt-in.

About migrating data, I don't think that's our job, but we can point users
to plugins if you know some.

El mar, 27 abr 2021 a las 8:03, Niklas Apache ()
escribió:

> Hey folks,
>
> we recently merged a PR [1] which significantly changes how cordova-
> android loads web content in the webview and now need to decide how to
> move proceed.
>
> Google introduced the WebViewAssetLoader to make it possible to use web
> content from a standard http(s) scheme instead of file:. This was done
> to remove security risks [2] and some apps with routing frameworks like
> React and Angular need this for proper routing.
>
> Because cordova-android 10 now uses AndroidX we could implement the
> WebViewAssetLoader and remove some deprecated or security related
> WebSettings and move the platform forward to current Android standards.
>
> This change may break some apps now because the origin changes if the
> app now runs on https://localhost for example instead of file://.
> Changing the origin means losing access to web storage like
> localstorage, indexedb etc. First and foremost we need to announce that
> change with the release for developers to act but additionally we could
> do:
>
> 1.) Default back to file:// and make the WebViewAssetLoader opt-in via
> config.xml. This exposes apps to the security risk:
>
> > Note: Apps should not open file:// URLs from any external source in
> WebView, don't enable this if your app accepts arbitrary URLs from
> external sources. It's recommended to always use
>  androidx.webkit.WebViewAssetLoader
> <
> https://developer.android.com/reference/androidx/webkit/WebViewAssetLoader
> >
> to access files including assets and resources over http(s):// schemes,
> instead of file:// URLs. To prevent possible security issues targeting
> Build.VERSION_CODES.Q
> 
> and earlier, you should explicitly set this value to false.
>
> 2.) Add a migration for localstorage etc. to the platform to provide a
> smoother transition
>
> 3.) Use the WebViewAssetLoader only and don't migrate in the platform
> but point users to a plugin that helps them to manage their migration
>
> Personally I would favor to move to WebViewAssetLoader by default in
> this breaking release to get apps up to date and adapt to Androids
> changes. I don't know how many apps would be affected because I suspect
> many apps are using native storage solutions (SQLite etc.) or are
> running Ionics WebView with the https scheme already. I am doing both
> for my apps because of the many localstorage and non https scheme issues
> we had in the past and I suspect many did as well.
>
> Cordova Android 10 needs to be released rather sooner than later so
> please leave your feedback.
>
> Thank you very much and kind regards
> Niklas
>
> [1] https://github.com/apache/cordova-android/pull/1137
> [2]
>
> https://developer.android.com/reference/android/webkit/WebSettings#setAllowFileAccess(boolean)
>

[DISCUSS] Moving forward with the WebView on AndroidX and cordova-android 10

[Niklas Apache]

Hey folks,

we recently merged a PR [1] which significantly changes how cordova-
android loads web content in the webview and now need to decide how to
move proceed.

Google introduced the WebViewAssetLoader to make it possible to use web
content from a standard http(s) scheme instead of file:. This was done
to remove security risks [2] and some apps with routing frameworks like
React and Angular need this for proper routing.

Because cordova-android 10 now uses AndroidX we could implement the
WebViewAssetLoader and remove some deprecated or security related
WebSettings and move the platform forward to current Android standards.

This change may break some apps now because the origin changes if the
app now runs on https://localhost for example instead of file://.
Changing the origin means losing access to web storage like
localstorage, indexedb etc. First and foremost we need to announce that
change with the release for developers to act but additionally we could
do:

1.) Default back to file:// and make the WebViewAssetLoader opt-in via
config.xml. This exposes apps to the security risk:

> Note: Apps should not open file:// URLs from any external source in
WebView, don't enable this if your app accepts arbitrary URLs from
external sources. It's recommended to always use
 androidx.webkit.WebViewAssetLoader

to access files including assets and resources over http(s):// schemes,
instead of file:// URLs. To prevent possible security issues targeting
Build.VERSION_CODES.Q

and earlier, you should explicitly set this value to false. 

2.) Add a migration for localstorage etc. to the platform to provide a
smoother transition

3.) Use the WebViewAssetLoader only and don't migrate in the platform
but point users to a plugin that helps them to manage their migration

Personally I would favor to move to WebViewAssetLoader by default in
this breaking release to get apps up to date and adapt to Androids
changes. I don't know how many apps would be affected because I suspect
many apps are using native storage solutions (SQLite etc.) or are
running Ionics WebView with the https scheme already. I am doing both
for my apps because of the many localstorage and non https scheme issues
we had in the past and I suspect many did as well.

Cordova Android 10 needs to be released rather sooner than later so
please leave your feedback.

Thank you very much and kind regards
Niklas

[1] https://github.com/apache/cordova-android/pull/1137
[2]
https://developer.android.com/reference/android/webkit/WebSettings#setAllowFileAccess(boolean)