Re: [whatwg] Persistent and temporary storage
On Wed, Mar 18, 2015 at 1:38 AM, Krinkle krinklem...@gmail.com wrote: I'd like to share a use case and problem we have at Wikipedia with localStorage. Thanks, this is great feedback. I imagine HTTP2 might make it appropriate to phase out batches and just request modules individually (always) and let the network layer do the combining and separated caching in a more natural way. Yeah, hopefully. * A way to know if a url is cached or not (e.g. know whether a url will hit HTTP 304) without making the request. Maybe we can expose that same-origin, not sure. Depends a bit on the implementer feedback we get for fetch()' cache feature. But privacy-wise it's somewhat problematic to reveal what is in the cache as the cache is not unique per-origin. * A way to prioritise which entries should be kept in localStorage and allow for low-prio entries to be evicted if short on space. * A way to know how much localStorage is available in total. * Perhaps a way to create a limited store within localStorage or IndexDB that has limited/restricted capacity (with some unique identifier, capacity percentage-based, or a min/max byte size?). * A separate store for caching HTTP resources (the Service Worker's Cache API?) The current setup is basically a storage area per site with LRU semantics. It's not completely done yet as not all storage features share the same store, but they will eventually. Persistence is planned as per OP. We have two other ideas roughly along the lines of what you ask for: 1) Allow a site to mint new storage areas. If we keep doing LRU on storage areas rather than sites that would allow for e.g. a game engine staying preserved while the initial set of levels (one per storage area) that are no longer played are cleared. 2) A storage area that acts like a cache. Resources that are not frequently used get deleted before those that get frequently used get deleted. -- https://annevankesteren.nl/
Re: [whatwg] Persistent and temporary storage
On 18 Mar 2015, at 00:45, Karl Dubost k...@la-grange.net wrote: Hi Timo, Le 18 mars 2015 à 09:38, Krinkle krinklem...@gmail.com a écrit : 2. HTTP 304 hits are not free. We found that loading JS/CSS from LocalStorage was faster than hitting a HTTP 304. Making enough difference to justify this change. Did you publish your tests and the results somewhere across browsers and devices? This is quite interesting. We have :-) Here is the relevant commit that initially introduced the feature: https://github.com/wikimedia/mediawiki/commit/c719401661 (Note that several changes were made before the first deployment, so the diff isn't conclusive but the commit message is interesting. Refer to the relevant files in 'master' for the currently deployed implementation.) The research was done by A/B testing this feature for a small amount of our traffic (a few million page views) over the course of three days. The feature was subsequently enabled in late 2013 for all users. (That is, for user agents passing the localStorage feature test.) Published here: https://meta.wikimedia.org/wiki/Research:Module_storage_performance — Krinkle
Re: [whatwg] Persistent and temporary storage
I'd like to share a use case and problem we have at Wikipedia with localStorage. The MediaWiki software (which Wikipedia runs on) uses a framework called ResourceLoader for bundling and delivering modules to the client. [1][2] Last year it was changed to make use of localStorage in addition to optimised HTTP 304 handling. Mainly because of two issues we found: 1. Batching is bad for incremental updates. We combine requests for multiple modules in predictable batches. This means usually only 1 or 2 actual HTTP request are made for the main payload. However, when one of those module change in a deployment, that batch would no longer be same and have to be invalidated in its entirety. Causing the user to have to re-download all modules in the same batch as well. Extracting the payload client-side into individual modules put in LocalStorage allowed us to only re-request the module that changed from the server and evaluate the rest from localStorage (and update the entry afterward). This reduced bandwidth significantly and improved page load times overall. I imagine HTTP2 might make it appropriate to phase out batches and just request modules individually (always) and let the network layer do the combining and separated caching in a more natural way. 2. HTTP 304 hits are not free. We found that loading JS/CSS from LocalStorage was faster than hitting a HTTP 304. Making enough difference to justify this change. So that went wrong? Well. The nice thing about regular 304 caching is that as developers we're not worried about the size restriction of the store. Whether the browser limits this or not. Whether it's FIFO, LRU or just unlimited isn't an immediate user-visible concern (it probably should be, but that's for another discussion). When we started using localStorage, users that once visited pages with lots of functionality enabled found themselves having a full localStorage. This caused other - more essential - functionality to no longer work. E.g. Logic that previously used cookies to store small state values that were moved to localStorage (to reduce network overhead and because it made semantic sense), such as Boolean : Hide fundraising banner or Last 10 autocomplete values – no longer worked as localStorage was filled up with our faux HTTP cache for ResourceLoader. Which is unfortunate, since the module store could easily fall back to requesting from HTTP (and usually hit 304) whereas those state values would never save and cause user-visible problems and functionality not working as expected. We're working around it in different ways (some things resorted to cookies) but are still stalled on a long-term solution for this problem. We're considering to move our module store from localStorage to IndexedDB as that's not being used at the moment. It would provide the same separation as cookies/localStorage. In that localStorage would keep working even if IndexDB was full. Some thoughts: * A way to know if a url is cached or not (e.g. know whether a url will hit HTTP 304) without making the request. * A way to prioritise which entries should be kept in localStorage and allow for low-prio entries to be evicted if short on space. * A way to know how much localStorage is available in total. * Perhaps a way to create a limited store within localStorage or IndexDB that has limited/restricted capacity (with some unique identifier, capacity percentage-based, or a min/max byte size?). * A separate store for caching HTTP resources (the Service Worker's Cache API?) — Timo Tijhof Software Engineer Wikimedia Foundation PS: Sorry if this is the wrong avenue for this type of feedback. Thanks in advance. [1] https://en.wikipedia.org/wiki/MediaWiki [2] https://www.mediawiki.org/wiki/ResourceLoader/Features On 13 Mar 2015, at 12:50, Anne van Kesteren ann...@annevk.nl wrote: A big gap with native is dependable storage for applications. I started sketching the problem space on this wiki page: https://wiki.whatwg.org/wiki/Storage Feedback I got is that having some kind of allotted quota is useful for applications. That way they know how much they can put away. However, this clashes a bit with offering something that is competitive with native. We can't really ask the user to divide up their storage. And yet when the user asks an application to store e.g. a whole bunch of music offline we don't really want the user agent to get in the way if the user already granted persistence. -- https://annevankesteren.nl/
Re: [whatwg] Persistent and temporary storage
Hi Timo, Le 18 mars 2015 à 09:38, Krinkle krinklem...@gmail.com a écrit : 2. HTTP 304 hits are not free. We found that loading JS/CSS from LocalStorage was faster than hitting a HTTP 304. Making enough difference to justify this change. Did you publish your tests and the results somewhere across browsers and devices? This is quite interesting. Thanks. -- Karl Dubost http://www.la-grange.net/karl/
Re: [whatwg] Persistent and temporary storage
On 13/03/2015 21:39 , Nils Dagsson Moskopp wrote:
There exists also an issue of perverse incentives. If the browser
tells an application how much storage it can use, an application
developer is likely to try to use the maximum allowed space.
Sorry but I don't think this makes much sense. I can't think of a
developer ever writing code along the lines of:
if (device.hasLeftOverQuota()) {
// ooh! let's write us some more bytes to disk!
}
else {
// oh well, who needz this? LOL!
}
Quotas are useful to set user expectations. For instance, people who
create sites for periodicals are really interested in allowing their
users to download entire issues offline. It's very helpful if they can
say you've got enough space for 3 more issues or you're short on
space, do you want to remove some older issues.
This could also lead to web apps refusing to run if an user agent
does not report enough space.
Which is completely acceptable. If your app needs to store a large data
set locally, telling the user it won't work upfront is a much better
experience than letting them waste both time and data usage (which can
easily matter a fair bit on mobile) only to fail anyway.
--
Robin Berjon - http://berjon.com/ - @robinberjon
Re: [whatwg] Persistent and temporary storage
On Mon, Mar 16, 2015 at 1:38 AM, Anne van Kesteren ann...@annevk.nl wrote: On Fri, Mar 13, 2015 at 5:06 PM, Joshua Bell jsb...@chromium.org wrote: A handful of us working on Chrome have been having similar discussions around what we've been calling durable storage. In its simplest model a bit granted by the user to an origin, which then requires explicit user action before the data might be cleared under storage pressure, so it sounds like our thinking is broadly aligned, although we're still exploring various possibilities and their implications for permission prompts, cleanup UI, behavior under pressure, etc. Yeah, same here, wiki page outlines a tentative plan. Gotcha. And thanks again for opening up this discussion! Similarly, we've been trying to keep this orthogonal from quota (either the UA's logic for assigning a quota to an origin quota, or possible standardized quota APIs), although the UA may use similar signals for granting permissions/assigning quota. I think we've come around in that we need to expose quota in some way to give developers some expectations to how much they can fetch and then store in best effort mode. I think that matches our latest discussions too... But that for persistent it can be the whole disk. ... and we're waffling on that one. Going that far implies that the UA does a really good job on its own or with user interaction to respond when the storage is indeed getting full. Mobile OSes typically provide UI to inspect how much storage is in use and clear apps and/or portions of their storage. IMHO, we need to fully develop that UX in the UA before I'd be comfortable letting sites easily consume the whole disk. But we realize that artificially capping disk usage is a gap between web and native, and so solving that problem is high priority for us. And I don't think there are spec/standards implications here so we can move fast on the UA side, as long as we spec that QuotaExceededError can happen on various operations regardless of permissions, because even unlimited quota can be constrained by physical limits. (FYI, we've been using durable and non-durable to distance the discussion from the now-loaded temporary vs. persistent terms which surfaced in earlier API proposals, some of which are implemented in Chrome) Ah right. Current set of terms I have is best effort (default; fixed quota), persistent (requires some kind of user opt-in, probably through an API-triggered dialog, but maybe also done if you pin a tab or bookmark or some such; 'unlimited' quota), and temporary (exists outside of best effort/persistent, e.g. for storing social network resources, other volatile assets, requires some kind of API opt-in; fixed quota). If I'm reading the wiki page correctly, I'm intrigued by the temporary proposal. To confirm, you're envisioning a completely new lightweight storage API and there's no implied addition to the other storage APIs? If so... well, pros and cons. I'm not a huge fan of adding Yet Another Storage API. On the other hand, I'd rather do that then fork the existing storage APIs into temp/persistent and try and shoehorn priorities into those. If it helps I did a thought experiment a while ago on what would a stripped-down, Promise-based IDB-lite look like? at https://gist.github.com/inexorabletash/c8069c042b734519680c - it doesn't have the priority scheme, but that would be easy to add at the 'open' entry point. ... One thing we should discuss under the storage umbrella is how atomically we treat all storage for an origin. Customers we've talked to acknowledge the reality that even durable storage can be wiped in the face of user action (e.g. via settings UI to clear cookies etc) or file corruption. One of the situations they're concerned about is dealing with partial clearing of data, e.g. Indexed DB databases are present but the SW cache has been wiped, or vice versa. Currently, for quota-based storage eviction, we evict an origin's entire storage at once - that's easiest for sites to reason about, since it matches the first time user or returning user on new device scenarios that must already be supported. If we're taking a step back to think of storage as a whole, we may want to provide more spec-level assurance in this area. -- https://annevankesteren.nl/
Re: [whatwg] Persistent and temporary storage
On Mon, Mar 16, 2015 at 5:23 PM, Joshua Bell jsb...@chromium.org wrote: On Mon, Mar 16, 2015 at 1:38 AM, Anne van Kesteren ann...@annevk.nl wrote: But that for persistent it can be the whole disk. ... and we're waffling on that one. Going that far implies that the UA does a really good job on its own or with user interaction to respond when the storage is indeed getting full. Mobile OSes typically provide UI to inspect how much storage is in use and clear apps and/or portions of their storage. IMHO, we need to fully develop that UX in the UA before I'd be comfortable letting sites easily consume the whole disk. Yeah, getting better UX/UI is a prerequisite for us as well. Firefox has a page about:permissions today. Something like that, but more easily accessible and without its issues would go some way towards addressing this. And perhaps making sure the user discovers it when they are about to run out of disk space... If I'm reading the wiki page correctly, I'm intrigued by the temporary proposal. To confirm, you're envisioning a completely new lightweight storage API and there's no implied addition to the other storage APIs? If so... well, pros and cons. I'm not a huge fan of adding Yet Another Storage API. On the other hand, I'd rather do that then fork the existing storage APIs into temp/persistent and try and shoehorn priorities into those. I think that's really v2 material anyway. Another idea we had is having named storage areas and then assigning priorities to those, with initially the whole site being in a single area. If it helps I did a thought experiment a while ago on what would a stripped-down, Promise-based IDB-lite look like? at https://gist.github.com/inexorabletash/c8069c042b734519680c - it doesn't have the priority scheme, but that would be easy to add at the 'open' entry point. Yeah, this is also something that warrants more discussion. There's cookies, localStorage, indexed DB, various filesystem APIs, notifications API, history API, and indeed always the question whether we want a nicer indexed DB / asynchronous localStorage. Also seems like v2 material. Something we should consider after bridging the persistence gap. One thing we should discuss under the storage umbrella is how atomically we treat all storage for an origin. Customers we've talked to acknowledge the reality that even durable storage can be wiped in the face of user action (e.g. via settings UI to clear cookies etc) or file corruption. One of the situations they're concerned about is dealing with partial clearing of data, e.g. Indexed DB databases are present but the SW cache has been wiped, or vice versa. Currently, for quota-based storage eviction, we evict an origin's entire storage at once - that's easiest for sites to reason about, since it matches the first time user or returning user on new device scenarios that must already be supported. If we're taking a step back to think of storage as a whole, we may want to provide more spec-level assurance in this area. Yeah, I think until we'd introduce something like named storage areas or temporary storage APIs we should treat everything under the best-effort / persistent banner as atomic. We should probably define that in the specification that outlines this storage architecture and then the various storage APIs we have can tie into it. -- https://annevankesteren.nl/
Re: [whatwg] Persistent and temporary storage
On Fri, Mar 13, 2015 at 3:25 PM, Janusz Majnert j.majn...@samsung.com wrote: On 13.03.2015 15:01, Anne van Kesteren wrote: The reason developers want it is to know how much they can download and store without getting an exception. Which still doesn't guarantee they won't get an exception if the device runs out of space for whatever reason. That is true, it would be an estimate (and I think we should name it as such in the API). With deduplication and compression it can't be super accurate. It's still better to reason with an estimate than with nothing at all though, I think. -- https://annevankesteren.nl/
Re: [whatwg] Persistent and temporary storage
On Fri, Mar 13, 2015 at 5:06 PM, Joshua Bell jsb...@chromium.org wrote: A handful of us working on Chrome have been having similar discussions around what we've been calling durable storage. In its simplest model a bit granted by the user to an origin, which then requires explicit user action before the data might be cleared under storage pressure, so it sounds like our thinking is broadly aligned, although we're still exploring various possibilities and their implications for permission prompts, cleanup UI, behavior under pressure, etc. Yeah, same here, wiki page outlines a tentative plan. Similarly, we've been trying to keep this orthogonal from quota (either the UA's logic for assigning a quota to an origin quota, or possible standardized quota APIs), although the UA may use similar signals for granting permissions/assigning quota. I think we've come around in that we need to expose quota in some way to give developers some expectations to how much they can fetch and then store in best effort mode. But that for persistent it can be the whole disk. (FYI, we've been using durable and non-durable to distance the discussion from the now-loaded temporary vs. persistent terms which surfaced in earlier API proposals, some of which are implemented in Chrome) Ah right. Current set of terms I have is best effort (default; fixed quota), persistent (requires some kind of user opt-in, probably through an API-triggered dialog, but maybe also done if you pin a tab or bookmark or some such; 'unlimited' quota), and temporary (exists outside of best effort/persistent, e.g. for storing social network resources, other volatile assets, requires some kind of API opt-in; fixed quota). -- https://annevankesteren.nl/
Re: [whatwg] Persistent and temporary storage
On Mon, Mar 16, 2015 at 4:12 AM, Biju bijumaill...@gmail.com wrote: At present data stored in indexDB is written some where deep in the profile folder, which is difficult to find. I don't think we should expect our users to traverse the directory structure at all. We need to expose UI to manage sites in about:preferences or equivalent. Additionally if we can restrict these kind of file/storage access to just https site (or some sort of signing mechanism for installed app),, it will help reduce annoyance of asking for permission by ever other site we visit. I suspect this will be HTTPS restricted simply because it is about persistence, but soon every site will have easy access to HTTPS. So if indeed all sites end up asking for this we would need to think of a better model, but I doubt that will be the case. -- https://annevankesteren.nl/
Re: [whatwg] Persistent and temporary storage
2015-03-13 21:39 GMT+01:00 Nils Dagsson Moskopp n...@dieweltistgarnichtso.net: Janusz Majnert j.majn...@samsung.com writes: On 13.03.2015 15:01, Anne van Kesteren wrote: On Fri, Mar 13, 2015 at 2:58 PM, Janusz Majnert j.majn...@samsung.com wrote: The real question is why having a quota is useful? The reason developers want it is to know how much they can download and store without getting an exception. Which still doesn't guarantee they won't get an exception if the device runs out of space for whatever reason. There exists also an issue of perverse incentives. If the browser tells an application how much storage it can use, an application developer is likely to try to use the maximum allowed space. This could also lead to web apps refusing to run if an user agent does not report enough space. Just another reason not to expose quota information, but instead have heuristics to calculate reasonable limit. When app reaches this limit user should be nudged (not prompted) by UA to investigate and/or set a hard limit, delete data, etc. The heuristics may take into account: * how often the site is visited * is the site bookmarked or installed (via W3C Manifest for web application) * does the app have associated service workers (indication that the app might actually use the downloaded data for some offline experience) * how much space is left on device * what kind of device: desktop with broadband or mobile with GPRS etc. One solution I know that tries to deal with that is Linux's OOM killer: If you go over the quota, your program is likely to be eaten by a grue. Native apps are not controlled when it comes to storing data and nobody complains. Is there any documentation on how they handle the above scenario? Just write to disk until you hit failure? I think so. This is certainly the case with desktop apps. I also didn't find any mention of quota in Android download manager docs (http://developer.android.com/reference/android/app/DownloadManager.html) or in Tizen's Download API (https://developer.tizen.org/dev-guide/2.3.0/org.tizen.mobile.native.apireference/group__CAPI__WEB__DOWNLOAD__MODULE.html) Regards, -- Janusz Majnert Senior Software Engineer Samsung RD Institute Poland Samsung Electronics -- Nils Dagsson Moskopp // erlehmann http://dieweltistgarnichtso.net
Re: [whatwg] Persistent and temporary storage
At present data stored in indexDB is written some where deep in the
profile folder, which is difficult to find.
Instead, if the web application files are stored in
\Users\{windows_userid}\WebApps\www.mycompany.com\ it will be easy to
navigate and be managed by user.
UA should only allow web app to write/read after getting user consent.
If UA find the \WebApps\www.mycompany.com\ folder missing, user should
be asked consent again.
A subdomain should be able to write to a parent domain after changing
its document domain, according security rules.
Additionally if we can restrict these kind of file/storage access to
just https site (or some sort of signing mechanism for installed
app),, it will help reduce annoyance of asking for permission by ever
other site we visit.
Cheers
GC
Re: [whatwg] Persistent and temporary storage
On 15 Mar 2015 03:35, Glenn Maynard gl...@zewt.org wrote: On Fri, Mar 13, 2015 at 3:13 PM, Silvia Pfeiffer silviapfeiff...@gmail.com wrote: On 14 Mar 2015 05:49, Tab Atkins Jr. jackalm...@gmail.com wrote: Users install a relatively small number of apps, and the uninstall flow (which deletes their storage) is also trivial. Users visit a relatively large number of web-pages (and even more distinct origins, due to iframes and ads), and we don't have any good notion of uninstall yet on the web; the existing flows for deleting storage are terrible. First you need a notion of install. Not having to install web pages is a feature, not a bug. In fact, it's one of the defining features of the platform. Sure, but you can't uninstall something that hasn't first been installed. Silvia.
Re: [whatwg] Persistent and temporary storage
13 mar 2015 21:13 Silvia Pfeiffer silviapfeiff...@gmail.com napisał(a): On 14 Mar 2015 05:49, Tab Atkins Jr. jackalm...@gmail.com wrote: On Fri, Mar 13, 2015 at 6:58 AM, Janusz Majnert j.majn...@samsung.com wrote: On 13.03.2015 13:50, Anne van Kesteren wrote: A big gap with native is dependable storage for applications. I started sketching the problem space on this wiki page: https://wiki.whatwg.org/wiki/Storage Feedback I got is that having some kind of allotted quota is useful for applications. That way they know how much they can put away. However, this clashes a bit with offering something that is competitive with native. We can't really ask the user to divide up their storage. And yet when the user asks an application to store e.g. a whole bunch of music offline we don't really want the user agent to get in the way if the user already granted persistence. The real question is why having a quota is useful? Native apps are not controlled when it comes to storing data and nobody complains. Users install a relatively small number of apps, and the uninstall flow (which deletes their storage) is also trivial. Users visit a relatively large number of web-pages (and even more distinct origins, due to iframes and ads), and we don't have any good notion of uninstall yet on the web; the existing flows for deleting storage are terrible. First you need a notion of install. On an android KitKat, open browser tabs are listed in the same way as open apps, which is a first step. Should bookmarks and desktop icons be unified in a second step to indicate installation? Then, closing the tab of a non-bookmarked app would indicate ability to remove local storage (implicit uninstall, but still following typical browser caching strategies). Removing the bookmark/desktop icon would indicate then indicate explicit uninstall. There's ongoing work on W3C Manifest for web applications ( https://w3c.github.io/manifest/) which introduces the notion of installation for web apps. So this bit is covered. Cheers, Silvia. I think proper solution would be not to restrict the available space, but provide GUI for users to: * see how much space an app uses (if it exceeds some preset amount) * inspect the files in platform's file explorer Yeah, some improved UI flows along these lines would be hugely helpful for this kind of thing. ~TJ
Re: [whatwg] Persistent and temporary storage
On Fri, Mar 13, 2015 at 3:13 PM, Silvia Pfeiffer silviapfeiff...@gmail.com wrote: On 14 Mar 2015 05:49, Tab Atkins Jr. jackalm...@gmail.com wrote: Users install a relatively small number of apps, and the uninstall flow (which deletes their storage) is also trivial. Users visit a relatively large number of web-pages (and even more distinct origins, due to iframes and ads), and we don't have any good notion of uninstall yet on the web; the existing flows for deleting storage are terrible. First you need a notion of install. Not having to install web pages is a feature, not a bug. In fact, it's one of the defining features of the platform. -- Glenn Maynard
Re: [whatwg] Persistent and temporary storage
On 14 Mar 2015 05:49, Tab Atkins Jr. jackalm...@gmail.com wrote: On Fri, Mar 13, 2015 at 6:58 AM, Janusz Majnert j.majn...@samsung.com wrote: On 13.03.2015 13:50, Anne van Kesteren wrote: A big gap with native is dependable storage for applications. I started sketching the problem space on this wiki page: https://wiki.whatwg.org/wiki/Storage Feedback I got is that having some kind of allotted quota is useful for applications. That way they know how much they can put away. However, this clashes a bit with offering something that is competitive with native. We can't really ask the user to divide up their storage. And yet when the user asks an application to store e.g. a whole bunch of music offline we don't really want the user agent to get in the way if the user already granted persistence. The real question is why having a quota is useful? Native apps are not controlled when it comes to storing data and nobody complains. Users install a relatively small number of apps, and the uninstall flow (which deletes their storage) is also trivial. Users visit a relatively large number of web-pages (and even more distinct origins, due to iframes and ads), and we don't have any good notion of uninstall yet on the web; the existing flows for deleting storage are terrible. First you need a notion of install. On an android KitKat, open browser tabs are listed in the same way as open apps, which is a first step. Should bookmarks and desktop icons be unified in a second step to indicate installation? Then, closing the tab of a non-bookmarked app would indicate ability to remove local storage (implicit uninstall, but still following typical browser caching strategies). Removing the bookmark/desktop icon would indicate then indicate explicit uninstall. Cheers, Silvia. I think proper solution would be not to restrict the available space, but provide GUI for users to: * see how much space an app uses (if it exceeds some preset amount) * inspect the files in platform's file explorer Yeah, some improved UI flows along these lines would be hugely helpful for this kind of thing. ~TJ
Re: [whatwg] Persistent and temporary storage
Janusz Majnert j.majn...@samsung.com writes: On 13.03.2015 15:01, Anne van Kesteren wrote: On Fri, Mar 13, 2015 at 2:58 PM, Janusz Majnert j.majn...@samsung.com wrote: The real question is why having a quota is useful? The reason developers want it is to know how much they can download and store without getting an exception. Which still doesn't guarantee they won't get an exception if the device runs out of space for whatever reason. There exists also an issue of perverse incentives. If the browser tells an application how much storage it can use, an application developer is likely to try to use the maximum allowed space. This could also lead to web apps refusing to run if an user agent does not report enough space. One solution I know that tries to deal with that is Linux's OOM killer: If you go over the quota, your program is likely to be eaten by a grue. Native apps are not controlled when it comes to storing data and nobody complains. Is there any documentation on how they handle the above scenario? Just write to disk until you hit failure? I think so. This is certainly the case with desktop apps. I also didn't find any mention of quota in Android download manager docs (http://developer.android.com/reference/android/app/DownloadManager.html) or in Tizen's Download API (https://developer.tizen.org/dev-guide/2.3.0/org.tizen.mobile.native.apireference/group__CAPI__WEB__DOWNLOAD__MODULE.html) Regards, -- Janusz Majnert Senior Software Engineer Samsung RD Institute Poland Samsung Electronics -- Nils Dagsson Moskopp // erlehmann http://dieweltistgarnichtso.net
[whatwg] Persistent and temporary storage
A big gap with native is dependable storage for applications. I started sketching the problem space on this wiki page: https://wiki.whatwg.org/wiki/Storage Feedback I got is that having some kind of allotted quota is useful for applications. That way they know how much they can put away. However, this clashes a bit with offering something that is competitive with native. We can't really ask the user to divide up their storage. And yet when the user asks an application to store e.g. a whole bunch of music offline we don't really want the user agent to get in the way if the user already granted persistence. -- https://annevankesteren.nl/
Re: [whatwg] Persistent and temporary storage
On Fri, Mar 13, 2015 at 2:58 PM, Janusz Majnert j.majn...@samsung.com wrote: The real question is why having a quota is useful? The reason developers want it is to know how much they can download and store without getting an exception. Native apps are not controlled when it comes to storing data and nobody complains. Is there any documentation on how they handle the above scenario? Just write to disk until you hit failure? I think proper solution would be not to restrict the available space, but provide GUI for users to: * see how much space an app uses (if it exceeds some preset amount) * inspect the files in platform's file explorer Agreed that we need much more solid UI for users to control sites. I would expect the granularity to be more along the lines of delete all initially, though. -- https://annevankesteren.nl/
Re: [whatwg] Persistent and temporary storage
On 13.03.2015 13:50, Anne van Kesteren wrote: A big gap with native is dependable storage for applications. I started sketching the problem space on this wiki page: https://wiki.whatwg.org/wiki/Storage Feedback I got is that having some kind of allotted quota is useful for applications. That way they know how much they can put away. However, this clashes a bit with offering something that is competitive with native. We can't really ask the user to divide up their storage. And yet when the user asks an application to store e.g. a whole bunch of music offline we don't really want the user agent to get in the way if the user already granted persistence. The real question is why having a quota is useful? Native apps are not controlled when it comes to storing data and nobody complains. I think proper solution would be not to restrict the available space, but provide GUI for users to: * see how much space an app uses (if it exceeds some preset amount) * inspect the files in platform's file explorer Regards, -- Janusz Majnert Senior Software Engineer Samsung RD Institute Poland Samsung Electronics
Re: [whatwg] Persistent and temporary storage
On 13.03.2015 15:01, Anne van Kesteren wrote: On Fri, Mar 13, 2015 at 2:58 PM, Janusz Majnert j.majn...@samsung.com wrote: The real question is why having a quota is useful? The reason developers want it is to know how much they can download and store without getting an exception. Which still doesn't guarantee they won't get an exception if the device runs out of space for whatever reason. Native apps are not controlled when it comes to storing data and nobody complains. Is there any documentation on how they handle the above scenario? Just write to disk until you hit failure? I think so. This is certainly the case with desktop apps. I also didn't find any mention of quota in Android download manager docs (http://developer.android.com/reference/android/app/DownloadManager.html) or in Tizen's Download API (https://developer.tizen.org/dev-guide/2.3.0/org.tizen.mobile.native.apireference/group__CAPI__WEB__DOWNLOAD__MODULE.html) Regards, -- Janusz Majnert Senior Software Engineer Samsung RD Institute Poland Samsung Electronics
Re: [whatwg] Persistent and temporary storage
Very timely! A handful of us working on Chrome have been having similar discussions around what we've been calling durable storage. In its simplest model a bit granted by the user to an origin, which then requires explicit user action before the data might be cleared under storage pressure, so it sounds like our thinking is broadly aligned, although we're still exploring various possibilities and their implications for permission prompts, cleanup UI, behavior under pressure, etc. Similarly, we've been trying to keep this orthogonal from quota (either the UA's logic for assigning a quota to an origin quota, or possible standardized quota APIs), although the UA may use similar signals for granting permissions/assigning quota. (FYI, we've been using durable and non-durable to distance the discussion from the now-loaded temporary vs. persistent terms which surfaced in earlier API proposals, some of which are implemented in Chrome) On Fri, Mar 13, 2015 at 7:25 AM, Janusz Majnert j.majn...@samsung.com wrote: On 13.03.2015 15:01, Anne van Kesteren wrote: On Fri, Mar 13, 2015 at 2:58 PM, Janusz Majnert j.majn...@samsung.com wrote: The real question is why having a quota is useful? The reason developers want it is to know how much they can download and store without getting an exception. Which still doesn't guarantee they won't get an exception if the device runs out of space for whatever reason. Native apps are not controlled when it comes to storing data and nobody complains. Is there any documentation on how they handle the above scenario? Just write to disk until you hit failure? I think so. This is certainly the case with desktop apps. I also didn't find any mention of quota in Android download manager docs ( http://developer.android.com/reference/android/app/DownloadManager.html) or in Tizen's Download API (https://developer.tizen.org/ dev-guide/2.3.0/org.tizen.mobile.native.apireference/ group__CAPI__WEB__DOWNLOAD__MODULE.html) Regards, -- Janusz Majnert Senior Software Engineer Samsung RD Institute Poland Samsung Electronics
