[IndexedDB] Granting storage quotas

[Mark Seaborn]

Is there any plan for involving the user in storage allocation decisions for
IndexedDB? [1]

For comparison, the WebStorage API [2] doesn't have any special support for
the user to make allocation choices.  My understanding is that browsers have
a fixed storage limit per origin -- in Chromium, 5Mb per origin.  The
problem with this model is that it is both too permissive and too
restrictive.

 1) It is too restrictive because a user might want to grant a web app a
large quantity of storage -- perhaps gigabytes.  (See use cases below.)

 2) It is too permissive because it enforces no limit on the amount of space
a web app can use:  A web app from example.com can create an unlimited
number of puppet subdomains:  aaa.example.com, bbb.example.com, etc.  It can
use aaa.example.com's 5Mb allocation by loading a script from
aaa.example.com in an iframe and communicating with it using postMessage().

Both WebStorage and and IndexedDB have sections titled Blocking third-party
storage which permit browsers to disallow access to local storage from
iframes.  This could *potentially* address (2), although the specs frame
this as a privacy issue rather than a resource allocation issue.  However,
if browsers opted to block local storage access in iframes, this would
reduce composability of web apps, because WebStorage/IndexedDB provide no
other way for one site to delegate access to its storage to another site.
Furthermore, leaving this unspecified dilutes the value of having a spec,
because there would be no behaviour that all web apps can depend on.


My proposal:

Add a requestQuota() interface to IndexedDB that allows a web app to ask for
an amount of storage.

The form of the request affects how it can be displayed to the user:

 * Suppose this is a Javascript method call, e.g.
  indexedDB.requestQuota(size_in_megabytes)
   The request could be displayed as an info bar, which would be
   displayed asynchronously, i.e. without the user clicking.

 * Alternatively, the request could be attached to a DOM element
   (e.g. an input element), which would be displayed as a button.
   Clicking the button opens a dialog.

The dialog would display the amount of storage that the web app has
requested.  The user can change the amount of storage, or grant the request
without change.  The dialog might also display how much storage is currently
available on the user's device.

A site's quota would be zero by default, otherwise a web app can multiply
any non-zero limit by the number of domains is can invent, as described
above.

A quota would be an upper limit on the space a site can use, but it would
not be a guarantee that the space is available.


Use cases:

* An e-mail web app requests an amount of storage that is large enough to
store all your current e-mail, plus your e-mail for the next year at
projected rates.  As this runs out, it can request more.

* A backup web app requests an amount that is large enough to store your
data.

Mark

[1] http://www.w3.org/TR/IndexedDB/
[2] http://dev.w3.org/html5/webstorage/

Re: [IndexedDB] Granting storage quotas

[Mark Seaborn]

On Tue, Apr 13, 2010 at 4:53 PM, João Eiras jo...@opera.com wrote:

 On Tue, 13 Apr 2010 12:09:14 +0200, Mark Seaborn mseab...@chromium.org
 wrote:

  Is there any plan for involving the user in storage allocation decisions
 for
 IndexedDB? [1]

 For comparison, the WebStorage API [2] doesn't have any special support
 for
 the user to make allocation choices.  My understanding is that browsers
 have
 a fixed storage limit per origin -- in Chromium, 5Mb per origin.  The
 problem with this model is that it is both too permissive and too
 restrictive.


 Not really. The user agent can ask for quota from the user when the limit
 is being hit without the webpage even having to worry about it. Opera 10.50
 does that.


I tried this out in Opera with a script that doubles the amount of storage
it uses on each reload [1].  I got a dialog saying:

Web Storage Quota

foo.com wants to store data on your computer, but needs more space.

Current limit: 2.7 MB

Requested limit: 5.4 MB
[this is a dropdown, also containing:
8.1 MB
10.7 MB
21.4 MB
42.7 MB
Unlimited]

[Allow] [Reject]

This is still too restrictive because:

1) It doesn't allow a web app to ask for a storage allocation up front,
before it starts to consume the storage.
2) In Opera, the quota can only be increased in multiples of about 15, so it
takes three prompts to get up into the range of gigabytes.
3) The web app can't choose when the question is put to the user.
4) The web app doesn't know how much storage has been allocated, so it
doesn't know when a question will be asked.
5) In Opera, if the user chooses Reject, they don't get prompted again.
This means that asking the user at an appropriate time is important for the
continued functioning of the web app.  Prompting the user at the wrong time
will interrupt them with a page-modal dialog which they might want to get
rid of with Reject, which would potentially break the web app by leaving
it unable to get more storage.

The workaround for (1) is to write junk data into local storage in an
attempt to force the browser to produce a prompt.  This is obviously
inefficient.  Also, this strategy is dependent on knowing what the initial
quota is, but because of (4), a web app would have to guess based on
knowledge of particular browsers' policies.

(2) is easily addressed by providing an editable field and not just a
dropdown.

(1) and (3) would be addressed by adding a requestQuota() interface as I
suggested.

For (5), I couldn't find any options in the UI for resetting the
prompt/reject state so that Opera would prompt again.  Delete Private Data
- Delete Web storage databases did not reset the prompt/reject state.
However, by searching I found that opera:webstorage provides a page for
granting larger quotas and resetting the prompt/reject state.

(5) could be addressed by using a non-modal info bar, or by addressing (3).

I don't mean to pick on Opera, by the way.  I think it's positive that it
does allow the quota to be increased.

Regards,
Mark


[1] Here's the page I used to test this:

p
  You have viewed this page
  span id=countan untold number of/span
  time(s).
/p
p
  Space consumed: span id=bytes/span
/p
script
  try {
if (!localStorage.pageLoadCount) {
  localStorage.pageLoadCount = 0;
  localStorage.data = x;
}
localStorage.pageLoadCount = parseInt(localStorage.pageLoadCount) + 1;
document.getElementById('count').textContent =
localStorage.pageLoadCount;
var data = localStorage.data;
data = data + data;
localStorage.data = data;
document.getElementById('bytes').textContent = data.length +  chars;
  }
  catch (exn) {
document.getElementById('bytes').textContent = exn;
  }
/script

Re: [IndexedDB] Granting storage quotas

[Mark Seaborn]

On Tue, Apr 20, 2010 at 7:17 PM, Shawn Wilsher sdwi...@mozilla.com wrote:

 On 4/20/2010 4:11 AM, Mark Seaborn wrote:

 1) It doesn't allow a web app to ask for a storage allocation up front,
 before it starts to consume the storage.

 Why does that matter?


It doesn't support the use cases that I suggested at the start of the
thread:

* An e-mail web app requests an amount of storage that is large enough to
store all your current e-mail, plus your e-mail for the next year at
projected rates.  As this runs out, it can request more.
* A backup web app requests an amount that is large enough to store your
data.

Suppose I leave an e-mail web app running on my laptop while I'm not using
it, expecting it to sync my mail.  Then I take the laptop somewhere where
there's no network connectivity.  I don't want to find that the e-mail app
failed to fetch my e-mail because it ran out of storage and was stuck while
the browser displayed a prompt that I wasn't there to see.

The same goes for the backup example.  I wouldn't want to find that the
backup app has failed to back up my data because it got stuck at a prompt.
This wouldn't happen if the backup app had the ability to request, up front,
the amount of storage that it knows it will need.  This request might happen
at the point where the user specifies what files they want the app to copy.


  2) In Opera, the quota can only be increased in multiples of about 15, so
 it
 takes three prompts to get up into the range of gigabytes.

 But there is an unlimited option, yeah?


True, but it would be better not to have to give the web app carte blanche.
However, it is a minor point because it is easily fixable.


  3) The web app can't choose when the question is put to the user.
 4) The web app doesn't know how much storage has been allocated, so it
 doesn't know when a question will be asked.
 5) In Opera, if the user chooses Reject, they don't get prompted again.
 This means that asking the user at an appropriate time is important for
 the
 continued functioning of the web app.  Prompting the user at the wrong
 time
 will interrupt them with a page-modal dialog which they might want to get
 rid of with Reject, which would potentially break the web app by leaving
 it unable to get more storage.

 These all feel like user-agent specific worries on how the user agent wants
 to bring this to the attention of the user.


It's not user agent specific if the fix involves adding or changing a
programmatic interface, or involves defining behaviour that web apps are
likely to rely on to function correctly.

Regards,
Mark

Re: [Bug 9823] New: Add maxExecutionContexts property with number of hardware execution contexts

[Mark Seaborn]

On Thu, Jun 10, 2010 at 10:04 AM, Jonas Sicking jo...@sicking.cc wrote:

 For what it's worth, it's unlikely that we at mozilla will implement
 this anytime soon, if at all. We're currently working on trying to
 reduce the ability to fingerprint [1] and this would be a step in the
 wrong direction for us. This is based on discussions with security
 folks here, so it's possible that others at mozilla has different
 opinions, but I still think it's unlikely that this will get past our
 security reviews for now.


While I'm very much in favour of reducing the browser fingerprint, I suspect
that if you expose non-determinism via concurrent message-passing between
web workers, a web app can probably work out how many cores the machine
has.  It can spawn multiple web workers, send many messages, and look at the
message interleaving.  (Do web workers have access to any high resolution
timers that would make this easier?)

That said, just because it's possible to get this information doesn't mean
it should be made easy.

Cheers,
Mark

Re: Updates to File API

[Mark Seaborn]

On Wed, Jun 2, 2010 at 5:06 PM, Jian Li jia...@chromium.org wrote:

 I have one question regarding the scheme for Blob.url. The latest spec says
 that The proposed URL scheme is filedata:. Mozilla already ships with
 moz-filedata:. Since the URL is now part of the Blob and it could be used
 to refer to both file data blob and binary data blob, should we consider
 making the scheme as blobdata: for better generalization? In addition,
 we're thinking it will probably be a good practice to encode the security
 origin in the blob URL scheme, like blobdata:
 http://example.com/33c6401f-8779-4ea2-9a9b-1b725d6cd50b. This will make
 doing the security origin check easier when a page tries to access the blob
 url that is created in another process, under multi-process architecture.


Why do the filedata: URLs need to apply a same-origin check?  It seems
like this would unnecessarily reduce composability.  In practice, the URLs
returned by the File API would be unguessable anyway.  Why not use
unguessability of these tokens as the security mechanism?  So if a web app
wants to share the file with other, co-operating entities (e.g. in an iframe
or another tab), it can do so by sharing the URL; otherwise, it can withhold
the URL.

When would the currently-proposed same-origin checks apply?  Would I be
right in thinking that they only apply to XMLHttpRequests from Javascript,
and don't apply if the URL is linked from an img element?

Regards,
Mark