Re: PSA: Change the Latest Editor's Drafts of WebStorage, WebWorkers, WebMessaging, Server-Sent Events and WebSockets
On 10/19/15 1:52 PM, Xiaoqian Wu wrote: The Web Platform WG intents to update the Latest Editor's Drafts of a set of HTML5 API specs, including WebStorage, WebWorkers, WebMessaging, Server-Sent Events and WebSockets. As no one in WPWG has committed to work on these specs, the latest EDs will be the WHATWG version. Hi Xiaoqian, Is the plan to gut the current ED of all content except a link to the WHATWG LS plus a message like "For the latest version of this specification use the WHATWG's Living Standard."? -Thanks, AB
Re: PSA: Change the Latest Editor's Drafts of WebStorage, WebWorkers, WebMessaging, Server-Sent Events and WebSockets
> On 21 Oct, 2015, at 1:52 am, Arthur Barstow <art.bars...@gmail.com> wrote: > > On 10/19/15 1:52 PM, Xiaoqian Wu wrote: >> The Web Platform WG intents to update the Latest Editor's Drafts of a set of >> HTML5 API specs, including WebStorage, WebWorkers, WebMessaging, Server-Sent >> Events and WebSockets. As no one in WPWG has committed to work on these >> specs, the latest EDs will be the WHATWG version. > > Hi Xiaoqian, > > Is the plan to gut the current ED of all content except a link to the WHATWG > LS plus a message like "For the latest version of this specification use the > WHATWG's Living Standard.”? Hi Art, +1. Thanks for the good advice. — xiaoqian > > -Thanks, AB > >
PSA: Change the Latest Editor's Drafts of WebStorage, WebWorkers, WebMessaging, Server-Sent Events and WebSockets
Hi All, The Web Platform WG intents to update the Latest Editor's Drafts of a set of HTML5 API specs, including WebStorage, WebWorkers, WebMessaging, Server-Sent Events and WebSockets. As no one in WPWG has committed to work on these specs, the latest EDs will be the WHATWG version. Please find the list of suggested changes below. * Web Storage (CR): Editor's Draft will be https://html.spec.whatwg.org/multipage/webstorage.html <https://html.spec.whatwg.org/multipage/webstorage.html>; * WebWorkers (WD): Editor's Draft will be https://html.spec.whatwg.org/multipage/workers.html <https://html.spec.whatwg.org/multipage/workers.html>; * WebMessaging (REC): Editor's Draft will be https://html.spec.whatwg.org/multipage/comms.html <https://html.spec.whatwg.org/multipage/comms.html>; * WebSockets (CR): Editor's Draft will be https://html.spec.whatwg.org/multipage/comms.html#network <https://html.spec.whatwg.org/multipage/comms.html#network>; * Server-Sent Events (REC): Editor's Draft will be https://html.spec.whatwg.org/multipage/comms.html#server-sent-events <https://html.spec.whatwg.org/multipage/comms.html#server-sent-events>. The Web Platform WG will continue to maintain a stable version for each of these specs (while they are moving along the W3C Recommendation track), and deliver an updated version if necessary. If you have any concerns with this proposal, please let us know by 22 October. Thanks. -- xiaoqian
CfC: publish Candidate Recommendation of Web Storage 2nd Edition deadline June 3 [Was: CfC: Moving webstorage to REC 2nd Edition]
On 5/28/15 11:38 AM, Xiaoqian Wu wrote: Hi Art, and all, I believe we have enough review on the current editor's draft to publish directly a CR. We don't need to do the extra WD step. If implementors or others have additional comments, they can do so during the CR phase. So I propose that we publish a CR directly. This all sounds fine to me. However, strictly speaking I believe we need to record the group's consensus to publish a Candidate Recommendation. As such, let's consider this e-mail as a CfC to publish a CR to publish a CR of Web Storage 2nd edition. If anyone has comments or concerns about this CfC, please reply by June 3 at the latest. Silence will be considered as agreeing with the proposal and explicit responses are preferred. If no non-resolvable blocking issues are raised, this CfC will be considered as passing and we will proceed with this proposal. -Thanks, ArtB Thanks. — xiaoqian On 2015-5-14, at 7:30pm, Arthur Barstow art.bars...@gmail.com wrote: Hi All, Based on Xiaoqian's analysis [1] of Web Storage changes since the REC was published in 2013, this is a Call for Consensus to: 1. Publish a new WD of the spec and to seek wide review of the spec, per process 2014. The new WD will be placed in w3.org/TR/webstorage/. The draft WD is [2] and the review period will be three weeks. The draft includes a summary of the changes since the REC was published [3]. 2. Update the REC errata doc [4] to include a reference to the new WD and its changes since REC section, and the github commit history. If you have any comments or concerns about this CfC, please reply by May 21 at the latest. Silence will be considered as agreeing with the proposal and explicit responses are preferred. If no non-resolvable blocking issues are raised, this CfC will be considered as passing and we will proceed with this proposal. -Thanks, ArtB [1] https://lists.w3.org/Archives/Public/public-webapps/2015AprJun/0607.html [2] https://w3c.github.io/webstorage/ [3] https://w3c.github.io/webstorage/#status-of-this-document [4] http://dev.w3.org/html5/webstorage/publish/errata-v1.html
Re: CfC: Moving webstorage to REC 2nd Edition; deadline May 21
Hi Art, and all, I believe we have enough review on the current editor's draft to publish directly a CR. We don't need to do the extra WD step. If implementors or others have additional comments, they can do so during the CR phase. So I propose that we publish a CR directly. Thanks. — xiaoqian On 2015-5-14, at 7:30pm, Arthur Barstow art.bars...@gmail.com wrote: Hi All, Based on Xiaoqian's analysis [1] of Web Storage changes since the REC was published in 2013, this is a Call for Consensus to: 1. Publish a new WD of the spec and to seek wide review of the spec, per process 2014. The new WD will be placed in w3.org/TR/webstorage/. The draft WD is [2] and the review period will be three weeks. The draft includes a summary of the changes since the REC was published [3]. 2. Update the REC errata doc [4] to include a reference to the new WD and its changes since REC section, and the github commit history. If you have any comments or concerns about this CfC, please reply by May 21 at the latest. Silence will be considered as agreeing with the proposal and explicit responses are preferred. If no non-resolvable blocking issues are raised, this CfC will be considered as passing and we will proceed with this proposal. -Thanks, ArtB [1] https://lists.w3.org/Archives/Public/public-webapps/2015AprJun/0607.html [2] https://w3c.github.io/webstorage/ [3] https://w3c.github.io/webstorage/#status-of-this-document [4] http://dev.w3.org/html5/webstorage/publish/errata-v1.html
Re: CfC: Moving webstorage to REC 2nd Edition; deadline May 21
On 14 May 2015, at 14:30, Arthur Barstow art.bars...@gmail.com wrote: 1. Publish a new WD of the spec and to seek wide review of the spec, per process 2014. The new WD will be placed in w3.org/TR/webstorage/. This addresses the concern and clears the confusion around an outdated /TR. If you have any comments or concerns about this CfC, please reply by May 21 at the latest. Silence will be considered as agreeing with the proposal and explicit responses are preferred. If no non-resolvable blocking issues are raised, this CfC will be considered as passing and we will proceed with this proposal. +1. Thanks for the swift turnaround. Thanks, -Anssi
RE: CfC: Moving webstorage to REC 2nd Edition; deadline May 21
From: Kostiainen, Anssi [mailto:anssi.kostiai...@intel.com] Sent: Thursday, May 14, 2015 21:16 To: Arthur Barstow Cc: public-webapps Subject: Re: CfC: Moving webstorage to REC 2nd Edition; deadline May 21 On 14 May 2015, at 14:30, Arthur Barstow art.bars...@gmail.com wrote: 1. Publish a new WD of the spec and to seek wide review of the spec, per process 2014. The new WD will be placed in w3.org/TR/webstorage/. This addresses the concern and clears the confusion around an outdated /TR. Yes, the spec looks good to me. If you have any comments or concerns about this CfC, please reply by May 21 at the latest. Silence will be considered as agreeing with the proposal and explicit responses are preferred. If no non-resolvable blocking issues are raised, this CfC will be considered as passing and we will proceed with this proposal. +1. Thanks for the swift turnaround. I also regenerated an implementation report at http://w3c.github.io/test-results/webstorage/all.html ... for your reference. BTW: I have sent out the above info yesterday, but seems the mailing list system didn't get it. Thanks, Zhiqiang
CfC: Moving webstorage to REC 2nd Edition; deadline May 21
Hi All, Based on Xiaoqian's analysis [1] of Web Storage changes since the REC was published in 2013, this is a Call for Consensus to: 1. Publish a new WD of the spec and to seek wide review of the spec, per process 2014. The new WD will be placed in w3.org/TR/webstorage/. The draft WD is [2] and the review period will be three weeks. The draft includes a summary of the changes since the REC was published [3]. 2. Update the REC errata doc [4] to include a reference to the new WD and its changes since REC section, and the github commit history. If you have any comments or concerns about this CfC, please reply by May 21 at the latest. Silence will be considered as agreeing with the proposal and explicit responses are preferred. If no non-resolvable blocking issues are raised, this CfC will be considered as passing and we will proceed with this proposal. -Thanks, ArtB [1] https://lists.w3.org/Archives/Public/public-webapps/2015AprJun/0607.html [2] https://w3c.github.io/webstorage/ [3] https://w3c.github.io/webstorage/#status-of-this-document [4] http://dev.w3.org/html5/webstorage/publish/errata-v1.html
[testing] unstable tests: eventsource, workers, websockets, webmessaging, webstorage
James identified tests that aren't producing consistent results http://hoppipolla.co.uk/410/unstable.txt. For WebApps, the test suites indentified are: eventsource, workers, websockets, webmessaging and webstorage. Please review these tests and if needed, create an issue and/or submit a fix. -Thanks, AB Original Message Subject:Unstable tests Resent-Date:Tue, 25 Mar 2014 11:16:16 + Resent-From:public-test-in...@w3.org Date: Tue, 25 Mar 2014 11:15:51 + From: ext James Graham ja...@hoppipolla.co.uk To: public-test-in...@w3.org public-test-in...@w3.org Using data from a set of web-platform-tests runs in desktop Firefox, I have a list of tests that aren't producing consistent results. This data is based on 10 runs per platform for 5 different platforms (2 Linux, 3 OS X). Instability is considered per-platform (so a test that consistently produces one result on OSX and another on Linux is considered stable). The list of tests that produced unstable results in any configuration is at [1]. For the purposes of presentation I squashed the results down into a single list without platform information, but I can change that if needed. This set of tests represents about 2% of the top level test files in the repository. In order to use the testsuite in the Mozilla CI system (or in any other CI system), the rate of instability has to be much lower than that. Therefore it is necessary to determine why these tests are not giving consistent results and take appropriate action. In the best case the problems will be largely with the tests themselves, either doing something non-deterministic or just having too short a timeout, or whatever. In this case we need to fix the test. In some cases the instability may be due to non-determinism in Firefox, in which case I may (unfortunately) have to disable the test locally until the underlying issue is fixed. If you have any time to help investigate the issues with these tests, particularly for tests that you own (i.e. ones that you wrote), it would be much appreciated. [1] http://hoppipolla.co.uk/410/unstable.txt
[Bug 19540] Firing WebStorage storage event
https://www.w3.org/Bugs/Public/show_bug.cgi?id=19540 Ian 'Hixie' Hickson i...@hixie.ch changed: What|Removed |Added Status|NEW |RESOLVED Resolution|--- |FIXED --- Comment #3 from Ian 'Hixie' Hickson i...@hixie.ch --- Yeah, that could have been clearer. How is it now? Please don't hesitate to reopen this bug if it's still not clear. -- You are receiving this mail because: You are on the CC list for the bug.
[Bug 19540] Firing WebStorage storage event
https://www.w3.org/Bugs/Public/show_bug.cgi?id=19540 Ian 'Hixie' Hickson i...@hixie.ch changed: What|Removed |Added Status|NEW |RESOLVED Resolution|--- |FIXED -- You are receiving this mail because: You are on the CC list for the bug.
[Bug 19540] Firing WebStorage storage event
https://www.w3.org/Bugs/Public/show_bug.cgi?id=19540
Janusz Majnert jmajn...@gmail.com changed:
What|Removed |Added
Status|RESOLVED|REOPENED
Resolution|FIXED |---
--- Comment #2 from Janusz Majnert jmajn...@gmail.com ---
Looks good to me now. Two things though:
1. Typo: events are fired on the Window objecys.
2. I'm not sure how the condition if the methods did something evaluates for
this code:
localStorage.setItem('key1','value1');
localStorage.setItem('key1','value1');
When running the above code, should UA generate one or two storage events?
--
You are receiving this mail because:
You are on the CC list for the bug.
[Bug 19540] New: Firing WebStorage storage event
https://www.w3.org/Bugs/Public/show_bug.cgi?id=19540 Priority: P2 Bug ID: 19540 CC: i...@hixie.ch, m...@w3.org, public-webapps@w3.org Assignee: i...@hixie.ch Summary: Firing WebStorage storage event QA Contact: public-webapps-bugzi...@w3.org Severity: normal Classification: Unclassified OS: Linux Reporter: jmajn...@gmail.com Hardware: PC Status: NEW Version: unspecified Component: Web Storage (editor: Ian Hickson) Product: WebAppsWG When trying to implement local storage I found it hard to understand the rules for firing the storage event. In section 11.2.3 (http://www.whatwg.org/specs/web-apps/current-work/multipage/webstorage.html#the-localstorage-attribute) we have this sentence [1]: When the setItem(), removeItem(), and clear() methods are called on a Storage object x that is associated with a local storage area, if the methods did something, then in every Document object whose Window object's localStorage attribute's Storage object is associated with the same storage area, other than x, a storage event must be fired, as described below. as described below points to section 11.2.4, which reads [2]: The storage event is fired when a storage area changes, as described in the previous two sections (for session storage, for local storage). When this happens, the user agent must queue a task to fire an event with the name storage, which does not bubble and is not cancelable, and which uses the StorageEvent interface, at each Window object whose Document object has a Storage object that is affected. What I understood: Sentence [1] says that storage events should be fired on affected Document objects, except the one that originated the change. Sentences in [2] say that when a storage event is fired as described in [1], a task must be queued to fire storage events on all affected Window objects. It also says that Document objects have Storage objects, which I don't think is true. Is my understanding correct? What am I missing? -- You are receiving this mail because: You are on the CC list for the bug.
[webstorage] Back to LC? [Was] RfC: LCWD of Web Storage; deadline September 27
The comment period for WebStorage's September 1 LCWD ended yesterday. I didn't notice any e-mail threads specific to this LC but there are 3 opens bugs at the moment [1]. Additionally, on September 10 the ED was updated to replace initStorageEvent with the dictionary-based approach [2]. What is the implementation status of the [2] change (I assume this change means another LC is required)? -AB [1] http://www.w3.org/Bugs/Public/buglist.cgi?product=WebAppsWGcomponent=Web%20Storage%20%28editor%3A%20Ian%20Hickson%29resolution=--- [2] http://dev.w3.org/cvsweb/html5/webstorage/Overview.html.diff?r1=1.181;r2=1.182;f=h On 9/6/11 10:19 AM, ext Arthur Barstow wrote: A new LCWD of Web Storage was published: http://www.w3.org/TR/2011/WD-webstorage-20110901/ Please send all comments to public-webapps@w3.org by September 27.
Re: [webstorage] Plan to move the spec to Last Call Working Draft
Thanks Scott for volunteering to create a fix for 13020. Ideally, the Editor would address all of the open bugs and WebApps' version of Web Storage would be as close as possible to the WHATWG's version. However, given the conflicting constraints of moving this spec to LC now and the low priority of this spec for Ian, it appears we need to proceed otherwise. Scott - please create a fix we can review and base it on the latest ED [ED]. It may be helpful if your fix included PLH's fix for 12111 [1211-fix] and to put your version of the spec in the publish directory [PUB] e.g. create LC-webstorage-2011July.html. -Thanks, ArtB [ED] http://dev.w3.org/cvsweb/html5/webstorage/ [PUB] http://dev.w3.org/cvsweb/html5/webstorage/publish/ [1211-fix] http://www.w3.org/2011/06/Web%20Storage.html On 6/30/11 3:20 PM, ext Scott Wilson wrote: On 30 Jun 2011, at 14:55, Arthur Barstow wrote: Given the lack of support for stopping work on Web Storage [1], I'd let to get consensus on the plan to move it to Last Call Working Draft. Currently there are two open bugs: 1. Bug 12111: spec for Storage object getItem(key) method does not match implementation behavior. PLH created a version of the spec that addresses this bug [12111-fix]. 2. Bug 13020: No user agent will implement the storage mutex so this passage does not reflect reality. There are different opinions on the priority of Web Storage ... * Web Storage is a low priority and the Editor will get to it when he gets to it * Web Storage is a high priority because the lack of a LCWD will block at least the Widget Interface spec from progressing on the Rec track There are various options on what to do next, including: 1. Fix 12111 and 13020 and move Web Storage to LCWD +1 2. Leave Web Storage as is and eventually implementations will match the spec 3. Do #1 in one version of the spec and keep #2 as a separate version of the spec (e.g. L1 and L2). Comments on these options are welcome. If you prefer #1, please indicate if you are willing to create a fix/patch for bug 13020. Yes. -AB [1] http://lists.w3.org/Archives/Public/public-webapps/2011AprJun/1110.html [12111] http://www.w3.org/Bugs/Public/show_bug.cgi?id=12111 [12111-fix] http://www.w3.org/2011/06/Web%20Storage.html [13020] http://www.w3.org/Bugs/Public/show_bug.cgi?id=13020
[webstorage] Plan to move the spec to Last Call Working Draft
Given the lack of support for stopping work on Web Storage [1], I'd let to get consensus on the plan to move it to Last Call Working Draft. Currently there are two open bugs: 1. Bug 12111: spec for Storage object getItem(key) method does not match implementation behavior. PLH created a version of the spec that addresses this bug [12111-fix]. 2. Bug 13020: No user agent will implement the storage mutex so this passage does not reflect reality. There are different opinions on the priority of Web Storage ... * Web Storage is a low priority and the Editor will get to it when he gets to it * Web Storage is a high priority because the lack of a LCWD will block at least the Widget Interface spec from progressing on the Rec track There are various options on what to do next, including: 1. Fix 12111 and 13020 and move Web Storage to LCWD 2. Leave Web Storage as is and eventually implementations will match the spec 3. Do #1 in one version of the spec and keep #2 as a separate version of the spec (e.g. L1 and L2). Comments on these options are welcome. If you prefer #1, please indicate if you are willing to create a fix/patch for bug 13020. -AB [1] http://lists.w3.org/Archives/Public/public-webapps/2011AprJun/1110.html [12111] http://www.w3.org/Bugs/Public/show_bug.cgi?id=12111 [12111-fix] http://www.w3.org/2011/06/Web%20Storage.html [13020] http://www.w3.org/Bugs/Public/show_bug.cgi?id=13020
Re: [webstorage] Plan to move the spec to Last Call Working Draft
On 30 Jun 2011, at 14:55, Arthur Barstow wrote: Given the lack of support for stopping work on Web Storage [1], I'd let to get consensus on the plan to move it to Last Call Working Draft. Currently there are two open bugs: 1. Bug 12111: spec for Storage object getItem(key) method does not match implementation behavior. PLH created a version of the spec that addresses this bug [12111-fix]. 2. Bug 13020: No user agent will implement the storage mutex so this passage does not reflect reality. There are different opinions on the priority of Web Storage ... * Web Storage is a low priority and the Editor will get to it when he gets to it * Web Storage is a high priority because the lack of a LCWD will block at least the Widget Interface spec from progressing on the Rec track There are various options on what to do next, including: 1. Fix 12111 and 13020 and move Web Storage to LCWD +1 2. Leave Web Storage as is and eventually implementations will match the spec 3. Do #1 in one version of the spec and keep #2 as a separate version of the spec (e.g. L1 and L2). Comments on these options are welcome. If you prefer #1, please indicate if you are willing to create a fix/patch for bug 13020. Yes. -AB [1] http://lists.w3.org/Archives/Public/public-webapps/2011AprJun/1110.html [12111] http://www.w3.org/Bugs/Public/show_bug.cgi?id=12111 [12111-fix] http://www.w3.org/2011/06/Web%20Storage.html [13020] http://www.w3.org/Bugs/Public/show_bug.cgi?id=13020
Re: [webstorage] origin security check
On 14 Jun 2011, at 06:28, Marcos Caceres wrote: On Monday, June 13, 2011, Ian Hickson i...@hixie.ch wrote: On Mon, 13 Jun 2011, Marcos Caceres wrote: I thought maybe I could get away with: When getting or setting the preferences attribute, if the origin of a widget instance is mutable (e.g., if the user agent allows document.domain to be dynamically changed), then the user agent must perform the object initialization steps of [Web Storage] substituting the preferences attribute for the localStorage attribute where appropriate. But maybe I'll just do a copy and paste and just replace the appropriate bits of text. I guess that could work. By the way, how are you resolving the multiple-thread problem here? (Since you're introducing a new API, it presumably doesn't have to have the same bug as the localStorage API, where we're stuck for legacy reasons and are basically forced to either have a cross-thread blocking API or a racy API, depending on how it's implemented, both of which suck.) We are not solving it:( As widgets run as a single process, each instance in a unique origin, don't share data/cache with browser tabs/windows or other widgets, this issue does not come up much... At least no one has complained to me about it. We've seen clients setting the same preference in different threads resulting in a consistency problem, however we basically go with the view that its something we just deal with - i.e. its not guaranteed to be consistent but we make best effort. In general use in a widget context its not going to be frequent or critical - we only come across it in a testing context by creating duplicate views of a widget instance, showing them alongside each other, which is a pretty pointless thing for a user to do. -- Ian Hickson U+1047E)\._.,--,'``.fL http://ln.hixie.ch/ U+263A/, _.. \ _\ ;`._ ,. Things that are impossible just take longer. `._.-(,_..'--(,_..'`-.;.' -- Marcos Caceres http://datadriven.com.au
Re: [webstorage] origin security check
On Fri, Jun 10, 2011 at 8:19 PM, Ian Hickson i...@hixie.ch wrote: On Fri, 10 Jun 2011, Marcos Caceres wrote: I tried to create a generic HTML test for this using localStorage, but could not figure out a way to trigger the SECURITY_ERR. I asked a few people (Lachy, Snedders, and even the guy that implemented Web Storage at Opera!) to help me come up with a test. No one was not able to come up with a test for this, as there seems to be a general lack of understanding how the whole effective script origin is set (we looked at the spec, read it backwards, then forwards, then scratched our heads for a bit). Can you explain (with maybe some javascript) how one would cause the SECURITY_ERR exception to be thrown by setItem() and getItem()? var foo = localStorage; foo.test = ''; document.domain = document.domain; // changes effective origin foo.test; // throws localStorage; // would also throw Thanks for this. Got it now :) -- Marcos Caceres http://datadriven.com.au
Re: [webstorage] origin security check
On Mon, 13 Jun 2011, Marcos Caceres wrote: I thought maybe I could get away with: When getting or setting the preferences attribute, if the origin of a widget instance is mutable (e.g., if the user agent allows document.domain to be dynamically changed), then the user agent must perform the object initialization steps of [Web Storage] substituting the preferences attribute for the localStorage attribute where appropriate. But maybe I'll just do a copy and paste and just replace the appropriate bits of text. I guess that could work. By the way, how are you resolving the multiple-thread problem here? (Since you're introducing a new API, it presumably doesn't have to have the same bug as the localStorage API, where we're stuck for legacy reasons and are basically forced to either have a cross-thread blocking API or a racy API, depending on how it's implemented, both of which suck.) -- Ian Hickson U+1047E)\._.,--,'``.fL http://ln.hixie.ch/ U+263A/, _.. \ _\ ;`._ ,. Things that are impossible just take longer. `._.-(,_..'--(,_..'`-.;.'
Re: [webstorage] origin security check
On Monday, June 13, 2011, Ian Hickson i...@hixie.ch wrote: On Mon, 13 Jun 2011, Marcos Caceres wrote: I thought maybe I could get away with: When getting or setting the preferences attribute, if the origin of a widget instance is mutable (e.g., if the user agent allows document.domain to be dynamically changed), then the user agent must perform the object initialization steps of [Web Storage] substituting the preferences attribute for the localStorage attribute where appropriate. But maybe I'll just do a copy and paste and just replace the appropriate bits of text. I guess that could work. By the way, how are you resolving the multiple-thread problem here? (Since you're introducing a new API, it presumably doesn't have to have the same bug as the localStorage API, where we're stuck for legacy reasons and are basically forced to either have a cross-thread blocking API or a racy API, depending on how it's implemented, both of which suck.) We are not solving it:( As widgets run as a single process, each instance in a unique origin, don't share data/cache with browser tabs/windows or other widgets, this issue does not come up much... At least no one has complained to me about it. -- Ian Hickson U+1047E )\._.,--,'``. fL http://ln.hixie.ch/ U+263A /, _.. \ _\ ;`._ ,. Things that are impossible just take longer. `._.-(,_..'--(,_..'`-.;.' -- Marcos Caceres http://datadriven.com.au
Re: [webstorage] origin security check
On Thu, Jun 9, 2011 at 6:07 PM, Ian Hickson i...@hixie.ch wrote: On Thu, 9 Jun 2011, Marcos Caceres wrote: tiny quick editorial request, where the spec says: When the localStorage attribute is accessed, the user agent must run the following steps: Can you please change that to: When the localStorage attribute is accessed, the user agent must run the origin security check. And then independently define just label the algorithm origin security check (or name it something better). I need to use the same text in another spec and would prefer to link instead of copy/paste. Done. Thanks! :) Just out of interest, what's the context for this? These steps are pretty specific to localStorage (and are not the complete security story -- see the later section on security), so I'm surprised to hear these particular steps would be reused. Context is the widget.preference attribute, which implements Storage (but supports some widgety things, like read-only keys/values): http://dev.w3.org/2006/waf/widgets-api/#the-preferences-attribute I'm want to replace the following section with the link to the Storage spec: http://dev.w3.org/2006/waf/widgets-api/#preference-origin-security-check0 Kind regards, Marcos -- Marcos Caceres http://datadriven.com.au
Re: [webstorage] origin security check
Hi Ian, On Fri, Jun 10, 2011 at 9:26 AM, Marcos Caceres marcosscace...@gmail.com wrote: On Thu, Jun 9, 2011 at 6:07 PM, Ian Hickson i...@hixie.ch wrote: On Thu, 9 Jun 2011, Marcos Caceres wrote: tiny quick editorial request, where the spec says: When the localStorage attribute is accessed, the user agent must run the following steps: Can you please change that to: When the localStorage attribute is accessed, the user agent must run the origin security check. And then independently define just label the algorithm origin security check (or name it something better). I need to use the same text in another spec and would prefer to link instead of copy/paste. Done. Thanks! :) Just out of interest, what's the context for this? These steps are pretty specific to localStorage (and are not the complete security story -- see the later section on security), so I'm surprised to hear these particular steps would be reused. Context is the widget.preference attribute, which implements Storage (but supports some widgety things, like read-only keys/values): http://dev.w3.org/2006/waf/widgets-api/#the-preferences-attribute I'm want to replace the following section with the link to the Storage spec: http://dev.w3.org/2006/waf/widgets-api/#preference-origin-security-check0 I tried to create a generic HTML test for this using localStorage, but could not figure out a way to trigger the SECURITY_ERR. I asked a few people (Lachy, Snedders, and even the guy that implemented Web Storage at Opera!) to help me come up with a test. No one was not able to come up with a test for this, as there seems to be a general lack of understanding how the whole effective script origin is set (we looked at the spec, read it backwards, then forwards, then scratched our heads for a bit). Can you explain (with maybe some javascript) how one would cause the SECURITY_ERR exception to be thrown by setItem() and getItem()? Many thanks in advance! Marcos -- Marcos Caceres http://datadriven.com.au
Re: [webstorage] origin security check
On Fri, 10 Jun 2011, Marcos Caceres wrote: On Thu, Jun 9, 2011 at 6:07 PM, Ian Hickson i...@hixie.ch wrote: On Thu, 9 Jun 2011, Marcos Caceres wrote: tiny quick editorial request, where the spec says: When the localStorage attribute is accessed, the user agent must run the following steps: Can you please change that to: When the localStorage attribute is accessed, the user agent must run the origin security check. And then independently define just label the algorithm origin security check (or name it something better). I need to use the same text in another spec and would prefer to link instead of copy/paste. Done. Thanks! :) Just out of interest, what's the context for this? These steps are pretty specific to localStorage (and are not the complete security story -- see the later section on security), so I'm surprised to hear these particular steps would be reused. Context is the widget.preference attribute, which implements Storage (but supports some widgety things, like read-only keys/values): http://dev.w3.org/2006/waf/widgets-api/#the-preferences-attribute I'm want to replace the following section with the link to the Storage spec: http://dev.w3.org/2006/waf/widgets-api/#preference-origin-security-check0 The algorithm we're talking about here wouldn't work for that; steps 3 and 4 in particular would mean that .preferences always returned the same object as .localStorage. -- Ian Hickson U+1047E)\._.,--,'``.fL http://ln.hixie.ch/ U+263A/, _.. \ _\ ;`._ ,. Things that are impossible just take longer. `._.-(,_..'--(,_..'`-.;.'
Re: [webstorage] origin security check
On Fri, 10 Jun 2011, Marcos Caceres wrote: I tried to create a generic HTML test for this using localStorage, but could not figure out a way to trigger the SECURITY_ERR. I asked a few people (Lachy, Snedders, and even the guy that implemented Web Storage at Opera!) to help me come up with a test. No one was not able to come up with a test for this, as there seems to be a general lack of understanding how the whole effective script origin is set (we looked at the spec, read it backwards, then forwards, then scratched our heads for a bit). Can you explain (with maybe some javascript) how one would cause the SECURITY_ERR exception to be thrown by setItem() and getItem()? var foo = localStorage; foo.test = ''; document.domain = document.domain; // changes effective origin foo.test; // throws localStorage; // would also throw -- Ian Hickson U+1047E)\._.,--,'``.fL http://ln.hixie.ch/ U+263A/, _.. \ _\ ;`._ ,. Things that are impossible just take longer. `._.-(,_..'--(,_..'`-.;.'
[webstorage] origin security check
Hi Ian, tiny quick editorial request, where the spec says: When the localStorage attribute is accessed, the user agent must run the following steps: Can you please change that to: When the localStorage attribute is accessed, the user agent must run the origin security check. And then independently define just label the algorithm origin security check (or name it something better). I need to use the same text in another spec and would prefer to link instead of copy/paste. Many thanks! Marcos -- Marcos Caceres http://datadriven.com.au
Re: [webstorage] origin security check
On Thu, 9 Jun 2011, Marcos Caceres wrote: tiny quick editorial request, where the spec says: When the localStorage attribute is accessed, the user agent must run the following steps: Can you please change that to: When the localStorage attribute is accessed, the user agent must run the origin security check. And then independently define just label the algorithm origin security check (or name it something better). I need to use the same text in another spec and would prefer to link instead of copy/paste. Done. Just out of interest, what's the context for this? These steps are pretty specific to localStorage (and are not the complete security story -- see the later section on security), so I'm surprised to hear these particular steps would be reused. -- Ian Hickson U+1047E)\._.,--,'``.fL http://ln.hixie.ch/ U+263A/, _.. \ _\ ;`._ ,. Things that are impossible just take longer. `._.-(,_..'--(,_..'`-.;.'
[webstorage] Plan to address open Bugs?
All, What is the plan to address the following Web Storage bugs: 1. Bug-12111; spec for Storage object getItem(key) method does not match implementation behavior http://www.w3.org/Bugs/Public/show_bug.cgi?id=12111 2. Bug-12272; Improve section on DNS spoofing attacks to address user attacks http://www.w3.org/Bugs/Public/show_bug.cgi?id=12272 3. Bug-12090; It would be nice to have one Storage object that you could place wherever you want. http://www.w3.org/Bugs/Public/show_bug.cgi?id=12090 Which of these must be addressed before the WG considers the spec LC-ready? Original Message Subject:Re: [webstorage] Moving Web Storage back to Last Call WD Date: Fri, 18 Feb 2011 00:19:51 +0900 From: ext Michael[tm] Smith m...@w3.org To: Ian Hickson i...@hixie.ch CC: Arthur Barstow art.bars...@nokia.com, Tab Atkins Jr. jackalm...@gmail.com, public-webapps public-webapps@w3.org Ian Hicksoni...@hixie.ch, 2011-02-14 10:13 +: On Sat, 12 Feb 2011, Arthur Barstow wrote: What high priority work must be done such that this spec is ready to be re-published as a new Last Call Working draft? Tab, do you know of anything that is blocking redoing an LC? (Personally I'm fine with it going to REC yesterday, so...) Bugzilla shows no open bugs for this spec I just now raised a new one: spec for Storage object getItem(key) method does not match implementation behavior http://www.w3.org/Bugs/Public/show_bug.cgi?id=12111 -- Michael[tm] Smith http://people.w3.org/mike
Re: [webstorage] Plan to address open Bugs?
On Thu, 28 Apr 2011, Arthur Barstow wrote: What is the plan to address the following Web Storage bugs: 1. Bug-12111; spec for Storage object getItem(key) method does not match implementation behavior http://www.w3.org/Bugs/Public/show_bug.cgi?id=12111 2. Bug-12272; Improve section on DNS spoofing attacks to address user attacks http://www.w3.org/Bugs/Public/show_bug.cgi?id=12272 3. Bug-12090; It would be nice to have one Storage object that you could place wherever you want. http://www.w3.org/Bugs/Public/show_bug.cgi?id=12090 My plan is to address them in the order of they appear on this bug list: http://www.w3.org/Bugs/Public/buglist.cgi?query_format=advancedshort_desc_type=allwordssubstrshort_desc=long_desc_type=allwordssubstrlong_desc=bug_file_loc_type=allwordssubstrbug_file_loc=status_whiteboard_type=allwordssubstrstatus_whiteboard=keywords_type=allwordskeywords=bug_status=UNCONFIRMEDbug_status=NEWbug_status=ASSIGNEDbug_status=REOPENEDemailassigned_to1=1emailtype1=exactemail1=ian%40hixie.chemailtype2=substringemail2=bugidtype=includebug_id=votes=chfieldfrom=chfieldto=Nowchfieldvalue=cmdtype=doitorder=Last+Changedfield0-0-0=nooptype0-0-0=noopvalue0-0-0= In recent weeks I've been focusing on multitrack video and video conferencing, but I have now returned to just going through feedback. A precise ETA depends mostly on how many interrupts I get dealing with politics in the HTML WG. -- Ian Hickson U+1047E)\._.,--,'``.fL http://ln.hixie.ch/ U+263A/, _.. \ _\ ;`._ ,. Things that are impossible just take longer. `._.-(,_..'--(,_..'`-.;.'
Re: [webstorage] Plan to address open Bugs?
Well, I guess the good news is that (at the time of this writing), there aren't 355 bugs ;). All - Inputs and proposals for these bugs are encouraged! On Apr/28/2011 2:33 PM, ext Ian Hickson wrote: On Thu, 28 Apr 2011, Arthur Barstow wrote: What is the plan to address the following Web Storage bugs: 1. Bug-12111; spec for Storage object getItem(key) method does not match implementation behavior http://www.w3.org/Bugs/Public/show_bug.cgi?id=12111 2. Bug-12272; Improve section on DNS spoofing attacks to address user attacks http://www.w3.org/Bugs/Public/show_bug.cgi?id=12272 3. Bug-12090; It would be nice to have one Storage object that you could place wherever you want. http://www.w3.org/Bugs/Public/show_bug.cgi?id=12090 My plan is to address them in the order of they appear on this bug list: http://www.w3.org/Bugs/Public/buglist.cgi?query_format=advancedshort_desc_type=allwordssubstrshort_desc=long_desc_type=allwordssubstrlong_desc=bug_file_loc_type=allwordssubstrbug_file_loc=status_whiteboard_type=allwordssubstrstatus_whiteboard=keywords_type=allwordskeywords=bug_status=UNCONFIRMEDbug_status=NEWbug_status=ASSIGNEDbug_status=REOPENEDemailassigned_to1=1emailtype1=exactemail1=ian%40hixie.chemailtype2=substringemail2=bugidtype=includebug_id=votes=chfieldfrom=chfieldto=Nowchfieldvalue=cmdtype=doitorder=Last+Changedfield0-0-0=nooptype0-0-0=noopvalue0-0-0= In recent weeks I've been focusing on multitrack video and video conferencing, but I have now returned to just going through feedback. A precise ETA depends mostly on how many interrupts I get dealing with politics in the HTML WG.
Re: [webstorage] Moving Web Storage back to Last Call WD
Ian Hickson i...@hixie.ch, 2011-02-14 10:13 +: On Sat, 12 Feb 2011, Arthur Barstow wrote: What high priority work must be done such that this spec is ready to be re-published as a new Last Call Working draft? Tab, do you know of anything that is blocking redoing an LC? (Personally I'm fine with it going to REC yesterday, so...) Bugzilla shows no open bugs for this spec I just now raised a new one: spec for Storage object getItem(key) method does not match implementation behavior http://www.w3.org/Bugs/Public/show_bug.cgi?id=12111 -- Michael[tm] Smith http://people.w3.org/mike
Re: [webstorage] Moving Web Storage back to Last Call WD
On Sat, 12 Feb 2011, Arthur Barstow wrote: What high priority work must be done such that this spec is ready to be re-published as a new Last Call Working draft? Tab, do you know of anything that is blocking redoing an LC? (Personally I'm fine with it going to REC yesterday, so...) Bugzilla shows no open bugs for this spec [Bugs] and the latest ED includes the following: [[ The use of the storage mutex to avoid race conditions is currently considered by certain implementors to be too high a performance burden, to the point where allowing data corruption is considered preferable. Alternatives that do not require a user-agent-wide per-origin script lock are eagerly sought after. If reviewers have any suggestions, they are urged to send them to the addresses given in the previous section. [...] ]] In particular, what are the proposals, plans and timeline to address the above issues? There is no proposal to address the issue. The plan is to wait for someone to have an idea. The timeline is thus open-ended. HTH, -- Ian Hickson U+1047E)\._.,--,'``.fL http://ln.hixie.ch/ U+263A/, _.. \ _\ ;`._ ,. Things that are impossible just take longer. `._.-(,_..'--(,_..'`-.;.'
[webstorage] Moving Web Storage back to Last Call WD
Hixie, All, Regarding re-publishing the Web Storage spec [ED] as a new Last Call Working Draft ... Bugzilla shows no open bugs for this spec [Bugs] and the latest ED includes the following: [[ http://dev.w3.org/html5/webstorage/#issues The use of the storage mutex to avoid race conditions is currently considered by certain implementors to be too high a performance burden, to the point where allowing data corruption is considered preferable. Alternatives that do not require a user-agent-wide per-origin script lock are eagerly sought after. If reviewers have any suggestions, they are urged to send them to the addresses given in the previous section. More details regarding this issue are available in these e-mails (as well as numerous others http://lists.whatwg.org/mmsearch.cgi/whatwg-whatwg.org?config=whatwg-whatwg.orgrestrict=exclude=method=andformat=shortsort=revtimewords=storage+mutex): * http://lists.whatwg.org/htdig.cgi/whatwg-whatwg.org/2009-September/023059.html * http://lists.whatwg.org/htdig.cgi/whatwg-whatwg.org/2009-December/024277.html ]] What high priority work must be done such that this spec is ready to be re-published as a new Last Call Working draft? In particular, what are the proposals, plans and timeline to address the above issues? -Art Barstow [ED] http://dev.w3.org/html5/eventsource/ [Bugs] http://www.w3.org/Bugs/Public/buglist.cgi?query_format=advancedshort_desc_type=allwordssubstrshort_desc=product=WebAppsWGcomponent=Web+Storage+%28editor%3A+Ian+Hickson%29longdesc_type=allwordssubstrlongdesc=bug_file_loc_type=allwordssubstrbug_file_loc=status_whiteboard_type=allwordssubstrstatus_whiteboard=keywords_type=allwordskeywords=bug_status=NEWbug_status=ASSIGNEDbug_status=REOPENEDemailtype1=substringemail1=emailtype2=substringemail2=bug_id_type=anyexactbug_id=votes=chfieldfrom=chfieldto=Nowchfieldvalue=cmdtype=doitorder=Reuse+same+sort+as+last+timefield0-0-0=nooptype0-0-0=noopvalue0-0-0=
Re: [webstorage] Moving Web Storage back to Last Call WD
The Web Storage ED is actually: http://dev.w3.org/html5/webstorage/
Re: Structured clone in WebStorage
On Sat, Dec 4, 2010 at 1:23 AM, João Eiras joao.ei...@gmail.com wrote: On , Darin Fisher da...@chromium.org wrote: I will also add that I think WebStorage (well LocalStorage) is terrible from a performance point of view because it is synchronous, and I'd be very happy if we could discourage its usage and give people more reasons to adopt a better API like IndexedDB. -Darin I don't understand how storing values in a hash map is performant terrible. One of the smaller performance problems with LocalStorage: You have to take a snapshot of that object synchronously. Even with optimizations like copy on write, complex objects can take a while to make such a snapshot of. But the much bigger problem is that LocalStorage is shared between multiple windows. To maintain run to completion, LocalStorage (and cookies) are specced to require taking a storage mutex (a global lock) upon first use of LocalStorage (or cookies) and holding it until JavaScript completes running. So if two windows are both accessing LocalStorage they'll lock up each other and anything else running in their event loop. IE and Chrome are the only two browsers with multiple event loops at the moment. Google Chrome has no intention of ever implementing the storage mutex and my understanding is that IE is the same. So, to answer your question, most of the performance issue is a lock contention one. And because of this, LocalStorage is (and very likely will always be) racy on 2 major browsers. And as more browser adopt multi-process architectures, I expect they'll follow suit. J
Re: Structured clone in WebStorage
On Sun, Dec 5, 2010 at 9:47 PM, Jeremy Orlow jor...@chromium.org wrote: On Sat, Dec 4, 2010 at 1:23 AM, João Eiras joao.ei...@gmail.com wrote: On , Darin Fisher da...@chromium.org wrote: I will also add that I think WebStorage (well LocalStorage) is terrible from a performance point of view because it is synchronous, and I'd be very happy if we could discourage its usage and give people more reasons to adopt a better API like IndexedDB. -Darin I don't understand how storing values in a hash map is performant terrible. One of the smaller performance problems with LocalStorage: You have to take a snapshot of that object synchronously. Even with optimizations like copy on write, complex objects can take a while to make such a snapshot of. But the much bigger problem is that LocalStorage is shared between multiple windows. To maintain run to completion, LocalStorage (and cookies) are specced to require taking a storage mutex (a global lock) upon first use of LocalStorage (or cookies) and holding it until JavaScript completes running. So if two windows are both accessing LocalStorage they'll lock up each other and anything else running in their event loop. IE and Chrome are the only two browsers with multiple event loops at the moment. Google Chrome has no intention of ever implementing the storage mutex and my understanding is that IE is the same. So, to answer your question, most of the performance issue is a lock contention one. And because of this, LocalStorage is (and very likely will always be) racy on 2 major browsers. And as more browser adopt multi-process architectures, I expect they'll follow suit. You asked only about storing values, but note that reading values has problems as well. Either the browser needs to read all LocalStorage data into memory before running any scripts in the page or it's likely that your page will need to block on reading data from disk when it does use LocalStorage. J
Re: Structured clone in WebStorage
Have you guys considered what happens when a Blob is present? I'm very concerned about that case since WebStorage is a synchronous API. If storing a Blob in LocalStorage involves synchronous disk IO, then I'm pretty sure it is something I would object to implementing. I don't think there is a hand-wavy answer about performing a file copy asynchronously in the background. What if the file copy fails? -Darin On Thu, Dec 2, 2010 at 1:45 PM, Anne van Kesteren ann...@opera.com wrote: On Thu, 02 Dec 2010 21:51:29 +0100, Maciej Stachowiak m...@apple.com wrote: We think this feature would be straightforward to implement in Safari/WebKit, and we think it is a useful feature. We would like to implement it at some point. I can't give a specific timeline. I am not sure if it is straightforward to implement in Opera, but this applies to us as well. -- Anne van Kesteren http://annevankesteren.nl/
Re: Structured clone in WebStorage
I will also add that I think WebStorage (well LocalStorage) is terrible from a performance point of view because it is synchronous, and I'd be very happy if we could discourage its usage and give people more reasons to adopt a better API like IndexedDB. -Darin On Fri, Dec 3, 2010 at 10:41 AM, Darin Fisher da...@chromium.org wrote: Have you guys considered what happens when a Blob is present? I'm very concerned about that case since WebStorage is a synchronous API. If storing a Blob in LocalStorage involves synchronous disk IO, then I'm pretty sure it is something I would object to implementing. I don't think there is a hand-wavy answer about performing a file copy asynchronously in the background. What if the file copy fails? -Darin On Thu, Dec 2, 2010 at 1:45 PM, Anne van Kesteren ann...@opera.comwrote: On Thu, 02 Dec 2010 21:51:29 +0100, Maciej Stachowiak m...@apple.com wrote: We think this feature would be straightforward to implement in Safari/WebKit, and we think it is a useful feature. We would like to implement it at some point. I can't give a specific timeline. I am not sure if it is straightforward to implement in Opera, but this applies to us as well. -- Anne van Kesteren http://annevankesteren.nl/
Re: Structured clone in WebStorage
I recall talking to hixie about this at some point, and I recall that at that point localstorage was explicitly prevented from Blobs, although I can't see any sign of that rule anymore :-/ --Oliver On Dec 3, 2010, at 10:41 AM, Darin Fisher wrote: Have you guys considered what happens when a Blob is present? I'm very concerned about that case since WebStorage is a synchronous API. If storing a Blob in LocalStorage involves synchronous disk IO, then I'm pretty sure it is something I would object to implementing. I don't think there is a hand-wavy answer about performing a file copy asynchronously in the background. What if the file copy fails? -Darin On Thu, Dec 2, 2010 at 1:45 PM, Anne van Kesteren ann...@opera.com wrote: On Thu, 02 Dec 2010 21:51:29 +0100, Maciej Stachowiak m...@apple.com wrote: We think this feature would be straightforward to implement in Safari/WebKit, and we think it is a useful feature. We would like to implement it at some point. I can't give a specific timeline. I am not sure if it is straightforward to implement in Opera, but this applies to us as well. -- Anne van Kesteren http://annevankesteren.nl/
Re: Structured clone in WebStorage
On Fri, 3 Dec 2010, Oliver Hunt wrote: I recall talking to hixie about this at some point, and I recall that at that point localstorage was explicitly prevented from Blobs, although I can't see any sign of that rule anymore :-/ Blobs are fine, they're async anyway. To store one you'd just need to let the blob know that you need a checkpoint. The actual storage in localStorage can all be done in the background. The feature that was problematic with localStorage wasn't Blobs, it was ImageData, and that should still be blocked. -- Ian Hickson U+1047E)\._.,--,'``.fL http://ln.hixie.ch/ U+263A/, _.. \ _\ ;`._ ,. Things that are impossible just take longer. `._.-(,_..'--(,_..'`-.;.'
Re: Structured clone in WebStorage
On , Darin Fisher da...@chromium.org wrote: I will also add that I think WebStorage (well LocalStorage) is terrible from a performance point of view because it is synchronous, and I'd be very happy if we could discourage its usage and give people more reasons to adopt a better API like IndexedDB. -Darin I don't understand how storing values in a hash map is performant terrible.
Re: Structured clone in WebStorage
On Nov/29/2010 9:59 AM, ext Adrian Bateman wrote: On Wednesday, November 24, 2010 3:01 AM, Jeremy Orlow wrote: For over a year now, the WebStorage spec has stipulated that Local/SessionStorage store and retrieve objects per the structured clone algorithm rather than strings. And yet there isn't a single implementation who's implemented this. I've talked to people in the know from several of the other major browsers and, although no one is super against implementing it (including us), no one has it on any of their (even internal) roadmaps. It's just not a high enough priority for anyone at the moment. I feel pretty strongly that we should _at least_ put in some non-normative note that no browser vendor is currently planning on implementing this feature. Or, better yet, just remove it from the spec until support starts emerging. I agree. We have no plans to support this in the near future either. At the very least, I think this should be noted as a feature at risk in the Call for Implementations [1]. I don't have a strong preference for removing this feature or marking it as a Feature At Risk when the Candidate is published. It would be good to get feedback from other implementers (Maciej?, Jonas?, Anne?). If no one plans to implement it, perhaps it should just be removed. -Art Barstow
Re: Structured clone in WebStorage
On Thu, Dec 2, 2010 at 5:45 AM, Arthur Barstow art.bars...@nokia.com wrote: On Nov/29/2010 9:59 AM, ext Adrian Bateman wrote: On Wednesday, November 24, 2010 3:01 AM, Jeremy Orlow wrote: For over a year now, the WebStorage spec has stipulated that Local/SessionStorage store and retrieve objects per the structured clone algorithm rather than strings. And yet there isn't a single implementation who's implemented this. I've talked to people in the know from several of the other major browsers and, although no one is super against implementing it (including us), no one has it on any of their (even internal) roadmaps. It's just not a high enough priority for anyone at the moment. I feel pretty strongly that we should _at least_ put in some non-normative note that no browser vendor is currently planning on implementing this feature. Or, better yet, just remove it from the spec until support starts emerging. I agree. We have no plans to support this in the near future either. At the very least, I think this should be noted as a feature at risk in the Call for Implementations [1]. I don't have a strong preference for removing this feature or marking it as a Feature At Risk when the Candidate is published. It would be good to get feedback from other implementers (Maciej?, Jonas?, Anne?). If no one plans to implement it, perhaps it should just be removed. I won't be the person implementing it, but fwiw I highly value having structured clones actually work. Any time I talk about localStorage or similar, I get people asking about storing non-string data, and not wanting to have to futz around with rolling their own serialization. ~TJ
Re: Structured clone in WebStorage
On Thu, Dec 2, 2010 at 5:06 PM, Tab Atkins Jr. jackalm...@gmail.com wrote: On Thu, Dec 2, 2010 at 5:45 AM, Arthur Barstow art.bars...@nokia.com wrote: On Nov/29/2010 9:59 AM, ext Adrian Bateman wrote: On Wednesday, November 24, 2010 3:01 AM, Jeremy Orlow wrote: For over a year now, the WebStorage spec has stipulated that Local/SessionStorage store and retrieve objects per the structured clone algorithm rather than strings. And yet there isn't a single implementation who's implemented this. I've talked to people in the know from several of the other major browsers and, although no one is super against implementing it (including us), no one has it on any of their (even internal) roadmaps. It's just not a high enough priority for anyone at the moment. I feel pretty strongly that we should _at least_ put in some non-normative note that no browser vendor is currently planning on implementing this feature. Or, better yet, just remove it from the spec until support starts emerging. I agree. We have no plans to support this in the near future either. At the very least, I think this should be noted as a feature at risk in the Call for Implementations [1]. I don't have a strong preference for removing this feature or marking it as a Feature At Risk when the Candidate is published. It would be good to get feedback from other implementers (Maciej?, Jonas?, Anne?). If no one plans to implement it, perhaps it should just be removed. I won't be the person implementing it, but fwiw I highly value having structured clones actually work. Any time I talk about localStorage or similar, I get people asking about storing non-string data, and not wanting to have to futz around with rolling their own serialization. The spec should reflect reality and not be a collection of cool ideas that may or may not ever be implemented. I'm not arguing the merits of the feature, I'm arguing that until at least a single vendor starts implementing this, it's confusing for developers to see such text in the spec--especially so when said text has been there for over a year. J
Re: Structured clone in WebStorage
* Tab Atkins Jr. wrote: I won't be the person implementing it, but fwiw I highly value having structured clones actually work. Any time I talk about localStorage or similar, I get people asking about storing non-string data, and not wanting to have to futz around with rolling their own serialization. For most who'd consider rolling their own, JSON.stringify would seem to be a viable alternative. -- Björn Höhrmann · mailto:bjo...@hoehrmann.de · http://bjoern.hoehrmann.de Am Badedeich 7 · Telefon: +49(0)160/4415681 · http://www.bjoernsworld.de 25899 Dagebüll · PGP Pub. KeyID: 0xA4357E78 · http://www.websitedev.de/
Re: Structured clone in WebStorage
On Thu, Dec 2, 2010 at 5:45 AM, Arthur Barstow art.bars...@nokia.com wrote: On Nov/29/2010 9:59 AM, ext Adrian Bateman wrote: On Wednesday, November 24, 2010 3:01 AM, Jeremy Orlow wrote: For over a year now, the WebStorage spec has stipulated that Local/SessionStorage store and retrieve objects per the structured clone algorithm rather than strings. And yet there isn't a single implementation who's implemented this. I've talked to people in the know from several of the other major browsers and, although no one is super against implementing it (including us), no one has it on any of their (even internal) roadmaps. It's just not a high enough priority for anyone at the moment. I feel pretty strongly that we should _at least_ put in some non-normative note that no browser vendor is currently planning on implementing this feature. Or, better yet, just remove it from the spec until support starts emerging. I agree. We have no plans to support this in the near future either. At the very least, I think this should be noted as a feature at risk in the Call for Implementations [1]. I don't have a strong preference for removing this feature or marking it as a Feature At Risk when the Candidate is published. It would be good to get feedback from other implementers (Maciej?, Jonas?, Anne?). If no one plans to implement it, perhaps it should just be removed. I personally would like to see it implemented in Firefox (and other browsers), but I don't feel super strongly. It's something that we likely will be discussing in a few weeks here at Mozilla. One important data-point if the timeline for adding structured-clones to localStorage has similar or longer timeline than adding support for IndexedDB for the various browsers... / Jonas
Re: Structured clone in WebStorage
On Thu, Dec 2, 2010 at 6:29 PM, Jonas Sicking jo...@sicking.cc wrote: On Thu, Dec 2, 2010 at 5:45 AM, Arthur Barstow art.bars...@nokia.com wrote: On Nov/29/2010 9:59 AM, ext Adrian Bateman wrote: On Wednesday, November 24, 2010 3:01 AM, Jeremy Orlow wrote: For over a year now, the WebStorage spec has stipulated that Local/SessionStorage store and retrieve objects per the structured clone algorithm rather than strings. And yet there isn't a single implementation who's implemented this. I've talked to people in the know from several of the other major browsers and, although no one is super against implementing it (including us), no one has it on any of their (even internal) roadmaps. It's just not a high enough priority for anyone at the moment. I feel pretty strongly that we should _at least_ put in some non-normative note that no browser vendor is currently planning on implementing this feature. Or, better yet, just remove it from the spec until support starts emerging. I agree. We have no plans to support this in the near future either. At the very least, I think this should be noted as a feature at risk in the Call for Implementations [1]. I don't have a strong preference for removing this feature or marking it as a Feature At Risk when the Candidate is published. It would be good to get feedback from other implementers (Maciej?, Jonas?, Anne?). If no one plans to implement it, perhaps it should just be removed. I personally would like to see it implemented in Firefox (and other browsers), but I don't feel super strongly. It's something that we likely will be discussing in a few weeks here at Mozilla. My understanding is that many people across many browsers have thought it was a cool idea and would have been happy to have seen it implemented. But no one has done so. Which is why I think we should _at least_ add a non-normative note stating the situation to the spec. Once it's being implemented then, by all means, we can remove it. But who knows how much longer it'll be before anyone actually implements it. One important data-point if the timeline for adding structured-clones to localStorage has similar or longer timeline than adding support for IndexedDB for the various browsers... There is a 99.99% chance Chromium will be shipping IndexedDB before structured clone support for local storage. And there's almost no chance we'll be the first to ship structured clone support for local storage, but we'll likely follow if/when others do. J
Re: Structured clone in WebStorage
On Dec 2, 2010, at 5:45 AM, Arthur Barstow wrote: On Nov/29/2010 9:59 AM, ext Adrian Bateman wrote: On Wednesday, November 24, 2010 3:01 AM, Jeremy Orlow wrote: For over a year now, the WebStorage spec has stipulated that Local/SessionStorage store and retrieve objects per the structured clone algorithm rather than strings. And yet there isn't a single implementation who's implemented this. I've talked to people in the know from several of the other major browsers and, although no one is super against implementing it (including us), no one has it on any of their (even internal) roadmaps. It's just not a high enough priority for anyone at the moment. I feel pretty strongly that we should _at least_ put in some non-normative note that no browser vendor is currently planning on implementing this feature. Or, better yet, just remove it from the spec until support starts emerging. I agree. We have no plans to support this in the near future either. At the very least, I think this should be noted as a feature at risk in the Call for Implementations [1]. I don't have a strong preference for removing this feature or marking it as a Feature At Risk when the Candidate is published. It would be good to get feedback from other implementers (Maciej?, Jonas?, Anne?). If no one plans to implement it, perhaps it should just be removed. We think this feature would be straightforward to implement in Safari/WebKit, and we think it is a useful feature. We would like to implement it at some point. I can't give a specific timeline. Regards, Maciej
Re: Structured clone in WebStorage
On Dec 2, 2010, at 10:41 AM, Jeremy Orlow wrote: On Thu, Dec 2, 2010 at 6:29 PM, Jonas Sicking jo...@sicking.cc wrote: On Thu, Dec 2, 2010 at 5:45 AM, Arthur Barstow art.bars...@nokia.com wrote: On Nov/29/2010 9:59 AM, ext Adrian Bateman wrote: On Wednesday, November 24, 2010 3:01 AM, Jeremy Orlow wrote: For over a year now, the WebStorage spec has stipulated that Local/SessionStorage store and retrieve objects per the structured clone algorithm rather than strings. And yet there isn't a single implementation who's implemented this. I've talked to people in the know from several of the other major browsers and, although no one is super against implementing it (including us), no one has it on any of their (even internal) roadmaps. It's just not a high enough priority for anyone at the moment. I feel pretty strongly that we should _at least_ put in some non-normative note that no browser vendor is currently planning on implementing this feature. Or, better yet, just remove it from the spec until support starts emerging. I agree. We have no plans to support this in the near future either. At the very least, I think this should be noted as a feature at risk in the Call for Implementations [1]. I don't have a strong preference for removing this feature or marking it as a Feature At Risk when the Candidate is published. It would be good to get feedback from other implementers (Maciej?, Jonas?, Anne?). If no one plans to implement it, perhaps it should just be removed. I personally would like to see it implemented in Firefox (and other browsers), but I don't feel super strongly. It's something that we likely will be discussing in a few weeks here at Mozilla. My understanding is that many people across many browsers have thought it was a cool idea and would have been happy to have seen it implemented. But no one has done so. Which is why I think we should _at least_ add a non-normative note stating the situation to the spec. Once it's being implemented then, by all means, we can remove it. But who knows how much longer it'll be before anyone actually implements it. I don't think it is necessary for specs to include non-normative notes about the current implementation status of particular features. I would be ok with marking the feature at risk if it still lacks implementations by the time Web Storage goes to CR. Regards, Maciej
Re: Structured clone in WebStorage
On Thu, 02 Dec 2010 21:51:29 +0100, Maciej Stachowiak m...@apple.com wrote: We think this feature would be straightforward to implement in Safari/WebKit, and we think it is a useful feature. We would like to implement it at some point. I can't give a specific timeline. I am not sure if it is straightforward to implement in Opera, but this applies to us as well. -- Anne van Kesteren http://annevankesteren.nl/
RE: Structured clone in WebStorage
On Wednesday, November 24, 2010 3:01 AM, Jeremy Orlow wrote: For over a year now, the WebStorage spec has stipulated that Local/SessionStorage store and retrieve objects per the structured clone algorithm rather than strings. And yet there isn't a single implementation who's implemented this. I've talked to people in the know from several of the other major browsers and, although no one is super against implementing it (including us), no one has it on any of their (even internal) roadmaps. It's just not a high enough priority for anyone at the moment. I feel pretty strongly that we should _at least_ put in some non-normative note that no browser vendor is currently planning on implementing this feature. Or, better yet, just remove it from the spec until support starts emerging. I agree. We have no plans to support this in the near future either. At the very least, I think this should be noted as a feature at risk in the Call for Implementations [1]. Adrian. [1] http://www.w3.org/2005/10/Process-20051014/tr.html#cfi
Structured clone in WebStorage
For over a year now, the WebStorage spec has stipulated that Local/SessionStorage store and retrieve objects per the structured clone algorithm rather than strings. And yet there isn't a single implementation who's implemented this. I've talked to people in the know from several of the other major browsers and, although no one is super against implementing it (including us), no one has it on any of their (even internal) roadmaps. It's just not a high enough priority for anyone at the moment. Yet any person who reads the spec gets a very different impression [1]. I personally have talked to dozens of very confused individuals who actually read the spec and are confused as to why they can't store anything other than strings in any implementation. Our developer relations team has talked to many more. And everyone keeps thinking support for structured clones is just around the horizon. I feel pretty strongly that we should _at least_ put in some non-normative note that no browser vendor is currently planning on implementing this feature. Or, better yet, just remove it from the spec until support starts emerging. J [1] The WebStorage specs is one of the most approachable and thus people actually do read it. In fact, even for many expert web developers, it's the first spec they've ever read.
Re: [webstorage] deleting a database
On Tue, 30 Jun 2009, João Eiras wrote: The webstorage specification needs an API to delete a database, because there's no such way to do it currently. Something like deleteDatabase(name) would be good. And after calling it, all open transactions would be marked as invalid. The user agent could then delete the data on disk and do all the necessary clean up. Deleting all the data in the database (i.e. dropping all the tables) should be sufficient, no? There's nothing to store if you don't have any data in the database (in particular, there's no reason to store the version as far as I can tell -- you can just act as if the user deleted the database as soon as the database is empty). -- Ian Hickson U+1047E)\._.,--,'``.fL http://ln.hixie.ch/ U+263A/, _.. \ _\ ;`._ ,. Things that are impossible just take longer. `._.-(,_..'--(,_..'`-.;.'
Re: [webstorage] deleting a database
Deleting all the data in the database (i.e. dropping all the tables) should be sufficient, no? There's nothing to store if you don't have any data in the database (in particular, there's no reason to store the version as far as I can tell -- you can just act as if the user deleted the database as soon as the database is empty). Well, I thought about that already, but that's an optimization that can be done by the user agent. Should a specification suggest low level optimizations ? Its also needed to delete views. Tables is not enough, and currently I'm not thinking of anything else. So developers need to be educated: drop all tables, drop all views. Would be much more simple to just delete the database explicitly, and should be straightforward for most implementations given that those also need to support their delete private data functionality.
Re: WebStorage and WebDatabase - creation and exceptions
On Mon, 31 Aug 2009, Nikunj R. Mehta wrote: In WebDatabase: The user agent may raise a SECURITY_ERR exception instead of returning a Database object if the request violates a policy decision (e.g. if the user agent is configured to not allow the page to open databases). In WebStorage (emphasis mine): When a new HTMLDocument is created, the user agent must check to see if the document's top-level browsing context has allocated a session storage area for that document's origin. If it has not, a new storage area for that document's origin must be created. When the localStorage attribute is accessed, the user agent must check to see if it has allocated a local storage area for the origin of the Document of the Window object on which the method was invoked. If it has not, a new storage area for that origin must be created. A browser may not allow local storage for a certain origin, just like it may not allow cookies to be stored. What is the expected behavior in that case? Alternatively, why allow browsers to selectively block out WebDatabase and not other kinds of storage? On Mon, 21 Sep 2009, Jonas Sicking wrote: The inconsistency seems like an oversight to me. Though I'm not sure what the correct way to do this is. It seems silly to for *every* feature mention that the UA may do something different if the user has opted to disable a feature for whatever reason. If I as an implementor wanted to allow my users to disable addEventListener, I would do so even if the spec doesn't explicitly allow it. On the other hand, it might be somewhat useful to give implementors a hint on *how* to disable a feature if it decides to do so. For example for addEventListener it might be appropriate to simply do nothing rather than throw an exception. I've added the hint to the localStorage spec for consistency. -- Ian Hickson U+1047E)\._.,--,'``.fL http://ln.hixie.ch/ U+263A/, _.. \ _\ ;`._ ,. Things that are impossible just take longer. `._.-(,_..'--(,_..'`-.;.'
Re: [webstorage] deleting a database
On Thu, 26 Nov 2009, João Eiras wrote: Deleting all the data in the database (i.e. dropping all the tables) should be sufficient, no? There's nothing to store if you don't have any data in the database (in particular, there's no reason to store the version as far as I can tell -- you can just act as if the user deleted the database as soon as the database is empty). Well, I thought about that already, but that's an optimization that can be done by the user agent. Should a specification suggest low level optimizations? I don't think so, no. Its also needed to delete views. Tables is not enough, and currently I'm not thinking of anything else. So developers need to be educated: drop all tables, drop all views. Right, they need to delete whatever they added. Would be much more simple to just delete the database explicitly Yes, but is this something that will happen often enough to matter? should be straightforward for most implementations given that those also need to support their delete private data functionality. I have no doubt it would be easy to implement (modulo getting the interaction with existing transactions right). -- Ian Hickson U+1047E)\._.,--,'``.fL http://ln.hixie.ch/ U+263A/, _.. \ _\ ;`._ ,. Things that are impossible just take longer. `._.-(,_..'--(,_..'`-.;.'
Re: [webstorage] deleting a database
Would be much more simple to just delete the database explicitly
Yes, but is this something that will happen often enough to matter?
Unfortunately, that question has two answers. It should happen often, but
I doubt web developers will ever bother to clean up the data they don't
need :)
From my personal experience, it's good to do quick synchronous cleanup
during testing. An heuristic like delete database if data file is empty
would need to be implemented when the database is no longer in use by any
webpage, so the dom object is garbage-collected and the underlying
database deleted.
Still on the same topic, why not also allow database names to be listed
somewhere ? It fits really nicely the use case of developer tools with web
UIs. Something like
interface Window {
readonly DOMStringList databases;//or databaseNames
}
should be enough IMO.
Re: [webstorage] deleting a database
On Thu, 26 Nov 2009, João Eiras wrote:
Would be much more simple to just delete the database explicitly
Yes, but is this something that will happen often enough to matter?
Unfortunately, that question has two answers. It should happen often,
but I doubt web developers will ever bother to clean up the data they
don't need :)
From my personal experience, it's good to do quick synchronous cleanup
during testing. An heuristic like delete database if data file is
empty would need to be implemented when the database is no longer in
use by any webpage, so the dom object is garbage-collected and the
underlying database deleted.
During testing you can do it manually (why would you do it from code?). I
think in practice we wouldn't see production code deleting databases.
Still on the same topic, why not also allow database names to be listed
somewhere ? It fits really nicely the use case of developer tools with
web UIs. Something like
interface Window {
readonly DOMStringList databases;//or databaseNames
}
should be enough IMO.
If the API gains traction, that would be a sensible addition. I don't want
to add features at this point without more adoption, though.
--
Ian Hickson U+1047E)\._.,--,'``.fL
http://ln.hixie.ch/ U+263A/, _.. \ _\ ;`._ ,.
Things that are impossible just take longer. `._.-(,_..'--(,_..'`-.;.'
Re: [webstorage] deleting a database
On Wed, 25 Nov 2009, ATSUSHI TAKAYAMA wrote: (Re-sending because I accidentally replied personally...) Deleting all the data in the database (i.e. dropping all the tables) SQLite 3 does not vacuum the database file when you drop a table. http://www.sqlite.org/lang_droptable.html That seems like an implementation issue. If you repeat create and drop tables, you will hit the database file size limit after possibly a long time. If this occurred only on a client side (i.e. not in the test environment), web developers wouldn't know that an error is occurring. Therefore, it would be nice to have a method to tell how much size the database is using. I don't think that such a number would make much sense. Different implementations could have radically different space usage for the same data. If the database implementation is inefficient, that's an implementation issue, not a Web application issue. -- Ian Hickson U+1047E)\._.,--,'``.fL http://ln.hixie.ch/ U+263A/, _.. \ _\ ;`._ ,. Things that are impossible just take longer. `._.-(,_..'--(,_..'`-.;.'
Re: [webstorage] deleting a database
During testing you can do it manually (why would you do it from code?). Unit tests.
Re: [webstorage] deleting a database
(Re-sending because I accidentally replied personally...) Deleting all the data in the database (i.e. dropping all the tables) SQLite 3 does not vacuum the database file when you drop a table. http://www.sqlite.org/lang_droptable.html If you repeat create and drop tables, you will hit the database file size limit after possibly a long time. If this occurred only on a client side (i.e. not in the test environment), web developers wouldn't know that an error is occurring. Therefore, it would be nice to have a method to tell how much size the database is using. A. Takayama
Resending Re: WebStorage and WebDatabase - creation and exceptions
There was no response to this earlier, so resending it. Please answer the question: why allow browsers to selectively block out WebDatabase and not other kinds of storage? Nikunj On Aug 31, 2009, at 11:07 AM, Nikunj R. Mehta wrote: In WebDatabase: The user agent may raise a SECURITY_ERR exception instead of returning a Database object if the request violates a policy decision (e.g. if the user agent is configured to not allow the page to open databases). In WebStorage (emphasis mine): When a new HTMLDocument is created, the user agent must check to see if the document's top-level browsing context has allocated a session storage area for that document's origin. If it has not, a new storage area for that document's origin must be created. When the localStorage attribute is accessed, the user agent must check to see if it has allocated a local storage area for the origin of the Document of the Window object on which the method was invoked. If it has not, a new storage area for that origin must be created. A browser may not allow local storage for a certain origin, just like it may not allow cookies to be stored. What is the expected behavior in that case? Alternatively, why allow browsers to selectively block out WebDatabase and not other kinds of storage? Nikunj http://o-micron.blogspot.com Nikunj http://o-micron.blogspot.com
Re: [WebStorage] structured clones serialization
On Wed, Sep 16, 2009 at 2:20 AM, Ian Hickson i...@hixie.ch wrote: On Mon, 14 Sep 2009, Marcos Caceres wrote: With the recent move to structured clones in Web Storage, I'm wondering if a serialization for those clones needs to be specified? Nothing in HTML5 exposes a serialisation. I know that (well, the DOM is serialized as HTML or XHTML markup;)). My question was: does a serialization of structured clones needs to be specified? If no, why not? A use case for widgets is widget preferences, where the preference's in a widget's configuration document are used to populate a storage object. At the moment, we only support strings: widget ... preference name=foo value=bar/ /widget Becomes: window.widget.preferences.foo; However, enabling some kind of text-based serialization for structured-clones would allow authors to serialize more complex data. I recommend using JSON, though it doesn't allow everything to be expressed that can be cloned. Understood. This is our current approach. Some other comments for the current editor's draft: I wonder if, where there is overlap, the following two assertions could be merged into a general assertion applied to the an object that implements Storage (as it would be useful for widgets.preferences). [[ 4.2 The sessionStorage attribute User agents should expire data from the local storage areas only for security reasons or when requested to do so by the user. User agents should always avoid deleting data while a script that could access that data is running. 4.3 The localStorage attribute User agents should not expire data from a browsing context's session storage areas, but may do so when the user requests that such data be deleted, or when the UA detects that it has limited storage space, or for security reasons. User agents should always avoid deleting data while a script that could access that data is running. When a top-level browsing context is destroyed (and therefore permanently inaccessible to the user) the data stored in its session storage areas can be discarded with it, as the API described in this specification provides no way for that data to ever be subsequently retrieved. ]] There seems to be very little overlap; I'm not sure there's enough to really justify a separate section. (For example, if we were to remove the bit about security reasons for deleting data, I wouldn't bother putting it into a common section, I'd just remove it altogether. It doesn't really add much, the only reason I included it was completeness so that people reading the above paragraphs didn't think I was trying to override UA decisions on security.) Because each type of storage area can have its own rules, I recommend having a separate paragraph for each one. It's possible for Storage objects to be volatile to the point where items can be deleted while script is running, that's a decision for whatever spec uses it. Ok, that makes good sense. We will evaluate which behavior suits us best, and copy the appropriate bits from either localStorage or sessionStorage. Thanks for your help and time! Kind regards, Marcos -- Marcos Caceres http://datadriven.com.au
Re: [WebStorage] structured clones serialization
On Mon, 14 Sep 2009, Marcos Caceres wrote: With the recent move to structured clones in Web Storage, I'm wondering if a serialization for those clones needs to be specified? Nothing in HTML5 exposes a serialisation. A use case for widgets is widget preferences, where the preference's in a widget's configuration document are used to populate a storage object. At the moment, we only support strings: widget ... preference name=foo value=bar/ /widget Becomes: window.widget.preferences.foo; However, enabling some kind of text-based serialization for structured-clones would allow authors to serialize more complex data. I recommend using JSON, though it doesn't allow everything to be expressed that can be cloned. Some other comments for the current editor's draft: I wonder if, where there is overlap, the following two assertions could be merged into a general assertion applied to the an object that implements Storage (as it would be useful for widgets.preferences). [[ 4.2 The sessionStorage attribute User agents should expire data from the local storage areas only for security reasons or when requested to do so by the user. User agents should always avoid deleting data while a script that could access that data is running. 4.3 The localStorage attribute User agents should not expire data from a browsing context's session storage areas, but may do so when the user requests that such data be deleted, or when the UA detects that it has limited storage space, or for security reasons. User agents should always avoid deleting data while a script that could access that data is running. When a top-level browsing context is destroyed (and therefore permanently inaccessible to the user) the data stored in its session storage areas can be discarded with it, as the API described in this specification provides no way for that data to ever be subsequently retrieved. ]] There seems to be very little overlap; I'm not sure there's enough to really justify a separate section. (For example, if we were to remove the bit about security reasons for deleting data, I wouldn't bother putting it into a common section, I'd just remove it altogether. It doesn't really add much, the only reason I included it was completeness so that people reading the above paragraphs didn't think I was trying to override UA decisions on security.) Because each type of storage area can have its own rules, I recommend having a separate paragraph for each one. It's possible for Storage objects to be volatile to the point where items can be deleted while script is running, that's a decision for whatever spec uses it. In the spec it sez: The use of parseInt() is needed when dealing with numbers (integers in this case) because the storage APIs are all string-based. Is the above still true with the introduction of structured clones? Fixed. In the spec it sez: Storage areas (both session storage and local storage) store strings. To store structured data in a storage area, you must first convert it to a string. As above. I'm all for casual language in a spec (makes it fun to read!), but you should really make it clear that it's the author and not the UA that needs to do the conversion. It's gone now, since it's no longer true. Cheers, -- Ian Hickson U+1047E)\._.,--,'``.fL http://ln.hixie.ch/ U+263A/, _.. \ _\ ;`._ ,. Things that are impossible just take longer. `._.-(,_..'--(,_..'`-.;.'
[WebStorage] structured clones serialization
Hi Ian, With the recent move to structured clones in Web Storage, I'm wondering if a serialization for those clones needs to be specified? A use case for widgets is widget preferences, where the preference's in a widget's configuration document are used to populate a storage object. At the moment, we only support strings: widget ... preference name=foo value=bar/ /widget Becomes: window.widget.preferences.foo; However, enabling some kind of text-based serialization for structured-clones would allow authors to serialize more complex data. Some other comments for the current editor's draft: I wonder if, where there is overlap, the following two assertions could be merged into a general assertion applied to the an object that implements Storage (as it would be useful for widgets.preferences). [[ 4.2 The sessionStorage attribute User agents should expire data from the local storage areas only for security reasons or when requested to do so by the user. User agents should always avoid deleting data while a script that could access that data is running. 4.3 The localStorage attribute User agents should not expire data from a browsing context's session storage areas, but may do so when the user requests that such data be deleted, or when the UA detects that it has limited storage space, or for security reasons. User agents should always avoid deleting data while a script that could access that data is running. When a top-level browsing context is destroyed (and therefore permanently inaccessible to the user) the data stored in its session storage areas can be discarded with it, as the API described in this specification provides no way for that data to ever be subsequently retrieved. ]] In the spec it sez: The use of parseInt() is needed when dealing with numbers (integers in this case) because the storage APIs are all string-based. Is the above still true with the introduction of structured clones? In the spec it sez: Storage areas (both session storage and local storage) store strings. To store structured data in a storage area, you must first convert it to a string. As above. I'm all for casual language in a spec (makes it fun to read!), but you should really make it clear that it's the author and not the UA that needs to do the conversion. -- Marcos Caceres http://datadriven.com.au
WebStorage and WebDatabase - creation and exceptions
In WebDatabase: The user agent may raise a SECURITY_ERR exception instead of returning a Database object if the request violates a policy decision (e.g. if the user agent is configured to not allow the page to open databases). In WebStorage (emphasis mine): When a new HTMLDocument is created, the user agent must check to see if the document's top-level browsing context has allocated a session storage area for that document's origin. If it has not, a new storage area for that document's origin must be created. When the localStorage attribute is accessed, the user agent must check to see if it has allocated a local storage area for the origin of the Document of the Window object on which the method was invoked. If it has not, a new storage area for that origin must be created. A browser may not allow local storage for a certain origin, just like it may not allow cookies to be stored. What is the expected behavior in that case? Alternatively, why allow browsers to selectively block out WebDatabase and not other kinds of storage? Nikunj http://o-micron.blogspot.com
Re: [WebStorage] Keepin' it in the family?
On Sun, 30 Aug 2009, Arthur Barstow wrote: When do you plan a new publication of Web Storage (in W3C's /TR/ space) and when do you expect this spec to be ready for Last Call WD publication? We can publish a WD whenever you like. I hope to be ready for LC within a few weeks. -- Ian Hickson U+1047E)\._.,--,'``.fL http://ln.hixie.ch/ U+263A/, _.. \ _\ ;`._ ,. Things that are impossible just take longer. `._.-(,_..'--(,_..'`-.;.'
[WebStorage] Keepin' it in the family?
Hi Hixie, I recently noticed some substantial changes to Web Storage (allowing things other than strings to be stored). Did the discussion to make such a change happen on the web-apps working group's mailing list? If yes, please disregard this email. If not, could you please, in the future, CC discussions about Web Storage to public-webapps (as the work is supposed to be taking place in this working group). Selfishly, Widgets have a dependency on that spec, so some of us here need to be kept up-to-date :) Kind regards, Marcos -- Marcos Caceres http://datadriven.com.au
Re: [WebStorage] Keepin' it in the family?
On Thu, 20 Aug 2009, Marcos Caceres wrote: I recently noticed some substantial changes to Web Storage (allowing things other than strings to be stored). Did the discussion to make such a change happen on the web-apps working group's mailing list? If yes, please disregard this email. If not, could you please, in the future, CC discussions about Web Storage to public-webapps (as the work is supposed to be taking place in this working group). Selfishly, Widgets have a dependency on that spec, so some of us here need to be kept up-to-date :) Discussion of Web Storage happens all over the place (on public-webapps, on wha...@whatwg.org, on IRC channels, on bug systems, etc). Changes, however, are all posted to the various mechanisms that track changes to HTML5, in particular commit-watch...@lists.whatwg.org, @WHATWG on Twitter, and the SVN tracker: http://html5.org/tools/web-apps-tracker The recent change to what can be stored was a result of the work on the File API. -- Ian Hickson U+1047E)\._.,--,'``.fL http://ln.hixie.ch/ U+263A/, _.. \ _\ ;`._ ,. Things that are impossible just take longer. `._.-(,_..'--(,_..'`-.;.'
Re: [WebStorage] Keepin' it in the family?
Ian Hickson wrote: On Thu, 20 Aug 2009, Marcos Caceres wrote: I recently noticed some substantial changes to Web Storage (allowing things other than strings to be stored). Did the discussion to make such a change happen on the web-apps working group's mailing list? If yes, please disregard this email. If not, could you please, in the future, CC discussions about Web Storage to public-webapps (as the work is supposed to be taking place in this working group). Selfishly, Widgets have a dependency on that spec, so some of us here need to be kept up-to-date :) Discussion of Web Storage happens all over the place (on public-webapps, on wha...@whatwg.org, on IRC channels, on bug systems, etc). Changes, however, are all posted to the various mechanisms that track changes to HTML5, in particular commit-watch...@lists.whatwg.org, @WHATWG on Twitter, and the SVN tracker: http://html5.org/tools/web-apps-tracker Be nice if this tool also identified the spec to which the change applied (as Web Storage is not part of HTML5, or is it still part of the WHAT-WG draft?). The recent change to what can be stored was a result of the work on the File API. Ok, I see.
RE: [WebStorage] Concerns on spec section 'Processing Model'
On Thu, 30 Jul 2009, Laxmi Narsimha Rao Oruganti wrote: I am unable to understand on how we are expecting spec readers to interpret the spec lines. At one end, we are not open to make it clear that it is a hint/model, and on the other end we are ok to not implementing that way. Think about a person reading the spec, who is *not* in touch with this discussion forum; he immediately will conclude that he *must* implement in the same way as we describe in the spec. So, yes his implementation will be interoperable but we are essentially making whole world implement the way we want:(. I sincerely hope people put the readers hat than editors hat here. It is so sad to see people are hesitant to detail out implantation aspects as hints. I now have a great respect for IETF specs, where they detail out everything in clear. If we want the spec to have quality, then I suggest keep implementation details completely out, but if we think that it is not the way W3C specs are written. Then, I would suggest keep those implementation details as hints and have the spec text clear like for example The database system should allow only one writer at a time. *One way* vendors could ensure this is by taking up an exclusive lock on the database file . The conformance section says: # Conformance requirements phrased as algorithms or specific steps may be # implemented in any manner, so long as the end result is equivalent. (In # particular, the algorithms defined in this specification are intended to # be easy to follow, and not intended to be performant.) -- http://dev.w3.org/html5/webdatabase/#conformance-requirements Is that not clear enough? -- Ian Hickson U+1047E)\._.,--,'``.fL http://ln.hixie.ch/ U+263A/, _.. \ _\ ;`._ ,. Things that are impossible just take longer. `._.-(,_..'--(,_..'`-.;.'
RE: [WebStorage] Concerns on spec section 'Processing Model'
I am unable to understand on how we are expecting spec readers to interpret the spec lines. At one end, we are not open to make it clear that it is a hint/model, and on the other end we are ok to not implementing that way. Think about a person reading the spec, who is *not* in touch with this discussion forum; he immediately will conclude that he *must* implement in the same way as we describe in the spec. So, yes his implementation will be interoperable but we are essentially making whole world implement the way we want:(. I sincerely hope people put the readers hat than editors hat here. It is so sad to see people are hesitant to detail out implantation aspects as hints. I now have a great respect for IETF specs, where they detail out everything in clear. If we want the spec to have quality, then I suggest keep implementation details completely out, but if we think that it is not the way W3C specs are written. Then, I would suggest keep those implementation details as hints and have the spec text clear like for example The database system should allow only one writer at a time. *One way* vendors could ensure this is by taking up an exclusive lock on the database file . Hope you guys take the criticism in the right spirit. Thanks, Laxmi -Original Message- From: Ian Hickson [mailto:i...@hixie.ch] Sent: Thursday, July 30, 2009 1:24 AM To: Laxmi Narsimha Rao Oruganti; Nikunj R. Mehta; Aaron Boodman Cc: public-webapps@w3.org Subject: Re: [WebStorage] Concerns on spec section 'Processing Model' On Thu, 16 Jul 2009, Laxmi Narsimha Rao Oruganti wrote: - Why is the spec mandating a transaction to take an *exclusive write lock on the entire database*? It is to avoid the possibility that two transactions will conflict and then have one be forced to roll back. For example, we want to make sure that there is no way that a Web page will ever have a failure if it opens a transaction that does: read from row A. read from row B. read from row C. write to row A. write to row B. write to row C. ...even if the page is opened twice, and the two pages both do this transaction at the same time. Since client-side have very low contention, getting a lock on the entire database rather than requiring that the author explicitly lock specific rows or columns is good enough. On Fri, 24 Jul 2009, Nikunj R. Mehta wrote: If you want to provide an application programmer with a limited degree of freedom from a certain class of errors, then there is a different solution. It is called isolation level [1]. When opening a transaction, just provide the required isolation level. Heck, if you'd like, make SERIALIZABLE the default value. This doesn't prevent rollback in the above case as far as I can tell. But don't disallow other possibilities or create the illusion of silver bullets. Why not? The exclusive lock model described in the spec is just a model, it isn't intended to be actually require an exclusive lock. If an implementation can get the same result using some other mechanism, that's fine. The spec says: [[ If the mode is read/write, the transaction must have an exclusive write lock over the entire database ]] Therefore, correct me if I am wrong, but the spec prohibits the following: An implementation of the Database object allows more than one transaction to write in a database while another transaction has a write lock on the same database, it is a failure. Assuming you mean that you would cause one of the two transactions above to fail, then this is not a black-box equivalent implementation of what the spec says, and it is therefore not conforming. If so, then I want to formally object to that spec text because it is overly restrictive on implementers as well as on application programmers. Do you have any alternative proposal that doesn't risk either transaction failing in the above example? 3. A read-only transaction includes inside it a read-write transaction. This isn't possible with the current asynchronous API as far as I can tell. With the synchronous API, it would hang trying to open the read-write transaction for however long it takes the UA to realise that the script that is trying to get the read-write transaction is the same one as the one that has an open read-only transaction, and then it would fail with error code 7. Then again the spec is too restrictive because application programmers need the ability to upgrade their lock from read-only to read-write What are the use cases for this? and an application should never deadlock itself. I would expect implementations to catch this case, but even if they didn't, it would time out eventually anyway. I can add explicit text saying that this should check to see if there is a synchronous read/write transaction open in the same thread; would that help? Therefore, I formally object to the spec disallowing
RE: [WebStorage] Concerns on spec section 'Processing Model'
[Being in a different time zone (IST), was not able to actively engage on this thread :(] There seems to be Update Loss issue here. If the UI thread which is supposed to enqueue the statements was scheduled out in the middle of a transaction block. And the background thread got scheduled in, consumed the already queued items alone part of transaction and committed. That means, we have lost atomicity right. I am sure spec did not intend this. Am I missing something? Thanks, Laxmi -Original Message- From: Aaron Boodman [mailto:a...@google.com] Sent: Saturday, July 25, 2009 6:35 AM To: Nikunj R. Mehta Cc: Ian Hickson; public-webapps WG; Laxmi Narsimha Rao Oruganti Subject: Re: [WebStorage] Concerns on spec section 'Processing Model' On Fri, Jul 24, 2009 at 5:32 PM, Nikunj R. Mehtanikunj.me...@oracle.com wrote: The spec is also silent about what happens if I put a wait by making another asynchronous call inside my transaction callback logic. By inference, this would be allowed since all statements are executed inside callbacks, so why distinguish between transaction and other (non-SQLTransactionErrorCallback) types of callbacks. The processing model in 4.3.2 simply says that the SQL statements are queued up. It is unclear what if anything happens if the database runs out of statements to execute if the transaction logic takes time to add another statement to the queue before the database decides to commit. Am I wrong or is this an ambiguous, but correct interpretation? 4.3.2, step 6 says: While there are any statements queued up in the transaction, perform the following steps for each queued up statement in the transaction, oldest first. Each statement has a statement, optionally a result set callback, and optionally an error callback. ... then there are the steps to process the statement, handle errors ... And then step 8 is: Commit the transaction. So it seems pretty clear to me that the transaction commits automatically when there are no more statement queued and all executed successfully. Also I helped design this, and I can tell you that was the intent. Those who are worried about throwing complexity of transaction recovery on Web programmers should perhaps also be worried about the insane complexity of asynchronous transaction programming, that no one in the world should have to learn. The mainstream database developers don't have to deal with that. Why should poor Web programmers have to suffer this? That is something that is unfortunate. However, it is a browser vendor requirement to not allow synchronous IO on the UI thread, because it leads to blocked web pages and browser UI. So there's not really alternative. This is part of the reason for the introduction of web workers. They can allow synchronous IO because the entire program is running on a separate thread. Moreover, with an asynchronous database the spec doesn't allow an application to rollback a transaction, should certain application logic require that. This is yet another case of creating a storage API that is different from traditional database developers. It does allow it. If an exception is thrown by a developer inside a statement callback, the transaction will be rolled back: 4.3.2, step 6, substep 6: If the callback was invoked and raised an exception, jump to the last step in the overall steps. There seems to be a pattern of ignoring good API practices when interacting with a database and it appears intentional. Am I wrong in my interpretation? I think you may be. It is common when working with databases in C++ to use the RAII pattern (http://en.wikipedia.org/wiki/Resource_Acquisition_Is_Initialization) to explicitly scope transactions. This is done to prevent developers from forgetting to close transactions. The analogue to this pattern in JavaScript is callbacks. You can't use RAII in the same way as C++ with Java, but C# introduced the using statement (http://msdn.microsoft.com/en-us/library/yh598w02.aspx) to allow the same sorts of things. In particular, the .net IDbTransaction interface inherits IDisposable, so that you can use it with the using statement (http://msdn.microsoft.com/en-us/library/system.data.idbtransaction.aspx), and that is the recommended pattern. It does appear that it is possible to hold a transaction open all day with the DatabaseSync interface (http://dev.w3.org/html5/webdatabase/#databasesync). Specifically the SQLTransactionSync method has commit/rollback methods. The DatabaseSync interface was added after I worked on this, so I can't say why it doesn't use callbacks. In any case, I was talking about the async flavor which is what my example code referred to. Do you agree it is not possible to hang transactions open from Database (http://dev.w3.org/html5/webdatabase/#database)? If not, what am I missing? I can't agree simply because the spec says nothing about it. In fact, if anything the rest of the spec text around
RE: [WebStorage] Concerns on spec section 'Processing Model'
That is all the responsibility of database system. We don't need to tell database systems on how to do it, we just need to tell them on what to do. Today database systems do have lock manager which takes care of these responsibilities. Coming to the question of failing transaction unpredictably, even with current specification; transaction do fail. For example, if there exists a writer transaction which is already holding an exclusive lock, this new thread would fail to acquire lock. The failures would be there. Now the next question people would ask is on how do we make sure that partial changes are not causing problem in case of a failure in the middle of sequence of operations. That is the responsibility of transaction manager. Note that transaction manager treats the whole sequence as a single atomic unit. Are we missing something? Thanks, Laxmi -Original Message- From: Ian Hickson [mailto:i...@hixie.ch] Sent: Friday, July 24, 2009 6:55 AM To: Nikunj R. Mehta Cc: public-webapps WG; Laxmi Narsimha Rao Oruganti Subject: Re: [WebStorage] Concerns on spec section 'Processing Model' On Thu, 16 Jul 2009, Nikunj R. Mehta wrote: The spec should not restrict implementations to any one level of concurrency unless there are specific undesirable effects. Restricting the database to a single writer means that if there are separate workers or background threads working to update non-overlapping portions, then they have to wait for the lone current writer. Implementations can certainly compete to produce the level of concurrency that developers need. Specifically, I propose that the following text [[ If the mode is read/write, the transaction must have an exclusive write lock over the entire database. If the mode is read-only, the transaction must have a shared read lock over the entire database. The user agent should wait for an appropriate lock to be available. ]] be replaced with the following text [[ Multiple read-only transactions may share the same data as long as there is no transaction attempting to write the data being read. The user agent must wait for transactions that are reading some data before allowing a read/write transaction on the same data to continue. ]] Since there's no way for the author to say ahead of time which rows or cells the transactions are going to use, how can you do the above without ending up with some transactions failing unpredictably? -- Ian Hickson U+1047E)\._.,--,'``.fL http://ln.hixie.ch/ U+263A/, _.. \ _\ ;`._ ,. Things that are impossible just take longer. `._.-(,_..'--(,_..'`-.;.'
RE: [WebStorage] Concerns on spec section 'Processing Model'
On Fri, 24 Jul 2009, Laxmi Narsimha Rao Oruganti wrote: That is all the responsibility of database system. We don't need to tell database systems on how to do it, we just need to tell them on what to do. Today database systems do have lock manager which takes care of these responsibilities. Coming to the question of failing transaction unpredictably, even with current specification; transaction do fail. For example, if there exists a writer transaction which is already holding an exclusive lock, this new thread would fail to acquire lock. The failures would be there. Now the next question people would ask is on how do we make sure that partial changes are not causing problem in case of a failure in the middle of sequence of operations. That is the responsibility of transaction manager. Note that transaction manager treats the whole sequence as a single atomic unit. As I understand it, with what is specced now, if you try to get a write transaction lock, it will only fail if it times out, which would probably be a symptom of a more serious bug anyway. There's never going to be a forced rollback; once you have got a transaction lock, you are not going to ever have it fail on you unexpectedly. I think this is an important invariant, because otherwise script writers _will_ shoot themselves in the foot. These aren't professional database developers; Web authors span the gamut of developer experience from the novice who is writing code more by luck than by knowledge all the way to the UI designer who wound up stuck with the task for writing the UI logic but has no professional background in programing, let alone concurrency in databases. We can't be firing unexpected exceptions when their users happen to open two tabs to the same application at the same time, leaving data unsaved. -- Ian Hickson U+1047E)\._.,--,'``.fL http://ln.hixie.ch/ U+263A/, _.. \ _\ ;`._ ,. Things that are impossible just take longer. `._.-(,_..'--(,_..'`-.;.'
RE: [WebStorage] Concerns on spec section 'Processing Model'
Let me probe this further to get clarity. [Ian] As I understand it, with what is specced now, if you try to get a write transaction lock, it will only fail if it times out, which would probably be a symptom of a more serious bug anyway. [Ian] There's never going to be a forced rollback; once you have got a transaction lock, you are not going to ever have it fail on you unexpectedly. My understanding of your requirement is Database should allow only one active writer transaction. How the database systems achieve this need not be explained. Note that, this need not be achieved only by acquiring an exclusive lock on the database file. Think about a database implementation which is not a single file based (Log + Checkpoint design model) where there is one data file and a couple of log files. Spec-ing that they have to hold exclusive lock on database file is ambiguous between data file and log file. If you take BDB JE as an example, they don't even have data file. Their model is a sequence of log files. I have a question: - Many of the database systems today don't ask the user to specify the purpose of use (read/write) when opening a transaction. But the spec is expecting to specify the use of transaction for read/write. If we really need this to fly with multiple database systems, we should start that type of choice at connection opening time and not at transaction creation time. A connection typically accepts read/write or read only connection. Nikunj or others should comment on whether taking read/write Vs read-only choice at connection time is practiced in their corresponding products. Thanks, Laxmi -Original Message- From: Ian Hickson [mailto:i...@hixie.ch] Sent: Friday, July 24, 2009 2:07 PM To: Laxmi Narsimha Rao Oruganti Cc: Nikunj R. Mehta; public-webapps WG Subject: RE: [WebStorage] Concerns on spec section 'Processing Model' On Fri, 24 Jul 2009, Laxmi Narsimha Rao Oruganti wrote: That is all the responsibility of database system. We don't need to tell database systems on how to do it, we just need to tell them on what to do. Today database systems do have lock manager which takes care of these responsibilities. Coming to the question of failing transaction unpredictably, even with current specification; transaction do fail. For example, if there exists a writer transaction which is already holding an exclusive lock, this new thread would fail to acquire lock. The failures would be there. Now the next question people would ask is on how do we make sure that partial changes are not causing problem in case of a failure in the middle of sequence of operations. That is the responsibility of transaction manager. Note that transaction manager treats the whole sequence as a single atomic unit. As I understand it, with what is specced now, if you try to get a write transaction lock, it will only fail if it times out, which would probably be a symptom of a more serious bug anyway. There's never going to be a forced rollback; once you have got a transaction lock, you are not going to ever have it fail on you unexpectedly. I think this is an important invariant, because otherwise script writers _will_ shoot themselves in the foot. These aren't professional database developers; Web authors span the gamut of developer experience from the novice who is writing code more by luck than by knowledge all the way to the UI designer who wound up stuck with the task for writing the UI logic but has no professional background in programing, let alone concurrency in databases. We can't be firing unexpected exceptions when their users happen to open two tabs to the same application at the same time, leaving data unsaved. -- Ian Hickson U+1047E)\._.,--,'``.fL http://ln.hixie.ch/ U+263A/, _.. \ _\ ;`._ ,. Things that are impossible just take longer. `._.-(,_..'--(,_..'`-.;.'
Re: [WebStorage] Concerns on spec section 'Processing Model'
On Fri, Jul 24, 2009 at 1:54 PM, Nikunj R. Mehtanikunj.me...@oracle.com wrote: Experience has shown that there is no easy way out when dealing with transactions, and locking at the whole database level is no solution to failures. The thing that makes the web browser environment different an interesting is that multiple independent applications can end up having access to the same database if the run in the same origin. This could be multiple instances of the same app (eg multiple gmail windows) or just different apps that happen to be on the same origin (many Google apps run on www.google.com). Because these apps are isolated from each other, they have no way to cooperate to reduce conflicts. They also have no control over whether there are multiple copies of themselves (the user control this). Therefore if the platform does not protect against this, basically any statement can fail due to conflict. This was a big problem with Gears, and led to applications having to go to crazy contortions to do things like master election. When we designed the HTML5 version of the database API we specifically tried to avoid it. I do not agree that database-level locking is a big problem for web applications. They are typically serving as most a handful of clients. As long as you can specify read vs read/write transactions, I think the current design is the correct trade-off in terms of complexity and correctness. - a
Re: [WebStorage] Concerns on spec section 'Processing Model'
On Fri, Jul 24, 2009 at 2:06 PM, Aaron Boodmana...@google.com wrote: I do not agree that database-level locking is a big problem for web applications. Preemptive correction: I mean for the client-side of web applications. There are usually at most a handful of clients accessing an HTML5 database instance. - a
Re: [WebStorage] Concerns on spec section 'Processing Model'
On Fri, Jul 24, 2009 at 2:17 PM, Nikunj R. Mehtanikunj.me...@oracle.com wrote:
On Jul 24, 2009, at 1:36 AM, Ian Hickson wrote:
On Fri, 24 Jul 2009, Laxmi Narsimha Rao Oruganti wrote:
That is all the responsibility of database system. We don't need to
tell database systems on how to do it, we just need to tell them on what
to do. Today database systems do have lock manager which takes care of
these responsibilities.
Coming to the question of failing transaction unpredictably, even with
current specification; transaction do fail. For example, if there
exists a writer transaction which is already holding an exclusive lock,
this new thread would fail to acquire lock. The failures would be
there.
Now the next question people would ask is on how do we make sure that
partial changes are not causing problem in case of a failure in the
middle of sequence of operations. That is the responsibility of
transaction manager. Note that transaction manager treats the whole
sequence as a single atomic unit.
As I understand it, with what is specced now, if you try to get a write
transaction lock, it will only fail if it times out, which would probably
be a symptom of a more serious bug anyway.
Can you explain a more serious bug? The write lock may actually happen in
the middle of a read-only transaction, can't it? I don't see spec text
prohibiting that.
It's not clear to me what you're asking about in this paragraph. Write
transactions should be exclusive, read transactions can be shared.
Write transactions queue until all open read transactions complete.
I can't find the actual spec right now (where is its canonical home
now?) so I can't point at the exact text, but that is the goal I
believe.
There's never going to be a
forced rollback; once you have got a transaction lock, you are not going
to ever have it fail on you unexpectedly.
Even if you have a transaction lock,
1. the application logic could cause an exception
2. the application finds an unacceptable data condition and needs to
rollback the transaction
In these cases, the developer caused the exception himself, and it
only affects his own application. So it's not unexpected in the same
sense locking exceptions are.
3. face a disk failure
Yes, but there is nothing that the platform can do about this case. It
is an exception in the truest sense of the word.
4. encounter a bug in the underlying software
The platform shouldn't have API just in case underlying software has
bugs. Throwing an exception and unwinding the transaction is the most
sensible thing to do and what it does now.
In either of these cases, how would the application code be expected to
recover?
It can't, but these are exceptional circumstances that are generally
unexpected and indicate bigger problems. This is different from access
conflicts which are expected and do not indicate bigger problems. The
platform can and should make this easier.
These aren't professional database
developers; Web authors span the gamut of developer experience from the
novice who is writing code more by luck than by knowledge all the way to
the UI designer who wound up stuck with the task for writing the UI logic
but has no professional background in programing, let alone concurrency in
databases.
This is a strong reason to avoid SQL in the front-end.
I am also interested in a non-SQL storage API, but I think this is a
separate issue.
We can't be firing unexpected exceptions when their users
happen to open two tabs to the same application at the same time, leaving
data unsaved.
So you'd much rather tell an application user that they should close one of
the two tabs since they can't obtain a read-write lock in both. I still
don't understand how the exclusive database lock helps. Would you please
elaborate?
I think you are either misunderstanding the spec or it has a bug,
because this is not the intent. Requests to obtain read/write
transactions are asynchronous and are queued until they can be
granted:
myDatabase.transaction(function(tx) { // this callback doesn't occur
until the caller has exclusive access
});
- a
RE: [WebStorage] Concerns on spec section 'Processing Model'
On Fri, 24 Jul 2009, Laxmi Narsimha Rao Oruganti wrote: Let me probe this further to get clarity. As I understand it, with what is specced now, if you try to get a write transaction lock, it will only fail if it times out, which would probably be a symptom of a more serious bug anyway. There's never going to be a forced rollback; once you have got a transaction lock, you are not going to ever have it fail on you unexpectedly. My understanding of your requirement is Database should allow only one active writer transaction. How the database systems achieve this need not be explained. Sure, so long as the implementation is black-box indistinguishable from what the spec says, it can do whatever it wants. Note that, this need not be achieved only by acquiring an exclusive lock on the database file. Think about a database implementation which is not a single file based (Log + Checkpoint design model) where there is one data file and a couple of log files. Spec-ing that they have to hold exclusive lock on database file is ambiguous between data file and log file. If you take BDB JE as an example, they don't even have data file. Their model is a sequence of log files. The exclusive lock model described in the spec is just a model, it isn't intended to be actually require an exclusive lock. If an implementation can get the same result using some other mechanism, that's fine. On Fri, 24 Jul 2009, Nikunj R. Mehta wrote: Database developers (whether experienced DBAs or newcomer WebApp programmers) identify the data set they are using through statements they execute (within or outside transactions). It is the database's job to find out which records are being used. Sure. The concepts of transaction processing apply no matter the granularity of a data item, whether it is a record or a disk block, or a whole file. There are many kinds of failures (and yes, failures are always unpredictable) [1]. Let's focus on failures arising from concurrency control enforcement, which is probably the one most people worry about from a programming perspective. In the following discussion, I use the term locking , even though other protocols have been developed and are in use, to guarantee serializability, i.e., correct interleaving of concurrent transactions. A knowledgeable database programmer would read the smallest set of data in a transaction so as to avoid locking the entire database for concurrent operations. Moreover, this approach also minimizes starvation, i.e., the amount of time a program would need to wait to obtain permission to exclusively access data. Transactions can fail even if locking occurs at the whole database level. As example, consider the situation: 1. A read-only transaction is timed out because some read-write transaction went on for too long. 2. A read-write transaction is timed out because some read-only transaction went on for too long. These are the only failure modes possible currently, I believe. 3. A read-only transaction includes inside it a read-write transaction. This isn't possible with the current asynchronous API as far as I can tell. With the synchronous API, it would hang trying to open the read-write transaction for however long it takes the UA to realise that the script that is trying to get the read-write transaction is the same one as the one that has an open read-only transaction, and then it would fail with error code 7. Experience has shown that there is no easy way out when dealing with transactions, and locking at the whole database level is no solution to failures. It's not supposed to be a solution to failures, it's supposed to be, and is, as far as I can tell, a way to make unpredictable, transient, intermittent, and hard-to-debug concurrency errors into guaranteed, easy-to-debug errors. On Fri, 24 Jul 2009, Nikunj R. Mehta wrote: There's never going to be a forced rollback; once you have got a transaction lock, you are not going to ever have it fail on you unexpectedly. Even if you have a transaction lock, 1. the application logic could cause an exception 2. the application finds an unacceptable data condition and needs to rollback the transaction Sure, but both of those are under the control of the author. 3. face a disk failure This is an exceptional situation from which there is no good recovery. It isn't an expected situation resulting from a complicated API. 4. encounter a bug in the underlying software We can't do anything to prevent these in the spec. In either of these cases, how would the application code be expected to recover? In the first two and the last one, the author can debug the problem and fix or work around the bug. In the case of hardware failure, there is no sane recovery model. These are very different from concurrency bugs. I think this is an important invariant, because otherwise script
Re: [WebStorage] Concerns on spec section 'Processing Model'
On Jul 24, 2009, at 2:53 PM, Ian Hickson wrote: These are very different from concurrency bugs. There are only three concurrency bugs 1. The Lost Update Problem 2. The Temporary Update (or Dirty Read) Problem 3. The Incorrect Summary Problem. Neither of these is related to the granularity of locking. All of these are solved through the use of transactions. If an application uses transactions correctly, then it is free from concurrency bugs. Nikunj http://o-micron.blogspot.com
Re: [WebStorage] Concerns on spec section 'Processing Model'
On Fri, 24 Jul 2009, Nikunj R. Mehta wrote: On Jul 24, 2009, at 2:53 PM, Ian Hickson wrote: These are very different from concurrency bugs. There are only three concurrency bugs 1. The Lost Update Problem 2. The Temporary Update (or Dirty Read) Problem 3. The Incorrect Summary Problem. Neither of these is related to the granularity of locking. All of these are solved through the use of transactions. If an application uses transactions correctly, then it is free from concurrency bugs. If you have two applications in two tabs, and they both need to read row A, then write to row B, and they start doing these two tasks simultaneously, how do you prevent either from failing if you don't have database-wide locking? -- Ian Hickson U+1047E)\._.,--,'``.fL http://ln.hixie.ch/ U+263A/, _.. \ _\ ;`._ ,. Things that are impossible just take longer. `._.-(,_..'--(,_..'`-.;.'
Re: [WebStorage] Solution proposed (was: Concerns on spec section 'Processing Model')
If you want to provide an application programmer with a limited degree of freedom from a certain class of errors, then there is a different solution. It is called isolation level [1]. When opening a transaction, just provide the required isolation level. Heck, if you'd like, make SERIALIZABLE the default value. But don't disallow other possibilities or create the illusion of silver bullets. On Jul 24, 2009, at 2:53 PM, Ian Hickson wrote: On Fri, 24 Jul 2009, Laxmi Narsimha Rao Oruganti wrote: Let me probe this further to get clarity. As I understand it, with what is specced now, if you try to get a write transaction lock, it will only fail if it times out, which would probably be a symptom of a more serious bug anyway. There's never going to be a forced rollback; once you have got a transaction lock, you are not going to ever have it fail on you unexpectedly. My understanding of your requirement is Database should allow only one active writer transaction. How the database systems achieve this need not be explained. Sure, so long as the implementation is black-box indistinguishable from what the spec says, it can do whatever it wants. Note that, this need not be achieved only by acquiring an exclusive lock on the database file. Think about a database implementation which is not a single file based (Log + Checkpoint design model) where there is one data file and a couple of log files. Spec-ing that they have to hold exclusive lock on database file is ambiguous between data file and log file. If you take BDB JE as an example, they don't even have data file. Their model is a sequence of log files. The exclusive lock model described in the spec is just a model, it isn't intended to be actually require an exclusive lock. If an implementation can get the same result using some other mechanism, that's fine. The spec says: [[ If the mode is read/write, the transaction must have an exclusive write lock over the entire database ]] Therefore, correct me if I am wrong, but the spec prohibits the following: An implementation of the Database object allows more than one transaction to write in a database while another transaction has a write lock on the same database, it is a failure. If so, then I want to formally object to that spec text because it is overly restrictive on implementers as well as on application programmers. [snip] 3. A read-only transaction includes inside it a read-write transaction. This isn't possible with the current asynchronous API as far as I can tell. With the synchronous API, it would hang trying to open the read-write transaction for however long it takes the UA to realise that the script that is trying to get the read-write transaction is the same one as the one that has an open read-only transaction, and then it would fail with error code 7. Then again the spec is too restrictive because application programmers need the ability to upgrade their lock from read-only to read-write and an application should never deadlock itself. We would have failed the same dumb programmer if we didn't allow this. Therefore, I formally object to the spec disallowing an application to upgrade its database lock. Experience has shown that there is no easy way out when dealing with transactions, and locking at the whole database level is no solution to failures. It's not supposed to be a solution to failures, it's supposed to be, and is, as far as I can tell, a way to make unpredictable, transient, intermittent, and hard-to-debug concurrency errors into guaranteed, easy-to-debug errors. How is a timeout an easy-to-debug error? What is the meaning of a guaranteed error? How is a guaranteed error better than its opposite? Do you have any facts to back this up? If not, I would like to avoid using that judgement. I think this is an important invariant, because otherwise script writers _will_ shoot themselves in the foot. Even if the transaction lock doesn't fail, how would one deal with other transaction failures? I don't understand the relevance. If there's a hardware error, retrying isn't going to help. If there's a concurrency error, the only solution will be to design complex locking semantics outside the API, which would be a terrible burden to place on Web authors. As I explained in my simple example of updating a spreadsheet cell, users cannot avoid complex semantics when dealing with concurrency and sharing in the face of consistency needs. It is an end-to-end reliability requirement (in the same sense as that used by Saltzer, Reed and Clark), and unavoidable for all but the unreliable systems. These aren't professional database developers; Web authors span the gamut of developer experience from the novice who is writing code more by luck than by knowledge all the way to the UI designer who wound up stuck with the task for writing the UI logic
Re: [WebStorage] Concerns on spec section 'Processing Model'
On Fri, Jul 24, 2009 at 2:54 PM, Nikunj R. Mehtanikunj.me...@oracle.com wrote:
On Jul 24, 2009, at 2:06 PM, Aaron Boodman wrote:
On Fri, Jul 24, 2009 at 1:54 PM, Nikunj R. Mehtanikunj.me...@oracle.com
wrote:
Experience has shown that there is no easy way out when dealing with
transactions, and locking at the whole database level is no solution to
failures.
The thing that makes the web browser environment different an
interesting is that multiple independent applications can end up
having access to the same database if the run in the same origin.
Applications have the ability to specify which database they want to use. So
I don't see problems in apps sharing an origin.
Right, but say two Gmail tabs are opened independently. They both say
they want to access the messages database. They have no way to know
about each other except through shared storage (postMessage does not
work across multiple independent tabs).
Now they can conflict with each other. There is no way for the
developer to deal with this problem other than retrying or
implementing another concurrency system on top of shared storage. This
seems bad.
This could be multiple instances of the same app (eg multiple gmail
windows) or just different apps that happen to be on the same origin
(many Google apps run on www.google.com).
When running multiple instances of the same application, or when different
applications share the same data, you are beginning to deal with multi-user
applications (even though it may be the same security principal). In
multi-user applications, database transactions are the same as what they are
on the server. Applications have no choice but to be careful in performing
transactions. Let me illustrate this with an example.
Say that I had a spreadsheet app. The value of a cell was displayed to the
user as X. Now, I go in to one tab A and say add five to X. I also go in
to B and say add five to X. One of those operations will have to fail
because it finds that the version of X is not what it was when the
transaction started out. Even if you put a lock on the entire database, you
can't avoid that problem.
The issue of the data changing between the time when it was displayed
to the user and the time when an update is started is different than
the problem of the data changing while a multi-step update (a
transaction) is in progress.
The first problem is well known and understood by client-side web
developers because the web is stateless and the same can occur between
contacts with the server. It's also pretty self-evident that if you
copy data out of storage and into the UI that the two can change
independently.
The second problem is not well known by the same people and would be
surprising. Up until recently there was no local storage except
cookies and some proprietary things, and both were synchronous.
Because all browsers until recently were single-threaded, this
effectively meant that clients had storage-wide locks (there were
actually bugs with this in Firefox+cookies, I am told, but cookies
were not frequently used in a way that exposed it).
It seems that the way the spec is written, novice programmers would be led
to either
1. face lost updates because they assume the browser locks the entire
database, and so they won't bother to do their own analysis of whether data
has changed since the last time they saw it.
Some very novice users will not realize that their UI and local store
can change independently, or will, but won't realize that there can be
multiple copies of their apps.
I think the current design is a good trade-off because addressing that
problem would essentially mean binding the UI to the datastore, which
would introduce gigantic API complexity making it basically not
workable. And many developers will understand the problem from
experience with the web.
2. create single-instance-only apps , i.e., hold a write lock on the
database forever since they don't want to deal version checks.
I don't think you understand the spec - it isn't actually possible to
hold the lock forever. Locks aren't an explit part of the API, but are
implicit and released automatically when functions return.
Take a look at the transaction method again:
db.transaction(function() {
tx.executeSql(strSql, function() {
});
});
The transaction is implicitly released when the last sql statement is
completed (or fails). The only way you can keep this transaction open
is to execute more SQL.
Because these apps are isolated from each other, they have no way to
cooperate to reduce conflicts. They also have no control over whether
there are multiple copies of themselves (the user control this).
Sorry, but there is postMessage, localStorage, and the database itself. What
do you mean these apps are isolated and have no way to cooperate?
postMessage can't be used across independent tabs. Even if it could,
it is asynchronous and most vendors would be reluctant to put more in
the spec
Re: [WebStorage] Concerns on spec section 'Processing Model'
On Jul 24, 2009, at 3:11 PM, Ian Hickson wrote: On Fri, 24 Jul 2009, Nikunj R. Mehta wrote: On Jul 24, 2009, at 2:53 PM, Ian Hickson wrote: These are very different from concurrency bugs. There are only three concurrency bugs 1. The Lost Update Problem 2. The Temporary Update (or Dirty Read) Problem 3. The Incorrect Summary Problem. Neither of these is related to the granularity of locking. All of these are solved through the use of transactions. If an application uses transactions correctly, then it is free from concurrency bugs. If you have two applications in two tabs, and they both need to read row A, then write to row B, and they start doing these two tasks simultaneously, how do you prevent either from failing if you don't have database-wide locking? First of all, the value of row A never changes in this example. So it is immaterial whether the transaction locked the whole database or just row B. Your application that wrote this kind of a query/update has a concurrency bug, namely lost update. IOW, it is losing the first update because it did not check the value of B before modifying it and didn't modify row A when it modified row B. Therefore, your question itself has a concurrency bug. This is why I said that locking is not a silver bullet and multi-user concurrency should not be taken lightly. For a primer on isolation levels, transactions, and locks, please see [1]. This discussion is an indicator of both the complexity involved in designing standards such as these and the amount of background knowledge required to design a good standard. Proponents of existing spec language have chosen to never explicitly back up their statements with the body of knowledge that exists in the database sciences. Taken together, all this makes it unlikely that a good SQL standard can be developed by this WG in a short period of time that some might be expecting. Nikunj http://o-micron.blogspot.com [1] http://www.oracle.com/technology/oramag/oracle/05-nov/o65asktom.html
Re: [WebStorage] Concerns on spec section 'Processing Model'
On Jul 24, 2009, at 3:57 PM, Aaron Boodman wrote: So you are reduced to very awkward ways of cooperating -- using the database itself as a queue or for master election, or designing a separate transaction system between tabs which might be on separate threads, using an asynchronous API. Or you just accept that any statement can fail and retry everything. Or your app is just buggy if multiple instances are open. Did you consider for a moment that all this is merely a result of the SQLite feature to lock the entire database? Nikunj http://o-micron.blogspot.com
Re: [WebStorage] Concerns on spec section 'Processing Model'
On Fri, Jul 24, 2009 at 4:12 PM, Nikunj R. Mehtanikunj.me...@oracle.com wrote:
On Jul 24, 2009, at 3:57 PM, Aaron Boodman wrote:
2. create single-instance-only apps , i.e., hold a write lock on the
database forever since they don't want to deal version checks.
I don't think you understand the spec - it isn't actually possible to
hold the lock forever.
It is a little insulting for you to say that, but I will not take offense to
it.
I didn't mean any offense, I really don't think you understand the
spec completely :).
Locks aren't an explit part of the API, but are
implicit and released automatically when functions return.
Take a look at the transaction method again:
db.transaction(function(tx) {
tx.executeSql(strSql, function() {
});
});
The transaction is implicitly released when the last sql statement is
completed (or fails). The only way you can keep this transaction open
is to execute more SQL.
(corrected a slight typo in the example - it was missing the parameter
definition for tx)
Thanks for the correction. Code is for conversational purposes only. I
also may be forgetting some API details since I haven't looked at this
in awhile.
If I put in a timer or another asynchronous call inside the block and that
block used the variable tx, wouldn't it force the implementation to continue
holding the database lock? If so, there is no limit to how long I can hold
on to variables, and hence I could hold on to the database as an exclusive
reader/writer for as long as I wanted to. A novice programmer would probably
not even understand what a transaction means, except that they need a
handle to change stuff. That programmer could hold on to this handle for
the duration of the session.
No. The transaction is not closed on GC, it is closed when the last
statement that is part of the transaction completes. So holding a
reference to the tx variable does nothing one way or the other. The
only way to hang the transaction open would be to execute statements
over and over.
On Fri, Jul 24, 2009 at 4:13 PM, Nikunj R. Mehtanikunj.me...@oracle.com wrote:
On Jul 24, 2009, at 3:57 PM, Aaron Boodman wrote:
So you are
reduced to very awkward ways of cooperating -- using the database
itself as a queue or for master election, or designing a separate
transaction system between tabs which might be on separate threads,
using an asynchronous API. Or you just accept that any statement can
fail and retry everything. Or your app is just buggy if multiple
instances are open.
Did you consider for a moment that all this is merely a result of the SQLite
feature to lock the entire database?
No, having the database not be able to change out from under a
multi-step update was a design goal of the API. Implementing a complex
application without exclusive transactions would be very difficult.
I do understand that your position that there is a tradeoff: you give
up some performance because a skilled developer could do finer grained
locking and get better concurrency. And I think you are arguing that
there should be an option for non-exclusive write transactions, or at
least it should be up to the UA.
I still feel that with the number of clients a typical HTML5 database
will have, this is a non-issue and the spec makes the correct
tradeoff.
- a
Re: [WebStorage] Concerns on spec section 'Processing Model'
On Jul 24, 2009, at 3:57 PM, Aaron Boodman wrote: I do not agree that database-level locking is a big problem for web applications. Our problem is not with databases doing database-level locking. Our problem is that such behavior is a MUST. I think it is very desirable for it to appear to the developer that writes to the local datastore are atomic. Lots of complexity falls out if this is not true. It is implicit that transactions give atomicity (that's what A in ACID stands for). It would be mischaracterizing this discussion to say that we are arguing about atomicity. We are, however, talking about isolation (the I in ACID), or more precisely the degree of isolation. In some models (non-SQL) it may be easier to arrange a large update in the application layer and commit it all at once. In SQL, this is less true so it is important to provide API that makes conflicts impossible while a multi-step update is in progress. This problem exists in the WebStorage model [1]. More specifically, there is no way to perform multiple updates atomically in it. The proposal that I have sketched about B-trees [2] does not have this problem since it is possible to work with transactions to get the atomicity as well as a desired isolation level. I take it that there are no issues with that proposal since I have not heard anyone say so. Perhaps your real issue is that the current API does not work well for non SQL data stores. Not at all! It would be disingenuous to find an ulterior motive in my arguments. Nikunj http://o-micron.blogspot.com [1] http://dev.w3.org/html5/webstorage/ [2] http://www.w3.org/mid/f480f60a-5dae-4b73-922a-93ed401cf...@oracle.com
Re: [WebStorage] Concerns on spec section 'Processing Model'
On Fri, Jul 24, 2009 at 4:30 PM, Nikunj R. Mehtanikunj.me...@oracle.com wrote: On Jul 24, 2009, at 3:57 PM, Aaron Boodman wrote: In some models (non-SQL) it may be easier to arrange a large update in the application layer and commit it all at once. In SQL, this is less true so it is important to provide API that makes conflicts impossible while a multi-step update is in progress. This problem exists in the WebStorage model [1]. More specifically, there is no way to perform multiple updates atomically in it. I agree. The proposal that I have sketched about B-trees [2] does not have this problem since it is possible to work with transactions to get the atomicity as well as a desired isolation level. I take it that there are no issues with that proposal since I have not heard anyone say so. I haven't reviewed that. I only chimed into this conversation because it looked like there were some misunderstandings and I worked on it early-on. - a
Re: [WebStorage] Concerns on spec section 'Processing Model'
On Jul 24, 2009, at 4:25 PM, Aaron Boodman wrote:
On Fri, Jul 24, 2009 at 4:12 PM, Nikunj R. Mehtanikunj.me...@oracle.com
wrote:
On Jul 24, 2009, at 3:57 PM, Aaron Boodman wrote:
2. create single-instance-only apps , i.e., hold a write lock on
the
database forever since they don't want to deal version checks.
I don't think you understand the spec - it isn't actually possible
to
hold the lock forever.
It is a little insulting for you to say that, but I will not take
offense to
it.
I didn't mean any offense, I really don't think you understand the
spec completely :).
I beg to differ. Au contraire, I really don't think you understand
databases at all.
Locks aren't an explit part of the API, but are
implicit and released automatically when functions return.
This is completely incorrect. Read below for more details.
Take a look at the transaction method again:
db.transaction(function(tx) {
tx.executeSql(strSql, function() {
});
});
The transaction is implicitly released when the last sql statement
is
completed (or fails). The only way you can keep this transaction
open
is to execute more SQL.
(corrected a slight typo in the example - it was missing the
parameter
definition for tx)
Thanks for the correction. Code is for conversational purposes only. I
also may be forgetting some API details since I haven't looked at this
in awhile.
If I put in a timer or another asynchronous call inside the block
and that
block used the variable tx, wouldn't it force the implementation to
continue
holding the database lock? If so, there is no limit to how long I
can hold
on to variables, and hence I could hold on to the database as an
exclusive
reader/writer for as long as I wanted to. A novice programmer would
probably
not even understand what a transaction means, except that they need a
handle to change stuff. That programmer could hold on to this
handle for
the duration of the session.
No. The transaction is not closed on GC, it is closed when the last
statement that is part of the transaction completes. So holding a
reference to the tx variable does nothing one way or the other. The
only way to hang the transaction open would be to execute statements
over and over.
A transaction is not complete until I either commit or rollback the
transaction, which I can choose to do as late as I want to, e.g., at
window.onclose. Therefore locks on the database will not be released
for as long as the application wants to hold on to the transaction.
On Fri, Jul 24, 2009 at 4:13 PM, Nikunj R. Mehtanikunj.me...@oracle.com
wrote:
On Jul 24, 2009, at 3:57 PM, Aaron Boodman wrote:
So you are
reduced to very awkward ways of cooperating -- using the database
itself as a queue or for master election, or designing a separate
transaction system between tabs which might be on separate threads,
using an asynchronous API. Or you just accept that any statement can
fail and retry everything. Or your app is just buggy if multiple
instances are open.
Did you consider for a moment that all this is merely a result of
the SQLite
feature to lock the entire database?
No, having the database not be able to change out from under a
multi-step update was a design goal of the API. Implementing a complex
application without exclusive transactions would be very difficult.
I am not proposing to take away your choice. But please don't take
away mine.
It would be useful to see an explanation as to why the proposal I made
[[
add an isolation level parameter with a default value of SERIALIZABLE
and remove the exclusive database-level write lock requirement
]]
is worse than the current spec text. You can refer to SQL92 explain
the meaning of SERIALIZABLE. AFAIK, there are no interoperability
problems with transaction isolation levels.
Nikunj
http://o-micron.blogspot.com
Re: [WebStorage] Concerns on spec section 'Processing Model'
On Fri, Jul 24, 2009 at 4:45 PM, Nikunj R. Mehtanikunj.me...@oracle.com wrote: No. The transaction is not closed on GC, it is closed when the last statement that is part of the transaction completes. So holding a reference to the tx variable does nothing one way or the other. The only way to hang the transaction open would be to execute statements over and over. A transaction is not complete until I either commit or rollback the transaction, which I can choose to do as late as I want to, e.g., at window.onclose. Therefore locks on the database will not be released for as long as the application wants to hold on to the transaction. I don't think that this is true, at least in the Database interface: http://dev.w3.org/html5/webdatabase/#asynchronous-database-api There is no explicit commit() method. The commit happens implicitly after all queued statements have been executed successfully. It does appear that it is possible to hold a transaction open all day with the DatabaseSync interface (http://dev.w3.org/html5/webdatabase/#databasesync). Specifically the SQLTransactionSync method has commit/rollback methods. The DatabaseSync interface was added after I worked on this, so I can't say why it doesn't use callbacks. In any case, I was talking about the async flavor which is what my example code referred to. Do you agree it is not possible to hang transactions open from Database (http://dev.w3.org/html5/webdatabase/#database)? If not, what am I missing? - a
Re: [WebStorage] Concerns on spec section 'Processing Model'
On Jul 24, 2009, at 4:58 PM, Aaron Boodman wrote: On Fri, Jul 24, 2009 at 4:45 PM, Nikunj R. Mehtanikunj.me...@oracle.com wrote: No. The transaction is not closed on GC, it is closed when the last statement that is part of the transaction completes. So holding a reference to the tx variable does nothing one way or the other. The only way to hang the transaction open would be to execute statements over and over. A transaction is not complete until I either commit or rollback the transaction, which I can choose to do as late as I want to, e.g., at window.onclose. Therefore locks on the database will not be released for as long as the application wants to hold on to the transaction. I don't think that this is true, at least in the Database interface: http://dev.w3.org/html5/webdatabase/#asynchronous-database-api There is no explicit commit() method. The commit happens implicitly after all queued statements have been executed successfully. The spec is also silent about what happens if I put a wait by making another asynchronous call inside my transaction callback logic. By inference, this would be allowed since all statements are executed inside callbacks, so why distinguish between transaction and other (non-SQLTransactionErrorCallback) types of callbacks. The processing model in 4.3.2 simply says that the SQL statements are queued up. It is unclear what if anything happens if the database runs out of statements to execute if the transaction logic takes time to add another statement to the queue before the database decides to commit. Am I wrong or is this an ambiguous, but correct interpretation? Those who are worried about throwing complexity of transaction recovery on Web programmers should perhaps also be worried about the insane complexity of asynchronous transaction programming, that no one in the world should have to learn. The mainstream database developers don't have to deal with that. Why should poor Web programmers have to suffer this? Moreover, with an asynchronous database the spec doesn't allow an application to rollback a transaction, should certain application logic require that. This is yet another case of creating a storage API that is different from traditional database developers. There seems to be a pattern of ignoring good API practices when interacting with a database and it appears intentional. Am I wrong in my interpretation? It does appear that it is possible to hold a transaction open all day with the DatabaseSync interface (http://dev.w3.org/html5/webdatabase/#databasesync). Specifically the SQLTransactionSync method has commit/rollback methods. The DatabaseSync interface was added after I worked on this, so I can't say why it doesn't use callbacks. In any case, I was talking about the async flavor which is what my example code referred to. Do you agree it is not possible to hang transactions open from Database (http://dev.w3.org/html5/webdatabase/#database)? If not, what am I missing? I can't agree simply because the spec says nothing about it. In fact, if anything the rest of the spec text around asynchronous processing suggests that it is possible to hang transactions indefinitely. Nikunj http://o-micron.blogspot.com
Re: [WebStorage] Concerns on spec section 'Processing Model'
On Fri, Jul 24, 2009 at 4:45 PM, Nikunj R. Mehtanikunj.me...@oracle.com wrote: I am not proposing to take away your choice. But please don't take away mine. It would be useful to see an explanation as to why the proposal I made [[ add an isolation level parameter with a default value of SERIALIZABLE and remove the exclusive database-level write lock requirement ]] is worse than the current spec text. You can refer to SQL92 explain the meaning of SERIALIZABLE. AFAIK, there are no interoperability problems with transaction isolation levels. I'm personally not opposed to adding more isolation levels in addition to the current single option of SERIALIZABLE. It could be added as an argument to transaction(). I don't think it is a particularly high value feature, but I also don't see a big problem with it. And I can imagine that some particularly ambitious developers might want to take advantage of it. - a
RE: [WebStorage] Concerns on spec section 'Processing Model'
+ Ian Hey Ian, Can you please express your opinion on this as an editor of Web Storage (now Web Database) Spec? Thanks, Laxmi From: Nikunj R. Mehta [mailto:nikunj.me...@oracle.com] Sent: Thursday, July 16, 2009 11:45 PM To: public-webapps WG Cc: Laxmi Narsimha Rao Oruganti Subject: Re: [WebStorage] Concerns on spec section 'Processing Model' The spec should not restrict implementations to any one level of concurrency unless there are specific undesirable effects. Restricting the database to a single writer means that if there are separate workers or background threads working to update non-overlapping portions, then they have to wait for the lone current writer. Implementations can certainly compete to produce the level of concurrency that developers need. Specifically, I propose that the following text [[ If the mode is read/write, the transaction must have an exclusive write lock over the entire database. If the mode is read-only, the transaction must have a shared read lock over the entire database. The user agent should wait for an appropriate lock to be available. ]] be replaced with the following text [[ Multiple read-only transactions may share the same data as long as there is no transaction attempting to write the data being read. The user agent must wait for transactions that are reading some data before allowing a read/write transaction on the same data to continue. ]] Nikunj http://o-micron.blogspot.com On Jul 16, 2009, at 6:46 AM, Laxmi Narsimha Rao Oruganti wrote: [Adding the subject, sorry for spam!] Hey folks, I have few questions on Web Storage Spec. I have checked the content of both latest published spechttp://www.w3.org/TR/webstorage/ and latest editors spechttp://dev.w3.org/html5/webstorage/. And the questions are applicable to both the versions of the spec. Section: 4.4.2 Processing model Text: 1. Open a new SQL transaction to the database, and create a SQLTransactionhttp://www.w3.org/TR/webstorage/#sqltransaction object that represents that transaction. If the mode is read/write, the transaction must have an exclusive write lock over the entire database. If the mode is read-only, the transaction must have a shared read lock over the entire database. The user agent should wait for an appropriate lock to be available. Concerns: - Why is the spec mandating a transaction to take an *exclusive write lock on the entire database*? No database book design mandates it. In fact, many client databases out there don't do this. I guess SQLite does this kind. But that does not mean that all implementations have this nature. I am kind of worried that we are putting implementation in theory. For me they are too separate, there are many ways a database could be designed. Like, log+checkpoint approach, shadow copy, version store, journals ...etc. I guess spec should say what a browser should do and not how. I would be happy to get enlightened. Thanks, Laxmi
[WebStorage]
Hey folks, I have few questions on Web Storage Spec. I have checked the content of both latest published spechttp://www.w3.org/TR/webstorage/ and latest editors spechttp://dev.w3.org/html5/webstorage/. And the questions are applicable to both the versions of the spec. Section: 4.4.2 Processing model Text: 1. Open a new SQL transaction to the database, and create a SQLTransactionhttp://www.w3.org/TR/webstorage/#sqltransaction object that represents that transaction. If the mode is read/write, the transaction must have an exclusive write lock over the entire database. If the mode is read-only, the transaction must have a shared read lock over the entire database. The user agent should wait for an appropriate lock to be available. Concerns: - Why is the spec mandating a transaction to take an *exclusive write lock on the entire database*? No database book design mandates it. In fact, many client databases out there don't do this. I guess SQLite does this kind. But that does not mean that all implementations have this nature. I am kind of worried that we are putting implementation in theory. For me they are too separate, there are many ways a database could be designed. Like, log+checkpoint approach, shadow copy, version store, journals ...etc. I guess spec should say what a browser should do and not how. I would be happy to get enlightened. Thanks, Laxmi
[WebStorage] Concerns on spec section 'Processing Model'
[Adding the subject, sorry for spam!] Hey folks, I have few questions on Web Storage Spec. I have checked the content of both latest published spechttp://www.w3.org/TR/webstorage/ and latest editors spechttp://dev.w3.org/html5/webstorage/. And the questions are applicable to both the versions of the spec. Section: 4.4.2 Processing model Text: 1. Open a new SQL transaction to the database, and create a SQLTransactionhttp://www.w3.org/TR/webstorage/#sqltransaction object that represents that transaction. If the mode is read/write, the transaction must have an exclusive write lock over the entire database. If the mode is read-only, the transaction must have a shared read lock over the entire database. The user agent should wait for an appropriate lock to be available. Concerns: - Why is the spec mandating a transaction to take an *exclusive write lock on the entire database*? No database book design mandates it. In fact, many client databases out there don't do this. I guess SQLite does this kind. But that does not mean that all implementations have this nature. I am kind of worried that we are putting implementation in theory. For me they are too separate, there are many ways a database could be designed. Like, log+checkpoint approach, shadow copy, version store, journals ...etc. I guess spec should say what a browser should do and not how. I would be happy to get enlightened. Thanks, Laxmi
WebDatabase/WebStorage (was Re: Points of order on this WG)
I would like to suggest that these specs be renamed to better reflect what they are about. For one, using the term Web in the title draws attention as the one (or the primary one). Secondly, it says nothing about the constructs offered. For example, WebDatabase suggests that this is *the* spec for structured storage, when, in fact, this group has argued in favor of multiple approaches, including one on B-tree databases that I have proposed. My suggestion is to rename the WebDatabase spec as the SQLDatabase spec. That way any other approach can be called the XXXDatabase spec. Similarly, with WebStorage, it is not clear what is the meaning of Web in the title, especially since we are currently left with just key-value storage. Since Web does nothing, except to distract and possibly mislead people into thinking that the spec covers all possible storage needs, I would suggest that the editor drop the word Web from the spec title. I also have a suggestion for the title - Key Value Storage. I do realize that this might be moot given that WebStorage has already gone through FPWD. Still, it does us no harm to at least rectify the situation now rather than going to CR with this name. Nikunj http://o-micron.blogspot.com On Jul 15, 2009, at 3:56 AM, Ian Hickson wrote: On Thu, 25 Jun 2009, Maciej Stachowiak wrote: On Jun 24, 2009, at 11:35 PM, Ian Hickson wrote: [...] (if anything, I think we should split Web Storage into two further specs [...] [...] I would prefer to see SQL Storage split out of the rest of Web Storage. We seem to have rough consensus and strong multilateral implementor interest on LocalStorage and SessionStorage, and they should be allowed to move forward on the standards track quickly. SQL Storage remains contentious, and only Apple and Google have shown strong implementor interest so far. And it has no technical tie to the other storage drafts. Done. http://dev.w3.org/html5/webstorage/ http://dev.w3.org/html5/webdatabase/ I'll probably not ask for Web Database to go to last call in October (unlike the rest of the specs I'm working on), so that we can add the SQL definition before last call (which I plan to do either Q4 this year or early next year). -- Ian Hickson U+1047E) \._.,--,'``.fL http://ln.hixie.ch/ U+263A/, _.. \ _ \ ;`._ ,. Things that are impossible just take longer. `._.-(,_..'-- (,_..'`-.;.'
Re: [WebStorage] Concerns on spec section 'Processing Model'
The spec should not restrict implementations to any one level of concurrency unless there are specific undesirable effects. Restricting the database to a single writer means that if there are separate workers or background threads working to update non- overlapping portions, then they have to wait for the lone current writer. Implementations can certainly compete to produce the level of concurrency that developers need. Specifically, I propose that the following text [[ If the mode is read/write, the transaction must have an exclusive write lock over the entire database. If the mode is read-only, the transaction must have a shared read lock over the entire database. The user agent should wait for an appropriate lock to be available. ]] be replaced with the following text [[ Multiple read-only transactions may share the same data as long as there is no transaction attempting to write the data being read. The user agent must wait for transactions that are reading some data before allowing a read/write transaction on the same data to continue. ]] Nikunj http://o-micron.blogspot.com On Jul 16, 2009, at 6:46 AM, Laxmi Narsimha Rao Oruganti wrote: [Adding the subject, sorry for spam!] Hey folks, I have few questions on Web Storage Spec. I have checked the content of both latest published spec and latest editors spec. And the questions are applicable to both the versions of the spec. Section: 4.4.2 Processing model Text: 1. Open a new SQL transaction to the database, and create a SQLTransaction object that represents that transaction. If the mode is read/write, the transaction must have an exclusive write lock over the entire database. If the mode is read-only, the transaction must have a shared read lock over the entire database. The user agent should wait for an appropriate lock to be available. Concerns: - Why is the spec mandating a transaction to take an *exclusive write lock on the entire database*? No database book design mandates it. In fact, many client databases out there don’t do this. I guess SQLite does this kind. But that does not mean that all implementations have this nature. I am kind of worried that we are putting implementation in theory. For me they are too separate, there are many ways a database could be designed. Like, log+checkpoint approach, shadow copy, version store, journals … etc. I guess spec should say what a browser should do and not how. I would be happy to get enlightened. Thanks, Laxmi
Re: WebDatabase/WebStorage (was Re: Points of order on this WG)
For what it's worth I don't think using the word Web in the name makes the connection that this is *the* *only* specification for storage for the web. I'll also point out that specs can be renamed at any point in the future if it turns out that the name is confusing. I also think the name of the spec is largely irrelevant. That said, I don't think a name like SQLDatabase is a very good name since there are lots of SQL database specifications and implementations. Something like WebSQLDatabase would be better. IMHO. But like I said, I think it's largely irrelevant. / Jonas On Thu, Jul 16, 2009 at 9:34 AM, Nikunj R. Mehtanikunj.me...@oracle.com wrote: I would like to suggest that these specs be renamed to better reflect what they are about. For one, using the term Web in the title draws attention as the one (or the primary one). Secondly, it says nothing about the constructs offered. For example, WebDatabase suggests that this is *the* spec for structured storage, when, in fact, this group has argued in favor of multiple approaches, including one on B-tree databases that I have proposed. My suggestion is to rename the WebDatabase spec as the SQLDatabase spec. That way any other approach can be called the XXXDatabase spec. Similarly, with WebStorage, it is not clear what is the meaning of Web in the title, especially since we are currently left with just key-value storage. Since Web does nothing, except to distract and possibly mislead people into thinking that the spec covers all possible storage needs, I would suggest that the editor drop the word Web from the spec title. I also have a suggestion for the title - Key Value Storage. I do realize that this might be moot given that WebStorage has already gone through FPWD. Still, it does us no harm to at least rectify the situation now rather than going to CR with this name. Nikunj http://o-micron.blogspot.com On Jul 15, 2009, at 3:56 AM, Ian Hickson wrote: On Thu, 25 Jun 2009, Maciej Stachowiak wrote: On Jun 24, 2009, at 11:35 PM, Ian Hickson wrote: [...] (if anything, I think we should split Web Storage into two further specs [...] [...] I would prefer to see SQL Storage split out of the rest of Web Storage. We seem to have rough consensus and strong multilateral implementor interest on LocalStorage and SessionStorage, and they should be allowed to move forward on the standards track quickly. SQL Storage remains contentious, and only Apple and Google have shown strong implementor interest so far. And it has no technical tie to the other storage drafts. Done. http://dev.w3.org/html5/webstorage/ http://dev.w3.org/html5/webdatabase/ I'll probably not ask for Web Database to go to last call in October (unlike the rest of the specs I'm working on), so that we can add the SQL definition before last call (which I plan to do either Q4 this year or early next year). -- Ian Hickson U+1047E )\._.,--,'``. fL http://ln.hixie.ch/ U+263A /, _.. \ _\ ;`._ ,. Things that are impossible just take longer. `._.-(,_..'--(,_..'`-.;.'
Re: WebDatabase/WebStorage (was Re: Points of order on this WG)
On Jul 16, 2009, at 1:18 PM, Jonas Sicking wrote: Something like WebSQLDatabase would be better. It may be irrelevant in the long run, but definitely worth a lot early on, IMHO. I like your name suggestion. Nikunj
