Re: [whatwg] LocalStorage in workers
On Wed, Sep 16, 2009 at 4:47 PM, Michael Nordman wrote: > > Is it? Can you provide some use cases? :-) > Um...sure... an app sets up a shared worker whose function it is to sync > up/down changes to the data the application manages... > > * pageA makes changes, other pageB sees it by virtue of an event and > reflects change it it view of the world... worker sees the change to by > virtue of the same event and pushes it up. > > * worker receive delta from server... and makes the change locally... pageA > and B see that by virtue of the event. > > > What is the use case for silo'd worker storage? > I mentioned this earlier and also explained that a work-around is to do this via message passing rather than shared memory. As I explained in a couple emails, shared memory is just an "optimization". And, as Robert explained, it's not ever clear whether it's a performance optimization or not...it might just be a simpler way to program. When I asked if you had any use cases, I was asking whether there were any use cases that could not be solved efficiently/reasonably elegantly by worker-only storage.
Re: [whatwg] LocalStorage in workers
> Is it? Can you provide some use cases? :-) Um...sure... an app sets up a shared worker whose function it is to sync up/down changes to the data the application manages... * pageA makes changes, other pageB sees it by virtue of an event and reflects change it it view of the world... worker sees the change to by virtue of the same event and pushes it up. * worker receive delta from server... and makes the change locally... pageA and B see that by virtue of the event. What is the use case for silo'd worker storage?
Re: [whatwg] LocalStorage in workers
On Thu, Sep 17, 2009 at 11:17 AM, Jeremy Orlow wrote: > The use cases all revolve around having a backend in a worker that handles > offline and/or caching. It could either feed its data to the page via > messages or shared memory. The former requires at least worker-only and the > latter requires storage shared between the worker and the page. The latter > is technically an optimization, but I agree that it's a fairly major one. > I don't think copying data from a worker to a page through any kind of database is going to outperform copying Javascript objects or even serializing to strings and then deserializing. You don't even necessarily need to copy all the JS objects passed from one thread to another if you're willing to do some COW or other tricks. Maybe I'm wrong, but at least it seems a premature optimization to declare that shared database storage between page and worker is necessary for performance. Rob -- "He was pierced for our transgressions, he was crushed for our iniquities; the punishment that brought us peace was upon him, and by his wounds we are healed. We all, like sheep, have gone astray, each of us has turned to his own way; and the LORD has laid on him the iniquity of us all." [Isaiah 53:5-6]
Re: [whatwg] LocalStorage in workers
On Wed, Sep 16, 2009 at 4:05 PM, Michael Nordman wrote: > > > On Wed, Sep 16, 2009 at 3:30 PM, Jeremy Orlow wrote: > >> On Wed, Sep 16, 2009 at 3:21 PM, Robert O'Callahan >> wrote: >> >>> On Thu, Sep 17, 2009 at 9:56 AM, Jeremy Orlow wrote: >>> 1) Create a LocalStorage like API that can only be accessed in an async way via pages (kind of like WebDatabase). 2) Remove any atomicity/consistency guarantees from synchronous LocalStorage access within pages (like IE8 currently does) and add an async interface for when pages do need atomicity/consistency. 3) Come up with a completely different storage API that all the browser vendors are willing to implement that only allows Async access from within pages. WebSimpleDatabase might be a good starting point for this. >>> >>> 4) Create WorkerStorage so that shared workers have exclusive, >>> synchronous access to their own persistent storage via an API compatible >>> with LocalStorage. >>> >> >> Ah yes. That is also an option. >> >> And, now that I think about it (combined with Jonas' last point) I think >> it might be the best option since it has a very low implementation cost, it >> keeps the very simple API, and solves the primary problem of not blocking >> pages' event loops. >> > > But it fails to solve the problem of a providing a shared storage > repository for the applications use, which at least to me is the real > primary goal. > Is it? Can you provide some use cases? :-) As I stated, my conversations with developers led me to believe having access to storage within workers is most important and that having shared memory between pages and workers would make things easier on some of them. In other words, from my talks, it's a secondary goal. On Wed, Sep 16, 2009 at 3:57 PM, Jonas Sicking wrote: > On Wed, Sep 16, 2009 at 3:36 PM, Jeremy Orlow wrote: > > Code wise, what Robert suggested is MUCH simpler. Almost for free in > > WebKit. Creating an asynchronous access method and exposing this in the > > page is much more complex. It also defeats the main purpose of > LocalStorage > > (which is to be a simple, light weight way to store data). > > The only difference between Roberts and my suggestion is that I'm also > adding a asynch accessor in the window. That doesn't seem to make it > "MUCH simpler", or am I missing something? > Well, doing just a sync interface is "MUCH simpler", but I suppose there's no reason not to add both to the spec. To be clear, though, adding the sync interface to workers would be a much higher priority for me than the async interface. Enough so that there's a chance we'd ship a version of Chrome that did not yet implement the async interface. That seems OK to me, though. > I do agree that some of the additional optional > multiple-differently-named storage area does add additional > complexity, and maybe we should defer that to something like the > WebSimpleStorage spec. > > > I certainly agree that having some shared memory format between workers > and > > pages would be good, and there's some use cases which would > > certainly benefit, but most of the developers I've talked to so far were > > mostly concerned about having _some_ form of storage and the shared > memory > > aspects were more nice to have. > > What would the specifics of a worker-only storage be? Can multiple > different workers access it? (In which case they need to be protected > by a mutex). Is there one storage per worker URL? Or do all workers > from a particular domain share the same workerStorage? > > I'm also wondering what the use-cases for a worker-only storage is? > The use cases all revolve around having a backend in a worker that handles offline and/or caching. It could either feed its data to the page via messages or shared memory. The former requires at least worker-only and the latter requires storage shared between the worker and the page. The latter is technically an optimization, but I agree that it's a fairly major one. J
Re: [whatwg] LocalStorage in workers
On Wed, Sep 16, 2009 at 3:30 PM, Jeremy Orlow wrote: > On Wed, Sep 16, 2009 at 3:21 PM, Robert O'Callahan > wrote: > >> On Thu, Sep 17, 2009 at 9:56 AM, Jeremy Orlow wrote: >> >>> 1) Create a LocalStorage like API that can only be accessed in an async >>> way via pages (kind of like WebDatabase). >>> >>> 2) Remove any >>> atomicity/consistency guarantees from synchronous LocalStorage access within >>> pages (like IE8 currently does) and add an async interface for when pages do >>> need atomicity/consistency. >>> >>> 3) Come up with a completely different storage API that all the browser >>> vendors are willing to implement that only allows Async access from within >>> pages. WebSimpleDatabase might be a good starting point for this. >>> >> >> 4) Create WorkerStorage so that shared workers have exclusive, synchronous >> access to their own persistent storage via an API compatible with >> LocalStorage. >> > > Ah yes. That is also an option. > > And, now that I think about it (combined with Jonas' last point) I think it > might be the best option since it has a very low implementation cost, it > keeps the very simple API, and solves the primary problem of not blocking > pages' event loops. > But it fails to solve the problem of a providing a shared storage repository for the applications use, which at least to me is the real primary goal.
Re: [whatwg] LocalStorage in workers
On Wed, Sep 16, 2009 at 3:36 PM, Jeremy Orlow wrote: > On Wed, Sep 16, 2009 at 3:32 PM, Jonas Sicking wrote: >> >> On Wed, Sep 16, 2009 at 3:21 PM, Robert O'Callahan >> wrote: >> > On Thu, Sep 17, 2009 at 9:56 AM, Jeremy Orlow >> > wrote: >> >> >> >> 1) Create a LocalStorage like API that can only be accessed in an async >> >> way via pages (kind of like WebDatabase). >> >> 2) Remove any >> >> atomicity/consistency guarantees from synchronous LocalStorage access >> >> within >> >> pages (like IE8 currently does) and add an async interface for when >> >> pages do >> >> need atomicity/consistency. >> >> 3) Come up with a completely different storage API that all the browser >> >> vendors are willing to implement that only allows Async access from >> >> within >> >> pages. WebSimpleDatabase might be a good starting point for this. >> > >> > 4) Create WorkerStorage so that shared workers have exclusive, >> > synchronous >> > access to their own persistent storage via an API compatible with >> > LocalStorage. >> >> I think some of the use cases require that code running in Window >> objects can access the same storage area though. Consider for example >> an email web app that uses a WorkerStorage area for to store email >> data locally (for performance and for offline support), and then uses >> a worker to synchronize that with the server. >> >> Here the code running in the Window wants to access the storage in >> order to render the emails in the page, and the worker wants to access >> it to synchronize with the server. >> >> See my email earlier in this thread. If we change the name from >> 'clientStorage' to 'workerStorage', while still allowing the main >> window to asynchronously get a reference to the storage, then I think >> that about matches what you're proposing (and what item 1 is >> proposing). > > Code wise, what Robert suggested is MUCH simpler. Almost for free in > WebKit. Creating an asynchronous access method and exposing this in the > page is much more complex. It also defeats the main purpose of LocalStorage > (which is to be a simple, light weight way to store data). The only difference between Roberts and my suggestion is that I'm also adding a asynch accessor in the window. That doesn't seem to make it "MUCH simpler", or am I missing something? I do agree that some of the additional optional multiple-differently-named storage area does add additional complexity, and maybe we should defer that to something like the WebSimpleStorage spec. > I certainly agree that having some shared memory format between workers and > pages would be good, and there's some use cases which would > certainly benefit, but most of the developers I've talked to so far were > mostly concerned about having _some_ form of storage and the shared memory > aspects were more nice to have. What would the specifics of a worker-only storage be? Can multiple different workers access it? (In which case they need to be protected by a mutex). Is there one storage per worker URL? Or do all workers from a particular domain share the same workerStorage? I'm also wondering what the use-cases for a worker-only storage is? / Jonas
Re: [whatwg] LocalStorage in workers
On Wed, Sep 16, 2009 at 3:36 PM, Jeremy Orlow wrote: > Code wise, what Robert suggested is MUCH simpler. Almost for free in > WebKit. Creating an asynchronous access method and exposing this in the > page is much more complex. It also defeats the main purpose of LocalStorage > (which is to be a simple, light weight way to store data). > I do not buy that "creating an asynchronous access method and exposing this in the page" ... "defeats the main purpose of LocalStorage (which is to be a simple, light weight way to store data)" Having one async callback doesn't make the API hard to use. Callbacks are easy to work with in JS. Adding one is not the end of the world by a long shot. That said, I suppose it is probably wise to chase down option 3), if people are motivated, so that we don't end up with *three* name/value storage APIs. - a
Re: [whatwg] LocalStorage in workers
On Wed, Sep 16, 2009 at 3:32 PM, Jonas Sicking wrote: > On Wed, Sep 16, 2009 at 3:21 PM, Robert O'Callahan > wrote: > > On Thu, Sep 17, 2009 at 9:56 AM, Jeremy Orlow > wrote: > >> > >> 1) Create a LocalStorage like API that can only be accessed in an async > >> way via pages (kind of like WebDatabase). > >> 2) Remove any > >> atomicity/consistency guarantees from synchronous LocalStorage access > within > >> pages (like IE8 currently does) and add an async interface for when > pages do > >> need atomicity/consistency. > >> 3) Come up with a completely different storage API that all the browser > >> vendors are willing to implement that only allows Async access from > within > >> pages. WebSimpleDatabase might be a good starting point for this. > > > > 4) Create WorkerStorage so that shared workers have exclusive, > synchronous > > access to their own persistent storage via an API compatible with > > LocalStorage. > > I think some of the use cases require that code running in Window > objects can access the same storage area though. Consider for example > an email web app that uses a WorkerStorage area for to store email > data locally (for performance and for offline support), and then uses > a worker to synchronize that with the server. > > Here the code running in the Window wants to access the storage in > order to render the emails in the page, and the worker wants to access > it to synchronize with the server. > > See my email earlier in this thread. If we change the name from > 'clientStorage' to 'workerStorage', while still allowing the main > window to asynchronously get a reference to the storage, then I think > that about matches what you're proposing (and what item 1 is > proposing). > Code wise, what Robert suggested is MUCH simpler. Almost for free in WebKit. Creating an asynchronous access method and exposing this in the page is much more complex. It also defeats the main purpose of LocalStorage (which is to be a simple, light weight way to store data). I certainly agree that having some shared memory format between workers and pages would be good, and there's some use cases which would certainly benefit, but most of the developers I've talked to so far were mostly concerned about having _some_ form of storage and the shared memory aspects were more nice to have. J
Re: [whatwg] LocalStorage in workers
On Wed, Sep 16, 2009 at 3:21 PM, Robert O'Callahan wrote: > On Thu, Sep 17, 2009 at 9:56 AM, Jeremy Orlow wrote: >> >> 1) Create a LocalStorage like API that can only be accessed in an async >> way via pages (kind of like WebDatabase). >> 2) Remove any >> atomicity/consistency guarantees from synchronous LocalStorage access within >> pages (like IE8 currently does) and add an async interface for when pages do >> need atomicity/consistency. >> 3) Come up with a completely different storage API that all the browser >> vendors are willing to implement that only allows Async access from within >> pages. WebSimpleDatabase might be a good starting point for this. > > 4) Create WorkerStorage so that shared workers have exclusive, synchronous > access to their own persistent storage via an API compatible with > LocalStorage. I think some of the use cases require that code running in Window objects can access the same storage area though. Consider for example an email web app that uses a WorkerStorage area for to store email data locally (for performance and for offline support), and then uses a worker to synchronize that with the server. Here the code running in the Window wants to access the storage in order to render the emails in the page, and the worker wants to access it to synchronize with the server. See my email earlier in this thread. If we change the name from 'clientStorage' to 'workerStorage', while still allowing the main window to asynchronously get a reference to the storage, then I think that about matches what you're proposing (and what item 1 is proposing). / Jonas
Re: [whatwg] LocalStorage in workers
Thanks, Robert - I didn't want to second my own proposal :) I think that #4 is probably a reasonable bridge API until we come up with a consensus API for #3. For myself, I see this API as being very useful for persistent workers (yes, I'm still banging that drum :). -atw On Wed, Sep 16, 2009 at 3:21 PM, Robert O'Callahan wrote: > On Thu, Sep 17, 2009 at 9:56 AM, Jeremy Orlow wrote: > >> 1) Create a LocalStorage like API that can only be accessed in an async >> way via pages (kind of like WebDatabase). >> >> 2) Remove any >> atomicity/consistency guarantees from synchronous LocalStorage access within >> pages (like IE8 currently does) and add an async interface for when pages do >> need atomicity/consistency. >> >> 3) Come up with a completely different storage API that all the browser >> vendors are willing to implement that only allows Async access from within >> pages. WebSimpleDatabase might be a good starting point for this. >> > > 4) Create WorkerStorage so that shared workers have exclusive, synchronous > access to their own persistent storage via an API compatible with > LocalStorage. > > This sounds like it has a low implementation cost and solves many use cases > in a very simple way, right? > > Rob > -- > "He was pierced for our transgressions, he was crushed for our iniquities; > the punishment that brought us peace was upon him, and by his wounds we are > healed. We all, like sheep, have gone astray, each of us has turned to his > own way; and the LORD has laid on him the iniquity of us all." [Isaiah > 53:5-6] >
Re: [whatwg] LocalStorage in workers
On Wed, Sep 16, 2009 at 3:21 PM, Robert O'Callahan wrote: > On Thu, Sep 17, 2009 at 9:56 AM, Jeremy Orlow wrote: > >> 1) Create a LocalStorage like API that can only be accessed in an async >> way via pages (kind of like WebDatabase). >> >> 2) Remove any >> atomicity/consistency guarantees from synchronous LocalStorage access within >> pages (like IE8 currently does) and add an async interface for when pages do >> need atomicity/consistency. >> >> 3) Come up with a completely different storage API that all the browser >> vendors are willing to implement that only allows Async access from within >> pages. WebSimpleDatabase might be a good starting point for this. >> > > 4) Create WorkerStorage so that shared workers have exclusive, synchronous > access to their own persistent storage via an API compatible with > LocalStorage. > Ah yes. That is also an option. And, now that I think about it (combined with Jonas' last point) I think it might be the best option since it has a very low implementation cost, it keeps the very simple API, and solves the primary problem of not blocking pages' event loops.
Re: [whatwg] LocalStorage in workers
On Thu, Sep 17, 2009 at 9:56 AM, Jeremy Orlow wrote: > 1) Create a LocalStorage like API that can only be accessed in an async way > via pages (kind of like WebDatabase). > > 2) Remove any > atomicity/consistency guarantees from synchronous LocalStorage access within > pages (like IE8 currently does) and add an async interface for when pages do > need atomicity/consistency. > > 3) Come up with a completely different storage API that all the browser > vendors are willing to implement that only allows Async access from within > pages. WebSimpleDatabase might be a good starting point for this. > 4) Create WorkerStorage so that shared workers have exclusive, synchronous access to their own persistent storage via an API compatible with LocalStorage. This sounds like it has a low implementation cost and solves many use cases in a very simple way, right? Rob -- "He was pierced for our transgressions, he was crushed for our iniquities; the punishment that brought us peace was upon him, and by his wounds we are healed. We all, like sheep, have gone astray, each of us has turned to his own way; and the LORD has laid on him the iniquity of us all." [Isaiah 53:5-6]
Re: [whatwg] LocalStorage in workers
On Wed, Sep 16, 2009 at 2:56 PM, Jeremy Orlow wrote: > On Wed, Sep 16, 2009 at 2:27 PM, Jonas Sicking wrote: >> >> On Wed, Sep 16, 2009 at 12:58 PM, Drew Wilson wrote: >> > I'm saying that an async API is overkill and unwieldy if all you need is >> > WorkerLocalStorage. >> > If you're going to route your localstorage access through an async API >> > anyway, then you might as well proxy it to the parent page - there's >> > very >> > little advantage to doing it otherwise, other than access to lexically >> > scoped resources from within your callback. >> >> Actually, there's a pretty big difference. With the current state of >> affairs, if a worker wants to make a computation based on values in >> the localStorage, and store the result in localStorage, this is >> extremely hard. >> >> For example, say that a worker want to perform the following operation: >> >> localStorage.result = F(localStorage.n); >> >> where F(n) is the n:th value in the Fibonacci sequence. >> >> To do this today the worker would first have to call the main window >> to get the localStoreage.n value. It could then calculate the result >> of F(localStorage.n). It would then send a message to the main window >> to store the result in localStorage.result. However in the meantime >> localStorage.n might have changed, which causes an inconsistent state. >> >> So instead the worker has to send both the value of localStorage.n as >> well as the result to the window. The window can then check if >> localStorage.n has changed. If it has changed, the window has to send >> the new value back to the worker, and then the worker has to redo its >> calculation. >> >> This has several problems. It's bug prone since the developer might >> not realize the race condition. It's very hard to do correctly. And >> even when done correctly risks wasting a lot of cycles. >> >> An alternative solution is to do all calculations in the main window, >> which has synchronous access to localStorage. But the whole point of a >> worker is to avoid having to do heavy work in the window. >> >> However, with the solution Jeremy proposed, calculating the above >> algorithm can be done in the worker after the worker while the worker >> is inside the callback and thus have synchronous access to >> localStorage. >> >> Say that instead of calculating Fibonacci numbers, we were storing a >> database of emails in localStorage, and using a worker to synchronize >> that database to a server. In this case it seems extermely complex to >> have to communicate asynchronously through the window and deal with >> race conditions where the user is modifying the email database at the >> same time. > > True. > The problem is that some page from the same origin might also try to access > LocalStorage. If it does, it'll block the entire event loop until the > worker is finished. I can't think of how to "fix" this in a way that's not > racy. My originally proposal was written in the hope that developers would > be more cautious since they're doing things inside an async callback, but > the more I think about it, the more I think this isn't realistic. > I think we have 3 options: > 1) Create a LocalStorage like API that can only be accessed in an async way > via pages (kind of like WebDatabase). > 2) Remove any atomicity/consistency guarantees from synchronous LocalStorage > access within pages (like IE8 currently does) and add an async interface for > when pages do need atomicity/consistency. > 3) Come up with a completely different storage API that all the browser > vendors are willing to implement that only allows Async access from within > pages. WebSimpleDatabase might be a good starting point for this. > > 1 is probably the simplest to implement, but it seems pretty hacky and it's > likely not powerful enough for many advanced web apps (offline web mail > would be an example). If we do 2, many (most?) web developers will just use > the sync interface and write racy apps. 3 will take the longest time to do, > but is definitely the best long term solution. I think 2 is right out. 1 is what we should have done in the first place if we had thought about the multiple processes thing. The only thing that's bad about 1 is that we're creating two extremely similar features. So I'd say 1 is unfortunate rather than hacky. 3 is is something that I personally think we should do no matter what as I'm not a big fan of the current SQL interface. But I wonder if we might want to do 1 anyway. After all, localStorage and the SQL APIs were both suggested. Presumably to allow localStorage to handle the simple cases and SQL to handle the more complex ones. / Jonas
Re: [whatwg] LocalStorage in workers
On Wed, Sep 16, 2009 at 2:27 PM, Jonas Sicking wrote: > On Wed, Sep 16, 2009 at 12:58 PM, Drew Wilson wrote: > > I'm saying that an async API is overkill and unwieldy if all you need is > > WorkerLocalStorage. > > If you're going to route your localstorage access through an async API > > anyway, then you might as well proxy it to the parent page - there's very > > little advantage to doing it otherwise, other than access to lexically > > scoped resources from within your callback. > > Actually, there's a pretty big difference. With the current state of > affairs, if a worker wants to make a computation based on values in > the localStorage, and store the result in localStorage, this is > extremely hard. > > For example, say that a worker want to perform the following operation: > > localStorage.result = F(localStorage.n); > > where F(n) is the n:th value in the Fibonacci sequence. > > To do this today the worker would first have to call the main window > to get the localStoreage.n value. It could then calculate the result > of F(localStorage.n). It would then send a message to the main window > to store the result in localStorage.result. However in the meantime > localStorage.n might have changed, which causes an inconsistent state. > > So instead the worker has to send both the value of localStorage.n as > well as the result to the window. The window can then check if > localStorage.n has changed. If it has changed, the window has to send > the new value back to the worker, and then the worker has to redo its > calculation. > > This has several problems. It's bug prone since the developer might > not realize the race condition. It's very hard to do correctly. And > even when done correctly risks wasting a lot of cycles. > > An alternative solution is to do all calculations in the main window, > which has synchronous access to localStorage. But the whole point of a > worker is to avoid having to do heavy work in the window. > > However, with the solution Jeremy proposed, calculating the above > algorithm can be done in the worker after the worker while the worker > is inside the callback and thus have synchronous access to > localStorage. > > Say that instead of calculating Fibonacci numbers, we were storing a > database of emails in localStorage, and using a worker to synchronize > that database to a server. In this case it seems extermely complex to > have to communicate asynchronously through the window and deal with > race conditions where the user is modifying the email database at the > same time. > True. The problem is that some page from the same origin might also try to access LocalStorage. If it does, it'll block the entire event loop until the worker is finished. I can't think of how to "fix" this in a way that's not racy. My originally proposal was written in the hope that developers would be more cautious since they're doing things inside an async callback, but the more I think about it, the more I think this isn't realistic. I think we have 3 options: 1) Create a LocalStorage like API that can only be accessed in an async way via pages (kind of like WebDatabase). 2) Remove any atomicity/consistency guarantees from synchronous LocalStorage access within pages (like IE8 currently does) and add an async interface for when pages do need atomicity/consistency. 3) Come up with a completely different storage API that all the browser vendors are willing to implement that only allows Async access from within pages. WebSimpleDatabase might be a good starting point for this. 1 is probably the simplest to implement, but it seems pretty hacky and it's likely not powerful enough for many advanced web apps (offline web mail would be an example). If we do 2, many (most?) web developers will just use the sync interface and write racy apps. 3 will take the longest time to do, but is definitely the best long term solution. Do others agree with my list? What's the best option out of these? Honestly, I'm kind of leaning towards 3 at this point. J
Re: [whatwg] LocalStorage in workers
On Wed, Sep 16, 2009 at 12:58 PM, Drew Wilson wrote: > I'm saying that an async API is overkill and unwieldy if all you need is > WorkerLocalStorage. > If you're going to route your localstorage access through an async API > anyway, then you might as well proxy it to the parent page - there's very > little advantage to doing it otherwise, other than access to lexically > scoped resources from within your callback. Actually, there's a pretty big difference. With the current state of affairs, if a worker wants to make a computation based on values in the localStorage, and store the result in localStorage, this is extremely hard. For example, say that a worker want to perform the following operation: localStorage.result = F(localStorage.n); where F(n) is the n:th value in the Fibonacci sequence. To do this today the worker would first have to call the main window to get the localStoreage.n value. It could then calculate the result of F(localStorage.n). It would then send a message to the main window to store the result in localStorage.result. However in the meantime localStorage.n might have changed, which causes an inconsistent state. So instead the worker has to send both the value of localStorage.n as well as the result to the window. The window can then check if localStorage.n has changed. If it has changed, the window has to send the new value back to the worker, and then the worker has to redo its calculation. This has several problems. It's bug prone since the developer might not realize the race condition. It's very hard to do correctly. And even when done correctly risks wasting a lot of cycles. An alternative solution is to do all calculations in the main window, which has synchronous access to localStorage. But the whole point of a worker is to avoid having to do heavy work in the window. However, with the solution Jeremy proposed, calculating the above algorithm can be done in the worker after the worker while the worker is inside the callback and thus have synchronous access to localStorage. Say that instead of calculating Fibonacci numbers, we were storing a database of emails in localStorage, and using a worker to synchronize that database to a server. In this case it seems extermely complex to have to communicate asynchronously through the window and deal with race conditions where the user is modifying the email database at the same time. / Jonas
Re: [whatwg] LocalStorage in workers
On Wed, Sep 16, 2009 at 1:06 PM, James Robinson wrote: > On Wed, Sep 16, 2009 at 11:34 AM, Michael Nordman wrote: > >> >> >> On Wed, Sep 16, 2009 at 11:24 AM, James Robinson wrote: >> >>> On Wed, Sep 16, 2009 at 10:53 AM, Michael Nordman >>> wrote: >>> On Wed, Sep 16, 2009 at 9:58 AM, Drew Wilson wrote: > Jeremy, what's the use case here - do developers want workers to have > access to shared local storage with pages? Or do they just want workers to > have access to their own non-shared local storage? > Because we could just give workers their own separate > WorkerLocalStorage and let them have at it. A worker could block all the > other accesses to WorkerLocalStorage within that domain, but so be it - it > wouldn't affect page access, and we already had that issue with the (now > removed?) synchronous SQL API. > > I think a much better case can be made for WorkerLocalStorage than for > "give workers access to page LocalStorage", and the design issues are much > simpler. > Putting workers in their own storage silo doesn't really make much sense? Sure it may be simpler for browser vendors, but does that make life simpler for app developers, or just have them scratching their heads about how to read/write the same data set from either flavor of context in their application? I see no rhyme or reason for the arbitrary barrier except for browser vendors to work around the awkward implict locks on LocalStorage (the source of much grief). Consider this... would it make sense to cordon off the databases workers vs pages can see? I would think not, and i would hope others agree. >>> >>> The difference is that the database interface is purely asynchronous >>> whereas storage is synchronous. >>> >> >> Sure... we're talking about adding an async api that allows worker to >> access a local storage repository... should such a thing exist, why should >> it not provide access to the same repository as seen by pages? >> > > Not quite - Jeremy proposed giving workers access to a synchronous API > (localStorage.*) but to only allow it to be called within the context of a > callback that the UA can run when it chooses. It's another way to approach > the implicit locking since a UA would have to, in effect, hold the storage > mutex for the duration of the callback. The page's context could still be > blocked for an indefinite amount of time by a worker thread. > Exactly. And I acknowledged this in my original email. Unfortunately, I don't have a good solution to the problem. The only thing I can think of is some timeout, but these would be inherently racy. There are times when the system is under heavy load and _everything_ goes slower. I don't see how we could enforce that workers don't keep LocalStorage locked for long enough that UI threads become affected and still keep behavior deterministic from the web developers point of view. That said, IF (and this is a big if) we decide to make localStorage NOT have any run to completion semantics across event loops and instead add in an async interface for atomic access to localStorage, then this would work, however. > Drew suggested isolating the worker's access to a separate storage 'arena' > so that there wouldn't be shared, synchronous access between the page > context and a worker context. This way the synchronous Storage API can be > used essentially unchanged without having to deal with the more nasty parts > of synchronization. > Drew, the most important thing to the developers I talked to is that they need _some_ storage directly accessible to workers. Many were thinking in terms of a shared worker syncing and doing the bulk of the processing and then one or more pages acting as thin clients. Some of them wanted to use it like shared memory (i.e. workers syncing and doing some processing and then storing information in storage; pages then display based on that data). This is especially interesting because storage events can be used to trigger updates to content. As I think about it, I suppose most of the use cases could actually be solved by a storage for workers, whether or not pages can also access it. If the burden were low, it would be nice to make them accessible though. If we did this, keeping some form of storage events would be nice. Note that most of the developers I talked to thought LocalStorage was not powerful enough for their needs, but since it's the only API supported by all the browsers they figured they were stuck with it, so they'd have to find _some_ way to make it work. Somaybe it doesn't make sense to spend a lot of effort making LocalStorage (in some form or another) work in workers. Maybe we should instead put our effort into something like WebSimpleDatabase? J
Re: [whatwg] LocalStorage in workers
On Wed, Sep 16, 2009 at 11:34 AM, Michael Nordman wrote:
>
>
> On Wed, Sep 16, 2009 at 11:24 AM, James Robinson wrote:
>
>> On Wed, Sep 16, 2009 at 10:53 AM, Michael Nordman wrote:
>>
>>>
>>>
>>> On Wed, Sep 16, 2009 at 9:58 AM, Drew Wilson wrote:
>>>
Jeremy, what's the use case here - do developers want workers to have
access to shared local storage with pages? Or do they just want workers to
have access to their own non-shared local storage?
Because we could just give workers their own separate WorkerLocalStorage
and let them have at it. A worker could block all the other accesses to
WorkerLocalStorage within that domain, but so be it - it wouldn't affect
page access, and we already had that issue with the (now removed?)
synchronous SQL API.
I think a much better case can be made for WorkerLocalStorage than for
"give workers access to page LocalStorage", and the design issues are much
simpler.
>>>
>>> Putting workers in their own storage silo doesn't really make much sense?
>>> Sure it may be simpler for browser vendors, but does that make life simpler
>>> for app developers, or just have them scratching their heads about how to
>>> read/write the same data set from either flavor of context in their
>>> application?
>>>
>>> I see no rhyme or reason for the arbitrary barrier except for browser
>>> vendors to work around the awkward implict locks on LocalStorage (the source
>>> of much grief). Consider this... would it make sense to cordon off the
>>> databases workers vs pages can see? I would think not, and i would hope
>>> others agree.
>>>
>>
>> The difference is that the database interface is purely asynchronous
>> whereas storage is synchronous.
>>
>
> Sure... we're talking about adding an async api that allows worker to
> access a local storage repository... should such a thing exist, why should
> it not provide access to the same repository as seen by pages?
>
Not quite - Jeremy proposed giving workers access to a synchronous API
(localStorage.*) but to only allow it to be called within the context of a
callback that the UA can run when it chooses. It's another way to approach
the implicit locking since a UA would have to, in effect, hold the storage
mutex for the duration of the callback. The page's context could still be
blocked for an indefinite amount of time by a worker thread.
Drew suggested isolating the worker's access to a separate storage 'arena'
so that there wouldn't be shared, synchronous access between the page
context and a worker context. This way the synchronous Storage API can be
used essentially unchanged without having to deal with the more nasty parts
of synchronization.
- James
>
>> If multiple threads have synchronous access to the same shared resource
>> then there has to be a consistency model. ECMAScript does not provide for
>> one so it has to be done at a higher level. Since there was not a solution
>> in the first versions that shipped, the awkward implicit locks you mention
>> were suggested as a workaround. However it's far from clear that these
>> solve the problem and are implementable. It seems like the only logical
>> continuation of this path would be to add explicit, blocking synchronization
>> primitives for developers to deal with - which I think everyone agrees would
>> be a terrible idea. If you're worried about developers scratching their
>> heads about how to pass data between workers just think about happens-before
>> relationships and multi-threaded memory models.
>>
>> In a hypothetical world without synchronous access to LocalStorage/cookies
>> from workers, there is no shared memory between threads except via message
>> passing. This can seem a bit tricky for developers but is very easy to
>> reason about and prove correctness and the absence of deadlocks.
>>
>> - James
>>
>>
>>>
>>>
>>>
-atw
On Tue, Sep 15, 2009 at 8:27 PM, Jonas Sicking wrote:
> On Tue, Sep 15, 2009 at 6:56 PM, Jeremy Orlow
> wrote:
> > One possible solution is to add an asynchronous callback interface
> for
> > LocalStorage into workers. For example:
> > function myCallback(localStorage) {
> > localStorage.accountBalance = localStorage.accountBalance + 100;
> > }
> > executeLocalStorageCallback(myCallback); // TODO: Make this name
> better
> > :-)
> > The interface is simple. You can only access localStorage via a
> callback.
> > Any use outside of the callback is illegal and would raise an
> exception.
> > The callback would acquire the storage mutex during execution, but
> the
> > worker's execution would not block during this time. Of course, it's
> still
> > possible for a poorly behaving worker to do large amounts
> of computation in
> > the callback, but hopefully the fact they're executing in a callback
> makes
> > the developer more aware of the problem.
>
>
Re: [whatwg] LocalStorage in workers
I'm saying that an async API is overkill and unwieldy if all you need is
WorkerLocalStorage.
If you're going to route your localstorage access through an async API
anyway, then you might as well proxy it to the parent page - there's very
little advantage to doing it otherwise, other than access to lexically
scoped resources from within your callback.
But, yeah, if you want to provide access to shared worker/page storage, then
an async API would be the way to go - I'm just saying that if you don't
actually need shared storage, then you could maintain a more convenient
synchronous silo'd API.
Since Jeremy didn't really elaborate on the use case, and he's getting
feedback from app developers that I'm not privy to, I figured I'd ask him.
-atw
On Wed, Sep 16, 2009 at 11:34 AM, Michael Nordman wrote:
>
>
> On Wed, Sep 16, 2009 at 11:24 AM, James Robinson wrote:
>
>> On Wed, Sep 16, 2009 at 10:53 AM, Michael Nordman wrote:
>>
>>>
>>>
>>> On Wed, Sep 16, 2009 at 9:58 AM, Drew Wilson wrote:
>>>
Jeremy, what's the use case here - do developers want workers to have
access to shared local storage with pages? Or do they just want workers to
have access to their own non-shared local storage?
Because we could just give workers their own separate WorkerLocalStorage
and let them have at it. A worker could block all the other accesses to
WorkerLocalStorage within that domain, but so be it - it wouldn't affect
page access, and we already had that issue with the (now removed?)
synchronous SQL API.
I think a much better case can be made for WorkerLocalStorage than for
"give workers access to page LocalStorage", and the design issues are much
simpler.
>>>
>>> Putting workers in their own storage silo doesn't really make much sense?
>>> Sure it may be simpler for browser vendors, but does that make life simpler
>>> for app developers, or just have them scratching their heads about how to
>>> read/write the same data set from either flavor of context in their
>>> application?
>>>
>>> I see no rhyme or reason for the arbitrary barrier except for browser
>>> vendors to work around the awkward implict locks on LocalStorage (the source
>>> of much grief). Consider this... would it make sense to cordon off the
>>> databases workers vs pages can see? I would think not, and i would hope
>>> others agree.
>>>
>>
>> The difference is that the database interface is purely asynchronous
>> whereas storage is synchronous.
>>
>
> Sure... we're talking about adding an async api that allows worker to
> access a local storage repository... should such a thing exist, why should
> it not provide access to the same repository as seen by pages?
>
>
>> If multiple threads have synchronous access to the same shared resource
>> then there has to be a consistency model. ECMAScript does not provide for
>> one so it has to be done at a higher level. Since there was not a solution
>> in the first versions that shipped, the awkward implicit locks you mention
>> were suggested as a workaround. However it's far from clear that these
>> solve the problem and are implementable. It seems like the only logical
>> continuation of this path would be to add explicit, blocking synchronization
>> primitives for developers to deal with - which I think everyone agrees would
>> be a terrible idea. If you're worried about developers scratching their
>> heads about how to pass data between workers just think about happens-before
>> relationships and multi-threaded memory models.
>>
>> In a hypothetical world without synchronous access to LocalStorage/cookies
>> from workers, there is no shared memory between threads except via message
>> passing. This can seem a bit tricky for developers but is very easy to
>> reason about and prove correctness and the absence of deadlocks.
>>
>> - James
>>
>>
>>>
>>>
>>>
-atw
On Tue, Sep 15, 2009 at 8:27 PM, Jonas Sicking wrote:
> On Tue, Sep 15, 2009 at 6:56 PM, Jeremy Orlow
> wrote:
> > One possible solution is to add an asynchronous callback interface
> for
> > LocalStorage into workers. For example:
> > function myCallback(localStorage) {
> > localStorage.accountBalance = localStorage.accountBalance + 100;
> > }
> > executeLocalStorageCallback(myCallback); // TODO: Make this name
> better
> > :-)
> > The interface is simple. You can only access localStorage via a
> callback.
> > Any use outside of the callback is illegal and would raise an
> exception.
> > The callback would acquire the storage mutex during execution, but
> the
> > worker's execution would not block during this time. Of course, it's
> still
> > possible for a poorly behaving worker to do large amounts
> of computation in
> > the callback, but hopefully the fact they're executing in a callback
> makes
> > the developer more aware of the problem.
>
> First of
Re: [whatwg] LocalStorage in workers
On Wed, Sep 16, 2009 at 11:24 AM, James Robinson wrote:
> On Wed, Sep 16, 2009 at 10:53 AM, Michael Nordman wrote:
>
>>
>>
>> On Wed, Sep 16, 2009 at 9:58 AM, Drew Wilson wrote:
>>
>>> Jeremy, what's the use case here - do developers want workers to have
>>> access to shared local storage with pages? Or do they just want workers to
>>> have access to their own non-shared local storage?
>>> Because we could just give workers their own separate WorkerLocalStorage
>>> and let them have at it. A worker could block all the other accesses to
>>> WorkerLocalStorage within that domain, but so be it - it wouldn't affect
>>> page access, and we already had that issue with the (now removed?)
>>> synchronous SQL API.
>>>
>>> I think a much better case can be made for WorkerLocalStorage than for
>>> "give workers access to page LocalStorage", and the design issues are much
>>> simpler.
>>>
>>
>> Putting workers in their own storage silo doesn't really make much sense?
>> Sure it may be simpler for browser vendors, but does that make life simpler
>> for app developers, or just have them scratching their heads about how to
>> read/write the same data set from either flavor of context in their
>> application?
>>
>> I see no rhyme or reason for the arbitrary barrier except for browser
>> vendors to work around the awkward implict locks on LocalStorage (the source
>> of much grief). Consider this... would it make sense to cordon off the
>> databases workers vs pages can see? I would think not, and i would hope
>> others agree.
>>
>
> The difference is that the database interface is purely asynchronous
> whereas storage is synchronous.
>
Sure... we're talking about adding an async api that allows worker to access
a local storage repository... should such a thing exist, why should it not
provide access to the same repository as seen by pages?
> If multiple threads have synchronous access to the same shared resource
> then there has to be a consistency model. ECMAScript does not provide for
> one so it has to be done at a higher level. Since there was not a solution
> in the first versions that shipped, the awkward implicit locks you mention
> were suggested as a workaround. However it's far from clear that these
> solve the problem and are implementable. It seems like the only logical
> continuation of this path would be to add explicit, blocking synchronization
> primitives for developers to deal with - which I think everyone agrees would
> be a terrible idea. If you're worried about developers scratching their
> heads about how to pass data between workers just think about happens-before
> relationships and multi-threaded memory models.
>
> In a hypothetical world without synchronous access to LocalStorage/cookies
> from workers, there is no shared memory between threads except via message
> passing. This can seem a bit tricky for developers but is very easy to
> reason about and prove correctness and the absence of deadlocks.
>
> - James
>
>
>>
>>
>>
>>>
>>> -atw
>>>
>>> On Tue, Sep 15, 2009 at 8:27 PM, Jonas Sicking wrote:
>>>
On Tue, Sep 15, 2009 at 6:56 PM, Jeremy Orlow
wrote:
> One possible solution is to add an asynchronous callback interface for
> LocalStorage into workers. For example:
> function myCallback(localStorage) {
> localStorage.accountBalance = localStorage.accountBalance + 100;
> }
> executeLocalStorageCallback(myCallback); // TODO: Make this name
better
> :-)
> The interface is simple. You can only access localStorage via a
callback.
> Any use outside of the callback is illegal and would raise an
exception.
> The callback would acquire the storage mutex during execution, but
the
> worker's execution would not block during this time. Of course, it's
still
> possible for a poorly behaving worker to do large amounts
of computation in
> the callback, but hopefully the fact they're executing in a callback
makes
> the developer more aware of the problem.
First off, I agree that not having localStorage in workers is a big
problem that we need to address.
If I were designing the localStorage interface today I would use the
above interface that you suggest. Grabbing localStorage can only be
done asynchronously, and while you're using it, no one else can get a
reference to it. This way there are no race conditions, but also no
way for anyone to have to lock.
So one solution is to do that in parallel to the current localStorage
interface. Let's say we introduce a 'clientStorage' object. You can
only get a reference to it using a 'getClientStorage' function. This
function is available both to workers and windows. The storage is
separate from localStorage so no need to worry about the 'storage
mutex'.
There is of course a risk that a worker grabs on to the clientStorage
and holds it inde
Re: [whatwg] LocalStorage in workers
On Wed, Sep 16, 2009 at 10:53 AM, Michael Nordman wrote:
>
>
> On Wed, Sep 16, 2009 at 9:58 AM, Drew Wilson wrote:
>
>> Jeremy, what's the use case here - do developers want workers to have
>> access to shared local storage with pages? Or do they just want workers to
>> have access to their own non-shared local storage?
>> Because we could just give workers their own separate WorkerLocalStorage
>> and let them have at it. A worker could block all the other accesses to
>> WorkerLocalStorage within that domain, but so be it - it wouldn't affect
>> page access, and we already had that issue with the (now removed?)
>> synchronous SQL API.
>>
>> I think a much better case can be made for WorkerLocalStorage than for
>> "give workers access to page LocalStorage", and the design issues are much
>> simpler.
>>
>
> Putting workers in their own storage silo doesn't really make much sense?
> Sure it may be simpler for browser vendors, but does that make life simpler
> for app developers, or just have them scratching their heads about how to
> read/write the same data set from either flavor of context in their
> application?
>
> I see no rhyme or reason for the arbitrary barrier except for browser
> vendors to work around the awkward implict locks on LocalStorage (the source
> of much grief). Consider this... would it make sense to cordon off the
> databases workers vs pages can see? I would think not, and i would hope
> others agree.
>
The difference is that the database interface is purely asynchronous whereas
storage is synchronous.
If multiple threads have synchronous access to the same shared resource then
there has to be a consistency model. ECMAScript does not provide for one so
it has to be done at a higher level. Since there was not a solution in the
first versions that shipped, the awkward implicit locks you mention were
suggested as a workaround. However it's far from clear that these solve the
problem and are implementable. It seems like the only logical continuation
of this path would be to add explicit, blocking synchronization primitives
for developers to deal with - which I think everyone agrees would be a
terrible idea. If you're worried about developers scratching their heads
about how to pass data between workers just think about happens-before
relationships and multi-threaded memory models.
In a hypothetical world without synchronous access to LocalStorage/cookies
from workers, there is no shared memory between threads except via message
passing. This can seem a bit tricky for developers but is very easy to
reason about and prove correctness and the absence of deadlocks.
- James
>
>
>
>>
>> -atw
>>
>> On Tue, Sep 15, 2009 at 8:27 PM, Jonas Sicking wrote:
>>
>>> On Tue, Sep 15, 2009 at 6:56 PM, Jeremy Orlow
>>> wrote:
>>> > One possible solution is to add an asynchronous callback interface for
>>> > LocalStorage into workers. For example:
>>> > function myCallback(localStorage) {
>>> > localStorage.accountBalance = localStorage.accountBalance + 100;
>>> > }
>>> > executeLocalStorageCallback(myCallback); // TODO: Make this name
>>> better
>>> > :-)
>>> > The interface is simple. You can only access localStorage via a
>>> callback.
>>> > Any use outside of the callback is illegal and would raise an
>>> exception.
>>> > The callback would acquire the storage mutex during execution, but the
>>> > worker's execution would not block during this time. Of course, it's
>>> still
>>> > possible for a poorly behaving worker to do large amounts
>>> of computation in
>>> > the callback, but hopefully the fact they're executing in a callback
>>> makes
>>> > the developer more aware of the problem.
>>>
>>> First off, I agree that not having localStorage in workers is a big
>>> problem that we need to address.
>>>
>>> If I were designing the localStorage interface today I would use the
>>> above interface that you suggest. Grabbing localStorage can only be
>>> done asynchronously, and while you're using it, no one else can get a
>>> reference to it. This way there are no race conditions, but also no
>>> way for anyone to have to lock.
>>>
>>> So one solution is to do that in parallel to the current localStorage
>>> interface. Let's say we introduce a 'clientStorage' object. You can
>>> only get a reference to it using a 'getClientStorage' function. This
>>> function is available both to workers and windows. The storage is
>>> separate from localStorage so no need to worry about the 'storage
>>> mutex'.
>>>
>>> There is of course a risk that a worker grabs on to the clientStorage
>>> and holds it indefinitely. This would result in the main window (or
>>> another worker) never getting a reference to it. However it doesn't
>>> affect responsiveness of that window, it's just that the callback will
>>> never happen. While that's not ideal, it seems like a smaller problem
>>> than any other solution that I can think of. And the WebDatabase
>>> interfaces are suffering from the same problem if I u
Re: [whatwg] LocalStorage in workers
On Wed, Sep 16, 2009 at 9:58 AM, Drew Wilson wrote:
> Jeremy, what's the use case here - do developers want workers to have
> access to shared local storage with pages? Or do they just want workers to
> have access to their own non-shared local storage?
> Because we could just give workers their own separate WorkerLocalStorage
> and let them have at it. A worker could block all the other accesses to
> WorkerLocalStorage within that domain, but so be it - it wouldn't affect
> page access, and we already had that issue with the (now removed?)
> synchronous SQL API.
>
> I think a much better case can be made for WorkerLocalStorage than for
> "give workers access to page LocalStorage", and the design issues are much
> simpler.
>
Putting workers in their own storage silo doesn't really make much sense?
Sure it may be simpler for browser vendors, but does that make life simpler
for app developers, or just have them scratching their heads about how to
read/write the same data set from either flavor of context in their
application?
I see no rhyme or reason for the arbitrary barrier except for browser
vendors to work around the awkward implict locks on LocalStorage (the source
of much grief). Consider this... would it make sense to cordon off the
databases workers vs pages can see? I would think not, and i would hope
others agree.
>
> -atw
>
> On Tue, Sep 15, 2009 at 8:27 PM, Jonas Sicking wrote:
>
>> On Tue, Sep 15, 2009 at 6:56 PM, Jeremy Orlow
>> wrote:
>> > One possible solution is to add an asynchronous callback interface for
>> > LocalStorage into workers. For example:
>> > function myCallback(localStorage) {
>> > localStorage.accountBalance = localStorage.accountBalance + 100;
>> > }
>> > executeLocalStorageCallback(myCallback); // TODO: Make this name better
>> > :-)
>> > The interface is simple. You can only access localStorage via a
>> callback.
>> > Any use outside of the callback is illegal and would raise an
>> exception.
>> > The callback would acquire the storage mutex during execution, but the
>> > worker's execution would not block during this time. Of course, it's
>> still
>> > possible for a poorly behaving worker to do large amounts
>> of computation in
>> > the callback, but hopefully the fact they're executing in a callback
>> makes
>> > the developer more aware of the problem.
>>
>> First off, I agree that not having localStorage in workers is a big
>> problem that we need to address.
>>
>> If I were designing the localStorage interface today I would use the
>> above interface that you suggest. Grabbing localStorage can only be
>> done asynchronously, and while you're using it, no one else can get a
>> reference to it. This way there are no race conditions, but also no
>> way for anyone to have to lock.
>>
>> So one solution is to do that in parallel to the current localStorage
>> interface. Let's say we introduce a 'clientStorage' object. You can
>> only get a reference to it using a 'getClientStorage' function. This
>> function is available both to workers and windows. The storage is
>> separate from localStorage so no need to worry about the 'storage
>> mutex'.
>>
>> There is of course a risk that a worker grabs on to the clientStorage
>> and holds it indefinitely. This would result in the main window (or
>> another worker) never getting a reference to it. However it doesn't
>> affect responsiveness of that window, it's just that the callback will
>> never happen. While that's not ideal, it seems like a smaller problem
>> than any other solution that I can think of. And the WebDatabase
>> interfaces are suffering from the same problem if I understand things
>> correctly.
>>
>> There's a couple of other interesting things we could expose on top of
>> this:
>>
>> First, a synchronous API for workers. We could allow workers to
>> synchronously get a reference to clientStorage. If someone is
>> currently using clientStorage then the worker blocks until the storage
>> becomes available. We could either use a callback as the above, which
>> blocks until the clientStorage is acquired and only holds the storage
>> until the callback exists. Or we could expose clientStorage as a
>> property which holds the storage until control is returned to the
>> worker eventloop, or until some explicit release API is called. The
>> latter would be how localStorage is now defined, with the important
>> difference that localStorage exposes the synchronous API to windows.
>>
>> Second, allow several named storage areas. We could add an API like
>> getNamedClientStorage(name, callback). This would allow two different
>> workers to simultaneously store things in a storage areas, as long as
>> they don't need to use the *same* storage area. It would also allow a
>> worker and the main window to simultaneously use separate storage
>> areas.
>>
>> However we need to be careful if we add both above features. We can't
>> allow a worker to grab multiple storage areas at the same time since
>> that could
Re: [whatwg] LocalStorage in workers
Jeremy, what's the use case here - do developers want workers to have access
to shared local storage with pages? Or do they just want workers to have
access to their own non-shared local storage?
Because we could just give workers their own separate WorkerLocalStorage and
let them have at it. A worker could block all the other accesses to
WorkerLocalStorage within that domain, but so be it - it wouldn't affect
page access, and we already had that issue with the (now removed?)
synchronous SQL API.
I think a much better case can be made for WorkerLocalStorage than for "give
workers access to page LocalStorage", and the design issues are much
simpler.
-atw
On Tue, Sep 15, 2009 at 8:27 PM, Jonas Sicking wrote:
> On Tue, Sep 15, 2009 at 6:56 PM, Jeremy Orlow wrote:
> > One possible solution is to add an asynchronous callback interface for
> > LocalStorage into workers. For example:
> > function myCallback(localStorage) {
> > localStorage.accountBalance = localStorage.accountBalance + 100;
> > }
> > executeLocalStorageCallback(myCallback); // TODO: Make this name better
> > :-)
> > The interface is simple. You can only access localStorage via a
> callback.
> > Any use outside of the callback is illegal and would raise an exception.
> > The callback would acquire the storage mutex during execution, but the
> > worker's execution would not block during this time. Of course, it's
> still
> > possible for a poorly behaving worker to do large amounts
> of computation in
> > the callback, but hopefully the fact they're executing in a callback
> makes
> > the developer more aware of the problem.
>
> First off, I agree that not having localStorage in workers is a big
> problem that we need to address.
>
> If I were designing the localStorage interface today I would use the
> above interface that you suggest. Grabbing localStorage can only be
> done asynchronously, and while you're using it, no one else can get a
> reference to it. This way there are no race conditions, but also no
> way for anyone to have to lock.
>
> So one solution is to do that in parallel to the current localStorage
> interface. Let's say we introduce a 'clientStorage' object. You can
> only get a reference to it using a 'getClientStorage' function. This
> function is available both to workers and windows. The storage is
> separate from localStorage so no need to worry about the 'storage
> mutex'.
>
> There is of course a risk that a worker grabs on to the clientStorage
> and holds it indefinitely. This would result in the main window (or
> another worker) never getting a reference to it. However it doesn't
> affect responsiveness of that window, it's just that the callback will
> never happen. While that's not ideal, it seems like a smaller problem
> than any other solution that I can think of. And the WebDatabase
> interfaces are suffering from the same problem if I understand things
> correctly.
>
> There's a couple of other interesting things we could expose on top of
> this:
>
> First, a synchronous API for workers. We could allow workers to
> synchronously get a reference to clientStorage. If someone is
> currently using clientStorage then the worker blocks until the storage
> becomes available. We could either use a callback as the above, which
> blocks until the clientStorage is acquired and only holds the storage
> until the callback exists. Or we could expose clientStorage as a
> property which holds the storage until control is returned to the
> worker eventloop, or until some explicit release API is called. The
> latter would be how localStorage is now defined, with the important
> difference that localStorage exposes the synchronous API to windows.
>
> Second, allow several named storage areas. We could add an API like
> getNamedClientStorage(name, callback). This would allow two different
> workers to simultaneously store things in a storage areas, as long as
> they don't need to use the *same* storage area. It would also allow a
> worker and the main window to simultaneously use separate storage
> areas.
>
> However we need to be careful if we add both above features. We can't
> allow a worker to grab multiple storage areas at the same time since
> that could cause deadlocks. However with proper APIs I believe we can
> avoid that.
>
> / Jonas
>
Re: [whatwg] LocalStorage in workers
On Tue, Sep 15, 2009 at 6:56 PM, Jeremy Orlow wrote:
> One possible solution is to add an asynchronous callback interface for
> LocalStorage into workers. For example:
> function myCallback(localStorage) {
> localStorage.accountBalance = localStorage.accountBalance + 100;
> }
> executeLocalStorageCallback(myCallback); // TODO: Make this name better
> :-)
> The interface is simple. You can only access localStorage via a callback.
> Any use outside of the callback is illegal and would raise an exception.
> The callback would acquire the storage mutex during execution, but the
> worker's execution would not block during this time. Of course, it's still
> possible for a poorly behaving worker to do large amounts of computation in
> the callback, but hopefully the fact they're executing in a callback makes
> the developer more aware of the problem.
First off, I agree that not having localStorage in workers is a big
problem that we need to address.
If I were designing the localStorage interface today I would use the
above interface that you suggest. Grabbing localStorage can only be
done asynchronously, and while you're using it, no one else can get a
reference to it. This way there are no race conditions, but also no
way for anyone to have to lock.
So one solution is to do that in parallel to the current localStorage
interface. Let's say we introduce a 'clientStorage' object. You can
only get a reference to it using a 'getClientStorage' function. This
function is available both to workers and windows. The storage is
separate from localStorage so no need to worry about the 'storage
mutex'.
There is of course a risk that a worker grabs on to the clientStorage
and holds it indefinitely. This would result in the main window (or
another worker) never getting a reference to it. However it doesn't
affect responsiveness of that window, it's just that the callback will
never happen. While that's not ideal, it seems like a smaller problem
than any other solution that I can think of. And the WebDatabase
interfaces are suffering from the same problem if I understand things
correctly.
There's a couple of other interesting things we could expose on top of this:
First, a synchronous API for workers. We could allow workers to
synchronously get a reference to clientStorage. If someone is
currently using clientStorage then the worker blocks until the storage
becomes available. We could either use a callback as the above, which
blocks until the clientStorage is acquired and only holds the storage
until the callback exists. Or we could expose clientStorage as a
property which holds the storage until control is returned to the
worker eventloop, or until some explicit release API is called. The
latter would be how localStorage is now defined, with the important
difference that localStorage exposes the synchronous API to windows.
Second, allow several named storage areas. We could add an API like
getNamedClientStorage(name, callback). This would allow two different
workers to simultaneously store things in a storage areas, as long as
they don't need to use the *same* storage area. It would also allow a
worker and the main window to simultaneously use separate storage
areas.
However we need to be careful if we add both above features. We can't
allow a worker to grab multiple storage areas at the same time since
that could cause deadlocks. However with proper APIs I believe we can
avoid that.
/ Jonas
[whatwg] LocalStorage in workers
I've talked to a lot of web developers about workers and the one thing that
they keep expressing to me is how disappointing it is to not have access to
LocalStorage. I tell them about the "work around" which is to pass a
message back to the page (or, in the case of shared workers, any page) and
ask it to look up the value, but this approach is uninteresting to most of
them. Enough so that most of them have said they'd probably just wait until
workers were more "mature" (and/or WebDatabase was more widely available)
before bothering with them.
The problem is that LocalStorage is a synchronous interface. When you
access it from a normal web page, all other event loops that try to access
it will block until either you exit your script context or call
yieldForStorageUpdates manually. Of course, during this time, the UI of the
browser will be locked up and many UAs would present the user with a slow
script dialog. This is (hopefully) enough to keep web apps reasonably in
check.
Unfortunately, it's perfectly reasonable for a worker to run forever. For
example, it might calculate pi. If, for example, that worker periodically
saved the value of pi to localStorage (and didn't call
yieldForStorageUpdates) then you could block your other event loops forever.
Even if a worker held the lock for a couple hundred milliseconds, it could
still affect the responsiveness of a web browser in a perceivable for no
apparent reason (from the users perspective). For this reason, it was taken
out of the spec.
One possible solution is to add an asynchronous callback interface for
LocalStorage into workers. For example:
function myCallback(localStorage) {
localStorage.accountBalance = localStorage.accountBalance + 100;
}
executeLocalStorageCallback(myCallback); // TODO: Make this name better
:-)
The interface is simple. You can only access localStorage via a callback.
Any use outside of the callback is illegal and would raise an exception.
The callback would acquire the storage mutex during execution, but the
worker's execution would not block during this time. Of course, it's still
possible for a poorly behaving worker to do large amounts of computation in
the callback, but hopefully the fact they're executing in a callback makes
the developer more aware of the problem.
I admit that this is not a great solution and would definitely like to hear
alternate proposals. But, no matter what, I think we need to think
seriously about giving workers access to LocalStorage again.
J
