Re: [whatwg] localStorage + worker processes
On Sat, Mar 28, 2009 at 11:29 AM, Michael Nordman wrote: > On Tue, Mar 24, 2009 at 2:11 AM, Ian Hickson wrote: > >> On Fri, 20 Mar 2009, Jonas Sicking wrote: >> > I do think it would be great if workers had access to some type of >> > structured storage. However I agree that the fact that both the main >> > thread and workers have synchronous access to the same storage is not >> > acceptable since that means that we're violating the >> > shared-nothing-message-passing design that makes workers not have to >> > deal with locks and other traditional multithread hazards. >> >> Agreed. The Database API seems well-suited for this, though. > > > Again... its not just workers that are affected by this... speaking as > someone > that works on a multi-threaded browser, this is troubling. If its possible > to > spec features that allow script to poke at the world beyond the page > boundaries in a fashion that doesn't not require locking semantics beyond > (I assume that "not" was unintended.) > the scope of a single scriptable API call... that would be less troubling. > SQL storage where the single API call is "run this complex transaction" might be acceptable. But if the single API call is "access some shared state" and it's up to Web developers to deal with locking and synchronization --- that will never be acceptable. That burden must fall on browser developers, not Web developers. Rob -- "He was pierced for our transgressions, he was crushed for our iniquities; the punishment that brought us peace was upon him, and by his wounds we are healed. We all, like sheep, have gone astray, each of us has turned to his own way; and the LORD has laid on him the iniquity of us all." [Isaiah 53:5-6]
Re: [whatwg] localStorage + worker processes
> sessionLifetime + tabSpecificScope doesn't make much sense since > you get a new set of tabs when starting a new session Sorry... make that persistentLifetime + tabScope doesn't make sense. On Fri, Mar 27, 2009 at 3:29 PM, Michael Nordman wrote: > > > On Tue, Mar 24, 2009 at 2:11 AM, Ian Hickson wrote: > >> >> I've updated the specs as follows: >> >> - removed localStorage from Web Workers for now. >> >> - extended the implicit lock mechanism that we had for storage to also >> cover document.cookie, and made the language more explicit about how >> it works. >> >> - added navigator.releaseLock(). >> >> >> On Fri, 20 Mar 2009, Jeremy Orlow wrote: >> > >> > Anyhow, the very first example in the spec ( >> > http://dev.w3.org/html5/workers/#a-background-number-crunching-worker) >> > shows work that's being done in a infinite loop with postMessage being >> > called when each prime is found. If you called localStorage anywhere >> > within that loop (say, to periodically save all primes found), you would >> > not be able to safely call window.localStorage in any other worker or >> > the web page. This is because the "task that started the script" never >> > ends. And this its 'lock' (on other scripts using local storage) will >> > never be released. >> >> I've removed localStorage from the Web Workers spec for now. >> >> >> On Fri, 20 Mar 2009, Jonas Sicking wrote: >> > >> > I do think it would be great if workers had access to some type of >> > structured storage. However I agree that the fact that both the main >> > thread and workers have synchronous access to the same storage is not >> > acceptable since that means that we're violating the >> > shared-nothing-message-passing design that makes workers not have to >> > deal with locks and other traditional multithread hazards. >> >> Agreed. The Database API seems well-suited for this, though. > > > Again... its not just workers that are affected by this... speaking as > someone > that works on a multi-threaded browser, this is troubling. If its possible > to > spec features that allow script to poke at the world beyond the page > boundaries in a fashion that doesn't not require locking semantics beyond > the scope of a single scriptable API call... that would be less troubling. > > >> >> >> On Fri, 20 Mar 2009, Drew Wilson wrote: >> > >> > One alternative I'd like to propose is to remove access to localStorage >> > for dedicated workers, and give SharedWorkers access to localStorage, >> > but have that storage be partitioned by the worker name (i.e. the worker >> > can access it, but it's not shared with web pages or any other workers >> > and so you don't have any synchronicity issues). >> >> That's an interesting idea, and would be relatively easy to do. Do people >> think it is worth it? > > > I think there's some additional low-hanging fruit too. We're toying with > two, > independent axis: lifetime vs accessScope. > > 'sessionStorage' has sessionOnlyLifetime and tabSpecificScope > 'localStorage' has persistentLifetime and browserWideScope > > In this nomenclature, the new idea could be phrased as... > > 'page/workerStorage' has persistentLifetime and page/workerSpecificScope > > Other slots in the matrix formed by these two axis could make sense... > > sessionLifetime + page/workerSpecificScope > sessionLifetime + browserWideScope > > sessionLifetime + tabSpecificScope doesn't make much sense since > you get a new set of tabs when starting a new session > > >> >> >> >> On Fri, 20 Mar 2009, Aaron Boodman wrote: >> > >> > I think the best option is to make access to localstorage asynchronous >> > for workers. This reduces the amount of time a worker can hold the >> > localstore lock so that it shouldn't be a problem for normal pages. It >> > sucks to make such a simple and useful API aync though. >> >> I don't think making it async helps here, since the problem isn't that it >> is synchronous, but that workers don't return quickly (by design). >> >> >> On Sat, 21 Mar 2009, Aaron Boodman wrote: >> > >> > Actually, I don't believe that it is required that the callback run >> > asynchronously. All the callback is used for is establishing the lock >> > lifetime explicitly, and we assume that this will usually make the lock >> > lifetime short. So we can block while we wait for it to become >> > available. This is just like the behavior today without workers. >> >> Nothing is to stop someone from just having a long callback, though. >> >> >> On Sat, 21 Mar 2009, Jonas Sicking wrote: >> > >> > As I understand the current API (on main window) to be defined, as soon >> > as someone accesses the .localStorage property, the implementation is >> > supposed to acquire a lock. This lock would be held on to until that >> > script returns to the event loop for that thread. >> > >> > So if javascript in another window, running in another thread or >> > process, tries to access .localStorage for the same origin, the >> > .localStorage getter would
Re: [whatwg] localStorage + worker processes
On Tue, Mar 24, 2009 at 2:11 AM, Ian Hickson wrote: > > I've updated the specs as follows: > > - removed localStorage from Web Workers for now. > > - extended the implicit lock mechanism that we had for storage to also > cover document.cookie, and made the language more explicit about how > it works. > > - added navigator.releaseLock(). > > > On Fri, 20 Mar 2009, Jeremy Orlow wrote: > > > > Anyhow, the very first example in the spec ( > > http://dev.w3.org/html5/workers/#a-background-number-crunching-worker) > > shows work that's being done in a infinite loop with postMessage being > > called when each prime is found. If you called localStorage anywhere > > within that loop (say, to periodically save all primes found), you would > > not be able to safely call window.localStorage in any other worker or > > the web page. This is because the "task that started the script" never > > ends. And this its 'lock' (on other scripts using local storage) will > > never be released. > > I've removed localStorage from the Web Workers spec for now. > > > On Fri, 20 Mar 2009, Jonas Sicking wrote: > > > > I do think it would be great if workers had access to some type of > > structured storage. However I agree that the fact that both the main > > thread and workers have synchronous access to the same storage is not > > acceptable since that means that we're violating the > > shared-nothing-message-passing design that makes workers not have to > > deal with locks and other traditional multithread hazards. > > Agreed. The Database API seems well-suited for this, though. Again... its not just workers that are affected by this... speaking as someone that works on a multi-threaded browser, this is troubling. If its possible to spec features that allow script to poke at the world beyond the page boundaries in a fashion that doesn't not require locking semantics beyond the scope of a single scriptable API call... that would be less troubling. > > > On Fri, 20 Mar 2009, Drew Wilson wrote: > > > > One alternative I'd like to propose is to remove access to localStorage > > for dedicated workers, and give SharedWorkers access to localStorage, > > but have that storage be partitioned by the worker name (i.e. the worker > > can access it, but it's not shared with web pages or any other workers > > and so you don't have any synchronicity issues). > > That's an interesting idea, and would be relatively easy to do. Do people > think it is worth it? I think there's some additional low-hanging fruit too. We're toying with two, independent axis: lifetime vs accessScope. 'sessionStorage' has sessionOnlyLifetime and tabSpecificScope 'localStorage' has persistentLifetime and browserWideScope In this nomenclature, the new idea could be phrased as... 'page/workerStorage' has persistentLifetime and page/workerSpecificScope Other slots in the matrix formed by these two axis could make sense... sessionLifetime + page/workerSpecificScope sessionLifetime + browserWideScope sessionLifetime + tabSpecificScope doesn't make much sense since you get a new set of tabs when starting a new session > > > > On Fri, 20 Mar 2009, Aaron Boodman wrote: > > > > I think the best option is to make access to localstorage asynchronous > > for workers. This reduces the amount of time a worker can hold the > > localstore lock so that it shouldn't be a problem for normal pages. It > > sucks to make such a simple and useful API aync though. > > I don't think making it async helps here, since the problem isn't that it > is synchronous, but that workers don't return quickly (by design). > > > On Sat, 21 Mar 2009, Aaron Boodman wrote: > > > > Actually, I don't believe that it is required that the callback run > > asynchronously. All the callback is used for is establishing the lock > > lifetime explicitly, and we assume that this will usually make the lock > > lifetime short. So we can block while we wait for it to become > > available. This is just like the behavior today without workers. > > Nothing is to stop someone from just having a long callback, though. > > > On Sat, 21 Mar 2009, Jonas Sicking wrote: > > > > As I understand the current API (on main window) to be defined, as soon > > as someone accesses the .localStorage property, the implementation is > > supposed to acquire a lock. This lock would be held on to until that > > script returns to the event loop for that thread. > > > > So if javascript in another window, running in another thread or > > process, tries to access .localStorage for the same origin, the > > .localStorage getter would try to acquire the same lock and block until > > the first thread releases the lock. > > Right. > > > On Sat, 21 Mar 2009, Jonas Sicking wrote: > > > > The problem with synchronously grabbing the lock is that we can only > > ever have one feature that uses synchronous locks, otherwise we'll risk > > dead-locks. > > Indeed. This is a problem with the current API for localStorage in windows > as
Re: [whatwg] localStorage + worker processes
On Tue, Mar 24, 2009 at 10:11 PM, Ian Hickson wrote:
> - extended the implicit lock mechanism that we had for storage to also
> cover document.cookie, and made the language more explicit about how
> it works.
That's basically good. It's possible that people might want to implement
something that's equivalent to the storage mutex in observable behaviour,
but allows more parallelism, such as speculative execution or finer-grained
locking when the implementation can prove it's safe. I assume implementors
of HTML5 already understand that that's allowed.
- added navigator.releaseLock().
>
This name could be confusing to developers, because there is no
corresponding explicit acquireLock(), which there usually is in an API that
exposes releaseLock().
navigator.allowInterruption() maybe?
On Sat, 21 Mar 2009, Jonas Sicking wrote:
> >
> > As a side note, if we do go with this async lock acquiring, we could add
> > an API like:
> >
> > getLockedFeatures(callback, 'localStore', 'cookie');
> >
> > This would be an asynchronously grab locks to multiple features and only
> > call the callback once all of them have been acquired. This would allow
> > computations across data from multiple locations guaranteed to be in
> > sync. The implementation would be responsible for grabbing the locks in
> > a consistent order to prevent deadlocks.
>
> Why would we want more than one lock? Is the potential performance gain
> worth the complexity?
>
> The problem with going with an async approach is that it means changing
> the API, which is something we can't really do for cookie (and don't
> really want to do for localStorage, since IE8 has shipped it.) We we are
> going to need a synchronous locking mechanism anyway.
It would be possible to use something like getLockedFeatures for workers
while using implicit locking for the main thread.
On Mon, 23 Mar 2009, Robert O'Callahan wrote:
> > It has to be resolved in a way that doesn't expose asynchronous cookie
> > or localStorage changes to Web developers. There is abundant evidence
> > that race conditions and synchronization are too hard for developers to
> > deal with. The spec should forbid asynchronously visible changes to
> > cookies or localStorage. In fact, it should probably simply say that all
> > script execution is serializable: always equivalent to some execution
> > you could get with a single-threaded browser that runs all scripts to
> > completion. Allowance could be made for explicit yield points if we need
> > to, e.g. alert().
>
> Generally speaking I have tried to maintain this invariant, but I have
> failed with cookies, and with localStorage in workers.
Now, with the storage mutex, are there any cases you know of where
serializability fails? If there are, it may be worth noting them in the
spec. If there aren't, why not simply write serializability into the spec?
User agents that share event loops between origins can't actually have any
> more than one lock total. Consider a case where there are three windows
> from three different origins, A, B, and C, where C contains a couple of
> s, and where A, B, and C are independent, but C share an event
> loop with whatever content is in its iframes. (This is the situation
> Chrome and IE are in, as I understand it, with event loops being
> per-window not per-origin, and it may be required because access to the
> frames[] hierarchy is synchronous.) Now, assume A and B have both obtained
> their respective locks, and are busy doing some long script. C is free to
> run more tasks from its event loop, which could include navigating one
> iframe to a page on either A and the other iframe to a page on B, meaning
> that the event loop of C is now beholden to two locks. If there is any
> manner in which to synchronously cause another origin to run script, this
> now means that C can attempt to obtain both locks; if we now imagine
> another window just like C that instead obtains the locks in the reverse
> order, we get deadlock.
Interesting example!
When two sets of unrelated browser contexts become related (e.g., C loads A
into an iframe), I imagined you would join A's lock and C's lock into a
single lock covering the new set of related browser contexts, which is safe
to do if at most one of those locks is currently held. When this happens due
to a document being created with origin A in C's iframe, it happens
asynchronously in C, right? So at that point C's lock is not held by
currently running script in C (although it might be held by code in another
domain which is already related to C), and we can block the join operation
in C until one of the two locks is released.
Then in your example, suppose C loads A's document first. Then C's lock and
A's lock are joined to make a CA-lock. Then suppose D ("another window just
like C") loads B's document; D's lock and B's lock are merged to make the
DB-lock. Now suppose C loads B. The two remaining locks are merged to form a
single CADB-lock. No deadlock is poss
Re: [whatwg] localStorage + worker processes
I've updated the specs as follows:
- removed localStorage from Web Workers for now.
- extended the implicit lock mechanism that we had for storage to also
cover document.cookie, and made the language more explicit about how
it works.
- added navigator.releaseLock().
On Fri, 20 Mar 2009, Jeremy Orlow wrote:
>
> Anyhow, the very first example in the spec (
> http://dev.w3.org/html5/workers/#a-background-number-crunching-worker)
> shows work that's being done in a infinite loop with postMessage being
> called when each prime is found. If you called localStorage anywhere
> within that loop (say, to periodically save all primes found), you would
> not be able to safely call window.localStorage in any other worker or
> the web page. This is because the "task that started the script" never
> ends. And this its 'lock' (on other scripts using local storage) will
> never be released.
I've removed localStorage from the Web Workers spec for now.
On Fri, 20 Mar 2009, Jonas Sicking wrote:
>
> I do think it would be great if workers had access to some type of
> structured storage. However I agree that the fact that both the main
> thread and workers have synchronous access to the same storage is not
> acceptable since that means that we're violating the
> shared-nothing-message-passing design that makes workers not have to
> deal with locks and other traditional multithread hazards.
Agreed. The Database API seems well-suited for this, though.
On Fri, 20 Mar 2009, Drew Wilson wrote:
>
> One alternative I'd like to propose is to remove access to localStorage
> for dedicated workers, and give SharedWorkers access to localStorage,
> but have that storage be partitioned by the worker name (i.e. the worker
> can access it, but it's not shared with web pages or any other workers
> and so you don't have any synchronicity issues).
That's an interesting idea, and would be relatively easy to do. Do people
think it is worth it?
On Fri, 20 Mar 2009, Aaron Boodman wrote:
>
> I think the best option is to make access to localstorage asynchronous
> for workers. This reduces the amount of time a worker can hold the
> localstore lock so that it shouldn't be a problem for normal pages. It
> sucks to make such a simple and useful API aync though.
I don't think making it async helps here, since the problem isn't that it
is synchronous, but that workers don't return quickly (by design).
On Sat, 21 Mar 2009, Aaron Boodman wrote:
>
> Actually, I don't believe that it is required that the callback run
> asynchronously. All the callback is used for is establishing the lock
> lifetime explicitly, and we assume that this will usually make the lock
> lifetime short. So we can block while we wait for it to become
> available. This is just like the behavior today without workers.
Nothing is to stop someone from just having a long callback, though.
On Sat, 21 Mar 2009, Jonas Sicking wrote:
>
> As I understand the current API (on main window) to be defined, as soon
> as someone accesses the .localStorage property, the implementation is
> supposed to acquire a lock. This lock would be held on to until that
> script returns to the event loop for that thread.
>
> So if javascript in another window, running in another thread or
> process, tries to access .localStorage for the same origin, the
> .localStorage getter would try to acquire the same lock and block until
> the first thread releases the lock.
Right.
On Sat, 21 Mar 2009, Jonas Sicking wrote:
>
> The problem with synchronously grabbing the lock is that we can only
> ever have one feature that uses synchronous locks, otherwise we'll risk
> dead-locks.
Indeed. This is a problem with the current API for localStorage in windows
as well.
I've made the spec explicitly have a single shared lock for all features
that need locking (currently just .cookie and .localStorage).
On Sun, 22 Mar 2009, Michael Nordman wrote:
>
> Given an async api, would it be possible to store values into
> localStorage at onunload time? I expect that could be a useful time to
> use this API.
>
> function onunload() {
> getLocalStorage(function(storage) {
> // Will this ever execute?
> });
> }
>
> Locking the storage until script completion isn't really necessary in
> many cases. Maybe we're over engineering this? Suppose immutability
> across calls was generally not guaranteed by the existing API. And we
> add an async getLocalStorage(callback) which does provide immutability
> for the duration of the callback if that is desired.
The problem is that people will walk into race conditions without
realising it, and they are amongst the hardest problems to debug.
On Sun, 22 Mar 2009, Drew Wilson wrote:
>
> The problem is that .length is basically useless without some kind of
> immutability guarantees.
Indeed.
On Sun, 22 Mar 2009, Drew Wilson wrote:
>
> That's why I'm proposing that the most reasonable implementation is just
> to
Re: [whatwg] localStorage + worker processes
*Given that things have simmered down, it seems like it's time for a re-cap of the options. I believe this includes all the options currently on the table (in broad strokes anyhow). If I missed anything or grossly mischaracterized any idea, feel free to correct me.* 0: Do nothing. As far as I can tell, no one thinks this is a valid option. 1: Simply disallow localStorage in workers. Possibly allow in shared/persistent workers, but give them their own private storage area. 2: Explicit locking. Possibly only required for workers. Would require dead lock detection or for the spec to dictate locking precedence if another lock were introduced. 3: Expose localStorage to workers as asynchronous calls. Most similar to other concurrency solutions within workers. 4: Have some yield/checkpoint/commit type call on localStorage that would release any implicit/invisible lock. This could also be useful in conjunction with #3. *My opinion:* #1 would definitely work, but doesn't seem necessary. #2 seems to go against the grain of how concurrency is implemented everywhere else in the spec. In addition, it could open pandora's box in terms of locks being added in elsewhere and starting to cause some messy issues there. #3 and #4 preserve the simplicity of localStorage for the simple/common case--which is important above else, in my opinion. On Mon, Mar 23, 2009 at 2:48 PM, Robert O'Callahan wrote: > On Tue, Mar 24, 2009 at 10:40 AM, Jeremy Orlow wrote: > >> I really like the idea of some generic yield, though I wonder if there's >> some reason it hasn't been added earlier. People have been using the >> setTimeout(..., 0) trick for a while to get around slow script warnings (and >> general unresponsiveness)...so surely something like this must have come up >> before? If so, what were the drawbacks? >> > > An obvious issue is that you can easily end up with arbitrarily deep > nesting and stack overflows. > > Rob > -- > "He was pierced for our transgressions, he was crushed for our iniquities; > the punishment that brought us peace was upon him, and by his wounds we are > healed. We all, like sheep, have gone astray, each of us has turned to his > own way; and the LORD has laid on him the iniquity of us all." [Isaiah > 53:5-6] >
Re: [whatwg] localStorage + worker processes
On Tue, Mar 24, 2009 at 10:40 AM, Jeremy Orlow wrote: > I really like the idea of some generic yield, though I wonder if there's > some reason it hasn't been added earlier. People have been using the > setTimeout(..., 0) trick for a while to get around slow script warnings (and > general unresponsiveness)...so surely something like this must have come up > before? If so, what were the drawbacks? > An obvious issue is that you can easily end up with arbitrarily deep nesting and stack overflows. Rob -- "He was pierced for our transgressions, he was crushed for our iniquities; the punishment that brought us peace was upon him, and by his wounds we are healed. We all, like sheep, have gone astray, each of us has turned to his own way; and the LORD has laid on him the iniquity of us all." [Isaiah 53:5-6]
Re: [whatwg] localStorage + worker processes
I really like the idea of some generic yield, though I wonder if there's some reason it hasn't been added earlier. People have been using the setTimeout(..., 0) trick for a while to get around slow script warnings (and general unresponsiveness)...so surely something like this must have come up before? If so, what were the drawbacks? On Mon, Mar 23, 2009 at 2:24 PM, Robert O'Callahan wrote: > On Tue, Mar 24, 2009 at 7:41 AM, Jeremy Orlow wrote: > >> One thing that hasn't been considered yet is some sort of optional hint to >> say "I'm done" in terms of accessing localStorage. Maybe call it >> localStorage.checkpoint() or localStroage.commit()? >> >> As far as the browser implemenation is concerned, a call to this function >> would be the same as the script ending. The great thing about this is that >> if a developer found one problem location in their code, they could add >> it--but it'd be completely optional. >> > > You mean if they find a performance problem due to overlarge lock scope? > > That's not a terrible idea. It certainly makes far more sense to be safe by > default and add constructs and code to gain a bit more parallelism, than to > be unsafe and parallel by default and have to add constructs and code to get > safety. > > I'm not sure what the semantics of a standalone "commit()" would be, > though. I think a better construct might be some sort of "yield" which > explicitly returns to a (nested) browser event loop and basically acts as a > script completion point. Even browsers that only use a single thread can run > the event loop there so that testing in those browsers will reveal bugs. > > > Rob > -- > "He was pierced for our transgressions, he was crushed for our iniquities; > the punishment that brought us peace was upon him, and by his wounds we are > healed. We all, like sheep, have gone astray, each of us has turned to his > own way; and the LORD has laid on him the iniquity of us all." [Isaiah > 53:5-6] >
Re: [whatwg] localStorage + worker processes
On Tue, Mar 24, 2009 at 7:41 AM, Jeremy Orlow wrote: > One thing that hasn't been considered yet is some sort of optional hint to > say "I'm done" in terms of accessing localStorage. Maybe call it > localStorage.checkpoint() or localStroage.commit()? > > As far as the browser implemenation is concerned, a call to this function > would be the same as the script ending. The great thing about this is that > if a developer found one problem location in their code, they could add > it--but it'd be completely optional. > You mean if they find a performance problem due to overlarge lock scope? That's not a terrible idea. It certainly makes far more sense to be safe by default and add constructs and code to gain a bit more parallelism, than to be unsafe and parallel by default and have to add constructs and code to get safety. I'm not sure what the semantics of a standalone "commit()" would be, though. I think a better construct might be some sort of "yield" which explicitly returns to a (nested) browser event loop and basically acts as a script completion point. Even browsers that only use a single thread can run the event loop there so that testing in those browsers will reveal bugs. Rob -- "He was pierced for our transgressions, he was crushed for our iniquities; the punishment that brought us peace was upon him, and by his wounds we are healed. We all, like sheep, have gone astray, each of us has turned to his own way; and the LORD has laid on him the iniquity of us all." [Isaiah 53:5-6]
Re: [whatwg] localStorage + worker processes
On Mon, Mar 23, 2009 at 11:41 AM, Jeremy Orlow wrote: > One thing that hasn't been considered yet is some sort of optional hint to > say "I'm done" in terms of accessing localStorage. Maybe call it > localStorage.checkpoint() or localStroage.commit()? > As far as the browser implemenation is concerned, a call to this function > would be the same as the script ending. The great thing about this is that > if a developer found one problem location in their code, they could add > it--but it'd be completely optional. > This could be used in conjunction with most of the other ideas already > floating around. This is definitely an interesting idea. It does become less critical if we go with the async lock approach, but even then it's worth considering. / Jonas
Re: [whatwg] localStorage + worker processes
On Mon, Mar 23, 2009 at 11:41 AM, Jeremy Orlow wrote: > One thing that hasn't been considered yet is some sort of optional hint to > say "I'm done" in terms of accessing localStorage. Maybe call it > localStorage.checkpoint() or localStroage.commit()? > > As far as the browser implemenation is concerned, a call to this function > would be the same as the script ending. The great thing about this is that > if a developer found one problem location in their code, they could add > it--but it'd be completely optional. > > This could be used in conjunction with most of the other ideas already > floating around. > If we're stuck with the "implicit" lock (somewhat insidious in my "radical" view:), agree an explicit means of releasing that lock would be nice .Unlock() maybe? > > J > > > On Sun, Mar 22, 2009 at 4:24 PM, Drew Wilson wrote: > >> If you deny workers, you can enforce exclusive access to localStorage by >> applying a lock that extends from the first access of localStorage until the >> script re-enters the event loop. Page script is guaranteed to re-enter the >> event loop fairly quickly (lest it trigger the browser's "this script is >> taking too long to run" protection) so you won't get starvation. Since >> worker script never has to re-enter the event loop, this isn't a feasible >> solution for workers. >> >> That's why I'm proposing that the most reasonable implementation is just >> to have a simple lock like I describe above, and then either deny access to >> localStorage to dedicated workers (shared workers can silo the storage as I >> described previously), or else just enforce a limit to how long workers can >> hold the localStorage lock (if they hold it beyond some period, they get >> terminated just like page script that doesn't re-enter the event loop). >> >> -atw >> >> On Sun, Mar 22, 2009 at 12:07 PM, Michael Nordman wrote: >> >>> >>> I don't see how denying workers solves the problem. In a multi-threaded >>> browser, this has to be resolved reasonably even in the absence of workers. >>> >> >
Re: [whatwg] localStorage + worker processes
One thing that hasn't been considered yet is some sort of optional hint to say "I'm done" in terms of accessing localStorage. Maybe call it localStorage.checkpoint() or localStroage.commit()? As far as the browser implemenation is concerned, a call to this function would be the same as the script ending. The great thing about this is that if a developer found one problem location in their code, they could add it--but it'd be completely optional. This could be used in conjunction with most of the other ideas already floating around. J On Sun, Mar 22, 2009 at 4:24 PM, Drew Wilson wrote: > If you deny workers, you can enforce exclusive access to localStorage by > applying a lock that extends from the first access of localStorage until the > script re-enters the event loop. Page script is guaranteed to re-enter the > event loop fairly quickly (lest it trigger the browser's "this script is > taking too long to run" protection) so you won't get starvation. Since > worker script never has to re-enter the event loop, this isn't a feasible > solution for workers. > > That's why I'm proposing that the most reasonable implementation is just to > have a simple lock like I describe above, and then either deny access to > localStorage to dedicated workers (shared workers can silo the storage as I > described previously), or else just enforce a limit to how long workers can > hold the localStorage lock (if they hold it beyond some period, they get > terminated just like page script that doesn't re-enter the event loop). > > -atw > > On Sun, Mar 22, 2009 at 12:07 PM, Michael Nordman wrote: > >> >> I don't see how denying workers solves the problem. In a multi-threaded >> browser, this has to be resolved reasonably even in the absence of workers. >> >
Re: [whatwg] localStorage + worker processes
On Mon, Mar 23, 2009 at 8:07 AM, Michael Nordman wrote: > I don't see how denying workers solves the problem. In a multi-threaded > browser, this has to be resolved reasonably even in the absence of workers. > It has to be resolved in a way that doesn't expose asynchronous cookie or localStorage changes to Web developers. There is abundant evidence that race conditions and synchronization are too hard for developers to deal with. The spec should forbid asynchronously visible changes to cookies or localStorage. In fact, it should probably simply say that all script execution is serializable: always equivalent to some execution you could get with a single-threaded browser that runs all scripts to completion. Allowance could be made for explicit yield points if we need to, e.g. alert(). If IE7+ allows asynchronous cookie changes, that is a bad bug. Perhaps today they get away with it because people aren't writing long-running scripts that use cookies as read/write storage. We shouldn't take that as a sign that the shared-state threaded programming model somehow works better on the Web than it does everywhere else. Some sort of implicit locking with guaranteed deadlock freedom should be workable for parallel browser implementations. For example, partition browser contexts into "related" subsets, where context A is related to context B if a script running in context A can affect the execution of an already-running script in context B. Use one lock per subset, and have a script execution acquire the lock when it first touches localStorage or cookies, and drop the lock when it completes (or yields). Additional optimizations are possible. An asynchronous API like the multi-resource-acquisition callback Jonas just proposed, would be helpful to enable additional parallelism. By declaring up-front which resources need to be locked and guaranteeing that only those resources will be touched, you can use a finer-grained locking scheme without risking deadlock. Rob -- "He was pierced for our transgressions, he was crushed for our iniquities; the punishment that brought us peace was upon him, and by his wounds we are healed. We all, like sheep, have gone astray, each of us has turned to his own way; and the LORD has laid on him the iniquity of us all." [Isaiah 53:5-6]
Re: [whatwg] localStorage + worker processes
If you deny workers, you can enforce exclusive access to localStorage by applying a lock that extends from the first access of localStorage until the script re-enters the event loop. Page script is guaranteed to re-enter the event loop fairly quickly (lest it trigger the browser's "this script is taking too long to run" protection) so you won't get starvation. Since worker script never has to re-enter the event loop, this isn't a feasible solution for workers. That's why I'm proposing that the most reasonable implementation is just to have a simple lock like I describe above, and then either deny access to localStorage to dedicated workers (shared workers can silo the storage as I described previously), or else just enforce a limit to how long workers can hold the localStorage lock (if they hold it beyond some period, they get terminated just like page script that doesn't re-enter the event loop). -atw On Sun, Mar 22, 2009 at 12:07 PM, Michael Nordman wrote: > > I don't see how denying workers solves the problem. In a multi-threaded > browser, this has to be resolved reasonably even in the absence of workers. > > >
Re: [whatwg] localStorage + worker processes
On Sun, Mar 22, 2009 at 11:53 AM, Aaron Boodman wrote: > On Sun, Mar 22, 2009 at 11:21 AM, Drew Wilson wrote: > > I've thought about this more, and I'm afraid that if you start making the > > API cumbersome (forcing only async access) then apps are just going to > use > > document.cookies instead of localStorage. I'd hate to see us radically > > change the API to support the worker case - I'd rather get rid of > > localStorage support from workers, or else just enforce a max time that a > > worker can hold the lock. > > I don't believe that. Adding one async callback is no inconvenience > compared to the sad farce that is the document.cookie "API". Also, > localstorage has many benefits including structured storage and not > getting sent to the server in every request. I don't see how denying workers solves the problem. In a multi-threaded browser, this has to be resolved reasonably even in the absence of workers.
Re: [whatwg] localStorage + worker processes
On Sun, Mar 22, 2009 at 11:21 AM, Drew Wilson wrote: > I've thought about this more, and I'm afraid that if you start making the > API cumbersome (forcing only async access) then apps are just going to use > document.cookies instead of localStorage. I'd hate to see us radically > change the API to support the worker case - I'd rather get rid of > localStorage support from workers, or else just enforce a max time that a > worker can hold the lock. I don't believe that. Adding one async callback is no inconvenience compared to the sad farce that is the document.cookie "API". Also, localstorage has many benefits including structured storage and not getting sent to the server in every request. - a
Re: [whatwg] localStorage + worker processes
On Sun, Mar 22, 2009 at 11:21 AM, Drew Wilson wrote:
> The problem is that .length is basically useless without some kind of
> immutability guarantees.
Understood, and when you need that guarantee it would be available via the
getLocalStorage(callback) API.
>
> I've thought about this more, and I'm afraid that if you start making the
> API cumbersome (forcing only async access) then apps are just going to use
> document.cookies instead of localStorage. I'd hate to see us radically
> change the API to support the worker case - I'd rather get rid of
> localStorage support from workers, or else just enforce a max time that a
> worker can hold the lock.
>
> -atw
>
>
> On Sun, Mar 22, 2009 at 10:46 AM, Michael Nordman wrote:
>
>>
>>
>> On Sat, Mar 21, 2009 at 3:25 PM, Aaron Boodman wrote:
>>
>>> On Sat, Mar 21, 2009 at 1:51 PM, Jonas Sicking wrote:
>>> > The problem with synchronously grabbing the lock is that we can only
>>> > ever have one feature that uses synchronous locks, otherwise we'll
>>> > risk dead-locks.
>>> >
>>> > Say that we make document.cookie behave the same way (to prevent
>>> > multi-process browsers like IE8 and chrome from having race
>>> > conditions). So that if you call document.getCookiesWithLock(callback)
>>> > we'll synchronously grab a lock and call the callback function. This
>>> > would cause two pages like the ones below to potentially deadlock:
>>> >
>>> > Page 1:
>>> > getLocalStorage(function(storage) {
>>> > document.getCookiesWithLock(function(cookieContainer) {
>>> >storage.foo = cookieContainer.getCookie('cookieName');
>>> > });
>>> > ]);
>>> >
>>> > Page 2:
>>> > document.getCookiesWithLock(function(cookieContainer) {
>>> > getLocalStorage(function(storage) {
>>> >cookieContainer.setCookie('cookieName', storage.bar);
>>> > });
>>> > });
>>>
>>> Good point. Ok, I agree that an asynchronous callback makes most sense
>>> for this API.
>>>
>>
>> Given an async api, would it be possible to store values into localStorage
>> at onunload time? I expect that could be a useful time to use this API.
>>
>> function onunload() {
>> getLocalStorage(function(storage) {
>> // Will this ever execute?
>> });
>> }
>>
>> Locking the storage until script completion isn't really necessary in many
>> cases. Maybe we're over engineering this? Suppose immutability across calls
>> was generally not guaranteed by the existing API. And we add an async
>> getLocalStorage(callback) which does provide immutability for the duration
>> of the callback if that is desired.
>>
>>
>
Re: [whatwg] localStorage + worker processes
The problem is that .length is basically useless without some kind of
immutability guarantees.
I've thought about this more, and I'm afraid that if you start making the
API cumbersome (forcing only async access) then apps are just going to use
document.cookies instead of localStorage. I'd hate to see us radically
change the API to support the worker case - I'd rather get rid of
localStorage support from workers, or else just enforce a max time that a
worker can hold the lock.
-atw
On Sun, Mar 22, 2009 at 10:46 AM, Michael Nordman wrote:
>
>
> On Sat, Mar 21, 2009 at 3:25 PM, Aaron Boodman wrote:
>
>> On Sat, Mar 21, 2009 at 1:51 PM, Jonas Sicking wrote:
>> > The problem with synchronously grabbing the lock is that we can only
>> > ever have one feature that uses synchronous locks, otherwise we'll
>> > risk dead-locks.
>> >
>> > Say that we make document.cookie behave the same way (to prevent
>> > multi-process browsers like IE8 and chrome from having race
>> > conditions). So that if you call document.getCookiesWithLock(callback)
>> > we'll synchronously grab a lock and call the callback function. This
>> > would cause two pages like the ones below to potentially deadlock:
>> >
>> > Page 1:
>> > getLocalStorage(function(storage) {
>> > document.getCookiesWithLock(function(cookieContainer) {
>> >storage.foo = cookieContainer.getCookie('cookieName');
>> > });
>> > ]);
>> >
>> > Page 2:
>> > document.getCookiesWithLock(function(cookieContainer) {
>> > getLocalStorage(function(storage) {
>> >cookieContainer.setCookie('cookieName', storage.bar);
>> > });
>> > });
>>
>> Good point. Ok, I agree that an asynchronous callback makes most sense
>> for this API.
>>
>
> Given an async api, would it be possible to store values into localStorage
> at onunload time? I expect that could be a useful time to use this API.
>
> function onunload() {
> getLocalStorage(function(storage) {
> // Will this ever execute?
> });
> }
>
> Locking the storage until script completion isn't really necessary in many
> cases. Maybe we're over engineering this? Suppose immutability across calls
> was generally not guaranteed by the existing API. And we add an async
> getLocalStorage(callback) which does provide immutability for the duration
> of the callback if that is desired.
>
>
Re: [whatwg] localStorage + worker processes
On Sat, Mar 21, 2009 at 3:25 PM, Aaron Boodman wrote:
> On Sat, Mar 21, 2009 at 1:51 PM, Jonas Sicking wrote:
> > The problem with synchronously grabbing the lock is that we can only
> > ever have one feature that uses synchronous locks, otherwise we'll
> > risk dead-locks.
> >
> > Say that we make document.cookie behave the same way (to prevent
> > multi-process browsers like IE8 and chrome from having race
> > conditions). So that if you call document.getCookiesWithLock(callback)
> > we'll synchronously grab a lock and call the callback function. This
> > would cause two pages like the ones below to potentially deadlock:
> >
> > Page 1:
> > getLocalStorage(function(storage) {
> > document.getCookiesWithLock(function(cookieContainer) {
> >storage.foo = cookieContainer.getCookie('cookieName');
> > });
> > ]);
> >
> > Page 2:
> > document.getCookiesWithLock(function(cookieContainer) {
> > getLocalStorage(function(storage) {
> >cookieContainer.setCookie('cookieName', storage.bar);
> > });
> > });
>
> Good point. Ok, I agree that an asynchronous callback makes most sense
> for this API.
>
Given an async api, would it be possible to store values into localStorage
at onunload time? I expect that could be a useful time to use this API.
function onunload() {
getLocalStorage(function(storage) {
// Will this ever execute?
});
}
Locking the storage until script completion isn't really necessary in many
cases. Maybe we're over engineering this? Suppose immutability across calls
was generally not guaranteed by the existing API. And we add an async
getLocalStorage(callback) which does provide immutability for the duration
of the callback if that is desired.
Re: [whatwg] localStorage + worker processes
On Sat, Mar 21, 2009 at 1:51 PM, Jonas Sicking wrote:
> The problem with synchronously grabbing the lock is that we can only
> ever have one feature that uses synchronous locks, otherwise we'll
> risk dead-locks.
>
> Say that we make document.cookie behave the same way (to prevent
> multi-process browsers like IE8 and chrome from having race
> conditions). So that if you call document.getCookiesWithLock(callback)
> we'll synchronously grab a lock and call the callback function. This
> would cause two pages like the ones below to potentially deadlock:
>
> Page 1:
> getLocalStorage(function(storage) {
> document.getCookiesWithLock(function(cookieContainer) {
> storage.foo = cookieContainer.getCookie('cookieName');
> });
> ]);
>
> Page 2:
> document.getCookiesWithLock(function(cookieContainer) {
> getLocalStorage(function(storage) {
> cookieContainer.setCookie('cookieName', storage.bar);
> });
> });
Good point. Ok, I agree that an asynchronous callback makes most sense
for this API.
- a
Re: [whatwg] localStorage + worker processes
As a side note, if we do go with this async lock acquiring, we could
add an API like:
getLockedFeatures(callback, 'localStore', 'cookie');
This would be an asynchronously grab locks to multiple features and
only call the callback once all of them have been acquired. This would
allow computations across data from multiple locations guaranteed to
be in sync. The implementation would be responsible for grabbing the
locks in a consistent order to prevent deadlocks.
/ Jonas
On Sat, Mar 21, 2009 at 12:48 AM, Jonas Sicking wrote:
> On Fri, Mar 20, 2009 at 3:10 PM, Aaron Boodman wrote:
>> I think the best option is to make access to localstorage asynchronous
>> for workers. This reduces the amount of time a worker can hold the
>> localstore lock so that it shouldn't be a problem for normal pages. It
>> sucks to make such a simple and useful API aync though.
>
> As I understand the current API (on main window) to be defined, as
> soon as someone accesses the .localStorage property, the
> implementation is supposed to acquire a lock. This lock would be held
> on to until that script returns to the event loop for that thread.
>
> So if javascript in another window, running in another thread or
> process, tries to access .localStorage for the same origin, the
> .localStorage getter would try to acquire the same lock and block
> until the first thread releases the lock.
>
> This could in theory be applied to applied to workers as well. However
> as Jeremy points out that could result in the a worker script running
> for a very long time blocking the window thread.
>
> What we could do, is to have an API like
>
> getLocalStorage(callback);
>
> This function returns immediately, but will then call the callback
> function as soon as the localStorage becomes available and the lock
> been acquired. This would always happen asynchronously off the event
> loop, which means that once the callback returns the lock is released
> again.
>
> Of course, it would still mean that a window context or worker could
> hold on to the lock for an indefinite time, but as long as the asych
> getLocalStorage API is used, this means that no thread is blocked,
> just that they aren't able to get access to the localStorage.
>
> So for example, the following code would safely add 1 to the 'foo'
> property in localStorage:
>
> getLocalStorage(function(store) {
> store.foo = parseInt(store.foo, 10) + 1;
> });
>
> Additionally, we would have to define that if the store object passed
> to the callback function is accessed outside after the callback has
> ended this will throw an exception. If the object is reactivated next
> time a callback is entered, or if a new storage object is created also
> needs to be defined.
>
> This new API I believe is good enough to be used both from workers and
> window contexts.
>
> We could even keep the current API implemented in IE8, or we could
> just ask people to write a wrapper for IE8 like:
>
> function getLocalStorage(callback) {
> setTimeout(function() { callback(localStorage); }, 0);
> }
>
> in an implementation that implements correct locking for the
> synchronous API, this will even produce the correct locking behavior
> for the new API.
>
> / Jonas
>
Re: [whatwg] localStorage + worker processes
On Sat, Mar 21, 2009 at 9:41 AM, Drew Wilson wrote: > That might work. Is it feasible for user agents to enforce limits on how > long a callback is allowed to run holding the lock? That way workers can't > starve normal pages from accessing their local storage. I don't think it will be a big problem. As long as we ensure that all locks are per-origin, that means that an application can only starve itself. Something that it has clear incentives not to. / Jonas
Re: [whatwg] localStorage + worker processes
On Sat, Mar 21, 2009 at 9:38 AM, Aaron Boodman wrote:
> On Sat, Mar 21, 2009 at 12:48 AM, Jonas Sicking wrote:
>> What we could do, is to have an API like
>>
>> getLocalStorage(callback);
>>
>> This function returns immediately, but will then call the callback
>> function as soon as the localStorage becomes available and the lock
>> been acquired. This would always happen asynchronously off the event
>> loop, which means that once the callback returns the lock is released
>> again.
>
> Funny, a few of us from Chromium were discussing a similar solution privately.
>
> Actually, I don't believe that it is required that the callback run
> asynchronously. All the callback is used for is establishing the lock
> lifetime explicitly, and we assume that this will usually make the
> lock lifetime short. So we can block while we wait for it to become
> available. This is just like the behavior today without workers.
The problem with synchronously grabbing the lock is that we can only
ever have one feature that uses synchronous locks, otherwise we'll
risk dead-locks.
Say that we make document.cookie behave the same way (to prevent
multi-process browsers like IE8 and chrome from having race
conditions). So that if you call document.getCookiesWithLock(callback)
we'll synchronously grab a lock and call the callback function. This
would cause two pages like the ones below to potentially deadlock:
Page 1:
getLocalStorage(function(storage) {
document.getCookiesWithLock(function(cookieContainer) {
storage.foo = cookieContainer.getCookie('cookieName');
});
]);
Page 2:
document.getCookiesWithLock(function(cookieContainer) {
getLocalStorage(function(storage) {
cookieContainer.setCookie('cookieName', storage.bar);
});
});
The point here isn't to propose a new cookie API, but rather that as
soon as we introduce more than one lock that can be synchronously
acquired, we run the risk of deadlocks if people don't grab them in
the right order.
While we could make the implementation detect this and case the second
page to throw when a deadlock is detected, that just turns the
deadlock into a race condition since pages are unlikely to handle this
exception correctly given that it generally doesn't happen.
By making all lock acquiring async, we ensure that script can only
ever grab one lock at a time, thus preventing people from grabbing
multiple locks in different order and causing deadlocks.
There is actually another alternative. We could specify an order that
locks have to be acquired if you grab more than one. So for example we
could say that the order is
localStore > cookie > otherRandomFeature
In other words, if you want both the localStore and cookie features,
you have to grab first localStore and then cookie. Attempting to grab
localStore once you have acquired the cookie lock *always* results in
an exception being thrown, even if there currently is no deadlock.
Anyone specifying a new feature that uses locking would have to define
where in the locking order this lock is placed. However this API
design does not seem very web developer friendly.
/ Jonas
Re: [whatwg] localStorage + worker processes
On Sat, Mar 21, 2009 at 9:41 AM, Drew Wilson wrote: > That might work. Is it feasible for user agents to enforce limits on how > long a callback is allowed to run holding the lock? That way workers can't > starve normal pages from accessing their local storage. It seems like they could use the same time limit that is used for detecting/stopping runaway scripts in web pages. - a
Re: [whatwg] localStorage + worker processes
That might work. Is it feasible for user agents to enforce limits on how
long a callback is allowed to run holding the lock? That way workers can't
starve normal pages from accessing their local storage.
-atw
On Sat, Mar 21, 2009 at 12:48 AM, Jonas Sicking wrote:
> On Fri, Mar 20, 2009 at 3:10 PM, Aaron Boodman wrote:
> > I think the best option is to make access to localstorage asynchronous
> > for workers. This reduces the amount of time a worker can hold the
> > localstore lock so that it shouldn't be a problem for normal pages. It
> > sucks to make such a simple and useful API aync though.
>
> As I understand the current API (on main window) to be defined, as
> soon as someone accesses the .localStorage property, the
> implementation is supposed to acquire a lock. This lock would be held
> on to until that script returns to the event loop for that thread.
>
> So if javascript in another window, running in another thread or
> process, tries to access .localStorage for the same origin, the
> .localStorage getter would try to acquire the same lock and block
> until the first thread releases the lock.
>
> This could in theory be applied to applied to workers as well. However
> as Jeremy points out that could result in the a worker script running
> for a very long time blocking the window thread.
>
> What we could do, is to have an API like
>
> getLocalStorage(callback);
>
> This function returns immediately, but will then call the callback
> function as soon as the localStorage becomes available and the lock
> been acquired. This would always happen asynchronously off the event
> loop, which means that once the callback returns the lock is released
> again.
>
> Of course, it would still mean that a window context or worker could
> hold on to the lock for an indefinite time, but as long as the asych
> getLocalStorage API is used, this means that no thread is blocked,
> just that they aren't able to get access to the localStorage.
>
> So for example, the following code would safely add 1 to the 'foo'
> property in localStorage:
>
> getLocalStorage(function(store) {
> store.foo = parseInt(store.foo, 10) + 1;
> });
>
> Additionally, we would have to define that if the store object passed
> to the callback function is accessed outside after the callback has
> ended this will throw an exception. If the object is reactivated next
> time a callback is entered, or if a new storage object is created also
> needs to be defined.
>
> This new API I believe is good enough to be used both from workers and
> window contexts.
>
> We could even keep the current API implemented in IE8, or we could
> just ask people to write a wrapper for IE8 like:
>
> function getLocalStorage(callback) {
> setTimeout(function() { callback(localStorage); }, 0);
> }
>
> in an implementation that implements correct locking for the
> synchronous API, this will even produce the correct locking behavior
> for the new API.
>
> / Jonas
>
Re: [whatwg] localStorage + worker processes
On Sat, Mar 21, 2009 at 12:48 AM, Jonas Sicking wrote: > What we could do, is to have an API like > > getLocalStorage(callback); > > This function returns immediately, but will then call the callback > function as soon as the localStorage becomes available and the lock > been acquired. This would always happen asynchronously off the event > loop, which means that once the callback returns the lock is released > again. Funny, a few of us from Chromium were discussing a similar solution privately. Actually, I don't believe that it is required that the callback run asynchronously. All the callback is used for is establishing the lock lifetime explicitly, and we assume that this will usually make the lock lifetime short. So we can block while we wait for it to become available. This is just like the behavior today without workers. > This new API I believe is good enough to be used both from workers and > window contexts. We could maintain backward compatibility by not making it required on normal web pages, because our assumption has been that web pages won't hold the lock for long periods of time (because they wouldn't want to block the UI either). - a
Re: [whatwg] localStorage + worker processes
On Fri, Mar 20, 2009 at 3:10 PM, Aaron Boodman wrote:
> I think the best option is to make access to localstorage asynchronous
> for workers. This reduces the amount of time a worker can hold the
> localstore lock so that it shouldn't be a problem for normal pages. It
> sucks to make such a simple and useful API aync though.
As I understand the current API (on main window) to be defined, as
soon as someone accesses the .localStorage property, the
implementation is supposed to acquire a lock. This lock would be held
on to until that script returns to the event loop for that thread.
So if javascript in another window, running in another thread or
process, tries to access .localStorage for the same origin, the
.localStorage getter would try to acquire the same lock and block
until the first thread releases the lock.
This could in theory be applied to applied to workers as well. However
as Jeremy points out that could result in the a worker script running
for a very long time blocking the window thread.
What we could do, is to have an API like
getLocalStorage(callback);
This function returns immediately, but will then call the callback
function as soon as the localStorage becomes available and the lock
been acquired. This would always happen asynchronously off the event
loop, which means that once the callback returns the lock is released
again.
Of course, it would still mean that a window context or worker could
hold on to the lock for an indefinite time, but as long as the asych
getLocalStorage API is used, this means that no thread is blocked,
just that they aren't able to get access to the localStorage.
So for example, the following code would safely add 1 to the 'foo'
property in localStorage:
getLocalStorage(function(store) {
store.foo = parseInt(store.foo, 10) + 1;
});
Additionally, we would have to define that if the store object passed
to the callback function is accessed outside after the callback has
ended this will throw an exception. If the object is reactivated next
time a callback is entered, or if a new storage object is created also
needs to be defined.
This new API I believe is good enough to be used both from workers and
window contexts.
We could even keep the current API implemented in IE8, or we could
just ask people to write a wrapper for IE8 like:
function getLocalStorage(callback) {
setTimeout(function() { callback(localStorage); }, 0);
}
in an implementation that implements correct locking for the
synchronous API, this will even produce the correct locking behavior
for the new API.
/ Jonas
Re: [whatwg] localStorage + worker processes
On Fri, Mar 20, 2009 at 3:34 PM, Dmitry Titov wrote: > On Fri, Mar 20, 2009 at 2:39 PM, Drew Wilson wrote: >> >> One alternative I'd like to propose is to remove access to localStorage >> for dedicated workers, and give SharedWorkers access to localStorage, but >> have that storage be partitioned by the worker name (i.e. the worker can >> access it, but it's not shared with web pages or any other workers and so >> you don't have any synchronicity issues). > > This may be interesting. It allows to keep single API for localStorage. Most > of the usage is probably going to happen inside such a worker - in other > words, sharing the info with other pages of the same origin may not be > needed, and there is always async communication when a live page wants to > share data with a shared worker. Interesting... essentially a page/worker specific persistent repository... doesn't span in the way "localStorage" does.
Re: [whatwg] localStorage + worker processes
On Fri, Mar 20, 2009 at 2:39 PM, Drew Wilson wrote: > One alternative I'd like to propose is to remove access to localStorage for > dedicated workers, and give SharedWorkers access to localStorage, but have > that storage be partitioned by the worker name (i.e. the worker can access > it, but it's not shared with web pages or any other workers and so you don't > have any synchronicity issues). > This may be interesting. It allows to keep single API for localStorage. Most of the usage is probably going to happen inside such a worker - in other words, sharing the info with other pages of the same origin may not be needed, and there is always async communication when a live page wants to share data with a shared worker. Dmitry
Re: [whatwg] localStorage + worker processes
I think the best option is to make access to localstorage asynchronous for workers. This reduces the amount of time a worker can hold the localstore lock so that it shouldn't be a problem for normal pages. It sucks to make such a simple and useful API aync though. - a
Re: [whatwg] localStorage + worker processes
On Fri, Mar 20, 2009 at 2:19 PM, Oliver Hunt wrote: >> When discussing this standard we have to recognize that not all >> browsers actually have a "main thread". > > Yes they do -- we're talking about the main thread from the point of view of > javascript, which is not necessarily the UI thread. The spec requires a single thread only for each unit of related browsing contexts. LocalStorage extends to all browsing contexts, regardless of which unit they are in. Adam
Re: [whatwg] localStorage + worker processes
I agree with Jeremy that the spec is currently unimplementable if we give localStorage access to workers. I'd like to point out that "workers who want to access localStorage just need to send a message to their main window" breaks down for persistent workers (where you don't necessarily have an open browser window) and is pretty wonky for shared workers (you can send a message to a window, but that window may go away before your message is processed, so you end up having to build some kind of "send message, timeout, pick a new window to send it to, etc" message layer). One alternative I'd like to propose is to remove access to localStorage for dedicated workers, and give SharedWorkers access to localStorage, but have that storage be partitioned by the worker name (i.e. the worker can access it, but it's not shared with web pages or any other workers and so you don't have any synchronicity issues). I don't see how this would work for dedicated workers, though, since there's no name to partition storage access, but they could always fall back to postMessage(). -atw On Fri, Mar 20, 2009 at 2:19 PM, Oliver Hunt wrote: > When discussing this standard we have to recognize that not all >> browsers actually have a "main thread". Time will tell if more or less >> browsers of the future will have multi-threaded architectures, but the >> trend has been for more I think. >> >> Any aspects of the spec that asserts or assumes a "main thread" is >> questionable. >> > > Yes they do -- we're talking about the main thread from the point of view > of javascript, which is not necessarily the UI thread. > > The important thing is that with the current model is that JS on any thread > may be blocked by js executing in a worker, which leads to a page (in > effect) locking up -- the UI may still be functional but that particular > page will have hung. > > --Oliver >
Re: [whatwg] localStorage + worker processes
When discussing this standard we have to recognize that not all browsers actually have a "main thread". Time will tell if more or less browsers of the future will have multi-threaded architectures, but the trend has been for more I think. Any aspects of the spec that asserts or assumes a "main thread" is questionable. Yes they do -- we're talking about the main thread from the point of view of javascript, which is not necessarily the UI thread. The important thing is that with the current model is that JS on any thread may be blocked by js executing in a worker, which leads to a page (in effect) locking up -- the UI may still be functional but that particular page will have hung. --Oliver
Re: [whatwg] localStorage + worker processes
On Fri, Mar 20, 2009 at 1:47 PM, Jonas Sicking wrote: > On Fri, Mar 20, 2009 at 1:23 PM, Jeremy Orlow wrote: >> What is the need for localStorage access within workers? Technically if >> someone really needed to access it, they could always have a function in the >> web page for accessing it and then use postMessage. In other words, they >> could build their own ad-hoc async API pretty easily. Another alternative >> is to just build an async API into the spec (and remove synchronous access >> to localStorage). >> Thoughts? > > I do think it would be great if workers had access to some type of > structured storage. However I agree that the fact that both the main > thread and workers have synchronous access to the same storage is not > acceptable since that means that we're violating the > shared-nothing-message-passing design that makes workers not have to > deal with locks and other traditional multithread hazards. > > / Jonas When discussing this standard we have to recognize that not all browsers actually have a "main thread". Time will tell if more or less browsers of the future will have multi-threaded architectures, but the trend has been for more I think. Any aspects of the spec that asserts or assumes a "main thread" is questionable.
Re: [whatwg] localStorage + worker processes
On Fri, Mar 20, 2009 at 1:23 PM, Jeremy Orlow wrote: > What is the need for localStorage access within workers? Technically if > someone really needed to access it, they could always have a function in the > web page for accessing it and then use postMessage. In other words, they > could build their own ad-hoc async API pretty easily. Another alternative > is to just build an async API into the spec (and remove synchronous access > to localStorage). > Thoughts? I do think it would be great if workers had access to some type of structured storage. However I agree that the fact that both the main thread and workers have synchronous access to the same storage is not acceptable since that means that we're violating the shared-nothing-message-passing design that makes workers not have to deal with locks and other traditional multithread hazards. / Jonas
