Re: [whatwg] localStorage + worker processes
On Sat, Mar 28, 2009 at 11:29 AM, Michael Nordman micha...@google.comwrote: On Tue, Mar 24, 2009 at 2:11 AM, Ian Hickson i...@hixie.ch wrote: On Fri, 20 Mar 2009, Jonas Sicking wrote: I do think it would be great if workers had access to some type of structured storage. However I agree that the fact that both the main thread and workers have synchronous access to the same storage is not acceptable since that means that we're violating the shared-nothing-message-passing design that makes workers not have to deal with locks and other traditional multithread hazards. Agreed. The Database API seems well-suited for this, though. Again... its not just workers that are affected by this... speaking as someone that works on a multi-threaded browser, this is troubling. If its possible to spec features that allow script to poke at the world beyond the page boundaries in a fashion that doesn't not require locking semantics beyond (I assume that not was unintended.) the scope of a single scriptable API call... that would be less troubling. SQL storage where the single API call is run this complex transaction might be acceptable. But if the single API call is access some shared state and it's up to Web developers to deal with locking and synchronization --- that will never be acceptable. That burden must fall on browser developers, not Web developers. Rob -- He was pierced for our transgressions, he was crushed for our iniquities; the punishment that brought us peace was upon him, and by his wounds we are healed. We all, like sheep, have gone astray, each of us has turned to his own way; and the LORD has laid on him the iniquity of us all. [Isaiah 53:5-6]
Re: [whatwg] localStorage + worker processes
On Tue, Mar 24, 2009 at 2:11 AM, Ian Hickson i...@hixie.ch wrote: I've updated the specs as follows: - removed localStorage from Web Workers for now. - extended the implicit lock mechanism that we had for storage to also cover document.cookie, and made the language more explicit about how it works. - added navigator.releaseLock(). On Fri, 20 Mar 2009, Jeremy Orlow wrote: Anyhow, the very first example in the spec ( http://dev.w3.org/html5/workers/#a-background-number-crunching-worker) shows work that's being done in a infinite loop with postMessage being called when each prime is found. If you called localStorage anywhere within that loop (say, to periodically save all primes found), you would not be able to safely call window.localStorage in any other worker or the web page. This is because the task that started the script never ends. And this its 'lock' (on other scripts using local storage) will never be released. I've removed localStorage from the Web Workers spec for now. On Fri, 20 Mar 2009, Jonas Sicking wrote: I do think it would be great if workers had access to some type of structured storage. However I agree that the fact that both the main thread and workers have synchronous access to the same storage is not acceptable since that means that we're violating the shared-nothing-message-passing design that makes workers not have to deal with locks and other traditional multithread hazards. Agreed. The Database API seems well-suited for this, though. Again... its not just workers that are affected by this... speaking as someone that works on a multi-threaded browser, this is troubling. If its possible to spec features that allow script to poke at the world beyond the page boundaries in a fashion that doesn't not require locking semantics beyond the scope of a single scriptable API call... that would be less troubling. On Fri, 20 Mar 2009, Drew Wilson wrote: One alternative I'd like to propose is to remove access to localStorage for dedicated workers, and give SharedWorkers access to localStorage, but have that storage be partitioned by the worker name (i.e. the worker can access it, but it's not shared with web pages or any other workers and so you don't have any synchronicity issues). That's an interesting idea, and would be relatively easy to do. Do people think it is worth it? I think there's some additional low-hanging fruit too. We're toying with two, independent axis: lifetime vs accessScope. 'sessionStorage' has sessionOnlyLifetime and tabSpecificScope 'localStorage' has persistentLifetime and browserWideScope In this nomenclature, the new idea could be phrased as... 'page/workerStorage' has persistentLifetime and page/workerSpecificScope Other slots in the matrix formed by these two axis could make sense... sessionLifetime + page/workerSpecificScope sessionLifetime + browserWideScope sessionLifetime + tabSpecificScope doesn't make much sense since you get a new set of tabs when starting a new session On Fri, 20 Mar 2009, Aaron Boodman wrote: I think the best option is to make access to localstorage asynchronous for workers. This reduces the amount of time a worker can hold the localstore lock so that it shouldn't be a problem for normal pages. It sucks to make such a simple and useful API aync though. I don't think making it async helps here, since the problem isn't that it is synchronous, but that workers don't return quickly (by design). On Sat, 21 Mar 2009, Aaron Boodman wrote: Actually, I don't believe that it is required that the callback run asynchronously. All the callback is used for is establishing the lock lifetime explicitly, and we assume that this will usually make the lock lifetime short. So we can block while we wait for it to become available. This is just like the behavior today without workers. Nothing is to stop someone from just having a long callback, though. On Sat, 21 Mar 2009, Jonas Sicking wrote: As I understand the current API (on main window) to be defined, as soon as someone accesses the .localStorage property, the implementation is supposed to acquire a lock. This lock would be held on to until that script returns to the event loop for that thread. So if javascript in another window, running in another thread or process, tries to access .localStorage for the same origin, the .localStorage getter would try to acquire the same lock and block until the first thread releases the lock. Right. On Sat, 21 Mar 2009, Jonas Sicking wrote: The problem with synchronously grabbing the lock is that we can only ever have one feature that uses synchronous locks, otherwise we'll risk dead-locks. Indeed. This is a problem with the current API for localStorage in windows as well. I've made the spec explicitly have a single shared lock for all features that need locking (currently just .cookie and
Re: [whatwg] localStorage + worker processes
sessionLifetime + tabSpecificScope doesn't make much sense since you get a new set of tabs when starting a new session Sorry... make that persistentLifetime + tabScope doesn't make sense. On Fri, Mar 27, 2009 at 3:29 PM, Michael Nordman micha...@google.comwrote: On Tue, Mar 24, 2009 at 2:11 AM, Ian Hickson i...@hixie.ch wrote: I've updated the specs as follows: - removed localStorage from Web Workers for now. - extended the implicit lock mechanism that we had for storage to also cover document.cookie, and made the language more explicit about how it works. - added navigator.releaseLock(). On Fri, 20 Mar 2009, Jeremy Orlow wrote: Anyhow, the very first example in the spec ( http://dev.w3.org/html5/workers/#a-background-number-crunching-worker) shows work that's being done in a infinite loop with postMessage being called when each prime is found. If you called localStorage anywhere within that loop (say, to periodically save all primes found), you would not be able to safely call window.localStorage in any other worker or the web page. This is because the task that started the script never ends. And this its 'lock' (on other scripts using local storage) will never be released. I've removed localStorage from the Web Workers spec for now. On Fri, 20 Mar 2009, Jonas Sicking wrote: I do think it would be great if workers had access to some type of structured storage. However I agree that the fact that both the main thread and workers have synchronous access to the same storage is not acceptable since that means that we're violating the shared-nothing-message-passing design that makes workers not have to deal with locks and other traditional multithread hazards. Agreed. The Database API seems well-suited for this, though. Again... its not just workers that are affected by this... speaking as someone that works on a multi-threaded browser, this is troubling. If its possible to spec features that allow script to poke at the world beyond the page boundaries in a fashion that doesn't not require locking semantics beyond the scope of a single scriptable API call... that would be less troubling. On Fri, 20 Mar 2009, Drew Wilson wrote: One alternative I'd like to propose is to remove access to localStorage for dedicated workers, and give SharedWorkers access to localStorage, but have that storage be partitioned by the worker name (i.e. the worker can access it, but it's not shared with web pages or any other workers and so you don't have any synchronicity issues). That's an interesting idea, and would be relatively easy to do. Do people think it is worth it? I think there's some additional low-hanging fruit too. We're toying with two, independent axis: lifetime vs accessScope. 'sessionStorage' has sessionOnlyLifetime and tabSpecificScope 'localStorage' has persistentLifetime and browserWideScope In this nomenclature, the new idea could be phrased as... 'page/workerStorage' has persistentLifetime and page/workerSpecificScope Other slots in the matrix formed by these two axis could make sense... sessionLifetime + page/workerSpecificScope sessionLifetime + browserWideScope sessionLifetime + tabSpecificScope doesn't make much sense since you get a new set of tabs when starting a new session On Fri, 20 Mar 2009, Aaron Boodman wrote: I think the best option is to make access to localstorage asynchronous for workers. This reduces the amount of time a worker can hold the localstore lock so that it shouldn't be a problem for normal pages. It sucks to make such a simple and useful API aync though. I don't think making it async helps here, since the problem isn't that it is synchronous, but that workers don't return quickly (by design). On Sat, 21 Mar 2009, Aaron Boodman wrote: Actually, I don't believe that it is required that the callback run asynchronously. All the callback is used for is establishing the lock lifetime explicitly, and we assume that this will usually make the lock lifetime short. So we can block while we wait for it to become available. This is just like the behavior today without workers. Nothing is to stop someone from just having a long callback, though. On Sat, 21 Mar 2009, Jonas Sicking wrote: As I understand the current API (on main window) to be defined, as soon as someone accesses the .localStorage property, the implementation is supposed to acquire a lock. This lock would be held on to until that script returns to the event loop for that thread. So if javascript in another window, running in another thread or process, tries to access .localStorage for the same origin, the .localStorage getter would try to acquire the same lock and block until the first thread releases the lock. Right. On Sat, 21 Mar 2009, Jonas Sicking wrote: The problem with synchronously grabbing the lock is that we can only ever
Re: [whatwg] localStorage + worker processes
I've updated the specs as follows:
- removed localStorage from Web Workers for now.
- extended the implicit lock mechanism that we had for storage to also
cover document.cookie, and made the language more explicit about how
it works.
- added navigator.releaseLock().
On Fri, 20 Mar 2009, Jeremy Orlow wrote:
Anyhow, the very first example in the spec (
http://dev.w3.org/html5/workers/#a-background-number-crunching-worker)
shows work that's being done in a infinite loop with postMessage being
called when each prime is found. If you called localStorage anywhere
within that loop (say, to periodically save all primes found), you would
not be able to safely call window.localStorage in any other worker or
the web page. This is because the task that started the script never
ends. And this its 'lock' (on other scripts using local storage) will
never be released.
I've removed localStorage from the Web Workers spec for now.
On Fri, 20 Mar 2009, Jonas Sicking wrote:
I do think it would be great if workers had access to some type of
structured storage. However I agree that the fact that both the main
thread and workers have synchronous access to the same storage is not
acceptable since that means that we're violating the
shared-nothing-message-passing design that makes workers not have to
deal with locks and other traditional multithread hazards.
Agreed. The Database API seems well-suited for this, though.
On Fri, 20 Mar 2009, Drew Wilson wrote:
One alternative I'd like to propose is to remove access to localStorage
for dedicated workers, and give SharedWorkers access to localStorage,
but have that storage be partitioned by the worker name (i.e. the worker
can access it, but it's not shared with web pages or any other workers
and so you don't have any synchronicity issues).
That's an interesting idea, and would be relatively easy to do. Do people
think it is worth it?
On Fri, 20 Mar 2009, Aaron Boodman wrote:
I think the best option is to make access to localstorage asynchronous
for workers. This reduces the amount of time a worker can hold the
localstore lock so that it shouldn't be a problem for normal pages. It
sucks to make such a simple and useful API aync though.
I don't think making it async helps here, since the problem isn't that it
is synchronous, but that workers don't return quickly (by design).
On Sat, 21 Mar 2009, Aaron Boodman wrote:
Actually, I don't believe that it is required that the callback run
asynchronously. All the callback is used for is establishing the lock
lifetime explicitly, and we assume that this will usually make the lock
lifetime short. So we can block while we wait for it to become
available. This is just like the behavior today without workers.
Nothing is to stop someone from just having a long callback, though.
On Sat, 21 Mar 2009, Jonas Sicking wrote:
As I understand the current API (on main window) to be defined, as soon
as someone accesses the .localStorage property, the implementation is
supposed to acquire a lock. This lock would be held on to until that
script returns to the event loop for that thread.
So if javascript in another window, running in another thread or
process, tries to access .localStorage for the same origin, the
.localStorage getter would try to acquire the same lock and block until
the first thread releases the lock.
Right.
On Sat, 21 Mar 2009, Jonas Sicking wrote:
The problem with synchronously grabbing the lock is that we can only
ever have one feature that uses synchronous locks, otherwise we'll risk
dead-locks.
Indeed. This is a problem with the current API for localStorage in windows
as well.
I've made the spec explicitly have a single shared lock for all features
that need locking (currently just .cookie and .localStorage).
On Sun, 22 Mar 2009, Michael Nordman wrote:
Given an async api, would it be possible to store values into
localStorage at onunload time? I expect that could be a useful time to
use this API.
function onunload() {
getLocalStorage(function(storage) {
// Will this ever execute?
});
}
Locking the storage until script completion isn't really necessary in
many cases. Maybe we're over engineering this? Suppose immutability
across calls was generally not guaranteed by the existing API. And we
add an async getLocalStorage(callback) which does provide immutability
for the duration of the callback if that is desired.
The problem is that people will walk into race conditions without
realising it, and they are amongst the hardest problems to debug.
On Sun, 22 Mar 2009, Drew Wilson wrote:
The problem is that .length is basically useless without some kind of
immutability guarantees.
Indeed.
On Sun, 22 Mar 2009, Drew Wilson wrote:
That's why I'm proposing that the most reasonable implementation is just
to have a simple lock like I describe above
This is what I've done.
Re: [whatwg] localStorage + worker processes
On Tue, Mar 24, 2009 at 10:11 PM, Ian Hickson i...@hixie.ch wrote: - extended the implicit lock mechanism that we had for storage to also cover document.cookie, and made the language more explicit about how it works. That's basically good. It's possible that people might want to implement something that's equivalent to the storage mutex in observable behaviour, but allows more parallelism, such as speculative execution or finer-grained locking when the implementation can prove it's safe. I assume implementors of HTML5 already understand that that's allowed. - added navigator.releaseLock(). This name could be confusing to developers, because there is no corresponding explicit acquireLock(), which there usually is in an API that exposes releaseLock(). navigator.allowInterruption() maybe? On Sat, 21 Mar 2009, Jonas Sicking wrote: As a side note, if we do go with this async lock acquiring, we could add an API like: getLockedFeatures(callback, 'localStore', 'cookie'); This would be an asynchronously grab locks to multiple features and only call the callback once all of them have been acquired. This would allow computations across data from multiple locations guaranteed to be in sync. The implementation would be responsible for grabbing the locks in a consistent order to prevent deadlocks. Why would we want more than one lock? Is the potential performance gain worth the complexity? The problem with going with an async approach is that it means changing the API, which is something we can't really do for cookie (and don't really want to do for localStorage, since IE8 has shipped it.) We we are going to need a synchronous locking mechanism anyway. It would be possible to use something like getLockedFeatures for workers while using implicit locking for the main thread. On Mon, 23 Mar 2009, Robert O'Callahan wrote: It has to be resolved in a way that doesn't expose asynchronous cookie or localStorage changes to Web developers. There is abundant evidence that race conditions and synchronization are too hard for developers to deal with. The spec should forbid asynchronously visible changes to cookies or localStorage. In fact, it should probably simply say that all script execution is serializable: always equivalent to some execution you could get with a single-threaded browser that runs all scripts to completion. Allowance could be made for explicit yield points if we need to, e.g. alert(). Generally speaking I have tried to maintain this invariant, but I have failed with cookies, and with localStorage in workers. Now, with the storage mutex, are there any cases you know of where serializability fails? If there are, it may be worth noting them in the spec. If there aren't, why not simply write serializability into the spec? User agents that share event loops between origins can't actually have any more than one lock total. Consider a case where there are three windows from three different origins, A, B, and C, where C contains a couple of iframes, and where A, B, and C are independent, but C share an event loop with whatever content is in its iframes. (This is the situation Chrome and IE are in, as I understand it, with event loops being per-window not per-origin, and it may be required because access to the frames[] hierarchy is synchronous.) Now, assume A and B have both obtained their respective locks, and are busy doing some long script. C is free to run more tasks from its event loop, which could include navigating one iframe to a page on either A and the other iframe to a page on B, meaning that the event loop of C is now beholden to two locks. If there is any manner in which to synchronously cause another origin to run script, this now means that C can attempt to obtain both locks; if we now imagine another window just like C that instead obtains the locks in the reverse order, we get deadlock. Interesting example! When two sets of unrelated browser contexts become related (e.g., C loads A into an iframe), I imagined you would join A's lock and C's lock into a single lock covering the new set of related browser contexts, which is safe to do if at most one of those locks is currently held. When this happens due to a document being created with origin A in C's iframe, it happens asynchronously in C, right? So at that point C's lock is not held by currently running script in C (although it might be held by code in another domain which is already related to C), and we can block the join operation in C until one of the two locks is released. Then in your example, suppose C loads A's document first. Then C's lock and A's lock are joined to make a CA-lock. Then suppose D (another window just like C) loads B's document; D's lock and B's lock are merged to make the DB-lock. Now suppose C loads B. The two remaining locks are merged to form a single CADB-lock. No deadlock is possible. If it can be shown that it is not ever
Re: [whatwg] localStorage + worker processes
One thing that hasn't been considered yet is some sort of optional hint to say I'm done in terms of accessing localStorage. Maybe call it localStorage.checkpoint() or localStroage.commit()? As far as the browser implemenation is concerned, a call to this function would be the same as the script ending. The great thing about this is that if a developer found one problem location in their code, they could add it--but it'd be completely optional. This could be used in conjunction with most of the other ideas already floating around. J On Sun, Mar 22, 2009 at 4:24 PM, Drew Wilson atwil...@google.com wrote: If you deny workers, you can enforce exclusive access to localStorage by applying a lock that extends from the first access of localStorage until the script re-enters the event loop. Page script is guaranteed to re-enter the event loop fairly quickly (lest it trigger the browser's this script is taking too long to run protection) so you won't get starvation. Since worker script never has to re-enter the event loop, this isn't a feasible solution for workers. That's why I'm proposing that the most reasonable implementation is just to have a simple lock like I describe above, and then either deny access to localStorage to dedicated workers (shared workers can silo the storage as I described previously), or else just enforce a limit to how long workers can hold the localStorage lock (if they hold it beyond some period, they get terminated just like page script that doesn't re-enter the event loop). -atw On Sun, Mar 22, 2009 at 12:07 PM, Michael Nordman micha...@google.comwrote: I don't see how denying workers solves the problem. In a multi-threaded browser, this has to be resolved reasonably even in the absence of workers.
Re: [whatwg] localStorage + worker processes
On Mon, Mar 23, 2009 at 11:41 AM, Jeremy Orlow jor...@google.com wrote: One thing that hasn't been considered yet is some sort of optional hint to say I'm done in terms of accessing localStorage. Maybe call it localStorage.checkpoint() or localStroage.commit()? As far as the browser implemenation is concerned, a call to this function would be the same as the script ending. The great thing about this is that if a developer found one problem location in their code, they could add it--but it'd be completely optional. This could be used in conjunction with most of the other ideas already floating around. If we're stuck with the implicit lock (somewhat insidious in my radical view:), agree an explicit means of releasing that lock would be nice .Unlock() maybe? J On Sun, Mar 22, 2009 at 4:24 PM, Drew Wilson atwil...@google.com wrote: If you deny workers, you can enforce exclusive access to localStorage by applying a lock that extends from the first access of localStorage until the script re-enters the event loop. Page script is guaranteed to re-enter the event loop fairly quickly (lest it trigger the browser's this script is taking too long to run protection) so you won't get starvation. Since worker script never has to re-enter the event loop, this isn't a feasible solution for workers. That's why I'm proposing that the most reasonable implementation is just to have a simple lock like I describe above, and then either deny access to localStorage to dedicated workers (shared workers can silo the storage as I described previously), or else just enforce a limit to how long workers can hold the localStorage lock (if they hold it beyond some period, they get terminated just like page script that doesn't re-enter the event loop). -atw On Sun, Mar 22, 2009 at 12:07 PM, Michael Nordman micha...@google.comwrote: I don't see how denying workers solves the problem. In a multi-threaded browser, this has to be resolved reasonably even in the absence of workers.
Re: [whatwg] localStorage + worker processes
On Mon, Mar 23, 2009 at 11:41 AM, Jeremy Orlow jor...@google.com wrote: One thing that hasn't been considered yet is some sort of optional hint to say I'm done in terms of accessing localStorage. Maybe call it localStorage.checkpoint() or localStroage.commit()? As far as the browser implemenation is concerned, a call to this function would be the same as the script ending. The great thing about this is that if a developer found one problem location in their code, they could add it--but it'd be completely optional. This could be used in conjunction with most of the other ideas already floating around. This is definitely an interesting idea. It does become less critical if we go with the async lock approach, but even then it's worth considering. / Jonas
Re: [whatwg] localStorage + worker processes
On Tue, Mar 24, 2009 at 7:41 AM, Jeremy Orlow jor...@google.com wrote: One thing that hasn't been considered yet is some sort of optional hint to say I'm done in terms of accessing localStorage. Maybe call it localStorage.checkpoint() or localStroage.commit()? As far as the browser implemenation is concerned, a call to this function would be the same as the script ending. The great thing about this is that if a developer found one problem location in their code, they could add it--but it'd be completely optional. You mean if they find a performance problem due to overlarge lock scope? That's not a terrible idea. It certainly makes far more sense to be safe by default and add constructs and code to gain a bit more parallelism, than to be unsafe and parallel by default and have to add constructs and code to get safety. I'm not sure what the semantics of a standalone commit() would be, though. I think a better construct might be some sort of yield which explicitly returns to a (nested) browser event loop and basically acts as a script completion point. Even browsers that only use a single thread can run the event loop there so that testing in those browsers will reveal bugs. Rob -- He was pierced for our transgressions, he was crushed for our iniquities; the punishment that brought us peace was upon him, and by his wounds we are healed. We all, like sheep, have gone astray, each of us has turned to his own way; and the LORD has laid on him the iniquity of us all. [Isaiah 53:5-6]
Re: [whatwg] localStorage + worker processes
I really like the idea of some generic yield, though I wonder if there's some reason it hasn't been added earlier. People have been using the setTimeout(..., 0) trick for a while to get around slow script warnings (and general unresponsiveness)...so surely something like this must have come up before? If so, what were the drawbacks? On Mon, Mar 23, 2009 at 2:24 PM, Robert O'Callahan rob...@ocallahan.orgwrote: On Tue, Mar 24, 2009 at 7:41 AM, Jeremy Orlow jor...@google.com wrote: One thing that hasn't been considered yet is some sort of optional hint to say I'm done in terms of accessing localStorage. Maybe call it localStorage.checkpoint() or localStroage.commit()? As far as the browser implemenation is concerned, a call to this function would be the same as the script ending. The great thing about this is that if a developer found one problem location in their code, they could add it--but it'd be completely optional. You mean if they find a performance problem due to overlarge lock scope? That's not a terrible idea. It certainly makes far more sense to be safe by default and add constructs and code to gain a bit more parallelism, than to be unsafe and parallel by default and have to add constructs and code to get safety. I'm not sure what the semantics of a standalone commit() would be, though. I think a better construct might be some sort of yield which explicitly returns to a (nested) browser event loop and basically acts as a script completion point. Even browsers that only use a single thread can run the event loop there so that testing in those browsers will reveal bugs. Rob -- He was pierced for our transgressions, he was crushed for our iniquities; the punishment that brought us peace was upon him, and by his wounds we are healed. We all, like sheep, have gone astray, each of us has turned to his own way; and the LORD has laid on him the iniquity of us all. [Isaiah 53:5-6]
Re: [whatwg] localStorage + worker processes
On Tue, Mar 24, 2009 at 10:40 AM, Jeremy Orlow jor...@google.com wrote: I really like the idea of some generic yield, though I wonder if there's some reason it hasn't been added earlier. People have been using the setTimeout(..., 0) trick for a while to get around slow script warnings (and general unresponsiveness)...so surely something like this must have come up before? If so, what were the drawbacks? An obvious issue is that you can easily end up with arbitrarily deep nesting and stack overflows. Rob -- He was pierced for our transgressions, he was crushed for our iniquities; the punishment that brought us peace was upon him, and by his wounds we are healed. We all, like sheep, have gone astray, each of us has turned to his own way; and the LORD has laid on him the iniquity of us all. [Isaiah 53:5-6]
Re: [whatwg] localStorage + worker processes
*Given that things have simmered down, it seems like it's time for a re-cap of the options. I believe this includes all the options currently on the table (in broad strokes anyhow). If I missed anything or grossly mischaracterized any idea, feel free to correct me.* 0: Do nothing. As far as I can tell, no one thinks this is a valid option. 1: Simply disallow localStorage in workers. Possibly allow in shared/persistent workers, but give them their own private storage area. 2: Explicit locking. Possibly only required for workers. Would require dead lock detection or for the spec to dictate locking precedence if another lock were introduced. 3: Expose localStorage to workers as asynchronous calls. Most similar to other concurrency solutions within workers. 4: Have some yield/checkpoint/commit type call on localStorage that would release any implicit/invisible lock. This could also be useful in conjunction with #3. *My opinion:* #1 would definitely work, but doesn't seem necessary. #2 seems to go against the grain of how concurrency is implemented everywhere else in the spec. In addition, it could open pandora's box in terms of locks being added in elsewhere and starting to cause some messy issues there. #3 and #4 preserve the simplicity of localStorage for the simple/common case--which is important above else, in my opinion. On Mon, Mar 23, 2009 at 2:48 PM, Robert O'Callahan rob...@ocallahan.orgwrote: On Tue, Mar 24, 2009 at 10:40 AM, Jeremy Orlow jor...@google.com wrote: I really like the idea of some generic yield, though I wonder if there's some reason it hasn't been added earlier. People have been using the setTimeout(..., 0) trick for a while to get around slow script warnings (and general unresponsiveness)...so surely something like this must have come up before? If so, what were the drawbacks? An obvious issue is that you can easily end up with arbitrarily deep nesting and stack overflows. Rob -- He was pierced for our transgressions, he was crushed for our iniquities; the punishment that brought us peace was upon him, and by his wounds we are healed. We all, like sheep, have gone astray, each of us has turned to his own way; and the LORD has laid on him the iniquity of us all. [Isaiah 53:5-6]
Re: [whatwg] localStorage + worker processes
On Sat, Mar 21, 2009 at 3:25 PM, Aaron Boodman a...@google.com wrote:
On Sat, Mar 21, 2009 at 1:51 PM, Jonas Sicking jo...@sicking.cc wrote:
The problem with synchronously grabbing the lock is that we can only
ever have one feature that uses synchronous locks, otherwise we'll
risk dead-locks.
Say that we make document.cookie behave the same way (to prevent
multi-process browsers like IE8 and chrome from having race
conditions). So that if you call document.getCookiesWithLock(callback)
we'll synchronously grab a lock and call the callback function. This
would cause two pages like the ones below to potentially deadlock:
Page 1:
getLocalStorage(function(storage) {
document.getCookiesWithLock(function(cookieContainer) {
storage.foo = cookieContainer.getCookie('cookieName');
});
]);
Page 2:
document.getCookiesWithLock(function(cookieContainer) {
getLocalStorage(function(storage) {
cookieContainer.setCookie('cookieName', storage.bar);
});
});
Good point. Ok, I agree that an asynchronous callback makes most sense
for this API.
Given an async api, would it be possible to store values into localStorage
at onunload time? I expect that could be a useful time to use this API.
function onunload() {
getLocalStorage(function(storage) {
// Will this ever execute?
});
}
Locking the storage until script completion isn't really necessary in many
cases. Maybe we're over engineering this? Suppose immutability across calls
was generally not guaranteed by the existing API. And we add an async
getLocalStorage(callback) which does provide immutability for the duration
of the callback if that is desired.
Re: [whatwg] localStorage + worker processes
The problem is that .length is basically useless without some kind of
immutability guarantees.
I've thought about this more, and I'm afraid that if you start making the
API cumbersome (forcing only async access) then apps are just going to use
document.cookies instead of localStorage. I'd hate to see us radically
change the API to support the worker case - I'd rather get rid of
localStorage support from workers, or else just enforce a max time that a
worker can hold the lock.
-atw
On Sun, Mar 22, 2009 at 10:46 AM, Michael Nordman micha...@google.comwrote:
On Sat, Mar 21, 2009 at 3:25 PM, Aaron Boodman a...@google.com wrote:
On Sat, Mar 21, 2009 at 1:51 PM, Jonas Sicking jo...@sicking.cc wrote:
The problem with synchronously grabbing the lock is that we can only
ever have one feature that uses synchronous locks, otherwise we'll
risk dead-locks.
Say that we make document.cookie behave the same way (to prevent
multi-process browsers like IE8 and chrome from having race
conditions). So that if you call document.getCookiesWithLock(callback)
we'll synchronously grab a lock and call the callback function. This
would cause two pages like the ones below to potentially deadlock:
Page 1:
getLocalStorage(function(storage) {
document.getCookiesWithLock(function(cookieContainer) {
storage.foo = cookieContainer.getCookie('cookieName');
});
]);
Page 2:
document.getCookiesWithLock(function(cookieContainer) {
getLocalStorage(function(storage) {
cookieContainer.setCookie('cookieName', storage.bar);
});
});
Good point. Ok, I agree that an asynchronous callback makes most sense
for this API.
Given an async api, would it be possible to store values into localStorage
at onunload time? I expect that could be a useful time to use this API.
function onunload() {
getLocalStorage(function(storage) {
// Will this ever execute?
});
}
Locking the storage until script completion isn't really necessary in many
cases. Maybe we're over engineering this? Suppose immutability across calls
was generally not guaranteed by the existing API. And we add an async
getLocalStorage(callback) which does provide immutability for the duration
of the callback if that is desired.
Re: [whatwg] localStorage + worker processes
On Sun, Mar 22, 2009 at 11:21 AM, Drew Wilson atwil...@google.com wrote:
The problem is that .length is basically useless without some kind of
immutability guarantees.
Understood, and when you need that guarantee it would be available via the
getLocalStorage(callback) API.
I've thought about this more, and I'm afraid that if you start making the
API cumbersome (forcing only async access) then apps are just going to use
document.cookies instead of localStorage. I'd hate to see us radically
change the API to support the worker case - I'd rather get rid of
localStorage support from workers, or else just enforce a max time that a
worker can hold the lock.
-atw
On Sun, Mar 22, 2009 at 10:46 AM, Michael Nordman micha...@google.comwrote:
On Sat, Mar 21, 2009 at 3:25 PM, Aaron Boodman a...@google.com wrote:
On Sat, Mar 21, 2009 at 1:51 PM, Jonas Sicking jo...@sicking.cc wrote:
The problem with synchronously grabbing the lock is that we can only
ever have one feature that uses synchronous locks, otherwise we'll
risk dead-locks.
Say that we make document.cookie behave the same way (to prevent
multi-process browsers like IE8 and chrome from having race
conditions). So that if you call document.getCookiesWithLock(callback)
we'll synchronously grab a lock and call the callback function. This
would cause two pages like the ones below to potentially deadlock:
Page 1:
getLocalStorage(function(storage) {
document.getCookiesWithLock(function(cookieContainer) {
storage.foo = cookieContainer.getCookie('cookieName');
});
]);
Page 2:
document.getCookiesWithLock(function(cookieContainer) {
getLocalStorage(function(storage) {
cookieContainer.setCookie('cookieName', storage.bar);
});
});
Good point. Ok, I agree that an asynchronous callback makes most sense
for this API.
Given an async api, would it be possible to store values into localStorage
at onunload time? I expect that could be a useful time to use this API.
function onunload() {
getLocalStorage(function(storage) {
// Will this ever execute?
});
}
Locking the storage until script completion isn't really necessary in many
cases. Maybe we're over engineering this? Suppose immutability across calls
was generally not guaranteed by the existing API. And we add an async
getLocalStorage(callback) which does provide immutability for the duration
of the callback if that is desired.
Re: [whatwg] localStorage + worker processes
On Sun, Mar 22, 2009 at 11:21 AM, Drew Wilson atwil...@google.com wrote: I've thought about this more, and I'm afraid that if you start making the API cumbersome (forcing only async access) then apps are just going to use document.cookies instead of localStorage. I'd hate to see us radically change the API to support the worker case - I'd rather get rid of localStorage support from workers, or else just enforce a max time that a worker can hold the lock. I don't believe that. Adding one async callback is no inconvenience compared to the sad farce that is the document.cookie API. Also, localstorage has many benefits including structured storage and not getting sent to the server in every request. - a
Re: [whatwg] localStorage + worker processes
On Sun, Mar 22, 2009 at 11:53 AM, Aaron Boodman a...@google.com wrote: On Sun, Mar 22, 2009 at 11:21 AM, Drew Wilson atwil...@google.com wrote: I've thought about this more, and I'm afraid that if you start making the API cumbersome (forcing only async access) then apps are just going to use document.cookies instead of localStorage. I'd hate to see us radically change the API to support the worker case - I'd rather get rid of localStorage support from workers, or else just enforce a max time that a worker can hold the lock. I don't believe that. Adding one async callback is no inconvenience compared to the sad farce that is the document.cookie API. Also, localstorage has many benefits including structured storage and not getting sent to the server in every request. I don't see how denying workers solves the problem. In a multi-threaded browser, this has to be resolved reasonably even in the absence of workers.
Re: [whatwg] localStorage + worker processes
If you deny workers, you can enforce exclusive access to localStorage by applying a lock that extends from the first access of localStorage until the script re-enters the event loop. Page script is guaranteed to re-enter the event loop fairly quickly (lest it trigger the browser's this script is taking too long to run protection) so you won't get starvation. Since worker script never has to re-enter the event loop, this isn't a feasible solution for workers. That's why I'm proposing that the most reasonable implementation is just to have a simple lock like I describe above, and then either deny access to localStorage to dedicated workers (shared workers can silo the storage as I described previously), or else just enforce a limit to how long workers can hold the localStorage lock (if they hold it beyond some period, they get terminated just like page script that doesn't re-enter the event loop). -atw On Sun, Mar 22, 2009 at 12:07 PM, Michael Nordman micha...@google.comwrote: I don't see how denying workers solves the problem. In a multi-threaded browser, this has to be resolved reasonably even in the absence of workers.
Re: [whatwg] localStorage + worker processes
On Mon, Mar 23, 2009 at 8:07 AM, Michael Nordman micha...@google.comwrote: I don't see how denying workers solves the problem. In a multi-threaded browser, this has to be resolved reasonably even in the absence of workers. It has to be resolved in a way that doesn't expose asynchronous cookie or localStorage changes to Web developers. There is abundant evidence that race conditions and synchronization are too hard for developers to deal with. The spec should forbid asynchronously visible changes to cookies or localStorage. In fact, it should probably simply say that all script execution is serializable: always equivalent to some execution you could get with a single-threaded browser that runs all scripts to completion. Allowance could be made for explicit yield points if we need to, e.g. alert(). If IE7+ allows asynchronous cookie changes, that is a bad bug. Perhaps today they get away with it because people aren't writing long-running scripts that use cookies as read/write storage. We shouldn't take that as a sign that the shared-state threaded programming model somehow works better on the Web than it does everywhere else. Some sort of implicit locking with guaranteed deadlock freedom should be workable for parallel browser implementations. For example, partition browser contexts into related subsets, where context A is related to context B if a script running in context A can affect the execution of an already-running script in context B. Use one lock per subset, and have a script execution acquire the lock when it first touches localStorage or cookies, and drop the lock when it completes (or yields). Additional optimizations are possible. An asynchronous API like the multi-resource-acquisition callback Jonas just proposed, would be helpful to enable additional parallelism. By declaring up-front which resources need to be locked and guaranteeing that only those resources will be touched, you can use a finer-grained locking scheme without risking deadlock. Rob -- He was pierced for our transgressions, he was crushed for our iniquities; the punishment that brought us peace was upon him, and by his wounds we are healed. We all, like sheep, have gone astray, each of us has turned to his own way; and the LORD has laid on him the iniquity of us all. [Isaiah 53:5-6]
Re: [whatwg] localStorage + worker processes
On Fri, Mar 20, 2009 at 3:10 PM, Aaron Boodman a...@google.com wrote:
I think the best option is to make access to localstorage asynchronous
for workers. This reduces the amount of time a worker can hold the
localstore lock so that it shouldn't be a problem for normal pages. It
sucks to make such a simple and useful API aync though.
As I understand the current API (on main window) to be defined, as
soon as someone accesses the .localStorage property, the
implementation is supposed to acquire a lock. This lock would be held
on to until that script returns to the event loop for that thread.
So if javascript in another window, running in another thread or
process, tries to access .localStorage for the same origin, the
.localStorage getter would try to acquire the same lock and block
until the first thread releases the lock.
This could in theory be applied to applied to workers as well. However
as Jeremy points out that could result in the a worker script running
for a very long time blocking the window thread.
What we could do, is to have an API like
getLocalStorage(callback);
This function returns immediately, but will then call the callback
function as soon as the localStorage becomes available and the lock
been acquired. This would always happen asynchronously off the event
loop, which means that once the callback returns the lock is released
again.
Of course, it would still mean that a window context or worker could
hold on to the lock for an indefinite time, but as long as the asych
getLocalStorage API is used, this means that no thread is blocked,
just that they aren't able to get access to the localStorage.
So for example, the following code would safely add 1 to the 'foo'
property in localStorage:
getLocalStorage(function(store) {
store.foo = parseInt(store.foo, 10) + 1;
});
Additionally, we would have to define that if the store object passed
to the callback function is accessed outside after the callback has
ended this will throw an exception. If the object is reactivated next
time a callback is entered, or if a new storage object is created also
needs to be defined.
This new API I believe is good enough to be used both from workers and
window contexts.
We could even keep the current API implemented in IE8, or we could
just ask people to write a wrapper for IE8 like:
function getLocalStorage(callback) {
setTimeout(function() { callback(localStorage); }, 0);
}
in an implementation that implements correct locking for the
synchronous API, this will even produce the correct locking behavior
for the new API.
/ Jonas
Re: [whatwg] localStorage + worker processes
On Sat, Mar 21, 2009 at 12:48 AM, Jonas Sicking jo...@sicking.cc wrote: What we could do, is to have an API like getLocalStorage(callback); This function returns immediately, but will then call the callback function as soon as the localStorage becomes available and the lock been acquired. This would always happen asynchronously off the event loop, which means that once the callback returns the lock is released again. Funny, a few of us from Chromium were discussing a similar solution privately. Actually, I don't believe that it is required that the callback run asynchronously. All the callback is used for is establishing the lock lifetime explicitly, and we assume that this will usually make the lock lifetime short. So we can block while we wait for it to become available. This is just like the behavior today without workers. This new API I believe is good enough to be used both from workers and window contexts. We could maintain backward compatibility by not making it required on normal web pages, because our assumption has been that web pages won't hold the lock for long periods of time (because they wouldn't want to block the UI either). - a
Re: [whatwg] localStorage + worker processes
That might work. Is it feasible for user agents to enforce limits on how
long a callback is allowed to run holding the lock? That way workers can't
starve normal pages from accessing their local storage.
-atw
On Sat, Mar 21, 2009 at 12:48 AM, Jonas Sicking jo...@sicking.cc wrote:
On Fri, Mar 20, 2009 at 3:10 PM, Aaron Boodman a...@google.com wrote:
I think the best option is to make access to localstorage asynchronous
for workers. This reduces the amount of time a worker can hold the
localstore lock so that it shouldn't be a problem for normal pages. It
sucks to make such a simple and useful API aync though.
As I understand the current API (on main window) to be defined, as
soon as someone accesses the .localStorage property, the
implementation is supposed to acquire a lock. This lock would be held
on to until that script returns to the event loop for that thread.
So if javascript in another window, running in another thread or
process, tries to access .localStorage for the same origin, the
.localStorage getter would try to acquire the same lock and block
until the first thread releases the lock.
This could in theory be applied to applied to workers as well. However
as Jeremy points out that could result in the a worker script running
for a very long time blocking the window thread.
What we could do, is to have an API like
getLocalStorage(callback);
This function returns immediately, but will then call the callback
function as soon as the localStorage becomes available and the lock
been acquired. This would always happen asynchronously off the event
loop, which means that once the callback returns the lock is released
again.
Of course, it would still mean that a window context or worker could
hold on to the lock for an indefinite time, but as long as the asych
getLocalStorage API is used, this means that no thread is blocked,
just that they aren't able to get access to the localStorage.
So for example, the following code would safely add 1 to the 'foo'
property in localStorage:
getLocalStorage(function(store) {
store.foo = parseInt(store.foo, 10) + 1;
});
Additionally, we would have to define that if the store object passed
to the callback function is accessed outside after the callback has
ended this will throw an exception. If the object is reactivated next
time a callback is entered, or if a new storage object is created also
needs to be defined.
This new API I believe is good enough to be used both from workers and
window contexts.
We could even keep the current API implemented in IE8, or we could
just ask people to write a wrapper for IE8 like:
function getLocalStorage(callback) {
setTimeout(function() { callback(localStorage); }, 0);
}
in an implementation that implements correct locking for the
synchronous API, this will even produce the correct locking behavior
for the new API.
/ Jonas
Re: [whatwg] localStorage + worker processes
On Sat, Mar 21, 2009 at 9:41 AM, Drew Wilson atwil...@google.com wrote: That might work. Is it feasible for user agents to enforce limits on how long a callback is allowed to run holding the lock? That way workers can't starve normal pages from accessing their local storage. It seems like they could use the same time limit that is used for detecting/stopping runaway scripts in web pages. - a
Re: [whatwg] localStorage + worker processes
On Sat, Mar 21, 2009 at 9:38 AM, Aaron Boodman a...@google.com wrote:
On Sat, Mar 21, 2009 at 12:48 AM, Jonas Sicking jo...@sicking.cc wrote:
What we could do, is to have an API like
getLocalStorage(callback);
This function returns immediately, but will then call the callback
function as soon as the localStorage becomes available and the lock
been acquired. This would always happen asynchronously off the event
loop, which means that once the callback returns the lock is released
again.
Funny, a few of us from Chromium were discussing a similar solution privately.
Actually, I don't believe that it is required that the callback run
asynchronously. All the callback is used for is establishing the lock
lifetime explicitly, and we assume that this will usually make the
lock lifetime short. So we can block while we wait for it to become
available. This is just like the behavior today without workers.
The problem with synchronously grabbing the lock is that we can only
ever have one feature that uses synchronous locks, otherwise we'll
risk dead-locks.
Say that we make document.cookie behave the same way (to prevent
multi-process browsers like IE8 and chrome from having race
conditions). So that if you call document.getCookiesWithLock(callback)
we'll synchronously grab a lock and call the callback function. This
would cause two pages like the ones below to potentially deadlock:
Page 1:
getLocalStorage(function(storage) {
document.getCookiesWithLock(function(cookieContainer) {
storage.foo = cookieContainer.getCookie('cookieName');
});
]);
Page 2:
document.getCookiesWithLock(function(cookieContainer) {
getLocalStorage(function(storage) {
cookieContainer.setCookie('cookieName', storage.bar);
});
});
The point here isn't to propose a new cookie API, but rather that as
soon as we introduce more than one lock that can be synchronously
acquired, we run the risk of deadlocks if people don't grab them in
the right order.
While we could make the implementation detect this and case the second
page to throw when a deadlock is detected, that just turns the
deadlock into a race condition since pages are unlikely to handle this
exception correctly given that it generally doesn't happen.
By making all lock acquiring async, we ensure that script can only
ever grab one lock at a time, thus preventing people from grabbing
multiple locks in different order and causing deadlocks.
There is actually another alternative. We could specify an order that
locks have to be acquired if you grab more than one. So for example we
could say that the order is
localStore cookie otherRandomFeature
In other words, if you want both the localStore and cookie features,
you have to grab first localStore and then cookie. Attempting to grab
localStore once you have acquired the cookie lock *always* results in
an exception being thrown, even if there currently is no deadlock.
Anyone specifying a new feature that uses locking would have to define
where in the locking order this lock is placed. However this API
design does not seem very web developer friendly.
/ Jonas
Re: [whatwg] localStorage + worker processes
On Sat, Mar 21, 2009 at 9:41 AM, Drew Wilson atwil...@google.com wrote: That might work. Is it feasible for user agents to enforce limits on how long a callback is allowed to run holding the lock? That way workers can't starve normal pages from accessing their local storage. I don't think it will be a big problem. As long as we ensure that all locks are per-origin, that means that an application can only starve itself. Something that it has clear incentives not to. / Jonas
Re: [whatwg] localStorage + worker processes
As a side note, if we do go with this async lock acquiring, we could
add an API like:
getLockedFeatures(callback, 'localStore', 'cookie');
This would be an asynchronously grab locks to multiple features and
only call the callback once all of them have been acquired. This would
allow computations across data from multiple locations guaranteed to
be in sync. The implementation would be responsible for grabbing the
locks in a consistent order to prevent deadlocks.
/ Jonas
On Sat, Mar 21, 2009 at 12:48 AM, Jonas Sicking jo...@sicking.cc wrote:
On Fri, Mar 20, 2009 at 3:10 PM, Aaron Boodman a...@google.com wrote:
I think the best option is to make access to localstorage asynchronous
for workers. This reduces the amount of time a worker can hold the
localstore lock so that it shouldn't be a problem for normal pages. It
sucks to make such a simple and useful API aync though.
As I understand the current API (on main window) to be defined, as
soon as someone accesses the .localStorage property, the
implementation is supposed to acquire a lock. This lock would be held
on to until that script returns to the event loop for that thread.
So if javascript in another window, running in another thread or
process, tries to access .localStorage for the same origin, the
.localStorage getter would try to acquire the same lock and block
until the first thread releases the lock.
This could in theory be applied to applied to workers as well. However
as Jeremy points out that could result in the a worker script running
for a very long time blocking the window thread.
What we could do, is to have an API like
getLocalStorage(callback);
This function returns immediately, but will then call the callback
function as soon as the localStorage becomes available and the lock
been acquired. This would always happen asynchronously off the event
loop, which means that once the callback returns the lock is released
again.
Of course, it would still mean that a window context or worker could
hold on to the lock for an indefinite time, but as long as the asych
getLocalStorage API is used, this means that no thread is blocked,
just that they aren't able to get access to the localStorage.
So for example, the following code would safely add 1 to the 'foo'
property in localStorage:
getLocalStorage(function(store) {
store.foo = parseInt(store.foo, 10) + 1;
});
Additionally, we would have to define that if the store object passed
to the callback function is accessed outside after the callback has
ended this will throw an exception. If the object is reactivated next
time a callback is entered, or if a new storage object is created also
needs to be defined.
This new API I believe is good enough to be used both from workers and
window contexts.
We could even keep the current API implemented in IE8, or we could
just ask people to write a wrapper for IE8 like:
function getLocalStorage(callback) {
setTimeout(function() { callback(localStorage); }, 0);
}
in an implementation that implements correct locking for the
synchronous API, this will even produce the correct locking behavior
for the new API.
/ Jonas
Re: [whatwg] localStorage + worker processes
On Sat, Mar 21, 2009 at 1:51 PM, Jonas Sicking jo...@sicking.cc wrote:
The problem with synchronously grabbing the lock is that we can only
ever have one feature that uses synchronous locks, otherwise we'll
risk dead-locks.
Say that we make document.cookie behave the same way (to prevent
multi-process browsers like IE8 and chrome from having race
conditions). So that if you call document.getCookiesWithLock(callback)
we'll synchronously grab a lock and call the callback function. This
would cause two pages like the ones below to potentially deadlock:
Page 1:
getLocalStorage(function(storage) {
document.getCookiesWithLock(function(cookieContainer) {
storage.foo = cookieContainer.getCookie('cookieName');
});
]);
Page 2:
document.getCookiesWithLock(function(cookieContainer) {
getLocalStorage(function(storage) {
cookieContainer.setCookie('cookieName', storage.bar);
});
});
Good point. Ok, I agree that an asynchronous callback makes most sense
for this API.
- a
Re: [whatwg] localStorage + worker processes
On Fri, Mar 20, 2009 at 1:23 PM, Jeremy Orlow jor...@google.com wrote: What is the need for localStorage access within workers? Technically if someone really needed to access it, they could always have a function in the web page for accessing it and then use postMessage. In other words, they could build their own ad-hoc async API pretty easily. Another alternative is to just build an async API into the spec (and remove synchronous access to localStorage). Thoughts? I do think it would be great if workers had access to some type of structured storage. However I agree that the fact that both the main thread and workers have synchronous access to the same storage is not acceptable since that means that we're violating the shared-nothing-message-passing design that makes workers not have to deal with locks and other traditional multithread hazards. / Jonas
Re: [whatwg] localStorage + worker processes
On Fri, Mar 20, 2009 at 1:47 PM, Jonas Sicking jo...@sicking.cc wrote: On Fri, Mar 20, 2009 at 1:23 PM, Jeremy Orlow jor...@google.com wrote: What is the need for localStorage access within workers? Technically if someone really needed to access it, they could always have a function in the web page for accessing it and then use postMessage. In other words, they could build their own ad-hoc async API pretty easily. Another alternative is to just build an async API into the spec (and remove synchronous access to localStorage). Thoughts? I do think it would be great if workers had access to some type of structured storage. However I agree that the fact that both the main thread and workers have synchronous access to the same storage is not acceptable since that means that we're violating the shared-nothing-message-passing design that makes workers not have to deal with locks and other traditional multithread hazards. / Jonas When discussing this standard we have to recognize that not all browsers actually have a main thread. Time will tell if more or less browsers of the future will have multi-threaded architectures, but the trend has been for more I think. Any aspects of the spec that asserts or assumes a main thread is questionable.
Re: [whatwg] localStorage + worker processes
When discussing this standard we have to recognize that not all browsers actually have a main thread. Time will tell if more or less browsers of the future will have multi-threaded architectures, but the trend has been for more I think. Any aspects of the spec that asserts or assumes a main thread is questionable. Yes they do -- we're talking about the main thread from the point of view of javascript, which is not necessarily the UI thread. The important thing is that with the current model is that JS on any thread may be blocked by js executing in a worker, which leads to a page (in effect) locking up -- the UI may still be functional but that particular page will have hung. --Oliver
Re: [whatwg] localStorage + worker processes
I agree with Jeremy that the spec is currently unimplementable if we give localStorage access to workers. I'd like to point out that workers who want to access localStorage just need to send a message to their main window breaks down for persistent workers (where you don't necessarily have an open browser window) and is pretty wonky for shared workers (you can send a message to a window, but that window may go away before your message is processed, so you end up having to build some kind of send message, timeout, pick a new window to send it to, etc message layer). One alternative I'd like to propose is to remove access to localStorage for dedicated workers, and give SharedWorkers access to localStorage, but have that storage be partitioned by the worker name (i.e. the worker can access it, but it's not shared with web pages or any other workers and so you don't have any synchronicity issues). I don't see how this would work for dedicated workers, though, since there's no name to partition storage access, but they could always fall back to postMessage(). -atw On Fri, Mar 20, 2009 at 2:19 PM, Oliver Hunt oli...@apple.com wrote: When discussing this standard we have to recognize that not all browsers actually have a main thread. Time will tell if more or less browsers of the future will have multi-threaded architectures, but the trend has been for more I think. Any aspects of the spec that asserts or assumes a main thread is questionable. Yes they do -- we're talking about the main thread from the point of view of javascript, which is not necessarily the UI thread. The important thing is that with the current model is that JS on any thread may be blocked by js executing in a worker, which leads to a page (in effect) locking up -- the UI may still be functional but that particular page will have hung. --Oliver
Re: [whatwg] localStorage + worker processes
I think the best option is to make access to localstorage asynchronous for workers. This reduces the amount of time a worker can hold the localstore lock so that it shouldn't be a problem for normal pages. It sucks to make such a simple and useful API aync though. - a
Re: [whatwg] localStorage + worker processes
On Fri, Mar 20, 2009 at 2:39 PM, Drew Wilson atwil...@google.com wrote: One alternative I'd like to propose is to remove access to localStorage for dedicated workers, and give SharedWorkers access to localStorage, but have that storage be partitioned by the worker name (i.e. the worker can access it, but it's not shared with web pages or any other workers and so you don't have any synchronicity issues). This may be interesting. It allows to keep single API for localStorage. Most of the usage is probably going to happen inside such a worker - in other words, sharing the info with other pages of the same origin may not be needed, and there is always async communication when a live page wants to share data with a shared worker. Dmitry
Re: [whatwg] localStorage + worker processes
On Fri, Mar 20, 2009 at 3:34 PM, Dmitry Titov dim...@google.com wrote: On Fri, Mar 20, 2009 at 2:39 PM, Drew Wilson atwil...@google.com wrote: One alternative I'd like to propose is to remove access to localStorage for dedicated workers, and give SharedWorkers access to localStorage, but have that storage be partitioned by the worker name (i.e. the worker can access it, but it's not shared with web pages or any other workers and so you don't have any synchronicity issues). This may be interesting. It allows to keep single API for localStorage. Most of the usage is probably going to happen inside such a worker - in other words, sharing the info with other pages of the same origin may not be needed, and there is always async communication when a live page wants to share data with a shared worker. Interesting... essentially a page/worker specific persistent repository... doesn't span in the way localStorage does.
