Re: Persistent Storage vs. Database
On Fri, Mar 8, 2013 at 8:16 PM, Jonas Sicking wrote: > On Fri, Mar 8, 2013 at 2:27 PM, Andrew Fedoniouk > wrote: >> On Fri, Mar 8, 2013 at 11:30 AM, Kyle Huey wrote: >>> On Fri, Mar 8, 2013 at 11:02 AM, Andrew Fedoniouk >>> wrote: On Thu, Mar 7, 2013 at 10:36 PM, Kyle Huey wrote: > On Thu, Mar 7, 2013 at 10:20 PM, Andrew Fedoniouk > wrote: > >> At least it is easier than http://www.w3.org/TR/IndexedDB/ :) > > > Easier doesn't necessarily mean better. LocalStorage is certainly > easier to > use than any async storage system ;-) > At least my implementation does not use any events. Proposed system of events in IndexedDB is the antipattern indeed. Exactly for the same reasons as finalizer *events* you've mentioned above - there is no guarantee that all events will be delivered to the code awaiting and relaying on them. >>> >>> >>> That's not true at all. If you don't understand the difference between >>> finalizers and events you're not going to be able to make a very informed >>> criticism of IndexedDB. >>> >> >> I would appreciate if you will define term `event`. After that we can discuss >> it further. > > https://dvcs.w3.org/hg/domcore/raw-file/tip/Overview.html#interface-event Thanks, but these are DOM events. How they are related to DB events or whatever they are named? What does it mean bubbling/capturing for them? > >> As of common practice there are two types so called outbound calls: >> 1. Synchronous *callbacks* that are not strictly speaking events - just >> function >>references: "after/before-doing-this-call-that". 'finalized' is >> that kind of callback. >> 2. Events - these are objects and event dispatching system associated >>with them. For example UI events use capture/bubble dispatching >>system and event queue. Events operate independently from their >>handlers. >> >> Let's take a look on this example from IndexedDB spec: >> >> var request = indexedDB.open('AddressBook', 15); >> request.onsuccess = function(evt) {...}; >> request.onerror = function(evt) {...}; >> >> It looks *very* wrong to me: >> >> What should happen if db.open() will open the DB immediately? > > The event fires asynchronously. Since JS is single threaded that means > that the onsuccess and onerror properties will be set before the event > is fired. 1. Who said that JS *must* always be single threaded? 2. Is there any requirement that IndexedDB must be used only with JS? 3. Are you sure that opening DB or doing look-up on b-tree is always more expensive than posting and dispatching events? Windows for example uses 18ms precise timers ( used for dispatching non-UI events, or similar mechanism ). Are you saying that all operations in IndexedDB simply cannot work faster than that? If "yes" then who needs such brake by design? If to compare look-up in b-tree on memory pages that are already in cached and the task of posting/dispatching messages including handler function invocation in script then ratio will be around 1:50. What's the name of the design that puts such overhead upfront? > >> Will request.onsuccess be called in this case? > > Yes. > >> If indexedDB.open is purely async operation then >> why it has to be exactly async? > > Because it requires IO. > 1. Why exactly it requires IO? Why it cannot be implemented by using main memory or memory-mapped files at the end ( not exactly IO in classic sense )? 2. localStorage also uses IO, what's the problem you see with it at the moment? 3. Which operation in IndexedDB require significantly more time to execute than corresponding one from localStorage? 4. What's wrong with Web Workers that were designed specifically for cases when operations takes long time? I do not see what operation can take long time in IndexedDB but still would like to know answer. >> What may take time there other >> than opening local file in worst case? If that request is async then >> when precisely request.onsuccess will be called? >> >> I would understand if DB.open call will be defined this way: >> >> function onsuccess(evt) {...}; >> function onerror(evt) {...}; >> >> var request = indexedDB.open('AddressBook', 15, onsuccess, onerror ); >> >> (so pure callback operation). > > If I write > > var x = 0; > function onsuccess(evt) { x = 1 }; > function onerror(evt) {...}; > var request = indexedDB.open('AddressBook', 15, onsuccess, onerror ); > console.log(x); > > would you expect the console to sometimes show 1 and sometimes show 0? > If not, why not? I am not sure I understand you here. This form: indexedDB.open('AddressBook', 15, onsuccess, onerror ); does not limit UA engineers to chose the most optimal implementation. So such open() may or may not use async model. But current spec mandates the most ineffective variant. By design. But in general there is no need for callback/events for operations on such small DBs. SQLi
Re: Persistent Storage vs. Database
On Fri, Mar 8, 2013 at 2:27 PM, Andrew Fedoniouk wrote: > On Fri, Mar 8, 2013 at 11:30 AM, Kyle Huey wrote: >> On Fri, Mar 8, 2013 at 11:02 AM, Andrew Fedoniouk >> wrote: >>> >>> On Thu, Mar 7, 2013 at 10:36 PM, Kyle Huey wrote: >>> > On Thu, Mar 7, 2013 at 10:20 PM, Andrew Fedoniouk >>> > wrote: >>> > >>> >> At least it is easier than http://www.w3.org/TR/IndexedDB/ :) >>> > >>> > >>> > Easier doesn't necessarily mean better. LocalStorage is certainly >>> > easier to >>> > use than any async storage system ;-) >>> > >>> >>> At least my implementation does not use any events. Proposed >>> system of events in IndexedDB is the antipattern indeed. Exactly for >>> the same reasons as finalizer *events* you've mentioned above - there >>> is no guarantee that all events will be delivered to the code awaiting >>> and relaying on them. >> >> >> That's not true at all. If you don't understand the difference between >> finalizers and events you're not going to be able to make a very informed >> criticism of IndexedDB. >> > > I would appreciate if you will define term `event`. After that we can discuss > it further. https://dvcs.w3.org/hg/domcore/raw-file/tip/Overview.html#interface-event > As of common practice there are two types so called outbound calls: > 1. Synchronous *callbacks* that are not strictly speaking events - just > function >references: "after/before-doing-this-call-that". 'finalized' is > that kind of callback. > 2. Events - these are objects and event dispatching system associated >with them. For example UI events use capture/bubble dispatching >system and event queue. Events operate independently from their >handlers. > > Let's take a look on this example from IndexedDB spec: > > var request = indexedDB.open('AddressBook', 15); > request.onsuccess = function(evt) {...}; > request.onerror = function(evt) {...}; > > It looks *very* wrong to me: > > What should happen if db.open() will open the DB immediately? The event fires asynchronously. Since JS is single threaded that means that the onsuccess and onerror properties will be set before the event is fired. > Will request.onsuccess be called in this case? Yes. > If indexedDB.open is purely async operation then > why it has to be exactly async? Because it requires IO. > What may take time there other > than opening local file in worst case? If that request is async then > when precisely request.onsuccess will be called? > > I would understand if DB.open call will be defined this way: > > function onsuccess(evt) {...}; > function onerror(evt) {...}; > > var request = indexedDB.open('AddressBook', 15, onsuccess, onerror ); > > (so pure callback operation). If I write var x = 0; function onsuccess(evt) { x = 1 }; function onerror(evt) {...}; var request = indexedDB.open('AddressBook', 15, onsuccess, onerror ); console.log(x); would you expect the console to sometimes show 1 and sometimes show 0? If not, why not? / Jonas
Re: Persistent Storage vs. Database
On Fri, Mar 8, 2013 at 12:36 AM, Kyle Huey wrote: > On Thu, Mar 7, 2013 at 10:20 PM, Andrew Fedoniouk < > n...@terrainformatica.com> wrote: > >> Physical commit (write) of objects to storage happens on either >> a) GC cycle or b) on explicit storage.commit() call or on c) VM shutdown. >> > > Persisting data off a GC cycle (via finalizers or something else) is a > pretty well known antipattern.[0] > Correct, but just to be clear, there are other ways to get a similar effect. In particular, you can queue a task, add a job to the "global script clean-up jobs" list, or use a microtask. At least it is easier than http://www.w3.org/TR/IndexedDB/ :) > > Note that if you see "TR" in a URL, you're probably looking at an old, obsolete spec. This one is almost a year out of date. Click the "editor's draft" link at the top to get to the real spec. https://dvcs.w3.org/hg/IndexedDB/raw-file/tip/Overview.html On Fri, Mar 8, 2013 at 1:02 PM, Andrew Fedoniouk wrote: > At least my implementation does not use any events. Proposed > system of events in IndexedDB is the antipattern indeed. Exactly for > the same reasons as finalizer *events* you've mentioned above - there > is no guarantee that all events will be delivered to the code awaiting > and relaying on them. > This is wrong. Events are not an "antipattern" (calling things that seems a bit of a fad these days), and they're certainly not a "proposal". It's the standard, well-established API on the Web, used broadly across the whole platform. -- Glenn Maynard
Re: Persistent Storage vs. Database
On Fri, Mar 8, 2013 at 11:30 AM, Kyle Huey wrote: > On Fri, Mar 8, 2013 at 11:02 AM, Andrew Fedoniouk > wrote: >> >> On Thu, Mar 7, 2013 at 10:36 PM, Kyle Huey wrote: >> > On Thu, Mar 7, 2013 at 10:20 PM, Andrew Fedoniouk >> > wrote: >> > >> >> At least it is easier than http://www.w3.org/TR/IndexedDB/ :) >> > >> > >> > Easier doesn't necessarily mean better. LocalStorage is certainly >> > easier to >> > use than any async storage system ;-) >> > >> >> At least my implementation does not use any events. Proposed >> system of events in IndexedDB is the antipattern indeed. Exactly for >> the same reasons as finalizer *events* you've mentioned above - there >> is no guarantee that all events will be delivered to the code awaiting >> and relaying on them. > > > That's not true at all. If you don't understand the difference between > finalizers and events you're not going to be able to make a very informed > criticism of IndexedDB. > I would appreciate if you will define term `event`. After that we can discuss it further. As of common practice there are two types so called outbound calls: 1. Synchronous *callbacks* that are not strictly speaking events - just function references: "after/before-doing-this-call-that". 'finalized' is that kind of callback. 2. Events - these are objects and event dispatching system associated with them. For example UI events use capture/bubble dispatching system and event queue. Events operate independently from their handlers. Let's take a look on this example from IndexedDB spec: var request = indexedDB.open('AddressBook', 15); request.onsuccess = function(evt) {...}; request.onerror = function(evt) {...}; It looks *very* wrong to me: What should happen if db.open() will open the DB immediately? Will request.onsuccess be called in this case? If indexedDB.open is purely async operation then why it has to be exactly async? What may take time there other than opening local file in worst case? If that request is async then when precisely request.onsuccess will be called? I would understand if DB.open call will be defined this way: function onsuccess(evt) {...}; function onerror(evt) {...}; var request = indexedDB.open('AddressBook', 15, onsuccess, onerror ); (so pure callback operation). But purpose of such event-aganza is still not clear to me. Why not classical: try { request = indexedDB.open('AddressBook', 15 ); } catch(err) { ... } ? In principle: what are operations in IndexedDB that may take so long time that they need to be async? Size of client side DB has and will always have some reasonable cap limiting its size and so any lookup or update operation in indexed DB will take some finite and *very* short time (if to compare with the time needed to relayout average page). Why these so strange "events" are there at all? And so why all this has to be that complex? Andrew Fedoniouk. http://terrainformatica.com
Re: Persistent Storage vs. Database
On Thu, Mar 7, 2013 at 10:36 PM, Kyle Huey wrote: > On Thu, Mar 7, 2013 at 10:20 PM, Andrew Fedoniouk > wrote: >> >> Physical commit (write) of objects to storage happens on either >> a) GC cycle or b) on explicit storage.commit() call or on c) VM shutdown. > > > Persisting data off a GC cycle (via finalizers or something else) is a > pretty well known antipattern.[0] Raymond is right in general when he speaks about .NET runtime environment. But JS computing environment is quite different from the .NET one - at least these two have different lifespans of VMs and memory heaps. In my case persistence is closer to virtual memory mechanism - when there is not enough memory persistable objects get swapped to the storage from the heap. That is quite widespread pattern if we want to use this language. In any case there is always storage.commit() if you need guarantees and deterministic storage state. > >> At least it is easier than http://www.w3.org/TR/IndexedDB/ :) > > > Easier doesn't necessarily mean better. LocalStorage is certainly easier to > use than any async storage system ;-) > At least my implementation does not use any events. Proposed system of events in IndexedDB is the antipattern indeed. Exactly for the same reasons as finalizer *events* you've mentioned above - there is no guarantee that all events will be delivered to the code awaiting and relaying on them. -- Andrew Fedoniouk. http://terrainformatica.com > - Kyle > > [0] e.g. > http://blogs.msdn.com/b/oldnewthing/archive/2010/08/09/10047586.aspx
Re: Persistent Storage vs. Database
On Fri, Mar 8, 2013 at 11:02 AM, Andrew Fedoniouk wrote: > On Thu, Mar 7, 2013 at 10:36 PM, Kyle Huey wrote: > > On Thu, Mar 7, 2013 at 10:20 PM, Andrew Fedoniouk > > wrote: > > > >> At least it is easier than http://www.w3.org/TR/IndexedDB/ :) > > > > > > Easier doesn't necessarily mean better. LocalStorage is certainly > easier to > > use than any async storage system ;-) > > > > At least my implementation does not use any events. Proposed > system of events in IndexedDB is the antipattern indeed. Exactly for > the same reasons as finalizer *events* you've mentioned above - there > is no guarantee that all events will be delivered to the code awaiting > and relaying on them. > That's not true at all. If you don't understand the difference between finalizers and events you're not going to be able to make a very informed criticism of IndexedDB. - Kyle
Re: Persistent Storage vs. Database
On Thu, Mar 7, 2013 at 10:20 PM, Andrew Fedoniouk wrote: > Physical commit (write) of objects to storage happens on either > a) GC cycle or b) on explicit storage.commit() call or on c) VM shutdown. > Persisting data off a GC cycle (via finalizers or something else) is a pretty well known antipattern.[0] At least it is easier than http://www.w3.org/TR/IndexedDB/ :) Easier doesn't necessarily mean better. LocalStorage is certainly easier to use than any async storage system ;-) - Kyle [0] e.g. http://blogs.msdn.com/b/oldnewthing/archive/2010/08/09/10047586.aspx
Persistent Storage vs. Database
I am not sure if my approach in Sciter/TIScript for data persistence is applicable to JS engines used in browsers... but I'll try to explain it with the hope that its idea can be useful at least to some extent. The whole data persistence in TIScript [1,2] is defined by two objects [classes]: - Storage - the data storage, and - Storage.Index - index, persistable ordered key/value map. Each Storage instance has property named `root`. Any object that is reachable from this root is persistent - will survive VM shutdown. Example: var storage = ...; storage.root = { one: 1, two: [ 1,2,3 ], three: { a:"A", b: "B", c: "C" } }; this will initialize persistent storage by the structure above. After this, in current or next script session, access to the storage.root will return that object. From that object all its descendants are reachable as usual. Physical commit (write) of objects to storage happens on either a) GC cycle or b) on explicit storage.commit() call or on c) VM shutdown. Objects from storage are loaded on demand - when they accessed. Therefore the storage provides transparent (for the JS programmer) set of operations. Data access to persistent objects is made exactly in the same way as to other objects in script heap. The only limitation: not all objects in script heap can be persistent this way. Only "JSON" subset (array,object, numeric, bool, string and null), Dates and Indexes. For obvious reasons. This persistence schema uses minimum number of abstractions and has full set of operations needed for accessing and storing structured data. At least it is easier than http://www.w3.org/TR/IndexedDB/ :) My pardon if all above is not exactly in mainstream. Persistence in TIScript is based on Konstantin Knizhnik's DyBase [3]. [1] TIScript high level definition: http://www.codeproject.com/Articles/33662/TIScript-language-a-gentle-extension-of-JavaScript [2] TIScript source code: https://code.google.com/p/tiscript/ [3] DyBase http://www.garret.ru/dybase.html -- Andrew Fedoniouk. http://terrainformatica.com