Re: IDBObjectStore/IDBIndex.exists(key)
On Sat, Jun 21, 2014 at 9:45 PM, ben turner bent.mozi...@gmail.com wrote: I think this sounds like a fine idea. -Ben Turner On Sat, Jun 21, 2014 at 5:39 PM, Jonas Sicking jo...@sicking.cc wrote: Hi all, I found an old email with notes about features that we might want to put in v2. Almost all of them was recently brought up in the recent threads about IDBv2. However there was one thing on the list that I haven't seen brought up. It might be a nice perf improvement to add support for a IDBObjectStore/IDBIndex.exists(key) function. This sounds redundant with count(). Was count() added to the spec after that note was written? (count() seems to be a relatively late addition, given that it occurs last in the IDLs) This would require less IO and less object creation than simply using .get(). It is probably particularly useful when doing a filtering join operation between two indexes/object stores. But it is probably useful other times too. Is this something that others think would be useful? / Jonas
Re: IDBObjectStore/IDBIndex.exists(key)
On Sat, Jun 21, 2014 at 7:02 PM, Marc Fawzi marc.fa...@gmail.com wrote: I think the same thought pattern can be applied elsewhere in the API design for v2. Consider the scenario of trying to find whether a given index exists or not (upon upgradeneeded). For now, we have to write noisy code like [].slice.call(objectStore.indexNames()).indexOf(someIndex) Why couldn't indexNames be an array? Technically, objectStoreNames() and indexNames() are specified to return a DOMStringList, which is an array-like with a contains() method, so you can write: objectStore.indexNames().contains(someIndex) ... however, DOMStringList fell out of vogue pretty much as soon as it was created. ISTR that Firefox just returns a JS Array here. But there's been talk about adding contains() to Array.prototype: http://esdiscuss.org/topic/array-prototype-contains ... which seems likely for ES7 in some form or another. Ideally we'd add Array.prototype.contains() and then indexNames().contains() works cross-browser (and we can delete DOMStringList from Chrome!). and dare we ask for this to either return the index or null: objectStore.index(someIndex) ? I understand the argument for throwing an error here but I think a silent null is more practical. I don't think we can change that for compat reasons. Aside: The API design definitely assumes you know what you're doing, e.g. introspecting a database schema is abnormal, since you (as the site author) should of course know exactly what the schema is, except during upgrades when you have the monotonically increasing version number to reason against. Of course, the minute you build an abstraction layer on top of the IDB API it's no longer abnormal and the API feels clunky. So yes, anything that makes the API easier to consume would be terrific. More thoughts welcome! I gave specific counters to your two concrete suggestions above, but please don't let that stop you. These rough edges in the API should be smoothed out! I had a very hard time until I saw the light. There's some solid thought behind the existing API, but it's also not designed for web development in terms of how it implements a good idea, not wether or not the idea is good. Sorry for the mini rant. No need to apologize, it's appreciated. v2 thinking needs to include making the API more powerful, more usable, and more approachable. It took me a little too long to get this app done which is my first time using IndexedDB (with a half broken debugger in Chrome): https://github.com/idibidiart/AllSeeingEye On Sat, Jun 21, 2014 at 5:39 PM, Jonas Sicking jo...@sicking.cc wrote: Hi all, I found an old email with notes about features that we might want to put in v2. Almost all of them was recently brought up in the recent threads about IDBv2. However there was one thing on the list that I haven't seen brought up. It might be a nice perf improvement to add support for a IDBObjectStore/IDBIndex.exists(key) function. This would require less IO and less object creation than simply using .get(). It is probably particularly useful when doing a filtering join operation between two indexes/object stores. But it is probably useful other times too. Is this something that others think would be useful? / Jonas
Re: IDBObjectStore/IDBIndex.exists(key)
On Mon, Jun 23, 2014 at 9:59 AM, Joshua Bell jsb...@google.com wrote: On Sat, Jun 21, 2014 at 9:45 PM, ben turner bent.mozi...@gmail.com wrote: I think this sounds like a fine idea. -Ben Turner On Sat, Jun 21, 2014 at 5:39 PM, Jonas Sicking jo...@sicking.cc wrote: Hi all, I found an old email with notes about features that we might want to put in v2. Almost all of them was recently brought up in the recent threads about IDBv2. However there was one thing on the list that I haven't seen brought up. It might be a nice perf improvement to add support for a IDBObjectStore/IDBIndex.exists(key) function. This sounds redundant with count(). Was count() added to the spec after that note was written? (count() seems to be a relatively late addition, given that it occurs last in the IDLs) Hmm.. good point. It is indeed very possible that I wrote that note before we had count(). There is a small performance difference between them though when applied to indexes. Indexes could have multiple entries with the same key (but different primaryKey), in which case count() would have to find all such entries, whereas exists() would only need to find the first. But most of the time count() probably does well enough. / Jonas
Re: IDBObjectStore/IDBIndex.exists(key)
Joshua, you're on, and I'll be happy to make suggestions once I've thought them through... At least to some extent :) Jonas, There is a small performance difference between them though when applied to indexes. Indexes could have multiple entries with the same key (but different primaryKey), in which case count() would have to find all such entries, whereas exists() would only need to find the first. Isn't it also possible to open cursor on an index with IDBKeyRange.only(key) ? Wouldn't that confirm/deny existence and you may abandon the cursor/transaction after it. Having said that, and speaking naively here, a synchronous .exists() or .contains() would be useful as existence checks shouldn't have to be exclusively asynchronous as that complicates how we'd write: if this exists and that other thing doesn't exists then do xyz However, a good Promises implementation (see Dexie.js as a potential inspiration/candidate for such) may allow us to work with such asynchronous existence checks in a way that keeps the code complexity manageable. Take all this with a grain of salt. I'm learning how to use IDB as I go and these are my mental notes during this process, so not always making total sense. Marc Sent from my iPhone On Jun 23, 2014, at 12:21 PM, Jonas Sicking jo...@sicking.cc wrote: There is a small performance difference between them though when applied to indexes. Indexes could have multiple entries with the same key (but different primaryKey), in which case count() would have to find all such entries, whereas exists() would only need to find the first.
Re: IDBObjectStore/IDBIndex.exists(key)
On Mon, Jun 23, 2014 at 1:03 PM, Marc Fawzi marc.fa...@gmail.com wrote: Having said that, and speaking naively here, a synchronous .exists() or .contains() would be useful as existence checks shouldn't have to be exclusively asynchronous as that complicates how we'd write: if this exists and that other thing doesn't exists then do xyz Note that the .contains() discussion is entirely separate from the .exists() discussion. I.e. your subject is entirely off-topic to this thread. The .exists() function I proposed lives on IDBObjectStore and IDBIndex and is an asynchronous database operation. The .contains() function that you are talking about lives on an array-like object and just does some in-memory tests which means that it's synchronous. So the two are completely unrelated. / Jonas
Re: IDBObjectStore/IDBIndex.exists(key)
No, I was suggesting .exists() can be synchronous to make it useful I referred to it as .contains() too so sorry if that conflated them for you but it has nothing to do with the .contains Joshua was talking about. In short, an asynchronous .exists() as you proposed does seem redundant But I was wondering what about a synchronous .exists() (the same proposal you had but synchronous as opposed to asynchronous) Makes any sense? Sent from my iPhone On Jun 23, 2014, at 1:28 PM, Jonas Sicking jo...@sicking.cc wrote: On Mon, Jun 23, 2014 at 1:03 PM, Marc Fawzi marc.fa...@gmail.com wrote: Having said that, and speaking naively here, a synchronous .exists() or .contains() would be useful as existence checks shouldn't have to be exclusively asynchronous as that complicates how we'd write: if this exists and that other thing doesn't exists then do xyz Note that the .contains() discussion is entirely separate from the .exists() discussion. I.e. your subject is entirely off-topic to this thread. The .exists() function I proposed lives on IDBObjectStore and IDBIndex and is an asynchronous database operation. The .contains() function that you are talking about lives on an array-like object and just does some in-memory tests which means that it's synchronous. So the two are completely unrelated. / Jonas
Re: IDBObjectStore/IDBIndex.exists(key)
On Mon, Jun 23, 2014 at 1:38 PM, Marc Fawzi marc.fa...@gmail.com wrote: No, I was suggesting .exists() can be synchronous to make it useful I referred to it as .contains() too so sorry if that conflated them for you but it has nothing to do with the .contains Joshua was talking about. In short, an asynchronous .exists() as you proposed does seem redundant But I was wondering what about a synchronous .exists() (the same proposal you had but synchronous as opposed to asynchronous) Makes any sense? We can't make any database operations synchronous since that require synchronous IO which is really bad for perf. It's also different from all other database operations. So if that's your request then the answer is definitely no. / Jonas Sent from my iPhone On Jun 23, 2014, at 1:28 PM, Jonas Sicking jo...@sicking.cc wrote: On Mon, Jun 23, 2014 at 1:03 PM, Marc Fawzi marc.fa...@gmail.com wrote: Having said that, and speaking naively here, a synchronous .exists() or .contains() would be useful as existence checks shouldn't have to be exclusively asynchronous as that complicates how we'd write: if this exists and that other thing doesn't exists then do xyz Note that the .contains() discussion is entirely separate from the .exists() discussion. I.e. your subject is entirely off-topic to this thread. The .exists() function I proposed lives on IDBObjectStore and IDBIndex and is an asynchronous database operation. The .contains() function that you are talking about lives on an array-like object and just does some in-memory tests which means that it's synchronous. So the two are completely unrelated. / Jonas
Re: IDBObjectStore/IDBIndex.exists(key)
On Mon, Jun 23, 2014 at 1:38 PM, Marc Fawzi marc.fa...@gmail.com wrote: No, I was suggesting .exists() can be synchronous to make it useful I referred to it as .contains() too so sorry if that conflated them for you but it has nothing to do with the .contains Joshua was talking about. In short, an asynchronous .exists() as you proposed does seem redundant But I was wondering what about a synchronous .exists() (the same proposal you had but synchronous as opposed to asynchronous) We can do synchronous tests against the schema as it is feasible for implementations to maintain a copy of the current schema for an open connection in memory in the same thread/process as script. (Or at least, no implementer has complained.) Actually hitting the backing store to look up a particular value may require a thread/process hop, so must be an asynchronous operation. Actually pulling the *data* across and decoding it is an added expense, which is why count(), the proposed exists(), and key cursors exist as optimizations over get() and regular cursors. Makes any sense? Sent from my iPhone On Jun 23, 2014, at 1:28 PM, Jonas Sicking jo...@sicking.cc wrote: On Mon, Jun 23, 2014 at 1:03 PM, Marc Fawzi marc.fa...@gmail.com wrote: Having said that, and speaking naively here, a synchronous .exists() or .contains() would be useful as existence checks shouldn't have to be exclusively asynchronous as that complicates how we'd write: if this exists and that other thing doesn't exists then do xyz Note that the .contains() discussion is entirely separate from the .exists() discussion. I.e. your subject is entirely off-topic to this thread. The .exists() function I proposed lives on IDBObjectStore and IDBIndex and is an asynchronous database operation. The .contains() function that you are talking about lives on an array-like object and just does some in-memory tests which means that it's synchronous. So the two are completely unrelated. / Jonas
Re: IDBObjectStore/IDBIndex.exists(key)
We can do synchronous tests against the schema as it is feasible for implementations to maintain a copy of the current schema for an open connection in memory in the same thread/process as script. (Or at least, no implementer has complained.) Oh cool. So I could have a 3rd party component in my app that can then test the schema directly and run certain functions only if some combination of conditions are met and having those test be synchronous makes the tests simple. For example, does xyz object store exist and does it have the right indices. If so then the component would run. Else it wouldn't. I get the thing about the cost of looking up a value and why that has to be asynchronous. Jonas, I have no request per se, just super curious about the rationalizations around v2 APIs, so I might have questions or curiosity that are expressed indirectly as suggestion. Sometimes I say naive things and other times the suggestions may be directly useful or bring up some other thoughts. I'll try to minimize the confusion. But do look out there: almost every front end developer I've talked to things IndexedDB is less than usable and the library makers haven't yet provided something both truly solid and worth throwing the native APIs for, so I'm trying to understand things better for myself so I can help build a better library but would rather have the IDB native API come to a point in its evolution where front end developers would be able to consume it directly with no intervening layer, and that's why asking and making sometimes dumb and sometimes useful suggestions, to try and understand how the IDB designers think. If that actually makes any sense. Sent from my iPhone On Jun 23, 2014, at 2:22 PM, Joshua Bell jsb...@google.com wrote: We can do synchronous tests against the schema as it is feasible for implementations to maintain a copy of the current schema for an open connection in memory in the same thread/process as script. (Or at least, no implementer has complained.)
Re: IDBObjectStore/IDBIndex.exists(key)
I think the same thought pattern can be applied elsewhere in the API design for v2. Consider the scenario of trying to find whether a given index exists or not (upon upgradeneeded). For now, we have to write noisy code like [].slice.call(objectStore.indexNames()).indexOf(someIndex) Why couldn't indexNames be an array? and dare we ask for this to either return the index or null: objectStore.index(someIndex) ? I understand the argument for throwing an error here but I think a silent null is more practical. So yes, anything that makes the API easier to consume would be terrific. I had a very hard time until I saw the light. There's some solid thought behind the existing API, but it's also not designed for web development in terms of how it implements a good idea, not wether or not the idea is good. Sorry for the mini rant. It took me a little too long to get this app done which is my first time using IndexedDB (with a half broken debugger in Chrome): https://github.com/idibidiart/AllSeeingEye On Sat, Jun 21, 2014 at 5:39 PM, Jonas Sicking jo...@sicking.cc wrote: Hi all, I found an old email with notes about features that we might want to put in v2. Almost all of them was recently brought up in the recent threads about IDBv2. However there was one thing on the list that I haven't seen brought up. It might be a nice perf improvement to add support for a IDBObjectStore/IDBIndex.exists(key) function. This would require less IO and less object creation than simply using .get(). It is probably particularly useful when doing a filtering join operation between two indexes/object stores. But it is probably useful other times too. Is this something that others think would be useful? / Jonas
Re: IDBObjectStore/IDBIndex.exists(key)
I think this sounds like a fine idea. -Ben Turner On Sat, Jun 21, 2014 at 5:39 PM, Jonas Sicking jo...@sicking.cc wrote: Hi all, I found an old email with notes about features that we might want to put in v2. Almost all of them was recently brought up in the recent threads about IDBv2. However there was one thing on the list that I haven't seen brought up. It might be a nice perf improvement to add support for a IDBObjectStore/IDBIndex.exists(key) function. This would require less IO and less object creation than simply using .get(). It is probably particularly useful when doing a filtering join operation between two indexes/object stores. But it is probably useful other times too. Is this something that others think would be useful? / Jonas
