Re: [IndexedDB] KeyPaths and missing properties.
On Fri, May 21, 2010 at 12:39 AM, Jonas Sicking jo...@sicking.cc wrote: So you'd have to pass in the javascript expression as a string. This certainly works but is more than a little ugly. It also complicates the implementation a good bit since it now has to include a javascript engine. Not a huge issue given that all browsers has one anyway, but feels a bit iffy. Hm. I'm looking at the spec, and I can't find where (if anywhere) it talks about what's allowed as a key path. I guess I had assumed that it was essentially javascript (or maybe some subset) and that we'd at least need to be parsing it anyway. If we don't adopt this idea because we're worried about depending on javascript, then we should spec out the keyPath syntax pretty precisely to ensure we don't have such a dependency. snip I think the intent of the current syntax is that it's a '.' separated list of property names. But yes, it definitely needs to be precisely defined. I'll create a bug for this and the other items we've already decided on in this thread. I still think it's an interesting idea, though I'm not sold on it. Especially for the first version of the spec. It seems like there would be a lot of edge cases to define here. First of all, how is the value passed in to this expression? Do we say that it's available through some value variable? So that if you want to index on the foo property, you pass in an expression like value.foo? Or do we want the value to be the global object, so that if you wanted to index on the foo property the expression would simply be foo? Since we're already talking about requiring that data being inserted into objectStores with a keyPath (for its primary key or in one of its indexes), setting it as the global object seems reasonable. And it matches what's currently specced for the simple 1 entityStore entry to 1 index entry (per index) case. You also have to specify how the value is returned. I.e. does the expression need to end with 'return X'? Or do we index on the result of the value returned from the last executed expression? Also, I think that if the value is the global object, then I declaring variables inside the expression modifies the value as global variables are set on the global object. Also, what happens if the javascript expression modifies the value? Does the implementation have to clone the value before calling each index expression? In order of how much I like the idea: 1) In an ideal world, we'd spec it to be read only, but I'm not sure if most JS engines have an easy way to do something like that. 2) Another possibility is to make the order in which indexes are processed deterministic. That way, if someone does modify it, it'll at least be consistent. 3) Cloning is another possibility, but it seems like it'd have a performance impact. Maybe optimized implementations could copy-on-write it, though? I think 1 and 3 suffers from the same problem of not being something that JS engines usually need to do, and so likely isn't implemented. In both cases you have to be able to mark an object graph such that you take special action if it's modified. And for 2, do you do the structured clone before or after you run the keyPath expressions? Not until after you've created the structured clone will you know if the value can even be stored. In short, I'd like to see a comprehensive proposal :) Then I could take that to the JS team and ask if it's implementable. Per other threads, it looks like we should make sure IndexedDB does not depend on JavaScript, so please ignore my proposal. On Thu, May 20, 2010 at 4:56 PM, Jonas Sicking jo...@sicking.cc wrote: On Thu, May 20, 2010 at 7:55 AM, Andrei Popescu andr...@google.com wrote: Hi, On Thu, May 20, 2010 at 10:47 AM, Jeremy Orlow jor...@chromium.org wrote: On Thu, May 20, 2010 at 1:24 AM, Jonas Sicking jo...@sicking.cc wrote: It seems like there would be a lot of edge cases to define here. First of all, how is the value passed in to this expression? Do we say that it's available through some value variable? So that if you want to index on the foo property, you pass in an expression like value.foo? Or do we want the value to be the global object, so that if you wanted to index on the foo property the expression would simply be foo? Since we're already talking about requiring that data being inserted into objectStores with a keyPath (for its primary key or in one of its indexes), setting it as the global object seems reasonable. And it matches what's currently specced for the simple 1 entityStore entry to 1 index entry (per index) case. Also, what happens if the javascript expression modifies the value? Does the implementation have to clone the value before calling each index expression? In order of how much I like the idea: 1) In an ideal world, we'd spec it to be read
Re: [IndexedDB] KeyPaths and missing properties.
On Thu, May 20, 2010 at 1:24 AM, Jonas Sicking jo...@sicking.cc wrote:
On Wed, May 19, 2010 at 4:58 PM, Jeremy Orlow jor...@chromium.org wrote:
On Wed, May 19, 2010 at 9:38 PM, Jonas Sicking jo...@sicking.cc wrote:
On Wed, May 19, 2010 at 2:36 AM, Jeremy Orlow jor...@chromium.org
wrote:
Interesting you'd bring this up. Andrei and I were just looking at
indexes
as specced and wondered whether it still makes sense to allow indexes
to
not
have a keyPath.
I think so. Consider for example a objectStore that contains entries
like:
{ name: Elvis Presley, born: 1935-1-8, death: 1977-8-16 }
But index on how old the person was when he/she died. I guess you
could require that people store a property containing the age, however
it seems unfortunate to require modifying the stored data.
Worse, you might have entries like:
{ id: 5, givenName: Benny, otherNames: [Göran, Bror],
familyName: Andersson, age: 63, interest: Music }
in order to search for people, you might want to search for a name, no
matter if that is a given name, a family name, or second name.
Currently you can accomplish this by inserting 4 entries into an
index, all pointing to the same entry in the objectStore. I can't
think of a sane way to accomplish this using only keyPath.
I think the main use case here is the need to allow an unknown quantity
of
entries in the index to one entry in the objectStore. What if we allowed
keyEntry's to be any javascript expression and allow them to return
arrays
of indexible data. In this case, the expression would return an array of
otherNames + the givenName + the familyName. The keyPath already is
essentially just a javascript expression, so this doesn't seem like too
big
of a leap.
This would also support your age upon death use case. (Though I think
requiring the developer to store that age if they want it indexed isn't
too
bad of an answer either.)
Thoughts?
I think it's an interesting idea, and one that came up when we were
talking to developers way back before IndexedDB spec was started.
However then we more discussed it in the sense that libraries would be
using the technique, rather than that the API would use it.
I agree this is on the borderline between something we should leave to
libraries (for at least v1) and something we should spec now.
The problem with using a javascript expression is how do you provide
it? If you're allowed to pass in a function, that also means that
you're pulling in the full scope of that function. This obviously
doesn't work since the index generally outlives the scope and is used
in other browsing sessions and other tabs.
Sorry, I should have been more clear. I think a string (like what's
currently specced) is definitely the way to go. You're right about the
scoping issues, but what if we ran the expressions without access to the
global scope? In addition to solving this problem (index works when you
first create it, but next time you open the browser it does't) it'd also
keep developers from doing really horrible things...like doing synchronous
XHRs.
So you'd have to pass in the javascript expression as a string. This
certainly works but is more than a little ugly. It also complicates
the implementation a good bit since it now has to include a javascript
engine. Not a huge issue given that all browsers has one anyway, but
feels a bit iffy.
Hm. I'm looking at the spec, and I can't find where (if anywhere) it talks
about what's allowed as a key path. I guess I had assumed that it was
essentially javascript (or maybe some subset) and that we'd at least need to
be parsing it anyway. If we don't adopt this idea because we're worried
about depending on javascript, then we should spec out the keyPath syntax
pretty precisely to ensure we don't have such a dependency.
snip
Oh, interesting! Do you know why the structured clone algorithm is
specced
that way?
Anyway, I'd agree that disallowing it is probably the right answer.
Hmm.. it turns out I'm wrong. It does not in fact drop id property.
Ugh, that complicates serializing arrays quite a bit :(
I still think it's surprising to add properties to arrays, and so I'd
prefer to treat arrays like primitives and other non-plain-objects and
make it an error.
I'm fine with this. I can't think of any use cases it'd get in the way
with.
On Thu, May 20, 2010 at 1:39 AM, Jonas Sicking jo...@sicking.cc wrote:
I still think it's an interesting idea, though I'm not sold on it.
Especially for the first version of the spec.
It seems like there would be a lot of edge cases to define here. First
of all, how is the value passed in to this expression? Do we say that
it's available through some value variable? So that if you want to
index on the foo property, you pass in an expression like
value.foo? Or do we want the value to be the global object, so that
if you wanted to index on the foo property the
Re: [IndexedDB] KeyPaths and missing properties.
Hi,
On Thu, May 20, 2010 at 10:47 AM, Jeremy Orlow jor...@chromium.org wrote:
On Thu, May 20, 2010 at 1:24 AM, Jonas Sicking jo...@sicking.cc wrote:
It seems like there would be a lot of edge cases to define here. First
of all, how is the value passed in to this expression? Do we say that
it's available through some value variable? So that if you want to
index on the foo property, you pass in an expression like
value.foo? Or do we want the value to be the global object, so that
if you wanted to index on the foo property the expression would
simply be foo?
Since we're already talking about requiring that data being inserted into
objectStores with a keyPath (for its primary key or in one of its indexes),
setting it as the global object seems reasonable. And it matches what's
currently specced for the simple 1 entityStore entry to 1 index entry (per
index) case.
Also, what happens if the javascript expression modifies the value?
Does the implementation have to clone the value before calling each
index expression?
In order of how much I like the idea:
1) In an ideal world, we'd spec it to be read only, but I'm not sure if most
JS engines have an easy way to do something like that.
2) Another possibility is to make the order in which indexes are processed
deterministic. That way, if someone does modify it, it'll at least
be consistent.
3) Cloning is another possibility, but it seems like it'd have a performance
impact. Maybe optimized implementations could copy-on-write it, though?
While it's true that allowing the keyPath to be any javascript
expression would be very elegant and flexible (although probably quite
difficult to explain in the spec), maybe it's worth considering a
simpler solution? For instance, could the keyPath be simply an array
of strings, with each string denoting the name of a property of the
objects in the store. So in Jonas' example:
{ id: 5, givenName: Benny, otherNames: [Göran, Bror],
familyName: Andersson, age: 63, interest: Music }
The keyPath could be set to
[givenName, otherNames, familyName].
The indexable data for this record would therefore be {Benny,
Göran, Bror, Andersson}.
Thanks,
Andrei
Re: [IndexedDB] KeyPaths and missing properties.
On Thu, May 20, 2010 at 3:55 PM, Andrei Popescu andr...@google.com wrote:
Hi,
On Thu, May 20, 2010 at 10:47 AM, Jeremy Orlow jor...@chromium.org
wrote:
On Thu, May 20, 2010 at 1:24 AM, Jonas Sicking jo...@sicking.cc wrote:
It seems like there would be a lot of edge cases to define here. First
of all, how is the value passed in to this expression? Do we say that
it's available through some value variable? So that if you want to
index on the foo property, you pass in an expression like
value.foo? Or do we want the value to be the global object, so that
if you wanted to index on the foo property the expression would
simply be foo?
Since we're already talking about requiring that data being inserted into
objectStores with a keyPath (for its primary key or in one of its
indexes),
setting it as the global object seems reasonable. And it matches what's
currently specced for the simple 1 entityStore entry to 1 index entry
(per
index) case.
Also, what happens if the javascript expression modifies the value?
Does the implementation have to clone the value before calling each
index expression?
In order of how much I like the idea:
1) In an ideal world, we'd spec it to be read only, but I'm not sure if
most
JS engines have an easy way to do something like that.
2) Another possibility is to make the order in which indexes are
processed
deterministic. That way, if someone does modify it, it'll at least
be consistent.
3) Cloning is another possibility, but it seems like it'd have a
performance
impact. Maybe optimized implementations could copy-on-write it, though?
While it's true that allowing the keyPath to be any javascript
expression would be very elegant and flexible (although probably quite
difficult to explain in the spec), maybe it's worth considering a
simpler solution? For instance, could the keyPath be simply an array
of strings, with each string denoting the name of a property of the
objects in the store. So in Jonas' example:
{ id: 5, givenName: Benny, otherNames: [Göran, Bror],
familyName: Andersson, age: 63, interest: Music }
The keyPath could be set to
[givenName, otherNames, familyName].
The indexable data for this record would therefore be {Benny,
Göran, Bror, Andersson}.
If we're OK making the script depend on JavaScript, I still like my
suggestion a bit better. But assuming we're not, I think this would be a
fine way to solve the problem.
Btw, if we're not depending on JavaScript, then whether or not we change any
of this, we'll need to essentially spec our own language for the keyPath.
(As far as I can tell, there's no normative language for the parsing of
keyPath strings in the current spec. And if we're trying not to depend on
JavaScript, we'll basically have to explain the parsing and how it maps to
objects in a completely language independent fashion.)
J
Re: [IndexedDB] KeyPaths and missing properties.
On Thu, May 20, 2010 at 7:55 AM, Andrei Popescu andr...@google.com wrote:
Hi,
On Thu, May 20, 2010 at 10:47 AM, Jeremy Orlow jor...@chromium.org wrote:
On Thu, May 20, 2010 at 1:24 AM, Jonas Sicking jo...@sicking.cc wrote:
It seems like there would be a lot of edge cases to define here. First
of all, how is the value passed in to this expression? Do we say that
it's available through some value variable? So that if you want to
index on the foo property, you pass in an expression like
value.foo? Or do we want the value to be the global object, so that
if you wanted to index on the foo property the expression would
simply be foo?
Since we're already talking about requiring that data being inserted into
objectStores with a keyPath (for its primary key or in one of its indexes),
setting it as the global object seems reasonable. And it matches what's
currently specced for the simple 1 entityStore entry to 1 index entry (per
index) case.
Also, what happens if the javascript expression modifies the value?
Does the implementation have to clone the value before calling each
index expression?
In order of how much I like the idea:
1) In an ideal world, we'd spec it to be read only, but I'm not sure if most
JS engines have an easy way to do something like that.
2) Another possibility is to make the order in which indexes are processed
deterministic. That way, if someone does modify it, it'll at least
be consistent.
3) Cloning is another possibility, but it seems like it'd have a performance
impact. Maybe optimized implementations could copy-on-write it, though?
While it's true that allowing the keyPath to be any javascript
expression would be very elegant and flexible (although probably quite
difficult to explain in the spec), maybe it's worth considering a
simpler solution? For instance, could the keyPath be simply an array
of strings, with each string denoting the name of a property of the
objects in the store. So in Jonas' example:
{ id: 5, givenName: Benny, otherNames: [Göran, Bror],
familyName: Andersson, age: 63, interest: Music }
The keyPath could be set to
[givenName, otherNames, familyName].
The indexable data for this record would therefore be {Benny,
Göran, Bror, Andersson}.
This wouldn't solve the case when the key is the result of a
calculation, such as my age-at-time-of-death example. Consider also
wanting to store objects like
{ id: 5, name: Benny Andersson, address: 1 Ohai Street\n Wahiawa, HI 96786 }
but wanting to index on the first name or on the state in the address.
/ Jonas
Re: [IndexedDB] KeyPaths and missing properties.
So you'd have to pass in the javascript expression as a string. This certainly works but is more than a little ugly. It also complicates the implementation a good bit since it now has to include a javascript engine. Not a huge issue given that all browsers has one anyway, but feels a bit iffy. Hm. I'm looking at the spec, and I can't find where (if anywhere) it talks about what's allowed as a key path. I guess I had assumed that it was essentially javascript (or maybe some subset) and that we'd at least need to be parsing it anyway. If we don't adopt this idea because we're worried about depending on javascript, then we should spec out the keyPath syntax pretty precisely to ensure we don't have such a dependency. snip I think the intent of the current syntax is that it's a '.' separated list of property names. But yes, it definitely needs to be precisely defined. I still think it's an interesting idea, though I'm not sold on it. Especially for the first version of the spec. It seems like there would be a lot of edge cases to define here. First of all, how is the value passed in to this expression? Do we say that it's available through some value variable? So that if you want to index on the foo property, you pass in an expression like value.foo? Or do we want the value to be the global object, so that if you wanted to index on the foo property the expression would simply be foo? Since we're already talking about requiring that data being inserted into objectStores with a keyPath (for its primary key or in one of its indexes), setting it as the global object seems reasonable. And it matches what's currently specced for the simple 1 entityStore entry to 1 index entry (per index) case. You also have to specify how the value is returned. I.e. does the expression need to end with 'return X'? Or do we index on the result of the value returned from the last executed expression? Also, I think that if the value is the global object, then I declaring variables inside the expression modifies the value as global variables are set on the global object. Also, what happens if the javascript expression modifies the value? Does the implementation have to clone the value before calling each index expression? In order of how much I like the idea: 1) In an ideal world, we'd spec it to be read only, but I'm not sure if most JS engines have an easy way to do something like that. 2) Another possibility is to make the order in which indexes are processed deterministic. That way, if someone does modify it, it'll at least be consistent. 3) Cloning is another possibility, but it seems like it'd have a performance impact. Maybe optimized implementations could copy-on-write it, though? I think 1 and 3 suffers from the same problem of not being something that JS engines usually need to do, and so likely isn't implemented. In both cases you have to be able to mark an object graph such that you take special action if it's modified. And for 2, do you do the structured clone before or after you run the keyPath expressions? Not until after you've created the structured clone will you know if the value can even be stored. In short, I'd like to see a comprehensive proposal :) Then I could take that to the JS team and ask if it's implementable. / Jonas
Re: [IndexedDB] KeyPaths and missing properties.
Interesting you'd bring this up. Andrei and I were just looking at indexes
as specced and wondered whether it still makes sense to allow indexes to not
have a keyPath. And, if so, whether we should tie insertion into the
objectStore to insertion to the index. The main reason to make such changes
would be to enforce a notion of database consistency. So I guess the
fundamental question here is whether there should be a 1:1 mapping between
entries in an objectStore's indexes and the objectStore itself.
On Wed, May 19, 2010 at 3:06 AM, Jonas Sicking jo...@sicking.cc wrote:
Hi IndexedDB fans!
So another issue that's come up here. This question is orthogonal to
the other discussed issues and applies equally to both the existing
API, and the mozilla proposed API, so I figured it was safe to raise
right away.
What should happen if you insert a object into an objectStore which
has an index attached to it, but the property defined by the keyPath
in the index does not exist? Consider a database with an objectStore
called people with keyPath name. The objectStore has an
auto-populated index named phone numbers with keyPath phone. The
objectStore is initially empty.
What should happen if you attempt to store an object with value {
name: Benny, email: be...@sweden.com } ?
There are three possible behaviors I can think of:
1. The storing operation should fail and an error event should be fired.
2. The storing operation should succeed, but no records should be
added to the phone index.
3. The storing operation should succeed and a record should be added
to the phone index using null as key.
Which one is correct behavior? What if the phone numbers index is
flagged as unique?
I definitely think there is a use case for allowing records to be
stored even if there are indexes on missing properties. Consider for
example a objectStore containing people where not everyone has a known
phone number. It still seems useful to be able to search based on
phone number in this objectStore. To satisfy this use case either
solution 2 or 3 would work. 3 would additionally allow searching for
everyone without a phone number, though I'm not sure how important
that is. On the other hand 2 would be more space effective in that
only relevant records would be stored.
Possibly we'll want to add a 'required' flag to the createIndex
function specifying if the index should cause behavior 1 or behavior
2. So far I'm not a big fan of 3, but I'm all ears for input.
If we went with (1) then it seems as though developers could simply insert
items with .phone=null to get around the limitation.
3 seems like it'd be error prone and somewhat confusing.
If we decide there should be a 1:1 mapping then it seems we're choosing
between 1 and 3. And if not, then I think 2 is the clear answer.
The same question arises when an index is added to an objectStore
which has existing values that lack the property that the index uses
as keyPath. Whichever solution we go with above I think should be used
here too.
Agreed.
There is a similar issue if an objectStore uses in-line keys. Consider
a database with a objectStore called people with keyPath name and
no key generator. What if someone inserts an object with the value {
email: be...@sweden.com }? I would think that this should be an
error, but it doesn't appear that any of the steps in Object Store
Storage steps says to create an error.
This is probably just an omission in the spec. If so, someone should file
an issue on it.
Finally, there is the issue of what should happen if someone tries to
insert a number as a value into an objectStore with a keyPath and a
key generator. Consider the objectStore albums with keyPath id and
a key generator enabled. What should happen if someone tries to insert
the value 9 into this table? If an object was inserted, then this
would result in that objects 'id' property being set to the next
generated value from the generator, however it doesn't really make
sense to set a property on a number. Same question obviously applies
if the stored value is a boolean, a string, or any other primitive JS
type, or types such as regexps and File objects. I think that in all
these cases the operation should fail.
Agreed.
The question also applies if the stored value is an array. Technically
Javascript allows setting values on arrays, however the structured
clone algorithm drops these values, and so it seems to add needless
complexity for the implementation if the only time you need to
serialize/deserialize arrays with non-integer properties is in this
one case. Additionally, the id would be lost when the value is read as
that is done using the structured clone algorithm too.
Hm. I'm having trouble following this case. Any chance you could explain
it further?
Thanks!
J
Re: [IndexedDB] KeyPaths and missing properties.
On Wed, May 19, 2010 at 2:36 AM, Jeremy Orlow jor...@chromium.org wrote:
Interesting you'd bring this up. Andrei and I were just looking at indexes
as specced and wondered whether it still makes sense to allow indexes to not
have a keyPath.
I think so. Consider for example a objectStore that contains entries like:
{ name: Elvis Presley, born: 1935-1-8, death: 1977-8-16 }
But index on how old the person was when he/she died. I guess you
could require that people store a property containing the age, however
it seems unfortunate to require modifying the stored data.
Worse, you might have entries like:
{ id: 5, givenName: Benny, otherNames: [Göran, Bror],
familyName: Andersson, age: 63, interest: Music }
in order to search for people, you might want to search for a name, no
matter if that is a given name, a family name, or second name.
Currently you can accomplish this by inserting 4 entries into an
index, all pointing to the same entry in the objectStore. I can't
think of a sane way to accomplish this using only keyPath.
And, if so, whether we should tie insertion into the
objectStore to insertion to the index. The main reason to make such changes
would be to enforce a notion of database consistency.
You mean so that you would have to pass in index keys to the
insert/modify functions? In other words, require at an API level that
indexes were always up to date?
This is an interesting idea. I can't really say I feel strongly one
way or another. I guess it depends on what the API would look like.
The nice thing about what we have now is that the API is pretty
simple.
So I guess the
fundamental question here is whether there should be a 1:1 mapping between
entries in an objectStore's indexes and the objectStore itself.
I'm not sure what you mean by a 1:1 mapping? Do you mean that there
are the same number of entries in the index as there are in the
objectStore?
On Wed, May 19, 2010 at 3:06 AM, Jonas Sicking jo...@sicking.cc wrote:
Hi IndexedDB fans!
So another issue that's come up here. This question is orthogonal to
the other discussed issues and applies equally to both the existing
API, and the mozilla proposed API, so I figured it was safe to raise
right away.
What should happen if you insert a object into an objectStore which
has an index attached to it, but the property defined by the keyPath
in the index does not exist? Consider a database with an objectStore
called people with keyPath name. The objectStore has an
auto-populated index named phone numbers with keyPath phone. The
objectStore is initially empty.
What should happen if you attempt to store an object with value {
name: Benny, email: be...@sweden.com } ?
There are three possible behaviors I can think of:
1. The storing operation should fail and an error event should be fired.
2. The storing operation should succeed, but no records should be
added to the phone index.
3. The storing operation should succeed and a record should be added
to the phone index using null as key.
Which one is correct behavior? What if the phone numbers index is
flagged as unique?
I definitely think there is a use case for allowing records to be
stored even if there are indexes on missing properties. Consider for
example a objectStore containing people where not everyone has a known
phone number. It still seems useful to be able to search based on
phone number in this objectStore. To satisfy this use case either
solution 2 or 3 would work. 3 would additionally allow searching for
everyone without a phone number, though I'm not sure how important
that is. On the other hand 2 would be more space effective in that
only relevant records would be stored.
Possibly we'll want to add a 'required' flag to the createIndex
function specifying if the index should cause behavior 1 or behavior
2. So far I'm not a big fan of 3, but I'm all ears for input.
If we went with (1) then it seems as though developers could simply insert
items with .phone=null to get around the limitation.
People doing that doesn't seem to really change anything. The question
would just turn into what should we do if the key is absent, null or
undefined. The same 1, 2, 3 answers still seem like possible answers.
3 seems like it'd be error prone and somewhat confusing.
If we decide there should be a 1:1 mapping then it seems we're choosing
between 1 and 3. And if not, then I think 2 is the clear answer.
I'm still not sure what the a 1:1 mapping means, so not really
following your logic here.
There is a similar issue if an objectStore uses in-line keys. Consider
a database with a objectStore called people with keyPath name and
no key generator. What if someone inserts an object with the value {
email: be...@sweden.com }? I would think that this should be an
error, but it doesn't appear that any of the steps in Object Store
Storage steps says to create an error.
This is probably just an omission in the spec. If so, someone should
Re: [IndexedDB] KeyPaths and missing properties.
On Wed, May 19, 2010 at 9:38 PM, Jonas Sicking jo...@sicking.cc wrote:
On Wed, May 19, 2010 at 2:36 AM, Jeremy Orlow jor...@chromium.org wrote:
Interesting you'd bring this up. Andrei and I were just looking at
indexes
as specced and wondered whether it still makes sense to allow indexes to
not
have a keyPath.
I think so. Consider for example a objectStore that contains entries like:
{ name: Elvis Presley, born: 1935-1-8, death: 1977-8-16 }
But index on how old the person was when he/she died. I guess you
could require that people store a property containing the age, however
it seems unfortunate to require modifying the stored data.
Worse, you might have entries like:
{ id: 5, givenName: Benny, otherNames: [Göran, Bror],
familyName: Andersson, age: 63, interest: Music }
in order to search for people, you might want to search for a name, no
matter if that is a given name, a family name, or second name.
Currently you can accomplish this by inserting 4 entries into an
index, all pointing to the same entry in the objectStore. I can't
think of a sane way to accomplish this using only keyPath.
I think the main use case here is the need to allow an unknown quantity of
entries in the index to one entry in the objectStore. What if we allowed
keyEntry's to be any javascript expression and allow them to return arrays
of indexible data. In this case, the expression would return an array of
otherNames + the givenName + the familyName. The keyPath already is
essentially just a javascript expression, so this doesn't seem like too big
of a leap.
This would also support your age upon death use case. (Though I think
requiring the developer to store that age if they want it indexed isn't too
bad of an answer either.)
Thoughts?
And, if so, whether we should tie insertion into the
objectStore to insertion to the index. The main reason to make such
changes
would be to enforce a notion of database consistency.
You mean so that you would have to pass in index keys to the
insert/modify functions? In other words, require at an API level that
indexes were always up to date?
This is an interesting idea. I can't really say I feel strongly one
way or another. I guess it depends on what the API would look like.
The nice thing about what we have now is that the API is pretty
simple.
One possible way to implement it: add/modify/addOrModify could have an
optional parameter that takes an associative array. For example, {age: 63,
interest: music} if you were indexing on age and interest.
That said, I think I like my above proposal better.
So I guess the
fundamental question here is whether there should be a 1:1 mapping
between
entries in an objectStore's indexes and the objectStore itself.
I'm not sure what you mean by a 1:1 mapping? Do you mean that there
are the same number of entries in the index as there are in the
objectStore?
Yes. But I think your indexing on multiple names use case may have already
sold me on supporting something other than just 1:1 mappings.
On Wed, May 19, 2010 at 3:06 AM, Jonas Sicking jo...@sicking.cc
wrote:
Hi IndexedDB fans!
So another issue that's come up here. This question is orthogonal to
the other discussed issues and applies equally to both the existing
API, and the mozilla proposed API, so I figured it was safe to raise
right away.
What should happen if you insert a object into an objectStore which
has an index attached to it, but the property defined by the keyPath
in the index does not exist? Consider a database with an objectStore
called people with keyPath name. The objectStore has an
auto-populated index named phone numbers with keyPath phone. The
objectStore is initially empty.
What should happen if you attempt to store an object with value {
name: Benny, email: be...@sweden.com } ?
There are three possible behaviors I can think of:
1. The storing operation should fail and an error event should be fired.
2. The storing operation should succeed, but no records should be
added to the phone index.
3. The storing operation should succeed and a record should be added
to the phone index using null as key.
Which one is correct behavior? What if the phone numbers index is
flagged as unique?
I definitely think there is a use case for allowing records to be
stored even if there are indexes on missing properties. Consider for
example a objectStore containing people where not everyone has a known
phone number. It still seems useful to be able to search based on
phone number in this objectStore. To satisfy this use case either
solution 2 or 3 would work. 3 would additionally allow searching for
everyone without a phone number, though I'm not sure how important
that is. On the other hand 2 would be more space effective in that
only relevant records would be stored.
Possibly we'll want to add a 'required' flag to the createIndex
function specifying if
Re: [IndexedDB] KeyPaths and missing properties.
On Wed, May 19, 2010 at 4:58 PM, Jeremy Orlow jor...@chromium.org wrote:
On Wed, May 19, 2010 at 9:38 PM, Jonas Sicking jo...@sicking.cc wrote:
On Wed, May 19, 2010 at 2:36 AM, Jeremy Orlow jor...@chromium.org wrote:
Interesting you'd bring this up. Andrei and I were just looking at
indexes
as specced and wondered whether it still makes sense to allow indexes to
not
have a keyPath.
I think so. Consider for example a objectStore that contains entries like:
{ name: Elvis Presley, born: 1935-1-8, death: 1977-8-16 }
But index on how old the person was when he/she died. I guess you
could require that people store a property containing the age, however
it seems unfortunate to require modifying the stored data.
Worse, you might have entries like:
{ id: 5, givenName: Benny, otherNames: [Göran, Bror],
familyName: Andersson, age: 63, interest: Music }
in order to search for people, you might want to search for a name, no
matter if that is a given name, a family name, or second name.
Currently you can accomplish this by inserting 4 entries into an
index, all pointing to the same entry in the objectStore. I can't
think of a sane way to accomplish this using only keyPath.
I think the main use case here is the need to allow an unknown quantity of
entries in the index to one entry in the objectStore. What if we allowed
keyEntry's to be any javascript expression and allow them to return arrays
of indexible data. In this case, the expression would return an array of
otherNames + the givenName + the familyName. The keyPath already is
essentially just a javascript expression, so this doesn't seem like too big
of a leap.
This would also support your age upon death use case. (Though I think
requiring the developer to store that age if they want it indexed isn't too
bad of an answer either.)
Thoughts?
I think it's an interesting idea, and one that came up when we were
talking to developers way back before IndexedDB spec was started.
However then we more discussed it in the sense that libraries would be
using the technique, rather than that the API would use it.
The problem with using a javascript expression is how do you provide
it? If you're allowed to pass in a function, that also means that
you're pulling in the full scope of that function. This obviously
doesn't work since the index generally outlives the scope and is used
in other browsing sessions and other tabs.
So you'd have to pass in the javascript expression as a string. This
certainly works but is more than a little ugly. It also complicates
the implementation a good bit since it now has to include a javascript
engine. Not a huge issue given that all browsers has one anyway, but
feels a bit iffy.
I still think it's an interesting idea, though I'm not sold on it.
Especially for the first version of the spec.
And, if so, whether we should tie insertion into the
objectStore to insertion to the index. The main reason to make such
changes
would be to enforce a notion of database consistency.
You mean so that you would have to pass in index keys to the
insert/modify functions? In other words, require at an API level that
indexes were always up to date?
This is an interesting idea. I can't really say I feel strongly one
way or another. I guess it depends on what the API would look like.
The nice thing about what we have now is that the API is pretty
simple.
One possible way to implement it: add/modify/addOrModify could have an
optional parameter that takes an associative array. For example, {age: 63,
interest: music} if you were indexing on age and interest.
That said, I think I like my above proposal better.
age and interest being index names? Definitely an interesting idea.
On Wed, May 19, 2010 at 3:06 AM, Jonas Sicking jo...@sicking.cc wrote:
Hi IndexedDB fans!
So another issue that's come up here. This question is orthogonal to
the other discussed issues and applies equally to both the existing
API, and the mozilla proposed API, so I figured it was safe to raise
right away.
What should happen if you insert a object into an objectStore which
has an index attached to it, but the property defined by the keyPath
in the index does not exist? Consider a database with an objectStore
called people with keyPath name. The objectStore has an
auto-populated index named phone numbers with keyPath phone. The
objectStore is initially empty.
What should happen if you attempt to store an object with value {
name: Benny, email: be...@sweden.com } ?
There are three possible behaviors I can think of:
1. The storing operation should fail and an error event should be
fired.
2. The storing operation should succeed, but no records should be
added to the phone index.
3. The storing operation should succeed and a record should be added
to the phone index using null as key.
Which one is correct behavior? What if the phone numbers index is
flagged as unique?
Re: [IndexedDB] KeyPaths and missing properties.
On Wed, May 19, 2010 at 5:24 PM, Jonas Sicking jo...@sicking.cc wrote:
On Wed, May 19, 2010 at 4:58 PM, Jeremy Orlow jor...@chromium.org wrote:
On Wed, May 19, 2010 at 9:38 PM, Jonas Sicking jo...@sicking.cc wrote:
On Wed, May 19, 2010 at 2:36 AM, Jeremy Orlow jor...@chromium.org wrote:
Interesting you'd bring this up. Andrei and I were just looking at
indexes
as specced and wondered whether it still makes sense to allow indexes to
not
have a keyPath.
I think so. Consider for example a objectStore that contains entries like:
{ name: Elvis Presley, born: 1935-1-8, death: 1977-8-16 }
But index on how old the person was when he/she died. I guess you
could require that people store a property containing the age, however
it seems unfortunate to require modifying the stored data.
Worse, you might have entries like:
{ id: 5, givenName: Benny, otherNames: [Göran, Bror],
familyName: Andersson, age: 63, interest: Music }
in order to search for people, you might want to search for a name, no
matter if that is a given name, a family name, or second name.
Currently you can accomplish this by inserting 4 entries into an
index, all pointing to the same entry in the objectStore. I can't
think of a sane way to accomplish this using only keyPath.
I think the main use case here is the need to allow an unknown quantity of
entries in the index to one entry in the objectStore. What if we allowed
keyEntry's to be any javascript expression and allow them to return arrays
of indexible data. In this case, the expression would return an array of
otherNames + the givenName + the familyName. The keyPath already is
essentially just a javascript expression, so this doesn't seem like too big
of a leap.
This would also support your age upon death use case. (Though I think
requiring the developer to store that age if they want it indexed isn't too
bad of an answer either.)
Thoughts?
I think it's an interesting idea, and one that came up when we were
talking to developers way back before IndexedDB spec was started.
However then we more discussed it in the sense that libraries would be
using the technique, rather than that the API would use it.
The problem with using a javascript expression is how do you provide
it? If you're allowed to pass in a function, that also means that
you're pulling in the full scope of that function. This obviously
doesn't work since the index generally outlives the scope and is used
in other browsing sessions and other tabs.
So you'd have to pass in the javascript expression as a string. This
certainly works but is more than a little ugly. It also complicates
the implementation a good bit since it now has to include a javascript
engine. Not a huge issue given that all browsers has one anyway, but
feels a bit iffy.
I still think it's an interesting idea, though I'm not sold on it.
Especially for the first version of the spec.
It seems like there would be a lot of edge cases to define here. First
of all, how is the value passed in to this expression? Do we say that
it's available through some value variable? So that if you want to
index on the foo property, you pass in an expression like
value.foo? Or do we want the value to be the global object, so that
if you wanted to index on the foo property the expression would
simply be foo?
Also, what happens if the javascript expression modifies the value?
Does the implementation have to clone the value before calling each
index expression?
/ Jonas
