Re: [IndexedDB] KeyPaths and missing properties.
On Fri, May 21, 2010 at 12:39 AM, Jonas Sicking wrote: > >> So you'd have to pass in the javascript expression as a string. This > >> certainly works but is more than a little ugly. It also complicates > >> the implementation a good bit since it now has to include a javascript > >> engine. Not a huge issue given that all browsers has one anyway, but > >> feels a bit iffy. > > > > Hm. I'm looking at the spec, and I can't find where (if anywhere) it > talks > > about what's allowed as a key path. I guess I had assumed that it was > > essentially javascript (or maybe some subset) and that we'd at least need > to > > be parsing it anyway. If we don't adopt this idea because we're worried > > about depending on javascript, then we should spec out the keyPath syntax > > pretty precisely to ensure we don't have such a dependency. > > > > I think the intent of the current syntax is that it's a '.' separated > list of property names. But yes, it definitely needs to be precisely > defined. > I'll create a bug for this and the other items we've already decided on in this thread. >> > I still think it's an interesting idea, though I'm not sold on it. > >> > Especially for the first version of the spec. > >> > >> It seems like there would be a lot of edge cases to define here. First > >> of all, how is the value passed in to this expression? Do we say that > >> it's available through some "value" variable? So that if you want to > >> index on the "foo" property, you pass in an expression like > >> "value.foo"? Or do we want the value to be the global object, so that > >> if you wanted to index on the "foo" property the expression would > >> simply be "foo"? > > > > Since we're already talking about requiring that data being inserted into > > objectStores with a keyPath (for its primary key or in one of its > indexes), > > setting it as the global object seems reasonable. And it matches what's > > currently specced for the simple 1 entityStore entry to 1 index entry > (per > > index) case. > > You also have to specify how the value is returned. I.e. does the > expression need to end with 'return X'? Or do we index on the result > of the value returned from the last executed expression? > > Also, I think that if the value is the global object, then I declaring > variables inside the expression modifies the value as global variables > are set on the global object. > > >> Also, what happens if the javascript expression modifies the value? > >> Does the implementation have to clone the value before calling each > >> "index expression"? > > > > In order of how much I like the idea: > > 1) In an ideal world, we'd spec it to be read only, but I'm not sure if > most > > JS engines have an easy way to do something like that. > > 2) Another possibility is to make the order in which indexes are > processed > > deterministic. That way, if someone does modify it, it'll at least > > be consistent. > > 3) Cloning is another possibility, but it seems like it'd have a > performance > > impact. Maybe optimized implementations could copy-on-write it, though? > > I think 1 and 3 suffers from the same problem of not being something > that JS engines usually need to do, and so likely isn't implemented. > In both cases you have to be able to mark an object graph such that > you take special action if it's modified. > > And for 2, do you do the structured clone before or after you run the > keyPath expressions? Not until after you've created the structured > clone will you know if the value can even be stored. > > In short, I'd like to see a comprehensive proposal :) Then I could > take that to the JS team and ask if it's implementable. > Per other threads, it looks like we should make sure IndexedDB does not depend on JavaScript, so please ignore my proposal. On Thu, May 20, 2010 at 4:56 PM, Jonas Sicking wrote: > On Thu, May 20, 2010 at 7:55 AM, Andrei Popescu > wrote: > > Hi, > > > > On Thu, May 20, 2010 at 10:47 AM, Jeremy Orlow > wrote: > >> On Thu, May 20, 2010 at 1:24 AM, Jonas Sicking > wrote: > >>> It seems like there would be a lot of edge cases to define here. First > >>> of all, how is the value passed in to this expression? Do we say that > >>> it's available through some "value" variable? So that if you want to > >>> index on the "foo" property, you pass in an expression like > >>> "value.foo"? Or do we want the value to be the global object, so that > >>> if you wanted to index on the "foo" property the expression would > >>> simply be "foo"? > >> > >> Since we're already talking about requiring that data being inserted > into > >> objectStores with a keyPath (for its primary key or in one of its > indexes), > >> setting it as the global object seems reasonable. And it matches what's > >> currently specced for the simple 1 entityStore entry to 1 index entry > (per > >> index) case. > >>> > >>> Also, what happens if the javascript expression modifies the value? > >>> Does the implementation have to clone th
Re: [IndexedDB] KeyPaths and missing properties.
>> So you'd have to pass in the javascript expression as a string. This >> certainly works but is more than a little ugly. It also complicates >> the implementation a good bit since it now has to include a javascript >> engine. Not a huge issue given that all browsers has one anyway, but >> feels a bit iffy. > > Hm. I'm looking at the spec, and I can't find where (if anywhere) it talks > about what's allowed as a key path. I guess I had assumed that it was > essentially javascript (or maybe some subset) and that we'd at least need to > be parsing it anyway. If we don't adopt this idea because we're worried > about depending on javascript, then we should spec out the keyPath syntax > pretty precisely to ensure we don't have such a dependency. > I think the intent of the current syntax is that it's a '.' separated list of property names. But yes, it definitely needs to be precisely defined. >> > I still think it's an interesting idea, though I'm not sold on it. >> > Especially for the first version of the spec. >> >> It seems like there would be a lot of edge cases to define here. First >> of all, how is the value passed in to this expression? Do we say that >> it's available through some "value" variable? So that if you want to >> index on the "foo" property, you pass in an expression like >> "value.foo"? Or do we want the value to be the global object, so that >> if you wanted to index on the "foo" property the expression would >> simply be "foo"? > > Since we're already talking about requiring that data being inserted into > objectStores with a keyPath (for its primary key or in one of its indexes), > setting it as the global object seems reasonable. And it matches what's > currently specced for the simple 1 entityStore entry to 1 index entry (per > index) case. You also have to specify how the value is returned. I.e. does the expression need to end with 'return X'? Or do we index on the result of the value returned from the last executed expression? Also, I think that if the value is the global object, then I declaring variables inside the expression modifies the value as global variables are set on the global object. >> Also, what happens if the javascript expression modifies the value? >> Does the implementation have to clone the value before calling each >> "index expression"? > > In order of how much I like the idea: > 1) In an ideal world, we'd spec it to be read only, but I'm not sure if most > JS engines have an easy way to do something like that. > 2) Another possibility is to make the order in which indexes are processed > deterministic. That way, if someone does modify it, it'll at least > be consistent. > 3) Cloning is another possibility, but it seems like it'd have a performance > impact. Maybe optimized implementations could copy-on-write it, though? I think 1 and 3 suffers from the same problem of not being something that JS engines usually need to do, and so likely isn't implemented. In both cases you have to be able to mark an object graph such that you take special action if it's modified. And for 2, do you do the structured clone before or after you run the keyPath expressions? Not until after you've created the structured clone will you know if the value can even be stored. In short, I'd like to see a comprehensive proposal :) Then I could take that to the JS team and ask if it's implementable. / Jonas
Re: [IndexedDB] KeyPaths and missing properties.
On Thu, May 20, 2010 at 7:55 AM, Andrei Popescu wrote:
> Hi,
>
> On Thu, May 20, 2010 at 10:47 AM, Jeremy Orlow wrote:
>> On Thu, May 20, 2010 at 1:24 AM, Jonas Sicking wrote:
>>> It seems like there would be a lot of edge cases to define here. First
>>> of all, how is the value passed in to this expression? Do we say that
>>> it's available through some "value" variable? So that if you want to
>>> index on the "foo" property, you pass in an expression like
>>> "value.foo"? Or do we want the value to be the global object, so that
>>> if you wanted to index on the "foo" property the expression would
>>> simply be "foo"?
>>
>> Since we're already talking about requiring that data being inserted into
>> objectStores with a keyPath (for its primary key or in one of its indexes),
>> setting it as the global object seems reasonable. And it matches what's
>> currently specced for the simple 1 entityStore entry to 1 index entry (per
>> index) case.
>>>
>>> Also, what happens if the javascript expression modifies the value?
>>> Does the implementation have to clone the value before calling each
>>> "index expression"?
>>
>> In order of how much I like the idea:
>> 1) In an ideal world, we'd spec it to be read only, but I'm not sure if most
>> JS engines have an easy way to do something like that.
>> 2) Another possibility is to make the order in which indexes are processed
>> deterministic. That way, if someone does modify it, it'll at least
>> be consistent.
>> 3) Cloning is another possibility, but it seems like it'd have a performance
>> impact. Maybe optimized implementations could copy-on-write it, though?
>>
>
>
> While it's true that allowing the keyPath to be any javascript
> expression would be very elegant and flexible (although probably quite
> difficult to explain in the spec), maybe it's worth considering a
> simpler solution? For instance, could the keyPath be simply an array
> of strings, with each string denoting the name of a property of the
> objects in the store. So in Jonas' example:
>
> { id: 5, givenName: "Benny", otherNames: ["Göran", "Bror"],
> familyName: "Andersson", age: 63, interest: "Music" }
>
> The keyPath could be set to
>
> ["givenName", "otherNames", "familyName"].
>
> The indexable data for this record would therefore be {"Benny",
> "Göran", "Bror", "Andersson"}.
This wouldn't solve the case when the key is the result of a
calculation, such as my age-at-time-of-death example. Consider also
wanting to store objects like
{ id: 5, name: "Benny Andersson", address: "1 Ohai Street\n Wahiawa, HI 96786" }
but wanting to index on the first name or on the state in the address.
/ Jonas
Re: [IndexedDB] KeyPaths and missing properties.
On Thu, May 20, 2010 at 3:55 PM, Andrei Popescu wrote:
> Hi,
>
> On Thu, May 20, 2010 at 10:47 AM, Jeremy Orlow
> wrote:
> > On Thu, May 20, 2010 at 1:24 AM, Jonas Sicking wrote:
> >> It seems like there would be a lot of edge cases to define here. First
> >> of all, how is the value passed in to this expression? Do we say that
> >> it's available through some "value" variable? So that if you want to
> >> index on the "foo" property, you pass in an expression like
> >> "value.foo"? Or do we want the value to be the global object, so that
> >> if you wanted to index on the "foo" property the expression would
> >> simply be "foo"?
> >
> > Since we're already talking about requiring that data being inserted into
> > objectStores with a keyPath (for its primary key or in one of its
> indexes),
> > setting it as the global object seems reasonable. And it matches what's
> > currently specced for the simple 1 entityStore entry to 1 index entry
> (per
> > index) case.
> >>
> >> Also, what happens if the javascript expression modifies the value?
> >> Does the implementation have to clone the value before calling each
> >> "index expression"?
> >
> > In order of how much I like the idea:
> > 1) In an ideal world, we'd spec it to be read only, but I'm not sure if
> most
> > JS engines have an easy way to do something like that.
> > 2) Another possibility is to make the order in which indexes are
> processed
> > deterministic. That way, if someone does modify it, it'll at least
> > be consistent.
> > 3) Cloning is another possibility, but it seems like it'd have a
> performance
> > impact. Maybe optimized implementations could copy-on-write it, though?
> >
>
>
> While it's true that allowing the keyPath to be any javascript
> expression would be very elegant and flexible (although probably quite
> difficult to explain in the spec), maybe it's worth considering a
> simpler solution? For instance, could the keyPath be simply an array
> of strings, with each string denoting the name of a property of the
> objects in the store. So in Jonas' example:
>
> { id: 5, givenName: "Benny", otherNames: ["Göran", "Bror"],
> familyName: "Andersson", age: 63, interest: "Music" }
>
> The keyPath could be set to
>
> ["givenName", "otherNames", "familyName"].
>
> The indexable data for this record would therefore be {"Benny",
> "Göran", "Bror", "Andersson"}.
>
If we're OK making the script depend on JavaScript, I still like my
suggestion a bit better. But assuming we're not, I think this would be a
fine way to solve the problem.
Btw, if we're not depending on JavaScript, then whether or not we change any
of this, we'll need to essentially spec our own language for the keyPath.
(As far as I can tell, there's no normative language for the parsing of
keyPath strings in the current spec. And if we're trying not to depend on
JavaScript, we'll basically have to explain the parsing and how it maps to
objects in a completely language independent fashion.)
J
Re: [IndexedDB] KeyPaths and missing properties.
Hi,
On Thu, May 20, 2010 at 10:47 AM, Jeremy Orlow wrote:
> On Thu, May 20, 2010 at 1:24 AM, Jonas Sicking wrote:
>> It seems like there would be a lot of edge cases to define here. First
>> of all, how is the value passed in to this expression? Do we say that
>> it's available through some "value" variable? So that if you want to
>> index on the "foo" property, you pass in an expression like
>> "value.foo"? Or do we want the value to be the global object, so that
>> if you wanted to index on the "foo" property the expression would
>> simply be "foo"?
>
> Since we're already talking about requiring that data being inserted into
> objectStores with a keyPath (for its primary key or in one of its indexes),
> setting it as the global object seems reasonable. And it matches what's
> currently specced for the simple 1 entityStore entry to 1 index entry (per
> index) case.
>>
>> Also, what happens if the javascript expression modifies the value?
>> Does the implementation have to clone the value before calling each
>> "index expression"?
>
> In order of how much I like the idea:
> 1) In an ideal world, we'd spec it to be read only, but I'm not sure if most
> JS engines have an easy way to do something like that.
> 2) Another possibility is to make the order in which indexes are processed
> deterministic. That way, if someone does modify it, it'll at least
> be consistent.
> 3) Cloning is another possibility, but it seems like it'd have a performance
> impact. Maybe optimized implementations could copy-on-write it, though?
>
While it's true that allowing the keyPath to be any javascript
expression would be very elegant and flexible (although probably quite
difficult to explain in the spec), maybe it's worth considering a
simpler solution? For instance, could the keyPath be simply an array
of strings, with each string denoting the name of a property of the
objects in the store. So in Jonas' example:
{ id: 5, givenName: "Benny", otherNames: ["Göran", "Bror"],
familyName: "Andersson", age: 63, interest: "Music" }
The keyPath could be set to
["givenName", "otherNames", "familyName"].
The indexable data for this record would therefore be {"Benny",
"Göran", "Bror", "Andersson"}.
Thanks,
Andrei
Re: [IndexedDB] KeyPaths and missing properties.
On Thu, May 20, 2010 at 1:24 AM, Jonas Sicking wrote:
> On Wed, May 19, 2010 at 4:58 PM, Jeremy Orlow wrote:
> > On Wed, May 19, 2010 at 9:38 PM, Jonas Sicking wrote:
> >>
> >> On Wed, May 19, 2010 at 2:36 AM, Jeremy Orlow
> wrote:
> >> > Interesting you'd bring this up. Andrei and I were just looking at
> >> > indexes
> >> > as specced and wondered whether it still makes sense to allow indexes
> to
> >> > not
> >> > have a keyPath.
> >>
> >> I think so. Consider for example a objectStore that contains entries
> like:
> >>
> >> { name: "Elvis Presley", born: "1935-1-8", death: "1977-8-16" }
> >>
> >> But index on how old the person was when he/she died. I guess you
> >> could require that people store a property containing the age, however
> >> it seems unfortunate to require modifying the stored data.
> >>
> >> Worse, you might have entries like:
> >>
> >> { id: 5, givenName: "Benny", otherNames: ["Göran", "Bror"],
> >> familyName: "Andersson", age: 63, interest: "Music" }
> >>
> >> in order to search for people, you might want to search for a name, no
> >> matter if that is a given name, a family name, or second name.
> >> Currently you can accomplish this by inserting 4 entries into an
> >> index, all pointing to the same entry in the objectStore. I can't
> >> think of a sane way to accomplish this using only keyPath.
> >
> > I think the main use case here is the need to allow an unknown quantity
> of
> > entries in the index to one entry in the objectStore. What if we allowed
> > keyEntry's to be any javascript expression and allow them to return
> arrays
> > of indexible data. In this case, the expression would return an array of
> > otherNames + the givenName + the familyName. The keyPath already is
> > essentially just a javascript expression, so this doesn't seem like too
> big
> > of a leap.
> > This would also support your age upon death use case. (Though I think
> > requiring the developer to store that age if they want it indexed isn't
> too
> > bad of an answer either.)
> > Thoughts?
>
> I think it's an interesting idea, and one that came up when we were
> talking to developers way back before IndexedDB spec was started.
> However then we more discussed it in the sense that libraries would be
> using the technique, rather than that the API would use it.
>
I agree this is on the borderline between something we should leave to
libraries (for at least v1) and something we should spec now.
The problem with using a javascript expression is how do you provide
> it? If you're allowed to pass in a function, that also means that
> you're pulling in the full scope of that function. This obviously
> doesn't work since the index generally outlives the scope and is used
> in other browsing sessions and other tabs.
>
Sorry, I should have been more clear. I think a string (like what's
currently specced) is definitely the way to go. You're right about the
scoping issues, but what if we ran the expressions without access to the
global scope? In addition to solving this problem (index works when you
first create it, but next time you open the browser it does't) it'd also
keep developers from doing really horrible things...like doing synchronous
XHRs.
> So you'd have to pass in the javascript expression as a string. This
> certainly works but is more than a little ugly. It also complicates
> the implementation a good bit since it now has to include a javascript
> engine. Not a huge issue given that all browsers has one anyway, but
> feels a bit iffy.
>
Hm. I'm looking at the spec, and I can't find where (if anywhere) it talks
about what's allowed as a key path. I guess I had assumed that it was
essentially javascript (or maybe some subset) and that we'd at least need to
be parsing it anyway. If we don't adopt this idea because we're worried
about depending on javascript, then we should spec out the keyPath syntax
pretty precisely to ensure we don't have such a dependency.
> Oh, interesting! Do you know why the structured clone algorithm is
> specced
> > that way?
> > Anyway, I'd agree that disallowing it is probably the right answer.
>
> Hmm.. it turns out I'm wrong. It does not in fact drop id property.
> Ugh, that complicates serializing arrays quite a bit :(
>
> I still think it's surprising to add properties to arrays, and so I'd
> prefer to treat arrays like primitives and other non-plain-objects and
> make it an error.
>
I'm fine with this. I can't think of any use cases it'd get in the way
with.
On Thu, May 20, 2010 at 1:39 AM, Jonas Sicking wrote:
> > I still think it's an interesting idea, though I'm not sold on it.
> > Especially for the first version of the spec.
>
> It seems like there would be a lot of edge cases to define here. First
> of all, how is the value passed in to this expression? Do we say that
> it's available through some "value" variable? So that if you want to
> index on the "foo" property, you pass in an expression like
> "value.foo"? Or do we want the
Re: [IndexedDB] KeyPaths and missing properties.
On Wed, May 19, 2010 at 5:24 PM, Jonas Sicking wrote:
> On Wed, May 19, 2010 at 4:58 PM, Jeremy Orlow wrote:
>> On Wed, May 19, 2010 at 9:38 PM, Jonas Sicking wrote:
>>>
>>> On Wed, May 19, 2010 at 2:36 AM, Jeremy Orlow wrote:
>>> > Interesting you'd bring this up. Andrei and I were just looking at
>>> > indexes
>>> > as specced and wondered whether it still makes sense to allow indexes to
>>> > not
>>> > have a keyPath.
>>>
>>> I think so. Consider for example a objectStore that contains entries like:
>>>
>>> { name: "Elvis Presley", born: "1935-1-8", death: "1977-8-16" }
>>>
>>> But index on how old the person was when he/she died. I guess you
>>> could require that people store a property containing the age, however
>>> it seems unfortunate to require modifying the stored data.
>>>
>>> Worse, you might have entries like:
>>>
>>> { id: 5, givenName: "Benny", otherNames: ["Göran", "Bror"],
>>> familyName: "Andersson", age: 63, interest: "Music" }
>>>
>>> in order to search for people, you might want to search for a name, no
>>> matter if that is a given name, a family name, or second name.
>>> Currently you can accomplish this by inserting 4 entries into an
>>> index, all pointing to the same entry in the objectStore. I can't
>>> think of a sane way to accomplish this using only keyPath.
>>
>> I think the main use case here is the need to allow an unknown quantity of
>> entries in the index to one entry in the objectStore. What if we allowed
>> keyEntry's to be any javascript expression and allow them to return arrays
>> of indexible data. In this case, the expression would return an array of
>> otherNames + the givenName + the familyName. The keyPath already is
>> essentially just a javascript expression, so this doesn't seem like too big
>> of a leap.
>> This would also support your age upon death use case. (Though I think
>> requiring the developer to store that age if they want it indexed isn't too
>> bad of an answer either.)
>> Thoughts?
>
> I think it's an interesting idea, and one that came up when we were
> talking to developers way back before IndexedDB spec was started.
> However then we more discussed it in the sense that libraries would be
> using the technique, rather than that the API would use it.
>
> The problem with using a javascript expression is how do you provide
> it? If you're allowed to pass in a function, that also means that
> you're pulling in the full scope of that function. This obviously
> doesn't work since the index generally outlives the scope and is used
> in other browsing sessions and other tabs.
>
> So you'd have to pass in the javascript expression as a string. This
> certainly works but is more than a little ugly. It also complicates
> the implementation a good bit since it now has to include a javascript
> engine. Not a huge issue given that all browsers has one anyway, but
> feels a bit iffy.
>
> I still think it's an interesting idea, though I'm not sold on it.
> Especially for the first version of the spec.
It seems like there would be a lot of edge cases to define here. First
of all, how is the value passed in to this expression? Do we say that
it's available through some "value" variable? So that if you want to
index on the "foo" property, you pass in an expression like
"value.foo"? Or do we want the value to be the global object, so that
if you wanted to index on the "foo" property the expression would
simply be "foo"?
Also, what happens if the javascript expression modifies the value?
Does the implementation have to clone the value before calling each
"index expression"?
/ Jonas
Re: [IndexedDB] KeyPaths and missing properties.
On Wed, May 19, 2010 at 4:58 PM, Jeremy Orlow wrote:
> On Wed, May 19, 2010 at 9:38 PM, Jonas Sicking wrote:
>>
>> On Wed, May 19, 2010 at 2:36 AM, Jeremy Orlow wrote:
>> > Interesting you'd bring this up. Andrei and I were just looking at
>> > indexes
>> > as specced and wondered whether it still makes sense to allow indexes to
>> > not
>> > have a keyPath.
>>
>> I think so. Consider for example a objectStore that contains entries like:
>>
>> { name: "Elvis Presley", born: "1935-1-8", death: "1977-8-16" }
>>
>> But index on how old the person was when he/she died. I guess you
>> could require that people store a property containing the age, however
>> it seems unfortunate to require modifying the stored data.
>>
>> Worse, you might have entries like:
>>
>> { id: 5, givenName: "Benny", otherNames: ["Göran", "Bror"],
>> familyName: "Andersson", age: 63, interest: "Music" }
>>
>> in order to search for people, you might want to search for a name, no
>> matter if that is a given name, a family name, or second name.
>> Currently you can accomplish this by inserting 4 entries into an
>> index, all pointing to the same entry in the objectStore. I can't
>> think of a sane way to accomplish this using only keyPath.
>
> I think the main use case here is the need to allow an unknown quantity of
> entries in the index to one entry in the objectStore. What if we allowed
> keyEntry's to be any javascript expression and allow them to return arrays
> of indexible data. In this case, the expression would return an array of
> otherNames + the givenName + the familyName. The keyPath already is
> essentially just a javascript expression, so this doesn't seem like too big
> of a leap.
> This would also support your age upon death use case. (Though I think
> requiring the developer to store that age if they want it indexed isn't too
> bad of an answer either.)
> Thoughts?
I think it's an interesting idea, and one that came up when we were
talking to developers way back before IndexedDB spec was started.
However then we more discussed it in the sense that libraries would be
using the technique, rather than that the API would use it.
The problem with using a javascript expression is how do you provide
it? If you're allowed to pass in a function, that also means that
you're pulling in the full scope of that function. This obviously
doesn't work since the index generally outlives the scope and is used
in other browsing sessions and other tabs.
So you'd have to pass in the javascript expression as a string. This
certainly works but is more than a little ugly. It also complicates
the implementation a good bit since it now has to include a javascript
engine. Not a huge issue given that all browsers has one anyway, but
feels a bit iffy.
I still think it's an interesting idea, though I'm not sold on it.
Especially for the first version of the spec.
>> > And, if so, whether we should tie insertion into the
>> > objectStore to insertion to the index. The main reason to make such
>> > changes
>> > would be to enforce a notion of database consistency.
>>
>> You mean so that you would have to pass in index keys to the
>> insert/modify functions? In other words, require at an API level that
>> indexes were always up to date?
>>
>> This is an interesting idea. I can't really say I feel strongly one
>> way or another. I guess it depends on what the API would look like.
>> The nice thing about what we have now is that the API is pretty
>> simple.
>
> One possible way to implement it: add/modify/addOrModify could have an
> optional parameter that takes an associative array. For example, {age: 63,
> interest: "music"} if you were indexing on age and interest.
> That said, I think I like my above proposal better.
"age" and "interest" being index names? Definitely an interesting idea.
>> > On Wed, May 19, 2010 at 3:06 AM, Jonas Sicking wrote:
>> >>
>> >> Hi IndexedDB fans!
>> >>
>> >> So another issue that's come up here. This question is orthogonal to
>> >> the other discussed issues and applies equally to both the existing
>> >> API, and the mozilla proposed API, so I figured it was safe to raise
>> >> right away.
>> >>
>> >> What should happen if you insert a object into an objectStore which
>> >> has an index attached to it, but the property defined by the keyPath
>> >> in the index does not exist? Consider a database with an objectStore
>> >> called "people" with keyPath "name". The objectStore has an
>> >> auto-populated index named "phone numbers" with keyPath "phone". The
>> >> objectStore is initially empty.
>> >>
>> >> What should happen if you attempt to store an object with value {
>> >> name: "Benny", email: "be...@sweden.com" } ?
>> >>
>> >> There are three possible behaviors I can think of:
>> >>
>> >> 1. The storing operation should fail and an error event should be
>> >> fired.
>> >> 2. The storing operation should succeed, but no records should be
>> >> added to the "phone" index.
>> >> 3. The storing operation s
Re: [IndexedDB] KeyPaths and missing properties.
On Wed, May 19, 2010 at 9:38 PM, Jonas Sicking wrote:
> On Wed, May 19, 2010 at 2:36 AM, Jeremy Orlow wrote:
> > Interesting you'd bring this up. Andrei and I were just looking at
> indexes
> > as specced and wondered whether it still makes sense to allow indexes to
> not
> > have a keyPath.
>
> I think so. Consider for example a objectStore that contains entries like:
>
> { name: "Elvis Presley", born: "1935-1-8", death: "1977-8-16" }
>
> But index on how old the person was when he/she died. I guess you
> could require that people store a property containing the age, however
> it seems unfortunate to require modifying the stored data.
>
> Worse, you might have entries like:
>
> { id: 5, givenName: "Benny", otherNames: ["Göran", "Bror"],
> familyName: "Andersson", age: 63, interest: "Music" }
>
> in order to search for people, you might want to search for a name, no
> matter if that is a given name, a family name, or second name.
> Currently you can accomplish this by inserting 4 entries into an
> index, all pointing to the same entry in the objectStore. I can't
> think of a sane way to accomplish this using only keyPath.
>
I think the main use case here is the need to allow an unknown quantity of
entries in the index to one entry in the objectStore. What if we allowed
keyEntry's to be any javascript expression and allow them to return arrays
of indexible data. In this case, the expression would return an array of
otherNames + the givenName + the familyName. The keyPath already is
essentially just a javascript expression, so this doesn't seem like too big
of a leap.
This would also support your age upon death use case. (Though I think
requiring the developer to store that age if they want it indexed isn't too
bad of an answer either.)
Thoughts?
> > And, if so, whether we should tie insertion into the
> > objectStore to insertion to the index. The main reason to make such
> changes
> > would be to enforce a notion of database consistency.
>
> You mean so that you would have to pass in index keys to the
> insert/modify functions? In other words, require at an API level that
> indexes were always up to date?
>
> This is an interesting idea. I can't really say I feel strongly one
> way or another. I guess it depends on what the API would look like.
> The nice thing about what we have now is that the API is pretty
> simple.
>
One possible way to implement it: add/modify/addOrModify could have an
optional parameter that takes an associative array. For example, {age: 63,
interest: "music"} if you were indexing on age and interest.
That said, I think I like my above proposal better.
> > So I guess the
> > fundamental question here is whether there should be a 1:1 mapping
> between
> > entries in an objectStore's indexes and the objectStore itself.
>
> I'm not sure what you mean by "a 1:1 mapping"? Do you mean that there
> are the same number of entries in the index as there are in the
> objectStore?
>
Yes. But I think your indexing on multiple names use case may have already
sold me on supporting something other than just 1:1 mappings.
> > On Wed, May 19, 2010 at 3:06 AM, Jonas Sicking
> wrote:
> >>
> >> Hi IndexedDB fans!
> >>
> >> So another issue that's come up here. This question is orthogonal to
> >> the other discussed issues and applies equally to both the existing
> >> API, and the mozilla proposed API, so I figured it was safe to raise
> >> right away.
> >>
> >> What should happen if you insert a object into an objectStore which
> >> has an index attached to it, but the property defined by the keyPath
> >> in the index does not exist? Consider a database with an objectStore
> >> called "people" with keyPath "name". The objectStore has an
> >> auto-populated index named "phone numbers" with keyPath "phone". The
> >> objectStore is initially empty.
> >>
> >> What should happen if you attempt to store an object with value {
> >> name: "Benny", email: "be...@sweden.com" } ?
> >>
> >> There are three possible behaviors I can think of:
> >>
> >> 1. The storing operation should fail and an error event should be fired.
> >> 2. The storing operation should succeed, but no records should be
> >> added to the "phone" index.
> >> 3. The storing operation should succeed and a record should be added
> >> to the "phone" index using null as key.
> >>
> >> Which one is correct behavior? What if the "phone numbers" index is
> >> flagged as unique?
> >>
> >> I definitely think there is a use case for allowing records to be
> >> stored even if there are indexes on missing properties. Consider for
> >> example a objectStore containing people where not everyone has a known
> >> phone number. It still seems useful to be able to search based on
> >> phone number in this objectStore. To satisfy this use case either
> >> solution 2 or 3 would work. 3 would additionally allow searching for
> >> everyone without a phone number, though I'm not sure how important
> >> that is. On the other hand 2 would be more
Re: [IndexedDB] KeyPaths and missing properties.
On Wed, May 19, 2010 at 2:36 AM, Jeremy Orlow wrote:
> Interesting you'd bring this up. Andrei and I were just looking at indexes
> as specced and wondered whether it still makes sense to allow indexes to not
> have a keyPath.
I think so. Consider for example a objectStore that contains entries like:
{ name: "Elvis Presley", born: "1935-1-8", death: "1977-8-16" }
But index on how old the person was when he/she died. I guess you
could require that people store a property containing the age, however
it seems unfortunate to require modifying the stored data.
Worse, you might have entries like:
{ id: 5, givenName: "Benny", otherNames: ["Göran", "Bror"],
familyName: "Andersson", age: 63, interest: "Music" }
in order to search for people, you might want to search for a name, no
matter if that is a given name, a family name, or second name.
Currently you can accomplish this by inserting 4 entries into an
index, all pointing to the same entry in the objectStore. I can't
think of a sane way to accomplish this using only keyPath.
> And, if so, whether we should tie insertion into the
> objectStore to insertion to the index. The main reason to make such changes
> would be to enforce a notion of database consistency.
You mean so that you would have to pass in index keys to the
insert/modify functions? In other words, require at an API level that
indexes were always up to date?
This is an interesting idea. I can't really say I feel strongly one
way or another. I guess it depends on what the API would look like.
The nice thing about what we have now is that the API is pretty
simple.
> So I guess the
> fundamental question here is whether there should be a 1:1 mapping between
> entries in an objectStore's indexes and the objectStore itself.
I'm not sure what you mean by "a 1:1 mapping"? Do you mean that there
are the same number of entries in the index as there are in the
objectStore?
> On Wed, May 19, 2010 at 3:06 AM, Jonas Sicking wrote:
>>
>> Hi IndexedDB fans!
>>
>> So another issue that's come up here. This question is orthogonal to
>> the other discussed issues and applies equally to both the existing
>> API, and the mozilla proposed API, so I figured it was safe to raise
>> right away.
>>
>> What should happen if you insert a object into an objectStore which
>> has an index attached to it, but the property defined by the keyPath
>> in the index does not exist? Consider a database with an objectStore
>> called "people" with keyPath "name". The objectStore has an
>> auto-populated index named "phone numbers" with keyPath "phone". The
>> objectStore is initially empty.
>>
>> What should happen if you attempt to store an object with value {
>> name: "Benny", email: "be...@sweden.com" } ?
>>
>> There are three possible behaviors I can think of:
>>
>> 1. The storing operation should fail and an error event should be fired.
>> 2. The storing operation should succeed, but no records should be
>> added to the "phone" index.
>> 3. The storing operation should succeed and a record should be added
>> to the "phone" index using null as key.
>>
>> Which one is correct behavior? What if the "phone numbers" index is
>> flagged as unique?
>>
>> I definitely think there is a use case for allowing records to be
>> stored even if there are indexes on missing properties. Consider for
>> example a objectStore containing people where not everyone has a known
>> phone number. It still seems useful to be able to search based on
>> phone number in this objectStore. To satisfy this use case either
>> solution 2 or 3 would work. 3 would additionally allow searching for
>> everyone without a phone number, though I'm not sure how important
>> that is. On the other hand 2 would be more space effective in that
>> only relevant records would be stored.
>>
>> Possibly we'll want to add a 'required' flag to the createIndex
>> function specifying if the index should cause behavior 1 or behavior
>> 2. So far I'm not a big fan of 3, but I'm all ears for input.
>
> If we went with (1) then it seems as though developers could simply insert
> items with .phone=null to get around the limitation.
People doing that doesn't seem to really change anything. The question
would just turn into "what should we do if the key is absent, null or
undefined". The same 1, 2, 3 answers still seem like possible answers.
> 3 seems like it'd be error prone and somewhat confusing.
> If we decide there should be a 1:1 mapping then it seems we're choosing
> between 1 and 3. And if not, then I think 2 is the clear answer.
I'm still not sure what the "a 1:1 mapping" means, so not really
following your logic here.
>> There is a similar issue if an objectStore uses in-line keys. Consider
>> a database with a objectStore called "people" with keyPath "name" and
>> no key generator. What if someone inserts an object with the value {
>> email: "be...@sweden.com" }? I would think that this should be an
>> error, but it doesn't appear that any of the steps in "Objec
Re: [IndexedDB] KeyPaths and missing properties.
Interesting you'd bring this up. Andrei and I were just looking at indexes
as specced and wondered whether it still makes sense to allow indexes to not
have a keyPath. And, if so, whether we should tie insertion into the
objectStore to insertion to the index. The main reason to make such changes
would be to enforce a notion of database consistency. So I guess the
fundamental question here is whether there should be a 1:1 mapping between
entries in an objectStore's indexes and the objectStore itself.
On Wed, May 19, 2010 at 3:06 AM, Jonas Sicking wrote:
> Hi IndexedDB fans!
>
> So another issue that's come up here. This question is orthogonal to
> the other discussed issues and applies equally to both the existing
> API, and the mozilla proposed API, so I figured it was safe to raise
> right away.
>
> What should happen if you insert a object into an objectStore which
> has an index attached to it, but the property defined by the keyPath
> in the index does not exist? Consider a database with an objectStore
> called "people" with keyPath "name". The objectStore has an
> auto-populated index named "phone numbers" with keyPath "phone". The
> objectStore is initially empty.
>
> What should happen if you attempt to store an object with value {
> name: "Benny", email: "be...@sweden.com" } ?
>
> There are three possible behaviors I can think of:
>
> 1. The storing operation should fail and an error event should be fired.
> 2. The storing operation should succeed, but no records should be
> added to the "phone" index.
> 3. The storing operation should succeed and a record should be added
> to the "phone" index using null as key.
>
> Which one is correct behavior? What if the "phone numbers" index is
> flagged as unique?
>
> I definitely think there is a use case for allowing records to be
> stored even if there are indexes on missing properties. Consider for
> example a objectStore containing people where not everyone has a known
> phone number. It still seems useful to be able to search based on
> phone number in this objectStore. To satisfy this use case either
> solution 2 or 3 would work. 3 would additionally allow searching for
> everyone without a phone number, though I'm not sure how important
> that is. On the other hand 2 would be more space effective in that
> only relevant records would be stored.
>
> Possibly we'll want to add a 'required' flag to the createIndex
> function specifying if the index should cause behavior 1 or behavior
> 2. So far I'm not a big fan of 3, but I'm all ears for input.
>
If we went with (1) then it seems as though developers could simply insert
items with .phone=null to get around the limitation.
3 seems like it'd be error prone and somewhat confusing.
If we decide there should be a 1:1 mapping then it seems we're choosing
between 1 and 3. And if not, then I think 2 is the clear answer.
> The same question arises when an index is added to an objectStore
> which has existing values that lack the property that the index uses
> as keyPath. Whichever solution we go with above I think should be used
> here too.
>
Agreed.
> There is a similar issue if an objectStore uses in-line keys. Consider
> a database with a objectStore called "people" with keyPath "name" and
> no key generator. What if someone inserts an object with the value {
> email: "be...@sweden.com" }? I would think that this should be an
> error, but it doesn't appear that any of the steps in "Object Store
> Storage steps" says to create an error.
>
This is probably just an omission in the spec. If so, someone should file
an issue on it.
> Finally, there is the issue of what should happen if someone tries to
> insert a number as a value into an objectStore with a keyPath and a
> key generator. Consider the objectStore "albums" with keyPath "id" and
> a key generator enabled. What should happen if someone tries to insert
> the value 9 into this table? If an object was inserted, then this
> would result in that objects 'id' property being set to the next
> generated value from the generator, however it doesn't really make
> sense to set a property on a number. Same question obviously applies
> if the stored value is a boolean, a string, or any other primitive JS
> type, or types such as regexps and File objects. I think that in all
> these cases the operation should fail.
>
Agreed.
> The question also applies if the stored value is an array. Technically
> Javascript allows setting values on arrays, however the structured
> clone algorithm drops these values, and so it seems to add needless
> complexity for the implementation if the only time you need to
> serialize/deserialize arrays with non-integer properties is in this
> one case. Additionally, the id would be lost when the value is read as
> that is done using the structured clone algorithm too.
>
Hm. I'm having trouble following this case. Any chance you could explain
it further?
Thanks!
J
[IndexedDB] KeyPaths and missing properties.
Hi IndexedDB fans!
So another issue that's come up here. This question is orthogonal to
the other discussed issues and applies equally to both the existing
API, and the mozilla proposed API, so I figured it was safe to raise
right away.
What should happen if you insert a object into an objectStore which
has an index attached to it, but the property defined by the keyPath
in the index does not exist? Consider a database with an objectStore
called "people" with keyPath "name". The objectStore has an
auto-populated index named "phone numbers" with keyPath "phone". The
objectStore is initially empty.
What should happen if you attempt to store an object with value {
name: "Benny", email: "be...@sweden.com" } ?
There are three possible behaviors I can think of:
1. The storing operation should fail and an error event should be fired.
2. The storing operation should succeed, but no records should be
added to the "phone" index.
3. The storing operation should succeed and a record should be added
to the "phone" index using null as key.
Which one is correct behavior? What if the "phone numbers" index is
flagged as unique?
I definitely think there is a use case for allowing records to be
stored even if there are indexes on missing properties. Consider for
example a objectStore containing people where not everyone has a known
phone number. It still seems useful to be able to search based on
phone number in this objectStore. To satisfy this use case either
solution 2 or 3 would work. 3 would additionally allow searching for
everyone without a phone number, though I'm not sure how important
that is. On the other hand 2 would be more space effective in that
only relevant records would be stored.
Possibly we'll want to add a 'required' flag to the createIndex
function specifying if the index should cause behavior 1 or behavior
2. So far I'm not a big fan of 3, but I'm all ears for input.
The same question arises when an index is added to an objectStore
which has existing values that lack the property that the index uses
as keyPath. Whichever solution we go with above I think should be used
here too.
There is a similar issue if an objectStore uses in-line keys. Consider
a database with a objectStore called "people" with keyPath "name" and
no key generator. What if someone inserts an object with the value {
email: "be...@sweden.com" }? I would think that this should be an
error, but it doesn't appear that any of the steps in "Object Store
Storage steps" says to create an error.
Finally, there is the issue of what should happen if someone tries to
insert a number as a value into an objectStore with a keyPath and a
key generator. Consider the objectStore "albums" with keyPath "id" and
a key generator enabled. What should happen if someone tries to insert
the value 9 into this table? If an object was inserted, then this
would result in that objects 'id' property being set to the next
generated value from the generator, however it doesn't really make
sense to set a property on a number. Same question obviously applies
if the stored value is a boolean, a string, or any other primitive JS
type, or types such as regexps and File objects. I think that in all
these cases the operation should fail.
The question also applies if the stored value is an array. Technically
Javascript allows setting values on arrays, however the structured
clone algorithm drops these values, and so it seems to add needless
complexity for the implementation if the only time you need to
serialize/deserialize arrays with non-integer properties is in this
one case. Additionally, the id would be lost when the value is read as
that is done using the structured clone algorithm too.
/ Jonas
