RE: [IndexedDB] IDBCursor.update for cursors returned from IDBIndex.openCursor
I agree with Jonas on this. I think accessing the index values is an important
feature (in addition to joins you can imagine add an extra property or two to
the index key* to create a covering index and avoid fetching the object in a
perf-critical path).
That said, to me it's just about allowing retrieval. For update/delete it would
be perfectly reasonable to have to go to the store in my opinion.
-pablo
-Original Message-
From: public-webapps-requ...@w3.org [mailto:public-webapps-requ...@w3.org] On
Behalf Of Jonas Sicking
Sent: Friday, September 17, 2010 3:15 PM
On Fri, Sep 17, 2010 at 2:46 AM, Jeremy Orlow jor...@chromium.org wrote:
On Fri, Sep 17, 2010 at 1:06 AM, Jonas Sicking jo...@sicking.cc wrote:
On Thu, Sep 16, 2010 at 2:23 PM, Jeremy Orlow jor...@chromium.org wrote:
On Thu, Sep 16, 2010 at 8:53 PM, Jonas Sicking jo...@sicking.cc wrote:
On Thu, Sep 16, 2010 at 2:15 AM, Jeremy Orlow jor...@chromium.org
wrote:
Wait a sec. What are the use cases for non-object cursors anyway?
They
made perfect sense back when we allowed explicit index management,
but
now
they kind of seem like a premature optimization or possibly even dead
weight. Maybe we should just remove them altogether?
They are still useful for joins. Consider an objectStore employees:
{ id: 1, name: Sven, employed: 1-1-2010 }
{ id: 2, name: Bert, employed: 5-1-2009 }
{ id: 3, name: Adam, employed: 6-6-2008 }
And objectStore sales
{ seller: 1, candyName: lollipop, quantity: 5, date: 9-15-2010 }
{ seller: 1, candyName: swedish fish, quantity: 12, date: 9-15-2010
}
{ seller: 2, candyName: jelly belly, quantity: 3, date: 9-14-2010 }
{ seller: 3, candyName: heath bar, quantity: 3, date: 9-13-2010 }
If you want to display the amount of sales per person, sorted by names
of sales person, you could do this by first creating and index for
employees with keyPath name. You'd then use IDBIndex.openCursor to
iterate that index, and for each entry find all entries in the sales
objectStore where seller matches the cursors .value.
So in this case you don't actually need any data from the employees
objectStore, all the data is available in the index. Thus it is
sufficient, and faster, to use openCursor than openObjectCursor.
In general, it's a common optimization to stick enough data in an
index that you don't have to actually look up in the objectStore
itself. This is slightly less commonly doable since we have relatively
simple indexes so far. But still doable as the example above shows.
Once we add support for arrays as keys this will be much more common
as you can then stick arbitrary data into the index by simply adding
additional entries to all key arrays. And even more so once we
(probably in a future version) add support for computed indexes.
On Thu, Sep 16, 2010 at 8:57 PM, Jonas Sicking jo...@sicking.cc wrote:
On Thu, Sep 16, 2010 at 4:08 AM, Jeremy Orlow jor...@chromium.org
wrote:
Actually, for that matter, are remove and update needed at all? I
think
they may just be more cruft left over from the explicit index days.
As
far
as I can tell, any .delete or .remove should be doable via an
objectCursor +
.puts/.removes on the objectStore.
They are not strictly needed, but they are a decent convinence
feature, and with a proper implementation they can even be a
performance optimization. With a cursor iterating a b-tree you can let
the cursor keep a pointer to the b-tree entry. They way .delete and
.update doesn't have to do a b-tree lookup at all.
We're currently not able to do this since our backend (sqlite) doesn't
have good enough cursor support, but I suspect that this will change
at some point in the future. In the mean time it seems like a good
thing to allow people to use API that will be faster in the future.
All your arguments revolve around what the spec
and implementations might do
in the future.
I disagree. The IDBIndex.openCursor example I included uses only
existing API, and is a performance improvement in at least our current
implementation. Would be interested to hear if it's not a performance
improvement in others.
It's not in ours because we join to the ObjectStore's data table either way.
But that's not at all why I'm bringing this up.
Why?
Typically we add API surface area only for use cases that
are currently impossible to satisfy or proven performance bottlenecks. I
agree that it's likely implementations will want to do optimizations
like
this in the future, but until they do, it'll be hard to really
understand
the implications and complications that might arrise.
That's not entirely true. All the databases I have worked with have
had significant performance degradations when having to look up the
main table contents rather than simply looking at the contents in the
index. I doubt that we'll be able to create a backend where
Re: [IndexedDB] IDBCursor.update for cursors returned from IDBIndex.openCursor
On 9/17/2010 3:14 PM, Jonas Sicking wrote: How do you then guarantee that a transaction that spans multiple objectStores either fully succeeds or is fully rolled back? Especially in the event of a crash during commit. If you don't use write ahead logging, and connect to each database with ATTACH, SQLite makes sure it is atomic for you (I don't recall how offhand though). Not using write ahead logging does mean that writers will block readers though. Cheers, Shawn smime.p7s Description: S/MIME Cryptographic Signature
Re: [IndexedDB] IDBCursor.update for cursors returned from IDBIndex.openCursor
On Fri, Sep 17, 2010 at 1:06 AM, Jonas Sicking jo...@sicking.cc wrote:
On Thu, Sep 16, 2010 at 2:23 PM, Jeremy Orlow jor...@chromium.org wrote:
On Thu, Sep 16, 2010 at 8:53 PM, Jonas Sicking jo...@sicking.cc wrote:
On Thu, Sep 16, 2010 at 2:15 AM, Jeremy Orlow jor...@chromium.org
wrote:
Wait a sec. What are the use cases for non-object cursors anyway?
They
made perfect sense back when we allowed explicit index management, but
now
they kind of seem like a premature optimization or possibly even dead
weight. Maybe we should just remove them altogether?
They are still useful for joins. Consider an objectStore employees:
{ id: 1, name: Sven, employed: 1-1-2010 }
{ id: 2, name: Bert, employed: 5-1-2009 }
{ id: 3, name: Adam, employed: 6-6-2008 }
And objectStore sales
{ seller: 1, candyName: lollipop, quantity: 5, date: 9-15-2010 }
{ seller: 1, candyName: swedish fish, quantity: 12, date: 9-15-2010
}
{ seller: 2, candyName: jelly belly, quantity: 3, date: 9-14-2010 }
{ seller: 3, candyName: heath bar, quantity: 3, date: 9-13-2010 }
If you want to display the amount of sales per person, sorted by names
of sales person, you could do this by first creating and index for
employees with keyPath name. You'd then use IDBIndex.openCursor to
iterate that index, and for each entry find all entries in the sales
objectStore where seller matches the cursors .value.
So in this case you don't actually need any data from the employees
objectStore, all the data is available in the index. Thus it is
sufficient, and faster, to use openCursor than openObjectCursor.
In general, it's a common optimization to stick enough data in an
index that you don't have to actually look up in the objectStore
itself. This is slightly less commonly doable since we have relatively
simple indexes so far. But still doable as the example above shows.
Once we add support for arrays as keys this will be much more common
as you can then stick arbitrary data into the index by simply adding
additional entries to all key arrays. And even more so once we
(probably in a future version) add support for computed indexes.
On Thu, Sep 16, 2010 at 8:57 PM, Jonas Sicking jo...@sicking.cc wrote:
On Thu, Sep 16, 2010 at 4:08 AM, Jeremy Orlow jor...@chromium.org
wrote:
Actually, for that matter, are remove and update needed at all? I
think
they may just be more cruft left over from the explicit index days.
As
far
as I can tell, any .delete or .remove should be doable via an
objectCursor +
.puts/.removes on the objectStore.
They are not strictly needed, but they are a decent convinence
feature, and with a proper implementation they can even be a
performance optimization. With a cursor iterating a b-tree you can let
the cursor keep a pointer to the b-tree entry. They way .delete and
.update doesn't have to do a b-tree lookup at all.
We're currently not able to do this since our backend (sqlite) doesn't
have good enough cursor support, but I suspect that this will change
at some point in the future. In the mean time it seems like a good
thing to allow people to use API that will be faster in the future.
All your arguments revolve around what the spec and implementations might
do
in the future.
I disagree. The IDBIndex.openCursor example I included uses only
existing API, and is a performance improvement in at least our current
implementation. Would be interested to hear if it's not a performance
improvement in others.
It's not in ours because we join to the ObjectStore's data table either way.
But that's not at all why I'm bringing this up.
Typically we add API surface area only for use cases that
are currently impossible to satisfy or proven performance bottlenecks. I
agree that it's likely implementations will want to do optimizations like
this in the future, but until they do, it'll be hard to really understand
the implications and complications that might arrise.
That's not entirely true. All the databases I have worked with have
had significant performance degradations when having to look up the
main table contents rather than simply looking at the contents in the
index. I doubt that we'll be able to create a backend where that is
not true. So I think we should assume that object cursors are slower
than plain cursors.
I agree this is true.
Further, I think we should get users on APIs that we are likely to
implement with a higher performance. For example, I think sqlite
doesn't support having multiple write transactions to the same
database, even if those are to different tables.
FWIW: The work around for this is putting each object store in its own
database.
Thus the whole API of
specifying which objectStores you want to include in a transaction is
purely for future optimizations in at least implementations backed by
sqlite.
It's funny you mention this because this level of
Re: [IndexedDB] IDBCursor.update for cursors returned from IDBIndex.openCursor
On Fri, Sep 17, 2010 at 2:46 AM, Jeremy Orlow jor...@chromium.org wrote:
On Fri, Sep 17, 2010 at 1:06 AM, Jonas Sicking jo...@sicking.cc wrote:
On Thu, Sep 16, 2010 at 2:23 PM, Jeremy Orlow jor...@chromium.org wrote:
On Thu, Sep 16, 2010 at 8:53 PM, Jonas Sicking jo...@sicking.cc wrote:
On Thu, Sep 16, 2010 at 2:15 AM, Jeremy Orlow jor...@chromium.org
wrote:
Wait a sec. What are the use cases for non-object cursors anyway?
They
made perfect sense back when we allowed explicit index management,
but
now
they kind of seem like a premature optimization or possibly even dead
weight. Maybe we should just remove them altogether?
They are still useful for joins. Consider an objectStore employees:
{ id: 1, name: Sven, employed: 1-1-2010 }
{ id: 2, name: Bert, employed: 5-1-2009 }
{ id: 3, name: Adam, employed: 6-6-2008 }
And objectStore sales
{ seller: 1, candyName: lollipop, quantity: 5, date: 9-15-2010 }
{ seller: 1, candyName: swedish fish, quantity: 12, date: 9-15-2010
}
{ seller: 2, candyName: jelly belly, quantity: 3, date: 9-14-2010 }
{ seller: 3, candyName: heath bar, quantity: 3, date: 9-13-2010 }
If you want to display the amount of sales per person, sorted by names
of sales person, you could do this by first creating and index for
employees with keyPath name. You'd then use IDBIndex.openCursor to
iterate that index, and for each entry find all entries in the sales
objectStore where seller matches the cursors .value.
So in this case you don't actually need any data from the employees
objectStore, all the data is available in the index. Thus it is
sufficient, and faster, to use openCursor than openObjectCursor.
In general, it's a common optimization to stick enough data in an
index that you don't have to actually look up in the objectStore
itself. This is slightly less commonly doable since we have relatively
simple indexes so far. But still doable as the example above shows.
Once we add support for arrays as keys this will be much more common
as you can then stick arbitrary data into the index by simply adding
additional entries to all key arrays. And even more so once we
(probably in a future version) add support for computed indexes.
On Thu, Sep 16, 2010 at 8:57 PM, Jonas Sicking jo...@sicking.cc wrote:
On Thu, Sep 16, 2010 at 4:08 AM, Jeremy Orlow jor...@chromium.org
wrote:
Actually, for that matter, are remove and update needed at all? I
think
they may just be more cruft left over from the explicit index days.
As
far
as I can tell, any .delete or .remove should be doable via an
objectCursor +
.puts/.removes on the objectStore.
They are not strictly needed, but they are a decent convinence
feature, and with a proper implementation they can even be a
performance optimization. With a cursor iterating a b-tree you can let
the cursor keep a pointer to the b-tree entry. They way .delete and
.update doesn't have to do a b-tree lookup at all.
We're currently not able to do this since our backend (sqlite) doesn't
have good enough cursor support, but I suspect that this will change
at some point in the future. In the mean time it seems like a good
thing to allow people to use API that will be faster in the future.
All your arguments revolve around what the spec
and implementations might do
in the future.
I disagree. The IDBIndex.openCursor example I included uses only
existing API, and is a performance improvement in at least our current
implementation. Would be interested to hear if it's not a performance
improvement in others.
It's not in ours because we join to the ObjectStore's data table either way.
But that's not at all why I'm bringing this up.
Why?
Typically we add API surface area only for use cases that
are currently impossible to satisfy or proven performance bottlenecks. I
agree that it's likely implementations will want to do optimizations
like
this in the future, but until they do, it'll be hard to really
understand
the implications and complications that might arrise.
That's not entirely true. All the databases I have worked with have
had significant performance degradations when having to look up the
main table contents rather than simply looking at the contents in the
index. I doubt that we'll be able to create a backend where that is
not true. So I think we should assume that object cursors are slower
than plain cursors.
I agree this is true.
Further, I think we should get users on APIs that we are likely to
implement with a higher performance. For example, I think sqlite
doesn't support having multiple write transactions to the same
database, even if those are to different tables.
FWIW: The work around for this is putting each object store in its own
database.
How do you then guarantee that a transaction that spans multiple
objectStores either fully succeeds or is fully rolled back?
Re: [IndexedDB] IDBCursor.update for cursors returned from IDBIndex.openCursor
On Wed, Sep 15, 2010 at 10:45 PM, Jonas Sicking jo...@sicking.cc wrote:
Heh, I've also been thinking about this exact issue lately. There is a
similar question for IDBCursor.delete.
On Wed, Sep 15, 2010 at 8:55 AM, Jeremy Orlow jor...@chromium.org wrote:
I think it's clear what IDBCursor does when created from
IDBObjectStore.openCursor or IDBIndex.openObjectCursor: it modifies the
objectStore's value and updates all indexes (or does nothing and returns
an
error if all of that can't be done while satisfying the constraints).
Agreed.
But what about IDBCursor.update when created from IDBIndex.openCursor? I
see two options: we could modify the value within the objectStore's value
that corresponds to the objectStore's key path or we could do like above
and
simply modify the objectStore's value.
There's also a third option: Throw an exception. Maybe that's what
you're referring to by make this unsupported below?
More concretely, if we have an object store with a id key path and an
index with a fname key path, and our index.openCursor() created cursor
is
currently on the {id: 22, fname: Fred} value (and thus cursor.key ==
Fred and cursor.value == 22), let's say I wanted to change the object
to
be {id: 23, fname: Fred}. In other words, I want id to change from 22
to
23. Which of the following should I write?
1) calling cursor.update(23) or
2) calling cursor.update({id: 23, fname: Fred})
The former seems to match the behavior of the IDBObjectStore.openCursor
and
IDBIndex.openObjectCursor better (i.e. it modifies the cursor.value).
The
latter intuitively seems like it'd be more useful. But to be honest, I
can't think of any use cases for either. Can anyone else? If not, maybe
we
should just make this unsupported for now?
The only use case I have thought of is wanting to update some set of
entries, where the best way to find these entries is through an index.
For example updating every entry with a specific shipping-id. You can
use IDBIndex.openObjectCursor for this, but that's slower than
IDBIndex.openCursor. So in the rare instance when you can make the
modification without inspecting the existing value (i.e. you only need
to write, read-modify-write), then IDBIndex.openCursor +
IDBCursor.update() would be a perf optimization.
On the other hand, it might be just as quick to call IDBObjectStore.put().
Since the use case if pretty weak (when would you be able to update an
entry without first reading the entry), and that you can seemingly get
the same performance using IDBObjectStore.put(), I would be fine with
making this unsupported.
As for IDBCursor.delete(), I can see a somewhat stronger use case
there. For example removing all entries with a specific shipping-id or
some such. If you can determine which entries should be removed purely
on the information in the index, then using IDBIndex.openCursor is
definitely faster than IDBIndex.openObjectCursor. So on one hand it
would be nice to allow people to use that. On the other hand, I
suspect you can get the same performance using IDBObjectStore.delete()
and we might want to be consistent with IDBCursor.update().
In this case I'm actually leaning towards allowing IDBCursor.delete(),
but I could go either way.
Wait a sec. What are the use cases for non-object cursors anyway? They
made perfect sense back when we allowed explicit index management, but now
they kind of seem like a premature optimization or possibly even dead
weight. Maybe we should just remove them altogether?
J
Re: [IndexedDB] IDBCursor.update for cursors returned from IDBIndex.openCursor
On Thu, Sep 16, 2010 at 10:15 AM, Jeremy Orlow jor...@chromium.org wrote:
On Wed, Sep 15, 2010 at 10:45 PM, Jonas Sicking jo...@sicking.cc wrote:
Heh, I've also been thinking about this exact issue lately. There is a
similar question for IDBCursor.delete.
On Wed, Sep 15, 2010 at 8:55 AM, Jeremy Orlow jor...@chromium.org
wrote:
I think it's clear what IDBCursor does when created from
IDBObjectStore.openCursor or IDBIndex.openObjectCursor: it modifies the
objectStore's value and updates all indexes (or does nothing and returns
an
error if all of that can't be done while satisfying the constraints).
Agreed.
But what about IDBCursor.update when created from IDBIndex.openCursor?
I
see two options: we could modify the value within the objectStore's
value
that corresponds to the objectStore's key path or we could do like above
and
simply modify the objectStore's value.
There's also a third option: Throw an exception. Maybe that's what
you're referring to by make this unsupported below?
More concretely, if we have an object store with a id key path and an
index with a fname key path, and our index.openCursor() created cursor
is
currently on the {id: 22, fname: Fred} value (and thus cursor.key ==
Fred and cursor.value == 22), let's say I wanted to change the object
to
be {id: 23, fname: Fred}. In other words, I want id to change from 22
to
23. Which of the following should I write?
1) calling cursor.update(23) or
2) calling cursor.update({id: 23, fname: Fred})
The former seems to match the behavior of the IDBObjectStore.openCursor
and
IDBIndex.openObjectCursor better (i.e. it modifies the cursor.value).
The
latter intuitively seems like it'd be more useful. But to be honest, I
can't think of any use cases for either. Can anyone else? If not,
maybe we
should just make this unsupported for now?
The only use case I have thought of is wanting to update some set of
entries, where the best way to find these entries is through an index.
For example updating every entry with a specific shipping-id. You can
use IDBIndex.openObjectCursor for this, but that's slower than
IDBIndex.openCursor. So in the rare instance when you can make the
modification without inspecting the existing value (i.e. you only need
to write, read-modify-write), then IDBIndex.openCursor +
IDBCursor.update() would be a perf optimization.
On the other hand, it might be just as quick to call IDBObjectStore.put().
Since the use case if pretty weak (when would you be able to update an
entry without first reading the entry), and that you can seemingly get
the same performance using IDBObjectStore.put(), I would be fine with
making this unsupported.
As for IDBCursor.delete(), I can see a somewhat stronger use case
there. For example removing all entries with a specific shipping-id or
some such. If you can determine which entries should be removed purely
on the information in the index, then using IDBIndex.openCursor is
definitely faster than IDBIndex.openObjectCursor. So on one hand it
would be nice to allow people to use that. On the other hand, I
suspect you can get the same performance using IDBObjectStore.delete()
and we might want to be consistent with IDBCursor.update().
In this case I'm actually leaning towards allowing IDBCursor.delete(),
but I could go either way.
Wait a sec. What are the use cases for non-object cursors anyway? They
made perfect sense back when we allowed explicit index management, but now
they kind of seem like a premature optimization or possibly even dead
weight. Maybe we should just remove them altogether?
Actually, for that matter, are remove and update needed at all? I think
they may just be more cruft left over from the explicit index days. As far
as I can tell, any .delete or .remove should be doable via an objectCursor +
.puts/.removes on the objectStore.
J
Re: [IndexedDB] IDBCursor.update for cursors returned from IDBIndex.openCursor
On Thu, Sep 16, 2010 at 10:15 AM, Jeremy Orlow jor...@chromium.org wrote:
On Wed, Sep 15, 2010 at 10:45 PM, Jonas Sicking jo...@sicking.cc wrote:
Heh, I've also been thinking about this exact issue lately. There is a
similar question for IDBCursor.delete.
On Wed, Sep 15, 2010 at 8:55 AM, Jeremy Orlow jor...@chromium.org wrote:
I think it's clear what IDBCursor does when created from
IDBObjectStore.openCursor or IDBIndex.openObjectCursor: it modifies the
objectStore's value and updates all indexes (or does nothing and returns
an
error if all of that can't be done while satisfying the constraints).
Agreed.
But what about IDBCursor.update when created from IDBIndex.openCursor?
I
see two options: we could modify the value within the objectStore's
value
that corresponds to the objectStore's key path or we could do like above
and
simply modify the objectStore's value.
There's also a third option: Throw an exception. Maybe that's what
you're referring to by make this unsupported below?
More concretely, if we have an object store with a id key path and an
index with a fname key path, and our index.openCursor() created cursor
is
currently on the {id: 22, fname: Fred} value (and thus cursor.key ==
Fred and cursor.value == 22), let's say I wanted to change the object
to
be {id: 23, fname: Fred}. In other words, I want id to change from 22
to
23. Which of the following should I write?
1) calling cursor.update(23) or
2) calling cursor.update({id: 23, fname: Fred})
The former seems to match the behavior of the IDBObjectStore.openCursor
and
IDBIndex.openObjectCursor better (i.e. it modifies the cursor.value).
The
latter intuitively seems like it'd be more useful. But to be honest, I
can't think of any use cases for either. Can anyone else? If not,
maybe we
should just make this unsupported for now?
The only use case I have thought of is wanting to update some set of
entries, where the best way to find these entries is through an index.
For example updating every entry with a specific shipping-id. You can
use IDBIndex.openObjectCursor for this, but that's slower than
IDBIndex.openCursor. So in the rare instance when you can make the
modification without inspecting the existing value (i.e. you only need
to write, read-modify-write), then IDBIndex.openCursor +
IDBCursor.update() would be a perf optimization.
On the other hand, it might be just as quick to call IDBObjectStore.put().
Since the use case if pretty weak (when would you be able to update an
entry without first reading the entry), and that you can seemingly get
the same performance using IDBObjectStore.put(), I would be fine with
making this unsupported.
As for IDBCursor.delete(), I can see a somewhat stronger use case
there. For example removing all entries with a specific shipping-id or
some such. If you can determine which entries should be removed purely
on the information in the index, then using IDBIndex.openCursor is
definitely faster than IDBIndex.openObjectCursor. So on one hand it
would be nice to allow people to use that. On the other hand, I
suspect you can get the same performance using IDBObjectStore.delete()
and we might want to be consistent with IDBCursor.update().
In this case I'm actually leaning towards allowing IDBCursor.delete(),
but I could go either way.
Wait a sec. What are the use cases for non-object cursors anyway? They
made perfect sense back when we allowed explicit index management, but now
they kind of seem like a premature optimization or possibly even dead
weight. Maybe we should just remove them altogether?
I guess the reason for having non-object cursors is just performance:
it's probably faster to iterate a non-object cursor since you're only
iterating over the primary keys of the records in the object store and
not over the full records. But I can't really come up with a
convincing usecase to justify this. I think it's fine to remove them.
Andrei
Re: [IndexedDB] IDBCursor.update for cursors returned from IDBIndex.openCursor
On Thu, Sep 16, 2010 at 2:15 AM, Jeremy Orlow jor...@chromium.org wrote:
On Wed, Sep 15, 2010 at 10:45 PM, Jonas Sicking jo...@sicking.cc wrote:
Heh, I've also been thinking about this exact issue lately. There is a
similar question for IDBCursor.delete.
On Wed, Sep 15, 2010 at 8:55 AM, Jeremy Orlow jor...@chromium.org wrote:
I think it's clear what IDBCursor does when created from
IDBObjectStore.openCursor or IDBIndex.openObjectCursor: it modifies the
objectStore's value and updates all indexes (or does nothing and returns
an
error if all of that can't be done while satisfying the constraints).
Agreed.
But what about IDBCursor.update when created from IDBIndex.openCursor?
I
see two options: we could modify the value within the objectStore's
value
that corresponds to the objectStore's key path or we could do like above
and
simply modify the objectStore's value.
There's also a third option: Throw an exception. Maybe that's what
you're referring to by make this unsupported below?
More concretely, if we have an object store with a id key path and an
index with a fname key path, and our index.openCursor() created cursor
is
currently on the {id: 22, fname: Fred} value (and thus cursor.key ==
Fred and cursor.value == 22), let's say I wanted to change the object
to
be {id: 23, fname: Fred}. In other words, I want id to change from 22
to
23. Which of the following should I write?
1) calling cursor.update(23) or
2) calling cursor.update({id: 23, fname: Fred})
The former seems to match the behavior of the IDBObjectStore.openCursor
and
IDBIndex.openObjectCursor better (i.e. it modifies the cursor.value).
The
latter intuitively seems like it'd be more useful. But to be honest, I
can't think of any use cases for either. Can anyone else? If not,
maybe we
should just make this unsupported for now?
The only use case I have thought of is wanting to update some set of
entries, where the best way to find these entries is through an index.
For example updating every entry with a specific shipping-id. You can
use IDBIndex.openObjectCursor for this, but that's slower than
IDBIndex.openCursor. So in the rare instance when you can make the
modification without inspecting the existing value (i.e. you only need
to write, read-modify-write), then IDBIndex.openCursor +
IDBCursor.update() would be a perf optimization.
On the other hand, it might be just as quick to call IDBObjectStore.put().
Since the use case if pretty weak (when would you be able to update an
entry without first reading the entry), and that you can seemingly get
the same performance using IDBObjectStore.put(), I would be fine with
making this unsupported.
As for IDBCursor.delete(), I can see a somewhat stronger use case
there. For example removing all entries with a specific shipping-id or
some such. If you can determine which entries should be removed purely
on the information in the index, then using IDBIndex.openCursor is
definitely faster than IDBIndex.openObjectCursor. So on one hand it
would be nice to allow people to use that. On the other hand, I
suspect you can get the same performance using IDBObjectStore.delete()
and we might want to be consistent with IDBCursor.update().
In this case I'm actually leaning towards allowing IDBCursor.delete(),
but I could go either way.
Wait a sec. What are the use cases for non-object cursors anyway? They
made perfect sense back when we allowed explicit index management, but now
they kind of seem like a premature optimization or possibly even dead
weight. Maybe we should just remove them altogether?
They are still useful for joins. Consider an objectStore employees:
{ id: 1, name: Sven, employed: 1-1-2010 }
{ id: 2, name: Bert, employed: 5-1-2009 }
{ id: 3, name: Adam, employed: 6-6-2008 }
And objectStore sales
{ seller: 1, candyName: lollipop, quantity: 5, date: 9-15-2010 }
{ seller: 1, candyName: swedish fish, quantity: 12, date: 9-15-2010 }
{ seller: 2, candyName: jelly belly, quantity: 3, date: 9-14-2010 }
{ seller: 3, candyName: heath bar, quantity: 3, date: 9-13-2010 }
If you want to display the amount of sales per person, sorted by names
of sales person, you could do this by first creating and index for
employees with keyPath name. You'd then use IDBIndex.openCursor to
iterate that index, and for each entry find all entries in the sales
objectStore where seller matches the cursors .value.
So in this case you don't actually need any data from the employees
objectStore, all the data is available in the index. Thus it is
sufficient, and faster, to use openCursor than openObjectCursor.
In general, it's a common optimization to stick enough data in an
index that you don't have to actually look up in the objectStore
itself. This is slightly less commonly doable since we have relatively
simple indexes so far. But still doable as the example above shows.
Once we add support for arrays as keys this
Re: [IndexedDB] IDBCursor.update for cursors returned from IDBIndex.openCursor
On Thu, Sep 16, 2010 at 4:08 AM, Jeremy Orlow jor...@chromium.org wrote:
On Thu, Sep 16, 2010 at 10:15 AM, Jeremy Orlow jor...@chromium.org wrote:
On Wed, Sep 15, 2010 at 10:45 PM, Jonas Sicking jo...@sicking.cc wrote:
Heh, I've also been thinking about this exact issue lately. There is a
similar question for IDBCursor.delete.
On Wed, Sep 15, 2010 at 8:55 AM, Jeremy Orlow jor...@chromium.org
wrote:
I think it's clear what IDBCursor does when created from
IDBObjectStore.openCursor or IDBIndex.openObjectCursor: it modifies the
objectStore's value and updates all indexes (or does nothing and
returns an
error if all of that can't be done while satisfying the constraints).
Agreed.
But what about IDBCursor.update when created from IDBIndex.openCursor?
I
see two options: we could modify the value within the objectStore's
value
that corresponds to the objectStore's key path or we could do like
above and
simply modify the objectStore's value.
There's also a third option: Throw an exception. Maybe that's what
you're referring to by make this unsupported below?
More concretely, if we have an object store with a id key path and an
index with a fname key path, and our index.openCursor() created
cursor is
currently on the {id: 22, fname: Fred} value (and thus cursor.key ==
Fred and cursor.value == 22), let's say I wanted to change the object
to
be {id: 23, fname: Fred}. In other words, I want id to change from
22 to
23. Which of the following should I write?
1) calling cursor.update(23) or
2) calling cursor.update({id: 23, fname: Fred})
The former seems to match the behavior of the IDBObjectStore.openCursor
and
IDBIndex.openObjectCursor better (i.e. it modifies the cursor.value).
The
latter intuitively seems like it'd be more useful. But to be honest, I
can't think of any use cases for either. Can anyone else? If not,
maybe we
should just make this unsupported for now?
The only use case I have thought of is wanting to update some set of
entries, where the best way to find these entries is through an index.
For example updating every entry with a specific shipping-id. You can
use IDBIndex.openObjectCursor for this, but that's slower than
IDBIndex.openCursor. So in the rare instance when you can make the
modification without inspecting the existing value (i.e. you only need
to write, read-modify-write), then IDBIndex.openCursor +
IDBCursor.update() would be a perf optimization.
On the other hand, it might be just as quick to call
IDBObjectStore.put().
Since the use case if pretty weak (when would you be able to update an
entry without first reading the entry), and that you can seemingly get
the same performance using IDBObjectStore.put(), I would be fine with
making this unsupported.
As for IDBCursor.delete(), I can see a somewhat stronger use case
there. For example removing all entries with a specific shipping-id or
some such. If you can determine which entries should be removed purely
on the information in the index, then using IDBIndex.openCursor is
definitely faster than IDBIndex.openObjectCursor. So on one hand it
would be nice to allow people to use that. On the other hand, I
suspect you can get the same performance using IDBObjectStore.delete()
and we might want to be consistent with IDBCursor.update().
In this case I'm actually leaning towards allowing IDBCursor.delete(),
but I could go either way.
Wait a sec. What are the use cases for non-object cursors anyway? They
made perfect sense back when we allowed explicit index management, but now
they kind of seem like a premature optimization or possibly even dead
weight. Maybe we should just remove them altogether?
Actually, for that matter, are remove and update needed at all? I think
they may just be more cruft left over from the explicit index days. As far
as I can tell, any .delete or .remove should be doable via an objectCursor +
.puts/.removes on the objectStore.
They are not strictly needed, but they are a decent convinence
feature, and with a proper implementation they can even be a
performance optimization. With a cursor iterating a b-tree you can let
the cursor keep a pointer to the b-tree entry. They way .delete and
.update doesn't have to do a b-tree lookup at all.
We're currently not able to do this since our backend (sqlite) doesn't
have good enough cursor support, but I suspect that this will change
at some point in the future. In the mean time it seems like a good
thing to allow people to use API that will be faster in the future.
/ Jonas
Re: [IndexedDB] IDBCursor.update for cursors returned from IDBIndex.openCursor
On Thu, Sep 16, 2010 at 8:53 PM, Jonas Sicking jo...@sicking.cc wrote:
On Thu, Sep 16, 2010 at 2:15 AM, Jeremy Orlow jor...@chromium.org wrote:
Wait a sec. What are the use cases for non-object cursors anyway? They
made perfect sense back when we allowed explicit index management, but
now
they kind of seem like a premature optimization or possibly even dead
weight. Maybe we should just remove them altogether?
They are still useful for joins. Consider an objectStore employees:
{ id: 1, name: Sven, employed: 1-1-2010 }
{ id: 2, name: Bert, employed: 5-1-2009 }
{ id: 3, name: Adam, employed: 6-6-2008 }
And objectStore sales
{ seller: 1, candyName: lollipop, quantity: 5, date: 9-15-2010 }
{ seller: 1, candyName: swedish fish, quantity: 12, date: 9-15-2010 }
{ seller: 2, candyName: jelly belly, quantity: 3, date: 9-14-2010 }
{ seller: 3, candyName: heath bar, quantity: 3, date: 9-13-2010 }
If you want to display the amount of sales per person, sorted by names
of sales person, you could do this by first creating and index for
employees with keyPath name. You'd then use IDBIndex.openCursor to
iterate that index, and for each entry find all entries in the sales
objectStore where seller matches the cursors .value.
So in this case you don't actually need any data from the employees
objectStore, all the data is available in the index. Thus it is
sufficient, and faster, to use openCursor than openObjectCursor.
In general, it's a common optimization to stick enough data in an
index that you don't have to actually look up in the objectStore
itself. This is slightly less commonly doable since we have relatively
simple indexes so far. But still doable as the example above shows.
Once we add support for arrays as keys this will be much more common
as you can then stick arbitrary data into the index by simply adding
additional entries to all key arrays. And even more so once we
(probably in a future version) add support for computed indexes.
On Thu, Sep 16, 2010 at 8:57 PM, Jonas Sicking jo...@sicking.cc wrote:
On Thu, Sep 16, 2010 at 4:08 AM, Jeremy Orlow jor...@chromium.org wrote:
Actually, for that matter, are remove and update needed at all? I think
they may just be more cruft left over from the explicit index days. As
far
as I can tell, any .delete or .remove should be doable via an
objectCursor +
.puts/.removes on the objectStore.
They are not strictly needed, but they are a decent convinence
feature, and with a proper implementation they can even be a
performance optimization. With a cursor iterating a b-tree you can let
the cursor keep a pointer to the b-tree entry. They way .delete and
.update doesn't have to do a b-tree lookup at all.
We're currently not able to do this since our backend (sqlite) doesn't
have good enough cursor support, but I suspect that this will change
at some point in the future. In the mean time it seems like a good
thing to allow people to use API that will be faster in the future.
All your arguments revolve around what the spec and implementations might do
in the future. Typically we add API surface area only for use cases that
are currently impossible to satisfy or proven performance bottlenecks. I
agree that it's likely implementations will want to do optimizations like
this in the future, but until they do, it'll be hard to really understand
the implications and complications that might arrise.
Given that we can easily add to the API in the future but it's nearly
impossible to take it away without breaking sites, I think it's prudent to
remove this unnecessary surface area. There are MANY other proven
performance/concurrency features in other database systems that we've chosen
to leave out for now for this reason. I don't see what's special about this
case.
I think we should leave in openObjectCursor/getObject but remove
openCursor/get for now. We can then revisit any of these features as soon
as there are implementations (both in the UAs and in web sites) mature
enough for us to get real feedback on the features.
J
Re: [IndexedDB] IDBCursor.update for cursors returned from IDBIndex.openCursor
On Thu, Sep 16, 2010 at 2:23 PM, Jeremy Orlow jor...@chromium.org wrote: I think we should leave in openObjectCursor/getObject but remove openCursor/get for now. We can then revisit any of these features as soon as there are implementations (both in the UAs and in web sites) mature enough for us to get real feedback on the features. If you do so, could you migrate the names over? No sense having a useless Object hanging around in the name. Terse is better. ~TJ
Re: [IndexedDB] IDBCursor.update for cursors returned from IDBIndex.openCursor
On Thu, Sep 16, 2010 at 11:04 PM, Tab Atkins Jr. jackalm...@gmail.comwrote: On Thu, Sep 16, 2010 at 2:23 PM, Jeremy Orlow jor...@chromium.org wrote: I think we should leave in openObjectCursor/getObject but remove openCursor/get for now. We can then revisit any of these features as soon as there are implementations (both in the UAs and in web sites) mature enough for us to get real feedback on the features. If you do so, could you migrate the names over? No sense having a useless Object hanging around in the name. Terse is better. When I wrote that, my concern was backing ourselves into a corner in terms of names if/when we do add back such an API. But now that I think about it, it seems as though the normal operation would be getting the objects (i.e. the value in the ObjectStore) and getting just the primary key would be more of an optimization. So yeah, I agree that getObject-get and openObjectCursor-openCursor is the right naming scheme here. J
Re: [IndexedDB] IDBCursor.update for cursors returned from IDBIndex.openCursor
On Thu, Sep 16, 2010 at 2:23 PM, Jeremy Orlow jor...@chromium.org wrote:
On Thu, Sep 16, 2010 at 8:53 PM, Jonas Sicking jo...@sicking.cc wrote:
On Thu, Sep 16, 2010 at 2:15 AM, Jeremy Orlow jor...@chromium.org wrote:
Wait a sec. What are the use cases for non-object cursors anyway? They
made perfect sense back when we allowed explicit index management, but
now
they kind of seem like a premature optimization or possibly even dead
weight. Maybe we should just remove them altogether?
They are still useful for joins. Consider an objectStore employees:
{ id: 1, name: Sven, employed: 1-1-2010 }
{ id: 2, name: Bert, employed: 5-1-2009 }
{ id: 3, name: Adam, employed: 6-6-2008 }
And objectStore sales
{ seller: 1, candyName: lollipop, quantity: 5, date: 9-15-2010 }
{ seller: 1, candyName: swedish fish, quantity: 12, date: 9-15-2010 }
{ seller: 2, candyName: jelly belly, quantity: 3, date: 9-14-2010 }
{ seller: 3, candyName: heath bar, quantity: 3, date: 9-13-2010 }
If you want to display the amount of sales per person, sorted by names
of sales person, you could do this by first creating and index for
employees with keyPath name. You'd then use IDBIndex.openCursor to
iterate that index, and for each entry find all entries in the sales
objectStore where seller matches the cursors .value.
So in this case you don't actually need any data from the employees
objectStore, all the data is available in the index. Thus it is
sufficient, and faster, to use openCursor than openObjectCursor.
In general, it's a common optimization to stick enough data in an
index that you don't have to actually look up in the objectStore
itself. This is slightly less commonly doable since we have relatively
simple indexes so far. But still doable as the example above shows.
Once we add support for arrays as keys this will be much more common
as you can then stick arbitrary data into the index by simply adding
additional entries to all key arrays. And even more so once we
(probably in a future version) add support for computed indexes.
On Thu, Sep 16, 2010 at 8:57 PM, Jonas Sicking jo...@sicking.cc wrote:
On Thu, Sep 16, 2010 at 4:08 AM, Jeremy Orlow jor...@chromium.org wrote:
Actually, for that matter, are remove and update needed at all? I think
they may just be more cruft left over from the explicit index days. As
far
as I can tell, any .delete or .remove should be doable via an
objectCursor +
.puts/.removes on the objectStore.
They are not strictly needed, but they are a decent convinence
feature, and with a proper implementation they can even be a
performance optimization. With a cursor iterating a b-tree you can let
the cursor keep a pointer to the b-tree entry. They way .delete and
.update doesn't have to do a b-tree lookup at all.
We're currently not able to do this since our backend (sqlite) doesn't
have good enough cursor support, but I suspect that this will change
at some point in the future. In the mean time it seems like a good
thing to allow people to use API that will be faster in the future.
All your arguments revolve around what the spec and implementations might do
in the future.
I disagree. The IDBIndex.openCursor example I included uses only
existing API, and is a performance improvement in at least our current
implementation. Would be interested to hear if it's not a performance
improvement in others.
Typically we add API surface area only for use cases that
are currently impossible to satisfy or proven performance bottlenecks. I
agree that it's likely implementations will want to do optimizations like
this in the future, but until they do, it'll be hard to really understand
the implications and complications that might arrise.
That's not entirely true. All the databases I have worked with have
had significant performance degradations when having to look up the
main table contents rather than simply looking at the contents in the
index. I doubt that we'll be able to create a backend where that is
not true. So I think we should assume that object cursors are slower
than plain cursors.
Further, I think we should get users on APIs that we are likely to
implement with a higher performance. For example, I think sqlite
doesn't support having multiple write transactions to the same
database, even if those are to different tables. Thus the whole API of
specifying which objectStores you want to include in a transaction is
purely for future optimizations in at least implementations backed by
sqlite.
I especially think these APIs are worth it given that it's low cost to
implement, and adds convenience value to users even if implementations
aren't faster yet.
/ Jonas
[IndexedDB] IDBCursor.update for cursors returned from IDBIndex.openCursor
I think it's clear what IDBCursor does when created from
IDBObjectStore.openCursor or IDBIndex.openObjectCursor: it modifies the
objectStore's value and updates all indexes (or does nothing and returns an
error if all of that can't be done while satisfying the constraints).
But what about IDBCursor.update when created from IDBIndex.openCursor? I
see two options: we could modify the value within the objectStore's value
that corresponds to the objectStore's key path or we could do like above and
simply modify the objectStore's value.
More concretely, if we have an object store with a id key path and an
index with a fname key path, and our index.openCursor() created cursor is
currently on the {id: 22, fname: Fred} value (and thus cursor.key ==
Fred and cursor.value == 22), let's say I wanted to change the object to
be {id: 23, fname: Fred}. In other words, I want id to change from 22 to
23. Which of the following should I write?
1) calling cursor.update(23) or
2) calling cursor.update({id: 23, fname: Fred})
The former seems to match the behavior of the IDBObjectStore.openCursor and
IDBIndex.openObjectCursor better (i.e. it modifies the cursor.value). The
latter intuitively seems like it'd be more useful. But to be honest, I
can't think of any use cases for either. Can anyone else? If not, maybe we
should just make this unsupported for now?
Btw, http://www.w3.org/TR/IndexedDB/#widl-IDBCursor-update probably needs to
include some other errors as well since we need to return something when a
constraint can't be met with the new value.
Also, I'll note that while writing this code, I've gotten confused a couple
times about what is the key and what is the value for Index cursors. I'm a
bit concerned web developers will as well.
J
