Re: [IndexedDB] Cursors and modifications
On Jul 16, 2010, at 5:47 AM, Pablo Castro wrote: From: Jonas Sicking [mailto:jo...@sicking.cc] Sent: Thursday, July 15, 2010 11:59 AM On Thu, Jul 15, 2010 at 11:02 AM, Pablo Castro pablo.cas...@microsoft.com wrote: From: jor...@google.com [mailto:jor...@google.com] On Behalf Of Jeremy Orlow Sent: Thursday, July 15, 2010 2:04 AM On Thu, Jul 15, 2010 at 2:44 AM, Jonas Sicking jo...@sicking.cc wrote: On Wed, Jul 14, 2010 at 6:20 PM, Pablo Castro pablo.cas...@microsoft.com wrote: If it's accurate, as a side note, for the async API it seems that this makes it more interesting to enforce callback order, so we can more easily explain what we mean by before. Indeed. What do you mean by enforce callback order? Are you saying that callbacks should be done in the order the requests are made (rather than prioritizing cursor callbacks)? (That's how I read it, but Jonas' Indeed makes me suspect I missed something. :-) That's right. If changes are visible as they are made within a transaction, then reordering the callbacks would have a visible effect. In particular if we prioritize the cursor callbacks then you'll tend to see a callback for a cursor move before you see a callback for say an add/modify, and it's not clear at that point whether the add/modify happened already and is visible (but the callback didn't land yet) or if the change hasn't happened yet. If callbacks are in order, you see changes within your transaction strictly in the order that each request is made, avoiding surprises in cursor callbacks. Oh, I took what you said just as that we need to have a defined callback order. Not anything in particular what that definition should be. Regarding when a modification happens, I think the design should be that changes logically happen as soon as the 'success' call is fired. Any success calls after that will see the modified values. Yep, I agree with this, a change happened for sure when you see the success callback. Before that you may or may not observe the change if you do a get or open a cursor to look at the record. I still think given the quite substantial speedups gained from prioritizing cursor callbacks, that it's the right thing to do. It arguably also has some benefits from a practical point of view when it comes to the very topic we're discussing. If we prioritize cursor callbacks, that makes it much easier to iterate a set of entries and update them, without having to worry about those updates messing up your iterator. I hear you on the perf implications, but I'm worried that non-sequential order for callbacks will be completely non-intuitive for users. In particular, if you're changing things as you scan a cursor, if then you cursor through the changes you're not sure if you'll see the changes or not (because the callback is the only definitive point where the change is visible. That seems quite problematic... One use case that is interesting is simultaneously walking over two different cursors, e.g., to process some compound join. In that case, the application determines how fast it wants to move on any of a number of open cursors. Would this be supported with this behavior? Nikunj
Re: [IndexedDB] Cursors and modifications
On Thu, Jul 22, 2010 at 3:49 AM, Nikunj Mehta nik...@o-micron.com wrote: On Jul 16, 2010, at 5:47 AM, Pablo Castro wrote: From: Jonas Sicking [mailto:jo...@sicking.cc] Sent: Thursday, July 15, 2010 11:59 AM On Thu, Jul 15, 2010 at 11:02 AM, Pablo Castro pablo.cas...@microsoft.com wrote: From: jor...@google.com [mailto:jor...@google.com] On Behalf Of Jeremy Orlow Sent: Thursday, July 15, 2010 2:04 AM On Thu, Jul 15, 2010 at 2:44 AM, Jonas Sicking jo...@sicking.cc wrote: On Wed, Jul 14, 2010 at 6:20 PM, Pablo Castro pablo.cas...@microsoft.com wrote: If it's accurate, as a side note, for the async API it seems that this makes it more interesting to enforce callback order, so we can more easily explain what we mean by before. Indeed. What do you mean by enforce callback order? Are you saying that callbacks should be done in the order the requests are made (rather than prioritizing cursor callbacks)? (That's how I read it, but Jonas' Indeed makes me suspect I missed something. :-) That's right. If changes are visible as they are made within a transaction, then reordering the callbacks would have a visible effect. In particular if we prioritize the cursor callbacks then you'll tend to see a callback for a cursor move before you see a callback for say an add/modify, and it's not clear at that point whether the add/modify happened already and is visible (but the callback didn't land yet) or if the change hasn't happened yet. If callbacks are in order, you see changes within your transaction strictly in the order that each request is made, avoiding surprises in cursor callbacks. Oh, I took what you said just as that we need to have a defined callback order. Not anything in particular what that definition should be. Regarding when a modification happens, I think the design should be that changes logically happen as soon as the 'success' call is fired. Any success calls after that will see the modified values. Yep, I agree with this, a change happened for sure when you see the success callback. Before that you may or may not observe the change if you do a get or open a cursor to look at the record. I still think given the quite substantial speedups gained from prioritizing cursor callbacks, that it's the right thing to do. It arguably also has some benefits from a practical point of view when it comes to the very topic we're discussing. If we prioritize cursor callbacks, that makes it much easier to iterate a set of entries and update them, without having to worry about those updates messing up your iterator. I hear you on the perf implications, but I'm worried that non-sequential order for callbacks will be completely non-intuitive for users. In particular, if you're changing things as you scan a cursor, if then you cursor through the changes you're not sure if you'll see the changes or not (because the callback is the only definitive point where the change is visible. That seems quite problematic... One use case that is interesting is simultaneously walking over two different cursors, e.g., to process some compound join. In that case, the application determines how fast it wants to move on any of a number of open cursors. Would this be supported with this behavior? Yes. cursor.continue() calls still execute in the order they are called, so you can alternate walking two separate cursors without any changes in callback order. / Jonas
Re: [IndexedDB] Cursors and modifications
On Thu, Jul 15, 2010 at 2:44 AM, Jonas Sicking jo...@sicking.cc wrote:
On Wed, Jul 14, 2010 at 6:20 PM, Pablo Castro
pablo.cas...@microsoft.com wrote:
Making sure I get the essence of this thread: we're saying that cursors
see live changes as they happen on objects that are after the object
you're currently standing on;
Yes.
and of course, any other activity within a transaction sees all the
changes that happened before that activity took place. Is that accurate?
Yes. All other activity sees all changes as soon as they have happened.
If it's accurate, as a side note, for the async API it seems that this
makes it more interesting to enforce callback order, so we can more easily
explain what we mean by before.
Indeed.
What do you mean by enforce callback order? Are you saying that callbacks
should be done in the order the requests are made (rather than prioritizing
cursor callbacks)? (That's how I read it, but Jonas' Indeed makes me
suspect I missed something. :-)
J
/ Jonas
From: jor...@google.com [mailto:jor...@google.com] On Behalf Of Jeremy
Orlow
Sent: Wednesday, July 14, 2010 9:27 AM
On Wed, Jul 14, 2010 at 5:17 PM, Jonas Sicking jo...@sicking.cc wrote:
On Wed, Jul 14, 2010 at 5:12 AM, Jeremy Orlow jor...@chromium.org
wrote:
On Thu, Jul 8, 2010 at 8:42 PM, Jonas Sicking jo...@sicking.cc wrote:
On Mon, Jul 5, 2010 at 9:45 AM, Andrei Popescu andr...@google.com
wrote:
On Sat, Jul 3, 2010 at 2:09 AM, Jonas Sicking jo...@sicking.cc
wrote:
On Fri, Jul 2, 2010 at 5:44 PM, Andrei Popescu andr...@google.com
wrote:
On Sat, Jul 3, 2010 at 1:14 AM, Jonas Sicking jo...@sicking.cc
wrote:
On Fri, Jul 2, 2010 at 4:40 PM, Pablo Castro
pablo.cas...@microsoft.com wrote:
From: public-webapps-requ...@w3.org
[mailto:public-webapps-requ...@w3.org] On Behalf Of Jonas
Sicking
Sent: Friday, July 02, 2010 4:00 PM
We ran into an complicated issue while implementing IndexedDB.
In
short, what should happen if an object store is modified while
a cursor is
iterating it? Note that the modification can be done within
the same
transaction, so the read/write locks preventing several
transactions from
accessing the same table isn't helping here.
Detailed problem description (this assumes the API proposed by
mozilla):
Consider a objectStore words containing the following
objects:
{ name: alpha }
{ name: bravo }
{ name: charlie }
{ name: delta }
and the following program (db is a previously opened
IDBDatabase):
var trans = db.transaction([words], READ_WRITE); var cursor;
var
result = []; trans.objectStore(words).openCursor().onsuccess
= function(e)
{
cursor = e.result;
result.push(cursor.value);
cursor.continue();
}
trans.objectStore(words).get(delta).onsuccess = function(e)
{
trans.objectStore(words).put({ name: delta,
myModifiedValue:
17 }); }
When the cursor reads the delta entry, will it see the
'myModifiedValue' property? Since we so far has defined that
the callback
order is defined to be the request order, that means that
put request
will be finished before the delta entry is iterated by the
cursor.
The problem is even more serious with cursors that iterate
indexes.
Here a modification can even affect the position of the
currently
iterated object in the index, and the modification can (if i'm
reading the
spec correctly) come from the cursor itself.
Consider the following objectStore people with keyPath name
containing the following objects:
{ name: Adam, count: 30 }
{ name: Bertil, count: 31 }
{ name: Cesar, count: 32 }
{ name: David, count: 33 }
{ name: Erik, count: 35 }
and an index countIndex with keyPath count. What would the
following code do?
results = [];
db.objectStore(people,
READ_WRITE).index(countIndex).openObjectCursor().onsuccess =
function (e) {
cursor = e.result;
if (!cursor) {
alert(results);
return;
}
if (cursor.value.name == Bertil) {
cursor.update({name: Bertil, count: 34 });
}
results.push(cursor.value.name);
cursor.continue();
};
What does this alert? Would it alert Adam,Bertil,Erik as the
cursor would stay on the Bertil object as it is moved in the
index? Or
would it alert Adam,Bertil,Cesar,David,Bertil,Erik as we
would iterate
Bertil again at its new position in the index?
My first reaction is that both from the expected behavior of
perspective (transaction is the scope of isolation) and from the
implementation perspective it would be better to see live changes
if they
happened in the same transaction as the cursor (over a store or
index). So
in your example you would iterate one of the rows twice.
Maintaining order
and membership stable would mean creating another scope of
isolation within
the transaction, which to me would be unusual
RE: [IndexedDB] Cursors and modifications
From: jor...@google.com [mailto:jor...@google.com] On Behalf Of Jeremy Orlow Sent: Thursday, July 15, 2010 2:04 AM On Thu, Jul 15, 2010 at 2:44 AM, Jonas Sicking jo...@sicking.cc wrote: On Wed, Jul 14, 2010 at 6:20 PM, Pablo Castro pablo.cas...@microsoft.com wrote: If it's accurate, as a side note, for the async API it seems that this makes it more interesting to enforce callback order, so we can more easily explain what we mean by before. Indeed. What do you mean by enforce callback order? Are you saying that callbacks should be done in the order the requests are made (rather than prioritizing cursor callbacks)? (That's how I read it, but Jonas' Indeed makes me suspect I missed something. :-) That's right. If changes are visible as they are made within a transaction, then reordering the callbacks would have a visible effect. In particular if we prioritize the cursor callbacks then you'll tend to see a callback for a cursor move before you see a callback for say an add/modify, and it's not clear at that point whether the add/modify happened already and is visible (but the callback didn't land yet) or if the change hasn't happened yet. If callbacks are in order, you see changes within your transaction strictly in the order that each request is made, avoiding surprises in cursor callbacks. -pablo
Re: [IndexedDB] Cursors and modifications
On Thu, Jul 15, 2010 at 11:02 AM, Pablo Castro pablo.cas...@microsoft.com wrote: From: jor...@google.com [mailto:jor...@google.com] On Behalf Of Jeremy Orlow Sent: Thursday, July 15, 2010 2:04 AM On Thu, Jul 15, 2010 at 2:44 AM, Jonas Sicking jo...@sicking.cc wrote: On Wed, Jul 14, 2010 at 6:20 PM, Pablo Castro pablo.cas...@microsoft.com wrote: If it's accurate, as a side note, for the async API it seems that this makes it more interesting to enforce callback order, so we can more easily explain what we mean by before. Indeed. What do you mean by enforce callback order? Are you saying that callbacks should be done in the order the requests are made (rather than prioritizing cursor callbacks)? (That's how I read it, but Jonas' Indeed makes me suspect I missed something. :-) That's right. If changes are visible as they are made within a transaction, then reordering the callbacks would have a visible effect. In particular if we prioritize the cursor callbacks then you'll tend to see a callback for a cursor move before you see a callback for say an add/modify, and it's not clear at that point whether the add/modify happened already and is visible (but the callback didn't land yet) or if the change hasn't happened yet. If callbacks are in order, you see changes within your transaction strictly in the order that each request is made, avoiding surprises in cursor callbacks. Oh, I took what you said just as that we need to have a defined callback order. Not anything in particular what that definition should be. Regarding when a modification happens, I think the design should be that changes logically happen as soon as the 'success' call is fired. Any success calls after that will see the modified values. I still think given the quite substantial speedups gained from prioritizing cursor callbacks, that it's the right thing to do. It arguably also has some benefits from a practical point of view when it comes to the very topic we're discussing. If we prioritize cursor callbacks, that makes it much easier to iterate a set of entries and update them, without having to worry about those updates messing up your iterator. / Jonas
RE: [IndexedDB] Cursors and modifications
From: Jonas Sicking [mailto:jo...@sicking.cc] Sent: Thursday, July 15, 2010 11:59 AM On Thu, Jul 15, 2010 at 11:02 AM, Pablo Castro pablo.cas...@microsoft.com wrote: From: jor...@google.com [mailto:jor...@google.com] On Behalf Of Jeremy Orlow Sent: Thursday, July 15, 2010 2:04 AM On Thu, Jul 15, 2010 at 2:44 AM, Jonas Sicking jo...@sicking.cc wrote: On Wed, Jul 14, 2010 at 6:20 PM, Pablo Castro pablo.cas...@microsoft.com wrote: If it's accurate, as a side note, for the async API it seems that this makes it more interesting to enforce callback order, so we can more easily explain what we mean by before. Indeed. What do you mean by enforce callback order? Are you saying that callbacks should be done in the order the requests are made (rather than prioritizing cursor callbacks)? (That's how I read it, but Jonas' Indeed makes me suspect I missed something. :-) That's right. If changes are visible as they are made within a transaction, then reordering the callbacks would have a visible effect. In particular if we prioritize the cursor callbacks then you'll tend to see a callback for a cursor move before you see a callback for say an add/modify, and it's not clear at that point whether the add/modify happened already and is visible (but the callback didn't land yet) or if the change hasn't happened yet. If callbacks are in order, you see changes within your transaction strictly in the order that each request is made, avoiding surprises in cursor callbacks. Oh, I took what you said just as that we need to have a defined callback order. Not anything in particular what that definition should be. Regarding when a modification happens, I think the design should be that changes logically happen as soon as the 'success' call is fired. Any success calls after that will see the modified values. Yep, I agree with this, a change happened for sure when you see the success callback. Before that you may or may not observe the change if you do a get or open a cursor to look at the record. I still think given the quite substantial speedups gained from prioritizing cursor callbacks, that it's the right thing to do. It arguably also has some benefits from a practical point of view when it comes to the very topic we're discussing. If we prioritize cursor callbacks, that makes it much easier to iterate a set of entries and update them, without having to worry about those updates messing up your iterator. I hear you on the perf implications, but I'm worried that non-sequential order for callbacks will be completely non-intuitive for users. In particular, if you're changing things as you scan a cursor, if then you cursor through the changes you're not sure if you'll see the changes or not (because the callback is the only definitive point where the change is visible. That seems quite problematic... -pablo
Re: [IndexedDB] Cursors and modifications
On Thu, Jul 8, 2010 at 8:42 PM, Jonas Sicking jo...@sicking.cc wrote:
On Mon, Jul 5, 2010 at 9:45 AM, Andrei Popescu andr...@google.com wrote:
On Sat, Jul 3, 2010 at 2:09 AM, Jonas Sicking jo...@sicking.cc wrote:
On Fri, Jul 2, 2010 at 5:44 PM, Andrei Popescu andr...@google.com
wrote:
On Sat, Jul 3, 2010 at 1:14 AM, Jonas Sicking jo...@sicking.cc
wrote:
On Fri, Jul 2, 2010 at 4:40 PM, Pablo Castro
pablo.cas...@microsoft.com wrote:
From: public-webapps-requ...@w3.org [mailto:
public-webapps-requ...@w3.org] On Behalf Of Jonas Sicking
Sent: Friday, July 02, 2010 4:00 PM
We ran into an complicated issue while implementing IndexedDB. In
short, what should happen if an object store is modified while a cursor is
iterating it? Note that the modification can be done within the same
transaction, so the read/write locks preventing several transactions from
accessing the same table isn't helping here.
Detailed problem description (this assumes the API proposed by
mozilla):
Consider a objectStore words containing the following objects:
{ name: alpha }
{ name: bravo }
{ name: charlie }
{ name: delta }
and the following program (db is a previously opened IDBDatabase):
var trans = db.transaction([words], READ_WRITE); var cursor; var
result = []; trans.objectStore(words).openCursor().onsuccess = function(e)
{
cursor = e.result;
result.push(cursor.value);
cursor.continue();
}
trans.objectStore(words).get(delta).onsuccess = function(e) {
trans.objectStore(words).put({ name: delta, myModifiedValue:
17 }); }
When the cursor reads the delta entry, will it see the
'myModifiedValue' property? Since we so far has defined that the callback
order is defined to be the request order, that means that put request
will be finished before the delta entry is iterated by the cursor.
The problem is even more serious with cursors that iterate indexes.
Here a modification can even affect the position of the currently
iterated object in the index, and the modification can (if i'm reading the
spec correctly) come from the cursor itself.
Consider the following objectStore people with keyPath name
containing the following objects:
{ name: Adam, count: 30 }
{ name: Bertil, count: 31 }
{ name: Cesar, count: 32 }
{ name: David, count: 33 }
{ name: Erik, count: 35 }
and an index countIndex with keyPath count. What would the
following code do?
results = [];
db.objectStore(people,
READ_WRITE).index(countIndex).openObjectCursor().onsuccess =
function (e) {
cursor = e.result;
if (!cursor) {
alert(results);
return;
}
if (cursor.value.name == Bertil) {
cursor.update({name: Bertil, count: 34 });
}
results.push(cursor.value.name);
cursor.continue();
};
What does this alert? Would it alert Adam,Bertil,Erik as the
cursor would stay on the Bertil object as it is moved in the index? Or
would it alert Adam,Bertil,Cesar,David,Bertil,Erik as we would iterate
Bertil again at its new position in the index?
My first reaction is that both from the expected behavior of
perspective (transaction is the scope of isolation) and from the
implementation perspective it would be better to see live changes if they
happened in the same transaction as the cursor (over a store or index). So
in your example you would iterate one of the rows twice. Maintaining order
and membership stable would mean creating another scope of isolation within
the transaction, which to me would be unusual and it would be probably quite
painful to implement without spilling a copy of the records to disk (at
least a copy of the keys/order if you don't care about protecting from
changes that don't affect membership/order; some databases call these keyset
cursors).
We could say that cursors always iterate snapshots, however this
introduces MVCC. Though it seems to me that SNAPSHOT_READ already does that.
Actually, even with MVCC you'd see your own changes, because they
happen in the same transaction so the buffer pool will use the same version
of the page. While it may be possible to reuse the MVCC infrastructure, it
would still require the introduction of a second scope for stability.
It's quite implementable using append-only b-trees. Though it might be
much to ask that implementations are forced to use that.
An alternative to what I suggested earlier is that all read operations
use read committed. I.e. they always see the data as it looked at
the beginning of the transaction. Would this be more compatible with
existing MVCC implementations?
Hmm, so if you modified the object store and then, later in the same
transaction, used a cursor to iterate the object store, the cursor
would not see the earlier modifications? That's not very intiutive to
me...or did I misunderstand?
If we go with read committed then yes, your understanding is correct.
Out of curiosity, how
Re: [IndexedDB] Cursors and modifications
On Wed, Jul 14, 2010 at 5:12 AM, Jeremy Orlow jor...@chromium.org wrote:
On Thu, Jul 8, 2010 at 8:42 PM, Jonas Sicking jo...@sicking.cc wrote:
On Mon, Jul 5, 2010 at 9:45 AM, Andrei Popescu andr...@google.com wrote:
On Sat, Jul 3, 2010 at 2:09 AM, Jonas Sicking jo...@sicking.cc wrote:
On Fri, Jul 2, 2010 at 5:44 PM, Andrei Popescu andr...@google.com
wrote:
On Sat, Jul 3, 2010 at 1:14 AM, Jonas Sicking jo...@sicking.cc
wrote:
On Fri, Jul 2, 2010 at 4:40 PM, Pablo Castro
pablo.cas...@microsoft.com wrote:
From: public-webapps-requ...@w3.org
[mailto:public-webapps-requ...@w3.org] On Behalf Of Jonas Sicking
Sent: Friday, July 02, 2010 4:00 PM
We ran into an complicated issue while implementing IndexedDB. In
short, what should happen if an object store is modified while a
cursor is
iterating it? Note that the modification can be done within the
same
transaction, so the read/write locks preventing several transactions
from
accessing the same table isn't helping here.
Detailed problem description (this assumes the API proposed by
mozilla):
Consider a objectStore words containing the following objects:
{ name: alpha }
{ name: bravo }
{ name: charlie }
{ name: delta }
and the following program (db is a previously opened IDBDatabase):
var trans = db.transaction([words], READ_WRITE); var cursor; var
result = []; trans.objectStore(words).openCursor().onsuccess =
function(e)
{
cursor = e.result;
result.push(cursor.value);
cursor.continue();
}
trans.objectStore(words).get(delta).onsuccess = function(e) {
trans.objectStore(words).put({ name: delta, myModifiedValue:
17 }); }
When the cursor reads the delta entry, will it see the
'myModifiedValue' property? Since we so far has defined that the
callback
order is defined to be the request order, that means that put
request
will be finished before the delta entry is iterated by the cursor.
The problem is even more serious with cursors that iterate
indexes.
Here a modification can even affect the position of the currently
iterated object in the index, and the modification can (if i'm
reading the
spec correctly) come from the cursor itself.
Consider the following objectStore people with keyPath name
containing the following objects:
{ name: Adam, count: 30 }
{ name: Bertil, count: 31 }
{ name: Cesar, count: 32 }
{ name: David, count: 33 }
{ name: Erik, count: 35 }
and an index countIndex with keyPath count. What would the
following code do?
results = [];
db.objectStore(people,
READ_WRITE).index(countIndex).openObjectCursor().onsuccess =
function (e) {
cursor = e.result;
if (!cursor) {
alert(results);
return;
}
if (cursor.value.name == Bertil) {
cursor.update({name: Bertil, count: 34 });
}
results.push(cursor.value.name);
cursor.continue();
};
What does this alert? Would it alert Adam,Bertil,Erik as the
cursor would stay on the Bertil object as it is moved in the
index? Or
would it alert Adam,Bertil,Cesar,David,Bertil,Erik as we would
iterate
Bertil again at its new position in the index?
My first reaction is that both from the expected behavior of
perspective (transaction is the scope of isolation) and from the
implementation perspective it would be better to see live changes if
they
happened in the same transaction as the cursor (over a store or
index). So
in your example you would iterate one of the rows twice. Maintaining
order
and membership stable would mean creating another scope of isolation
within
the transaction, which to me would be unusual and it would be probably
quite
painful to implement without spilling a copy of the records to disk (at
least a copy of the keys/order if you don't care about protecting from
changes that don't affect membership/order; some databases call these
keyset
cursors).
We could say that cursors always iterate snapshots, however this
introduces MVCC. Though it seems to me that SNAPSHOT_READ already
does that.
Actually, even with MVCC you'd see your own changes, because they
happen in the same transaction so the buffer pool will use the same
version
of the page. While it may be possible to reuse the MVCC
infrastructure, it
would still require the introduction of a second scope for stability.
It's quite implementable using append-only b-trees. Though it might
be
much to ask that implementations are forced to use that.
An alternative to what I suggested earlier is that all read
operations
use read committed. I.e. they always see the data as it looked at
the beginning of the transaction. Would this be more compatible with
existing MVCC implementations?
Hmm, so if you modified the object store and then, later in the same
transaction, used a cursor to iterate the object store, the cursor
would not see the earlier
RE: [IndexedDB] Cursors and modifications
Making sure I get the essence of this thread: we're saying that cursors see
live changes as they happen on objects that are after the object you're
currently standing on; and of course, any other activity within a transaction
sees all the changes that happened before that activity took place. Is that
accurate?
If it's accurate, as a side note, for the async API it seems that this makes it
more interesting to enforce callback order, so we can more easily explain what
we mean by before.
Thanks
-pablo
From: jor...@google.com [mailto:jor...@google.com] On Behalf Of Jeremy Orlow
Sent: Wednesday, July 14, 2010 9:27 AM
On Wed, Jul 14, 2010 at 5:17 PM, Jonas Sicking jo...@sicking.cc wrote:
On Wed, Jul 14, 2010 at 5:12 AM, Jeremy Orlow jor...@chromium.org wrote:
On Thu, Jul 8, 2010 at 8:42 PM, Jonas Sicking jo...@sicking.cc wrote:
On Mon, Jul 5, 2010 at 9:45 AM, Andrei Popescu andr...@google.com wrote:
On Sat, Jul 3, 2010 at 2:09 AM, Jonas Sicking jo...@sicking.cc wrote:
On Fri, Jul 2, 2010 at 5:44 PM, Andrei Popescu andr...@google.com
wrote:
On Sat, Jul 3, 2010 at 1:14 AM, Jonas Sicking jo...@sicking.cc
wrote:
On Fri, Jul 2, 2010 at 4:40 PM, Pablo Castro
pablo.cas...@microsoft.com wrote:
From: public-webapps-requ...@w3.org
[mailto:public-webapps-requ...@w3.org] On Behalf Of Jonas Sicking
Sent: Friday, July 02, 2010 4:00 PM
We ran into an complicated issue while implementing IndexedDB. In
short, what should happen if an object store is modified while a
cursor is
iterating it? Note that the modification can be done within the
same
transaction, so the read/write locks preventing several transactions
from
accessing the same table isn't helping here.
Detailed problem description (this assumes the API proposed by
mozilla):
Consider a objectStore words containing the following objects:
{ name: alpha }
{ name: bravo }
{ name: charlie }
{ name: delta }
and the following program (db is a previously opened IDBDatabase):
var trans = db.transaction([words], READ_WRITE); var cursor; var
result = []; trans.objectStore(words).openCursor().onsuccess =
function(e)
{
cursor = e.result;
result.push(cursor.value);
cursor.continue();
}
trans.objectStore(words).get(delta).onsuccess = function(e) {
trans.objectStore(words).put({ name: delta, myModifiedValue:
17 }); }
When the cursor reads the delta entry, will it see the
'myModifiedValue' property? Since we so far has defined that the
callback
order is defined to be the request order, that means that put
request
will be finished before the delta entry is iterated by the cursor.
The problem is even more serious with cursors that iterate
indexes.
Here a modification can even affect the position of the currently
iterated object in the index, and the modification can (if i'm
reading the
spec correctly) come from the cursor itself.
Consider the following objectStore people with keyPath name
containing the following objects:
{ name: Adam, count: 30 }
{ name: Bertil, count: 31 }
{ name: Cesar, count: 32 }
{ name: David, count: 33 }
{ name: Erik, count: 35 }
and an index countIndex with keyPath count. What would the
following code do?
results = [];
db.objectStore(people,
READ_WRITE).index(countIndex).openObjectCursor().onsuccess =
function (e) {
cursor = e.result;
if (!cursor) {
alert(results);
return;
}
if (cursor.value.name == Bertil) {
cursor.update({name: Bertil, count: 34 });
}
results.push(cursor.value.name);
cursor.continue();
};
What does this alert? Would it alert Adam,Bertil,Erik as the
cursor would stay on the Bertil object as it is moved in the
index? Or
would it alert Adam,Bertil,Cesar,David,Bertil,Erik as we would
iterate
Bertil again at its new position in the index?
My first reaction is that both from the expected behavior of
perspective (transaction is the scope of isolation) and from the
implementation perspective it would be better to see live changes if
they
happened in the same transaction as the cursor (over a store or
index). So
in your example you would iterate one of the rows twice. Maintaining
order
and membership stable would mean creating another scope of isolation
within
the transaction, which to me would be unusual and it would be probably
quite
painful to implement without spilling a copy of the records to disk (at
least a copy of the keys/order if you don't care about protecting from
changes that don't affect membership/order; some databases call these
keyset
cursors).
We could say that cursors always iterate snapshots, however this
introduces MVCC. Though it seems to me that SNAPSHOT_READ already
does that.
Actually, even with MVCC you'd see your own changes, because they
happen in the same transaction so the buffer pool will use the same
version
of the page.
Re: [IndexedDB] Cursors and modifications
On Wed, Jul 14, 2010 at 6:20 PM, Pablo Castro
pablo.cas...@microsoft.com wrote:
Making sure I get the essence of this thread: we're saying that cursors see
live changes as they happen on objects that are after the object you're
currently standing on;
Yes.
and of course, any other activity within a transaction sees all the changes
that happened before that activity took place. Is that accurate?
Yes. All other activity sees all changes as soon as they have happened.
If it's accurate, as a side note, for the async API it seems that this makes
it more interesting to enforce callback order, so we can more easily explain
what we mean by before.
Indeed.
/ Jonas
From: jor...@google.com [mailto:jor...@google.com] On Behalf Of Jeremy Orlow
Sent: Wednesday, July 14, 2010 9:27 AM
On Wed, Jul 14, 2010 at 5:17 PM, Jonas Sicking jo...@sicking.cc wrote:
On Wed, Jul 14, 2010 at 5:12 AM, Jeremy Orlow jor...@chromium.org wrote:
On Thu, Jul 8, 2010 at 8:42 PM, Jonas Sicking jo...@sicking.cc wrote:
On Mon, Jul 5, 2010 at 9:45 AM, Andrei Popescu andr...@google.com wrote:
On Sat, Jul 3, 2010 at 2:09 AM, Jonas Sicking jo...@sicking.cc wrote:
On Fri, Jul 2, 2010 at 5:44 PM, Andrei Popescu andr...@google.com
wrote:
On Sat, Jul 3, 2010 at 1:14 AM, Jonas Sicking jo...@sicking.cc
wrote:
On Fri, Jul 2, 2010 at 4:40 PM, Pablo Castro
pablo.cas...@microsoft.com wrote:
From: public-webapps-requ...@w3.org
[mailto:public-webapps-requ...@w3.org] On Behalf Of Jonas Sicking
Sent: Friday, July 02, 2010 4:00 PM
We ran into an complicated issue while implementing IndexedDB. In
short, what should happen if an object store is modified while a
cursor is
iterating it? Note that the modification can be done within the
same
transaction, so the read/write locks preventing several
transactions from
accessing the same table isn't helping here.
Detailed problem description (this assumes the API proposed by
mozilla):
Consider a objectStore words containing the following objects:
{ name: alpha }
{ name: bravo }
{ name: charlie }
{ name: delta }
and the following program (db is a previously opened IDBDatabase):
var trans = db.transaction([words], READ_WRITE); var cursor; var
result = []; trans.objectStore(words).openCursor().onsuccess =
function(e)
{
cursor = e.result;
result.push(cursor.value);
cursor.continue();
}
trans.objectStore(words).get(delta).onsuccess = function(e) {
trans.objectStore(words).put({ name: delta, myModifiedValue:
17 }); }
When the cursor reads the delta entry, will it see the
'myModifiedValue' property? Since we so far has defined that the
callback
order is defined to be the request order, that means that put
request
will be finished before the delta entry is iterated by the cursor.
The problem is even more serious with cursors that iterate
indexes.
Here a modification can even affect the position of the currently
iterated object in the index, and the modification can (if i'm
reading the
spec correctly) come from the cursor itself.
Consider the following objectStore people with keyPath name
containing the following objects:
{ name: Adam, count: 30 }
{ name: Bertil, count: 31 }
{ name: Cesar, count: 32 }
{ name: David, count: 33 }
{ name: Erik, count: 35 }
and an index countIndex with keyPath count. What would the
following code do?
results = [];
db.objectStore(people,
READ_WRITE).index(countIndex).openObjectCursor().onsuccess =
function (e) {
cursor = e.result;
if (!cursor) {
alert(results);
return;
}
if (cursor.value.name == Bertil) {
cursor.update({name: Bertil, count: 34 });
}
results.push(cursor.value.name);
cursor.continue();
};
What does this alert? Would it alert Adam,Bertil,Erik as the
cursor would stay on the Bertil object as it is moved in the
index? Or
would it alert Adam,Bertil,Cesar,David,Bertil,Erik as we would
iterate
Bertil again at its new position in the index?
My first reaction is that both from the expected behavior of
perspective (transaction is the scope of isolation) and from the
implementation perspective it would be better to see live changes if
they
happened in the same transaction as the cursor (over a store or
index). So
in your example you would iterate one of the rows twice. Maintaining
order
and membership stable would mean creating another scope of isolation
within
the transaction, which to me would be unusual and it would be
probably quite
painful to implement without spilling a copy of the records to disk
(at
least a copy of the keys/order if you don't care about protecting from
changes that don't affect membership/order; some databases call these
keyset
cursors).
We could say that cursors always iterate snapshots, however this
introduces MVCC. Though it seems to me that SNAPSHOT_READ already
Re: [IndexedDB] Cursors and modifications
On Mon, Jul 5, 2010 at 9:45 AM, Andrei Popescu andr...@google.com wrote:
On Sat, Jul 3, 2010 at 2:09 AM, Jonas Sicking jo...@sicking.cc wrote:
On Fri, Jul 2, 2010 at 5:44 PM, Andrei Popescu andr...@google.com wrote:
On Sat, Jul 3, 2010 at 1:14 AM, Jonas Sicking jo...@sicking.cc wrote:
On Fri, Jul 2, 2010 at 4:40 PM, Pablo Castro pablo.cas...@microsoft.com
wrote:
From: public-webapps-requ...@w3.org
[mailto:public-webapps-requ...@w3.org] On Behalf Of Jonas Sicking
Sent: Friday, July 02, 2010 4:00 PM
We ran into an complicated issue while implementing IndexedDB. In
short, what should happen if an object store is modified while a cursor
is iterating it? Note that the modification can be done within the
same transaction, so the read/write locks preventing several
transactions from accessing the same table isn't helping here.
Detailed problem description (this assumes the API proposed by mozilla):
Consider a objectStore words containing the following objects:
{ name: alpha }
{ name: bravo }
{ name: charlie }
{ name: delta }
and the following program (db is a previously opened IDBDatabase):
var trans = db.transaction([words], READ_WRITE); var cursor; var
result = []; trans.objectStore(words).openCursor().onsuccess =
function(e) {
cursor = e.result;
result.push(cursor.value);
cursor.continue();
}
trans.objectStore(words).get(delta).onsuccess = function(e) {
trans.objectStore(words).put({ name: delta, myModifiedValue: 17
}); }
When the cursor reads the delta entry, will it see the
'myModifiedValue' property? Since we so far has defined that the
callback order is defined to be the request order, that means that
put request will be finished before the delta entry is iterated by
the cursor.
The problem is even more serious with cursors that iterate indexes.
Here a modification can even affect the position of the currently
iterated object in the index, and the modification can (if i'm reading
the spec correctly) come from the cursor itself.
Consider the following objectStore people with keyPath name
containing the following objects:
{ name: Adam, count: 30 }
{ name: Bertil, count: 31 }
{ name: Cesar, count: 32 }
{ name: David, count: 33 }
{ name: Erik, count: 35 }
and an index countIndex with keyPath count. What would the
following code do?
results = [];
db.objectStore(people,
READ_WRITE).index(countIndex).openObjectCursor().onsuccess = function
(e) {
cursor = e.result;
if (!cursor) {
alert(results);
return;
}
if (cursor.value.name == Bertil) {
cursor.update({name: Bertil, count: 34 });
}
results.push(cursor.value.name);
cursor.continue();
};
What does this alert? Would it alert Adam,Bertil,Erik as the cursor
would stay on the Bertil object as it is moved in the index? Or would
it alert Adam,Bertil,Cesar,David,Bertil,Erik as we would iterate
Bertil again at its new position in the index?
My first reaction is that both from the expected behavior of perspective
(transaction is the scope of isolation) and from the implementation
perspective it would be better to see live changes if they happened in
the same transaction as the cursor (over a store or index). So in your
example you would iterate one of the rows twice. Maintaining order and
membership stable would mean creating another scope of isolation within
the transaction, which to me would be unusual and it would be probably
quite painful to implement without spilling a copy of the records to disk
(at least a copy of the keys/order if you don't care about protecting
from changes that don't affect membership/order; some databases call
these keyset cursors).
We could say that cursors always iterate snapshots, however this
introduces MVCC. Though it seems to me that SNAPSHOT_READ already does
that.
Actually, even with MVCC you'd see your own changes, because they happen
in the same transaction so the buffer pool will use the same version of
the page. While it may be possible to reuse the MVCC infrastructure, it
would still require the introduction of a second scope for stability.
It's quite implementable using append-only b-trees. Though it might be
much to ask that implementations are forced to use that.
An alternative to what I suggested earlier is that all read operations
use read committed. I.e. they always see the data as it looked at
the beginning of the transaction. Would this be more compatible with
existing MVCC implementations?
Hmm, so if you modified the object store and then, later in the same
transaction, used a cursor to iterate the object store, the cursor
would not see the earlier modifications? That's not very intiutive to
me...or did I misunderstand?
If we go with read committed then yes, your understanding is correct.
Out of curiosity, how are you feeling about the cursors iterate data
as it looked when cursor was opened solution?
I feel that that's the
Re: [IndexedDB] Cursors and modifications
On Sat, Jul 3, 2010 at 2:09 AM, Jonas Sicking jo...@sicking.cc wrote:
On Fri, Jul 2, 2010 at 5:44 PM, Andrei Popescu andr...@google.com wrote:
On Sat, Jul 3, 2010 at 1:14 AM, Jonas Sicking jo...@sicking.cc wrote:
On Fri, Jul 2, 2010 at 4:40 PM, Pablo Castro pablo.cas...@microsoft.com
wrote:
From: public-webapps-requ...@w3.org [mailto:public-webapps-requ...@w3.org]
On Behalf Of Jonas Sicking
Sent: Friday, July 02, 2010 4:00 PM
We ran into an complicated issue while implementing IndexedDB. In short,
what should happen if an object store is modified while a cursor is
iterating it? Note that the modification can be done within the same
transaction, so the read/write locks preventing several transactions
from accessing the same table isn't helping here.
Detailed problem description (this assumes the API proposed by mozilla):
Consider a objectStore words containing the following objects:
{ name: alpha }
{ name: bravo }
{ name: charlie }
{ name: delta }
and the following program (db is a previously opened IDBDatabase):
var trans = db.transaction([words], READ_WRITE); var cursor; var
result = []; trans.objectStore(words).openCursor().onsuccess =
function(e) {
cursor = e.result;
result.push(cursor.value);
cursor.continue();
}
trans.objectStore(words).get(delta).onsuccess = function(e) {
trans.objectStore(words).put({ name: delta, myModifiedValue: 17
}); }
When the cursor reads the delta entry, will it see the
'myModifiedValue' property? Since we so far has defined that the
callback order is defined to be the request order, that means that
put request will be finished before the delta entry is iterated by the
cursor.
The problem is even more serious with cursors that iterate indexes.
Here a modification can even affect the position of the currently
iterated object in the index, and the modification can (if i'm reading
the spec correctly) come from the cursor itself.
Consider the following objectStore people with keyPath name
containing the following objects:
{ name: Adam, count: 30 }
{ name: Bertil, count: 31 }
{ name: Cesar, count: 32 }
{ name: David, count: 33 }
{ name: Erik, count: 35 }
and an index countIndex with keyPath count. What would the following
code do?
results = [];
db.objectStore(people,
READ_WRITE).index(countIndex).openObjectCursor().onsuccess = function
(e) {
cursor = e.result;
if (!cursor) {
alert(results);
return;
}
if (cursor.value.name == Bertil) {
cursor.update({name: Bertil, count: 34 });
}
results.push(cursor.value.name);
cursor.continue();
};
What does this alert? Would it alert Adam,Bertil,Erik as the cursor
would stay on the Bertil object as it is moved in the index? Or would
it alert Adam,Bertil,Cesar,David,Bertil,Erik as we would iterate
Bertil again at its new position in the index?
My first reaction is that both from the expected behavior of perspective
(transaction is the scope of isolation) and from the implementation
perspective it would be better to see live changes if they happened in the
same transaction as the cursor (over a store or index). So in your example
you would iterate one of the rows twice. Maintaining order and membership
stable would mean creating another scope of isolation within the
transaction, which to me would be unusual and it would be probably quite
painful to implement without spilling a copy of the records to disk (at
least a copy of the keys/order if you don't care about protecting from
changes that don't affect membership/order; some databases call these
keyset cursors).
We could say that cursors always iterate snapshots, however this
introduces MVCC. Though it seems to me that SNAPSHOT_READ already does
that.
Actually, even with MVCC you'd see your own changes, because they happen
in the same transaction so the buffer pool will use the same version of
the page. While it may be possible to reuse the MVCC infrastructure, it
would still require the introduction of a second scope for stability.
It's quite implementable using append-only b-trees. Though it might be
much to ask that implementations are forced to use that.
An alternative to what I suggested earlier is that all read operations
use read committed. I.e. they always see the data as it looked at
the beginning of the transaction. Would this be more compatible with
existing MVCC implementations?
Hmm, so if you modified the object store and then, later in the same
transaction, used a cursor to iterate the object store, the cursor
would not see the earlier modifications? That's not very intiutive to
me...or did I misunderstand?
If we go with read committed then yes, your understanding is correct.
Out of curiosity, how are you feeling about the cursors iterate data
as it looked when cursor was opened solution?
I feel that that's the easiest solution to specify although it may
also be unintuitive if one calls 'put
[IndexedDB] Cursors and modifications
Hi All,
We ran into an complicated issue while implementing IndexedDB. In
short, what should happen if an object store is modified while a
cursor is iterating it? Note that the modification can be done within
the same transaction, so the read/write locks preventing several
transactions from accessing the same table isn't helping here.
Detailed problem description (this assumes the API proposed by mozilla):
Consider a objectStore words containing the following objects:
{ name: alpha }
{ name: bravo }
{ name: charlie }
{ name: delta }
and the following program (db is a previously opened IDBDatabase):
var trans = db.transaction([words], READ_WRITE);
var cursor;
var result = [];
trans.objectStore(words).openCursor().onsuccess = function(e) {
cursor = e.result;
result.push(cursor.value);
cursor.continue();
}
trans.objectStore(words).get(delta).onsuccess = function(e) {
trans.objectStore(words).put({ name: delta, myModifiedValue: 17 });
}
When the cursor reads the delta entry, will it see the
'myModifiedValue' property? Since we so far has defined that the
callback order is defined to be the request order, that means that put
request will be finished before the delta entry is iterated by the
cursor.
The problem is even more serious with cursors that iterate indexes.
Here a modification can even affect the position of the currently
iterated object in the index, and the modification can (if i'm reading
the spec correctly) come from the cursor itself.
Consider the following objectStore people with keyPath name
containing the following objects:
{ name: Adam, count: 30 }
{ name: Bertil, count: 31 }
{ name: Cesar, count: 32 }
{ name: David, count: 33 }
{ name: Erik, count: 35 }
and an index countIndex with keyPath count. What would the
following code do?
results = [];
db.objectStore(people,
READ_WRITE).index(countIndex).openObjectCursor().onsuccess =
function (e) {
cursor = e.result;
if (!cursor) {
alert(results);
return;
}
if (cursor.value.name == Bertil) {
cursor.update({name: Bertil, count: 34 });
}
results.push(cursor.value.name);
cursor.continue();
};
What does this alert? Would it alert Adam,Bertil,Erik as the cursor
would stay on the Bertil object as it is moved in the index? Or
would it alert Adam,Bertil,Cesar,David,Bertil,Erik as we would
iterate Bertil again at its new position in the index?
We could say that cursors always iterate snapshots, however this
introduces MVCC. Though it seems to me that SNAPSHOT_READ already does
that.
We could also say that cursors iterate live data though that can be
pretty confusing and forces the implementation to deal with entries
being added and removed during iteration, and it'd be tricky to define
all edge cases.
It's certainly debatable how much of a problem any of these edgecases
are for users. Note that all of this is only an issue if you modify
and read from the same records *in the same transaction*. I can't
think of a case where it isn't trivial to avoid these problems by
separating things into separate transactions. However it'd be nice to
avoid creating foot-guns for people to play with (think of the
children!).
However we still need to define *something*. I would suggest that we
define that cursors iterate snapshots. It seems the cleanest for users
and easiest to define. And once implementations add MVCC support it
should be easy to implement. I think we've come up with a decent plan
for how to do implement it in sqlite even without proper MVCC, so it
should be doable even then.
/ Jonas
RE: [IndexedDB] Cursors and modifications
From: public-webapps-requ...@w3.org [mailto:public-webapps-requ...@w3.org] On
Behalf Of Jonas Sicking
Sent: Friday, July 02, 2010 4:00 PM
We ran into an complicated issue while implementing IndexedDB. In short,
what should happen if an object store is modified while a cursor is
iterating it? Note that the modification can be done within the same
transaction, so the read/write locks preventing several transactions from
accessing the same table isn't helping here.
Detailed problem description (this assumes the API proposed by mozilla):
Consider a objectStore words containing the following objects:
{ name: alpha }
{ name: bravo }
{ name: charlie }
{ name: delta }
and the following program (db is a previously opened IDBDatabase):
var trans = db.transaction([words], READ_WRITE); var cursor; var result =
[]; trans.objectStore(words).openCursor().onsuccess = function(e) {
cursor = e.result;
result.push(cursor.value);
cursor.continue();
}
trans.objectStore(words).get(delta).onsuccess = function(e) {
trans.objectStore(words).put({ name: delta, myModifiedValue: 17 }); }
When the cursor reads the delta entry, will it see the 'myModifiedValue'
property? Since we so far has defined that the callback order is defined to
be the request order, that means that put request will be finished before
the delta entry is iterated by the cursor.
The problem is even more serious with cursors that iterate indexes.
Here a modification can even affect the position of the currently iterated
object in the index, and the modification can (if i'm reading the spec
correctly) come from the cursor itself.
Consider the following objectStore people with keyPath name
containing the following objects:
{ name: Adam, count: 30 }
{ name: Bertil, count: 31 }
{ name: Cesar, count: 32 }
{ name: David, count: 33 }
{ name: Erik, count: 35 }
and an index countIndex with keyPath count. What would the following
code do?
results = [];
db.objectStore(people,
READ_WRITE).index(countIndex).openObjectCursor().onsuccess = function (e) {
cursor = e.result;
if (!cursor) {
alert(results);
return;
}
if (cursor.value.name == Bertil) {
cursor.update({name: Bertil, count: 34 });
}
results.push(cursor.value.name);
cursor.continue();
};
What does this alert? Would it alert Adam,Bertil,Erik as the cursor would
stay on the Bertil object as it is moved in the index? Or would it alert
Adam,Bertil,Cesar,David,Bertil,Erik as we would iterate Bertil again at
its new position in the index?
My first reaction is that both from the expected behavior of perspective
(transaction is the scope of isolation) and from the implementation perspective
it would be better to see live changes if they happened in the same transaction
as the cursor (over a store or index). So in your example you would iterate one
of the rows twice. Maintaining order and membership stable would mean creating
another scope of isolation within the transaction, which to me would be unusual
and it would be probably quite painful to implement without spilling a copy of
the records to disk (at least a copy of the keys/order if you don't care about
protecting from changes that don't affect membership/order; some databases call
these keyset cursors).
We could say that cursors always iterate snapshots, however this introduces
MVCC. Though it seems to me that SNAPSHOT_READ already does that.
Actually, even with MVCC you'd see your own changes, because they happen in the
same transaction so the buffer pool will use the same version of the page.
While it may be possible to reuse the MVCC infrastructure, it would still
require the introduction of a second scope for stability.
We could also say that cursors iterate live data though that can be pretty
confusing and forces the implementation to deal with entries being added and
removed during iteration, and it'd be tricky to define all edge cases.
Would this be any different from the implementation perspective than dealing
with changes that happen through other transactions once they are committed?
Typically at least in non-MVCC systems committed changes that are further
ahead in a cursor scan end up showing up even when the cursor was opened
before the other transaction committed.
It's certainly debatable how much of a problem any of these edgecases are
for users. Note that all of this is only an issue if you modify and read
from the same records *in the same transaction*. I can't think of a case
where it isn't trivial to avoid these problems by separating things into
separate transactions. However it'd be nice to avoid creating foot-guns
for people to play with (think of the children!).
However we still need to define *something*. I would suggest that we define
that cursors iterate snapshots. It seems the cleanest for users and easiest
to define. And once implementations add MVCC support it should be
Re: [IndexedDB] Cursors and modifications
On Fri, Jul 2, 2010 at 4:40 PM, Pablo Castro pablo.cas...@microsoft.com wrote:
From: public-webapps-requ...@w3.org [mailto:public-webapps-requ...@w3.org] On
Behalf Of Jonas Sicking
Sent: Friday, July 02, 2010 4:00 PM
We ran into an complicated issue while implementing IndexedDB. In short,
what should happen if an object store is modified while a cursor is
iterating it? Note that the modification can be done within the same
transaction, so the read/write locks preventing several transactions from
accessing the same table isn't helping here.
Detailed problem description (this assumes the API proposed by mozilla):
Consider a objectStore words containing the following objects:
{ name: alpha }
{ name: bravo }
{ name: charlie }
{ name: delta }
and the following program (db is a previously opened IDBDatabase):
var trans = db.transaction([words], READ_WRITE); var cursor; var result =
[]; trans.objectStore(words).openCursor().onsuccess = function(e) {
cursor = e.result;
result.push(cursor.value);
cursor.continue();
}
trans.objectStore(words).get(delta).onsuccess = function(e) {
trans.objectStore(words).put({ name: delta, myModifiedValue: 17 }); }
When the cursor reads the delta entry, will it see the 'myModifiedValue'
property? Since we so far has defined that the callback order is defined to
be the request order, that means that put request will be finished
before the delta entry is iterated by the cursor.
The problem is even more serious with cursors that iterate indexes.
Here a modification can even affect the position of the currently iterated
object in the index, and the modification can (if i'm reading the spec
correctly) come from the cursor itself.
Consider the following objectStore people with keyPath name
containing the following objects:
{ name: Adam, count: 30 }
{ name: Bertil, count: 31 }
{ name: Cesar, count: 32 }
{ name: David, count: 33 }
{ name: Erik, count: 35 }
and an index countIndex with keyPath count. What would the following
code do?
results = [];
db.objectStore(people,
READ_WRITE).index(countIndex).openObjectCursor().onsuccess = function (e)
{
cursor = e.result;
if (!cursor) {
alert(results);
return;
}
if (cursor.value.name == Bertil) {
cursor.update({name: Bertil, count: 34 });
}
results.push(cursor.value.name);
cursor.continue();
};
What does this alert? Would it alert Adam,Bertil,Erik as the cursor would
stay on the Bertil object as it is moved in the index? Or would it alert
Adam,Bertil,Cesar,David,Bertil,Erik as we would iterate Bertil again at
its new position in the index?
My first reaction is that both from the expected behavior of perspective
(transaction is the scope of isolation) and from the implementation
perspective it would be better to see live changes if they happened in the
same transaction as the cursor (over a store or index). So in your example
you would iterate one of the rows twice. Maintaining order and membership
stable would mean creating another scope of isolation within the transaction,
which to me would be unusual and it would be probably quite painful to
implement without spilling a copy of the records to disk (at least a copy of
the keys/order if you don't care about protecting from changes that don't
affect membership/order; some databases call these keyset cursors).
We could say that cursors always iterate snapshots, however this introduces
MVCC. Though it seems to me that SNAPSHOT_READ already does that.
Actually, even with MVCC you'd see your own changes, because they happen in
the same transaction so the buffer pool will use the same version of the
page. While it may be possible to reuse the MVCC infrastructure, it would
still require the introduction of a second scope for stability.
It's quite implementable using append-only b-trees. Though it might be
much to ask that implementations are forced to use that.
An alternative to what I suggested earlier is that all read operations
use read committed. I.e. they always see the data as it looked at
the beginning of the transaction. Would this be more compatible with
existing MVCC implementations?
I'd imagine this should be as easy to implement as SNAPSHOT_READ.
We could also say that cursors iterate live data though that can be pretty
confusing and forces the implementation to deal with entries being added
and removed during iteration, and it'd be tricky to define all edge
cases.
Would this be any different from the implementation perspective than dealing
with changes that happen through other transactions once they are committed?
Typically at least in non-MVCC systems committed changes that are further
ahead in a cursor scan end up showing up even when the cursor was opened
before the other transaction committed.
IndexedDB doesn't allow write transactions to a given store to start
while there are read transactions using that store,
Re: [IndexedDB] Cursors and modifications
On Sat, Jul 3, 2010 at 1:14 AM, Jonas Sicking jo...@sicking.cc wrote:
On Fri, Jul 2, 2010 at 4:40 PM, Pablo Castro pablo.cas...@microsoft.com
wrote:
From: public-webapps-requ...@w3.org [mailto:public-webapps-requ...@w3.org]
On Behalf Of Jonas Sicking
Sent: Friday, July 02, 2010 4:00 PM
We ran into an complicated issue while implementing IndexedDB. In short,
what should happen if an object store is modified while a cursor is
iterating it? Note that the modification can be done within the same
transaction, so the read/write locks preventing several transactions from
accessing the same table isn't helping here.
Detailed problem description (this assumes the API proposed by mozilla):
Consider a objectStore words containing the following objects:
{ name: alpha }
{ name: bravo }
{ name: charlie }
{ name: delta }
and the following program (db is a previously opened IDBDatabase):
var trans = db.transaction([words], READ_WRITE); var cursor; var result
= []; trans.objectStore(words).openCursor().onsuccess = function(e) {
cursor = e.result;
result.push(cursor.value);
cursor.continue();
}
trans.objectStore(words).get(delta).onsuccess = function(e) {
trans.objectStore(words).put({ name: delta, myModifiedValue: 17 }); }
When the cursor reads the delta entry, will it see the 'myModifiedValue'
property? Since we so far has defined that the callback order is defined
to be the request order, that means that put request will be finished
before the delta entry is iterated by the cursor.
The problem is even more serious with cursors that iterate indexes.
Here a modification can even affect the position of the currently iterated
object in the index, and the modification can (if i'm reading the spec
correctly) come from the cursor itself.
Consider the following objectStore people with keyPath name
containing the following objects:
{ name: Adam, count: 30 }
{ name: Bertil, count: 31 }
{ name: Cesar, count: 32 }
{ name: David, count: 33 }
{ name: Erik, count: 35 }
and an index countIndex with keyPath count. What would the following
code do?
results = [];
db.objectStore(people,
READ_WRITE).index(countIndex).openObjectCursor().onsuccess = function
(e) {
cursor = e.result;
if (!cursor) {
alert(results);
return;
}
if (cursor.value.name == Bertil) {
cursor.update({name: Bertil, count: 34 });
}
results.push(cursor.value.name);
cursor.continue();
};
What does this alert? Would it alert Adam,Bertil,Erik as the cursor
would stay on the Bertil object as it is moved in the index? Or would it
alert Adam,Bertil,Cesar,David,Bertil,Erik as we would iterate Bertil
again at its new position in the index?
My first reaction is that both from the expected behavior of perspective
(transaction is the scope of isolation) and from the implementation
perspective it would be better to see live changes if they happened in the
same transaction as the cursor (over a store or index). So in your example
you would iterate one of the rows twice. Maintaining order and membership
stable would mean creating another scope of isolation within the
transaction, which to me would be unusual and it would be probably quite
painful to implement without spilling a copy of the records to disk (at
least a copy of the keys/order if you don't care about protecting from
changes that don't affect membership/order; some databases call these keyset
cursors).
We could say that cursors always iterate snapshots, however this
introduces MVCC. Though it seems to me that SNAPSHOT_READ already does
that.
Actually, even with MVCC you'd see your own changes, because they happen in
the same transaction so the buffer pool will use the same version of the
page. While it may be possible to reuse the MVCC infrastructure, it would
still require the introduction of a second scope for stability.
It's quite implementable using append-only b-trees. Though it might be
much to ask that implementations are forced to use that.
An alternative to what I suggested earlier is that all read operations
use read committed. I.e. they always see the data as it looked at
the beginning of the transaction. Would this be more compatible with
existing MVCC implementations?
Hmm, so if you modified the object store and then, later in the same
transaction, used a cursor to iterate the object store, the cursor
would not see the earlier modifications? That's not very intiutive to
me...or did I misunderstand?
I'd imagine this should be as easy to implement as SNAPSHOT_READ.
We could also say that cursors iterate live data though that can be pretty
confusing and forces the implementation to deal with entries being added
and removed during iteration, and it'd be tricky to define all edge
cases.
Would this be any different from the implementation perspective than dealing
with changes that happen through other transactions once they are
Re: [IndexedDB] Cursors and modifications
On Fri, Jul 2, 2010 at 5:44 PM, Andrei Popescu andr...@google.com wrote:
On Sat, Jul 3, 2010 at 1:14 AM, Jonas Sicking jo...@sicking.cc wrote:
On Fri, Jul 2, 2010 at 4:40 PM, Pablo Castro pablo.cas...@microsoft.com
wrote:
From: public-webapps-requ...@w3.org [mailto:public-webapps-requ...@w3.org]
On Behalf Of Jonas Sicking
Sent: Friday, July 02, 2010 4:00 PM
We ran into an complicated issue while implementing IndexedDB. In short,
what should happen if an object store is modified while a cursor is
iterating it? Note that the modification can be done within the same
transaction, so the read/write locks preventing several transactions from
accessing the same table isn't helping here.
Detailed problem description (this assumes the API proposed by mozilla):
Consider a objectStore words containing the following objects:
{ name: alpha }
{ name: bravo }
{ name: charlie }
{ name: delta }
and the following program (db is a previously opened IDBDatabase):
var trans = db.transaction([words], READ_WRITE); var cursor; var result
= []; trans.objectStore(words).openCursor().onsuccess = function(e) {
cursor = e.result;
result.push(cursor.value);
cursor.continue();
}
trans.objectStore(words).get(delta).onsuccess = function(e) {
trans.objectStore(words).put({ name: delta, myModifiedValue: 17 });
}
When the cursor reads the delta entry, will it see the
'myModifiedValue' property? Since we so far has defined that the callback
order is defined to be the request order, that means that put request
will be finished before the delta entry is iterated by the cursor.
The problem is even more serious with cursors that iterate indexes.
Here a modification can even affect the position of the currently
iterated object in the index, and the modification can (if i'm reading
the spec correctly) come from the cursor itself.
Consider the following objectStore people with keyPath name
containing the following objects:
{ name: Adam, count: 30 }
{ name: Bertil, count: 31 }
{ name: Cesar, count: 32 }
{ name: David, count: 33 }
{ name: Erik, count: 35 }
and an index countIndex with keyPath count. What would the following
code do?
results = [];
db.objectStore(people,
READ_WRITE).index(countIndex).openObjectCursor().onsuccess = function
(e) {
cursor = e.result;
if (!cursor) {
alert(results);
return;
}
if (cursor.value.name == Bertil) {
cursor.update({name: Bertil, count: 34 });
}
results.push(cursor.value.name);
cursor.continue();
};
What does this alert? Would it alert Adam,Bertil,Erik as the cursor
would stay on the Bertil object as it is moved in the index? Or would
it alert Adam,Bertil,Cesar,David,Bertil,Erik as we would iterate
Bertil again at its new position in the index?
My first reaction is that both from the expected behavior of perspective
(transaction is the scope of isolation) and from the implementation
perspective it would be better to see live changes if they happened in the
same transaction as the cursor (over a store or index). So in your example
you would iterate one of the rows twice. Maintaining order and membership
stable would mean creating another scope of isolation within the
transaction, which to me would be unusual and it would be probably quite
painful to implement without spilling a copy of the records to disk (at
least a copy of the keys/order if you don't care about protecting from
changes that don't affect membership/order; some databases call these
keyset cursors).
We could say that cursors always iterate snapshots, however this
introduces MVCC. Though it seems to me that SNAPSHOT_READ already does
that.
Actually, even with MVCC you'd see your own changes, because they happen in
the same transaction so the buffer pool will use the same version of the
page. While it may be possible to reuse the MVCC infrastructure, it would
still require the introduction of a second scope for stability.
It's quite implementable using append-only b-trees. Though it might be
much to ask that implementations are forced to use that.
An alternative to what I suggested earlier is that all read operations
use read committed. I.e. they always see the data as it looked at
the beginning of the transaction. Would this be more compatible with
existing MVCC implementations?
Hmm, so if you modified the object store and then, later in the same
transaction, used a cursor to iterate the object store, the cursor
would not see the earlier modifications? That's not very intiutive to
me...or did I misunderstand?
If we go with read committed then yes, your understanding is correct.
Out of curiosity, how are you feeling about the cursors iterate data
as it looked when cursor was opened solution?
I'd imagine this should be as easy to implement as SNAPSHOT_READ.
We could also say that cursors iterate live data though that can be
pretty confusing and forces the
Re: [IndexedDB] Cursors and modifications
On Sat, Jul 3, 2010 at 11:09 AM, Jonas Sicking jo...@sicking.cc wrote:
On Fri, Jul 2, 2010 at 5:44 PM, Andrei Popescu andr...@google.com wrote:
On Sat, Jul 3, 2010 at 1:14 AM, Jonas Sicking jo...@sicking.cc wrote:
On Fri, Jul 2, 2010 at 4:40 PM, Pablo Castro
pablo.cas...@microsoft.com wrote:
From: public-webapps-requ...@w3.org [mailto:
public-webapps-requ...@w3.org] On Behalf Of Jonas Sicking
Sent: Friday, July 02, 2010 4:00 PM
We ran into an complicated issue while implementing IndexedDB. In
short, what should happen if an object store is modified while a cursor is
iterating it? Note that the modification can be done within the same
transaction, so the read/write locks preventing several transactions from
accessing the same table isn't helping here.
Detailed problem description (this assumes the API proposed by
mozilla):
Consider a objectStore words containing the following objects:
{ name: alpha }
{ name: bravo }
{ name: charlie }
{ name: delta }
and the following program (db is a previously opened IDBDatabase):
var trans = db.transaction([words], READ_WRITE); var cursor; var
result = []; trans.objectStore(words).openCursor().onsuccess = function(e)
{
cursor = e.result;
result.push(cursor.value);
cursor.continue();
}
trans.objectStore(words).get(delta).onsuccess = function(e) {
trans.objectStore(words).put({ name: delta, myModifiedValue: 17
}); }
When the cursor reads the delta entry, will it see the
'myModifiedValue' property? Since we so far has defined that the callback
order is defined to be the request order, that means that put request
will be finished before the delta entry is iterated by the cursor.
The problem is even more serious with cursors that iterate indexes.
Here a modification can even affect the position of the currently
iterated object in the index, and the modification can (if i'm reading the
spec correctly) come from the cursor itself.
Consider the following objectStore people with keyPath name
containing the following objects:
{ name: Adam, count: 30 }
{ name: Bertil, count: 31 }
{ name: Cesar, count: 32 }
{ name: David, count: 33 }
{ name: Erik, count: 35 }
and an index countIndex with keyPath count. What would the
following code do?
results = [];
db.objectStore(people,
READ_WRITE).index(countIndex).openObjectCursor().onsuccess =
function (e) {
cursor = e.result;
if (!cursor) {
alert(results);
return;
}
if (cursor.value.name == Bertil) {
cursor.update({name: Bertil, count: 34 });
}
results.push(cursor.value.name);
cursor.continue();
};
What does this alert? Would it alert Adam,Bertil,Erik as the cursor
would stay on the Bertil object as it is moved in the index? Or would it
alert Adam,Bertil,Cesar,David,Bertil,Erik as we would iterate Bertil
again at its new position in the index?
My first reaction is that both from the expected behavior of
perspective (transaction is the scope of isolation) and from the
implementation perspective it would be better to see live changes if they
happened in the same transaction as the cursor (over a store or index). So
in your example you would iterate one of the rows twice. Maintaining order
and membership stable would mean creating another scope of isolation within
the transaction, which to me would be unusual and it would be probably quite
painful to implement without spilling a copy of the records to disk (at
least a copy of the keys/order if you don't care about protecting from
changes that don't affect membership/order; some databases call these keyset
cursors).
We could say that cursors always iterate snapshots, however this
introduces MVCC. Though it seems to me that SNAPSHOT_READ already does that.
Actually, even with MVCC you'd see your own changes, because they
happen in the same transaction so the buffer pool will use the same version
of the page. While it may be possible to reuse the MVCC infrastructure, it
would still require the introduction of a second scope for stability.
It's quite implementable using append-only b-trees. Though it might be
much to ask that implementations are forced to use that.
An alternative to what I suggested earlier is that all read operations
use read committed. I.e. they always see the data as it looked at
the beginning of the transaction. Would this be more compatible with
existing MVCC implementations?
Hmm, so if you modified the object store and then, later in the same
transaction, used a cursor to iterate the object store, the cursor
would not see the earlier modifications? That's not very intiutive to
me...or did I misunderstand?
If we go with read committed then yes, your understanding is correct.
I agree with Andrei that this proposal seems highly unintuitive.
More generally, I strongly am against doing anything other than maintaining
serializable database semantics
Re: [IndexedDB] Cursors and modifications
On Sat, Jul 3, 2010 at 11:09 AM, Jonas Sicking jo...@sicking.cc wrote:
On Fri, Jul 2, 2010 at 5:44 PM, Andrei Popescu andr...@google.com wrote:
On Sat, Jul 3, 2010 at 1:14 AM, Jonas Sicking jo...@sicking.cc wrote:
On Fri, Jul 2, 2010 at 4:40 PM, Pablo Castro
pablo.cas...@microsoft.com wrote:
From: public-webapps-requ...@w3.org [mailto:
public-webapps-requ...@w3.org] On Behalf Of Jonas Sicking
Sent: Friday, July 02, 2010 4:00 PM
We ran into an complicated issue while implementing IndexedDB. In
short, what should happen if an object store is modified while a cursor is
iterating it? Note that the modification can be done within the same
transaction, so the read/write locks preventing several transactions from
accessing the same table isn't helping here.
Detailed problem description (this assumes the API proposed by
mozilla):
Consider a objectStore words containing the following objects:
{ name: alpha }
{ name: bravo }
{ name: charlie }
{ name: delta }
and the following program (db is a previously opened IDBDatabase):
var trans = db.transaction([words], READ_WRITE); var cursor; var
result = []; trans.objectStore(words).openCursor().onsuccess = function(e)
{
cursor = e.result;
result.push(cursor.value);
cursor.continue();
}
trans.objectStore(words).get(delta).onsuccess = function(e) {
trans.objectStore(words).put({ name: delta, myModifiedValue: 17
}); }
When the cursor reads the delta entry, will it see the
'myModifiedValue' property? Since we so far has defined that the callback
order is defined to be the request order, that means that put request
will be finished before the delta entry is iterated by the cursor.
The problem is even more serious with cursors that iterate indexes.
Here a modification can even affect the position of the currently
iterated object in the index, and the modification can (if i'm reading the
spec correctly) come from the cursor itself.
Consider the following objectStore people with keyPath name
containing the following objects:
{ name: Adam, count: 30 }
{ name: Bertil, count: 31 }
{ name: Cesar, count: 32 }
{ name: David, count: 33 }
{ name: Erik, count: 35 }
and an index countIndex with keyPath count. What would the
following code do?
results = [];
db.objectStore(people,
READ_WRITE).index(countIndex).openObjectCursor().onsuccess =
function (e) {
cursor = e.result;
if (!cursor) {
alert(results);
return;
}
if (cursor.value.name == Bertil) {
cursor.update({name: Bertil, count: 34 });
}
results.push(cursor.value.name);
cursor.continue();
};
What does this alert? Would it alert Adam,Bertil,Erik as the cursor
would stay on the Bertil object as it is moved in the index? Or would it
alert Adam,Bertil,Cesar,David,Bertil,Erik as we would iterate Bertil
again at its new position in the index?
My first reaction is that both from the expected behavior of
perspective (transaction is the scope of isolation) and from the
implementation perspective it would be better to see live changes if they
happened in the same transaction as the cursor (over a store or index). So
in your example you would iterate one of the rows twice. Maintaining order
and membership stable would mean creating another scope of isolation within
the transaction, which to me would be unusual and it would be probably quite
painful to implement without spilling a copy of the records to disk (at
least a copy of the keys/order if you don't care about protecting from
changes that don't affect membership/order; some databases call these keyset
cursors).
We could say that cursors always iterate snapshots, however this
introduces MVCC. Though it seems to me that SNAPSHOT_READ already does that.
Actually, even with MVCC you'd see your own changes, because they
happen in the same transaction so the buffer pool will use the same version
of the page. While it may be possible to reuse the MVCC infrastructure, it
would still require the introduction of a second scope for stability.
It's quite implementable using append-only b-trees. Though it might be
much to ask that implementations are forced to use that.
An alternative to what I suggested earlier is that all read operations
use read committed. I.e. they always see the data as it looked at
the beginning of the transaction. Would this be more compatible with
existing MVCC implementations?
Hmm, so if you modified the object store and then, later in the same
transaction, used a cursor to iterate the object store, the cursor
would not see the earlier modifications? That's not very intiutive to
me...or did I misunderstand?
If we go with read committed then yes, your understanding is correct.
Out of curiosity, how are you feeling about the cursors iterate data
as it looked when cursor was opened solution?
I'm actually starting to warm up to the idea some, but
Re: [IndexedDB] Cursors and modifications
On Fri, Jul 2, 2010 at 7:27 PM, Jeremy Orlow jor...@chromium.org wrote:
On Sat, Jul 3, 2010 at 11:09 AM, Jonas Sicking jo...@sicking.cc wrote:
On Fri, Jul 2, 2010 at 5:44 PM, Andrei Popescu andr...@google.com wrote:
On Sat, Jul 3, 2010 at 1:14 AM, Jonas Sicking jo...@sicking.cc wrote:
On Fri, Jul 2, 2010 at 4:40 PM, Pablo Castro
pablo.cas...@microsoft.com wrote:
From: public-webapps-requ...@w3.org
[mailto:public-webapps-requ...@w3.org] On Behalf Of Jonas Sicking
Sent: Friday, July 02, 2010 4:00 PM
We ran into an complicated issue while implementing IndexedDB. In
short, what should happen if an object store is modified while a
cursor is
iterating it? Note that the modification can be done within the same
transaction, so the read/write locks preventing several transactions
from
accessing the same table isn't helping here.
Detailed problem description (this assumes the API proposed by
mozilla):
Consider a objectStore words containing the following objects:
{ name: alpha }
{ name: bravo }
{ name: charlie }
{ name: delta }
and the following program (db is a previously opened IDBDatabase):
var trans = db.transaction([words], READ_WRITE); var cursor; var
result = []; trans.objectStore(words).openCursor().onsuccess =
function(e)
{
cursor = e.result;
result.push(cursor.value);
cursor.continue();
}
trans.objectStore(words).get(delta).onsuccess = function(e) {
trans.objectStore(words).put({ name: delta, myModifiedValue:
17 }); }
When the cursor reads the delta entry, will it see the
'myModifiedValue' property? Since we so far has defined that the
callback
order is defined to be the request order, that means that put
request
will be finished before the delta entry is iterated by the cursor.
The problem is even more serious with cursors that iterate indexes.
Here a modification can even affect the position of the currently
iterated object in the index, and the modification can (if i'm reading
the
spec correctly) come from the cursor itself.
Consider the following objectStore people with keyPath name
containing the following objects:
{ name: Adam, count: 30 }
{ name: Bertil, count: 31 }
{ name: Cesar, count: 32 }
{ name: David, count: 33 }
{ name: Erik, count: 35 }
and an index countIndex with keyPath count. What would the
following code do?
results = [];
db.objectStore(people,
READ_WRITE).index(countIndex).openObjectCursor().onsuccess =
function (e) {
cursor = e.result;
if (!cursor) {
alert(results);
return;
}
if (cursor.value.name == Bertil) {
cursor.update({name: Bertil, count: 34 });
}
results.push(cursor.value.name);
cursor.continue();
};
What does this alert? Would it alert Adam,Bertil,Erik as the
cursor would stay on the Bertil object as it is moved in the index?
Or
would it alert Adam,Bertil,Cesar,David,Bertil,Erik as we would
iterate
Bertil again at its new position in the index?
My first reaction is that both from the expected behavior of
perspective (transaction is the scope of isolation) and from the
implementation perspective it would be better to see live changes if they
happened in the same transaction as the cursor (over a store or index).
So
in your example you would iterate one of the rows twice. Maintaining
order
and membership stable would mean creating another scope of isolation
within
the transaction, which to me would be unusual and it would be probably
quite
painful to implement without spilling a copy of the records to disk (at
least a copy of the keys/order if you don't care about protecting from
changes that don't affect membership/order; some databases call these
keyset
cursors).
We could say that cursors always iterate snapshots, however this
introduces MVCC. Though it seems to me that SNAPSHOT_READ already does
that.
Actually, even with MVCC you'd see your own changes, because they
happen in the same transaction so the buffer pool will use the same
version
of the page. While it may be possible to reuse the MVCC infrastructure,
it
would still require the introduction of a second scope for stability.
It's quite implementable using append-only b-trees. Though it might be
much to ask that implementations are forced to use that.
An alternative to what I suggested earlier is that all read operations
use read committed. I.e. they always see the data as it looked at
the beginning of the transaction. Would this be more compatible with
existing MVCC implementations?
Hmm, so if you modified the object store and then, later in the same
transaction, used a cursor to iterate the object store, the cursor
would not see the earlier modifications? That's not very intiutive to
me...or did I misunderstand?
If we go with read committed then yes, your understanding is correct.
Out of curiosity, how
