Re: [IndexedDB] Atomic schema changes
On Sun, Jun 27, 2010 at 12:27 AM, Jonas Sicking jo...@sicking.cc wrote:
But, I feel pretty strongly that a setVersion/schema change
transaction
should not simply kill off anything else currently running. The
reason
is
that it's not hard for apps to recover from a connection failing, but
it
is
hard to handle a transaction failing in an unpredictable way.
Especially
static transactions (which should rarely fail committing since
serialization
can be guaranteed before the transaction starts).
That might be a good idea for v1. I was planning on doing a separate
thread for setVersion, but maybe it's tied enough to the topic of
schema changes that it makes sense to bring up here.
What I suggest is that when setVersion is called, we fire
'versionchange' event on all other open IDBDatabase objects. This
event contains information of what the desired new version number is.
If no other IDBDatabase objects are open for the specific database, no
'versionchange' events are fired. This allows pages using the old
schema version to automatically save any pending data (for example any
draft emails) and display UI to the user suggesting that the tab be
closed. If possible without dataloss, the tab could even reload itself
to automatically load an updated version of the page which uses the
new schema version.
The 'versionchange' event would use an interface like
interface IDBVersionEvent : IDBEvent {
readonly attribute string version;
};
First of all, what I was originally advocating (sorry for not being
clear)
is that we should kill the database connection but not until all active
transactions are complete. Though we should probably block new
transactions
from starting once setVersion is called.
But I really like your versionchange event idea regardless. I agree
that
letting the app sync any data that might be in memory (for example, a
draft
email) is important. And the idea that the web app could refresh itself
(or
download new application code or something) seems pretty cool and
useful.
I'm fine with it firing on all frames except the one that initiated
(like
storage events). If we go with the kill the connection once all active
transactions are done and block new ones from starting, we'd want to
start
the blocking only after all versionchange events have finished.
The main reason that I like the idea of not stating the version change
until
all active connections have closed is that not all apps will handle
versionchange. My original idea was that we should just break such web
apps
and let the user refresh, but now that you've pointed out the potential
for
data loss I'm not sure that's an option. Savvy web apps can kill all
existing database connections when they get the versionchange and thus
avoid
stalling things.
Additionally, if there are open IDBDatabase objects, we fire a
'blocked' event at the IDBRequest object returned from the setVersion
call. This allows the page to display UI to the user asking the user
to close all other relevant tabs.
Once all other IDBDatabase objects are closed, we create a transaction
and fire the normal 'success' event.
While there are pending version change requests, no success events are
fired for calls to IDBFactory.open for the relevant database.
We might want to support forcefully killing off open IDBDatabase
objects in the future, but I think that can wait until after v1.
Really? I can't see an app like gmail ever asking users to close tabs.
I
bet they'd sooner run all the application logic in an iframe and
navigate it
away when doing a schema change.
And I don't see many people correctly implementing a blocked event
handler.
If anything, it should be an error code.
It doesn't seem that hard to have an explicit way to tell the database
explicitly OK, I'm done. Or, at very least, we could make it so that
when
there's an existing setVersion happening, all new connection requests
stall.
That way all pages can reload themselves but they won't connect to the
database until the upgrade is complete.
But really...all of this is really hacky. I'm starting to wonder if we
should just kill the database connections on a setVersion as I
originally
tried to suggest.
I'm pretty concerned though that sites will need to take asynchronous
actions in order to save all required data. While gmail happily
automatically saves every few minutes, and presumably could
immediately do so upon a 'versionchange' event, I don't think all
editors are willing t. For example many editors ask the user if they
want to save the current changes when they are closed, in order to not
overwrite correct data.
Additionally, there is always the risk that developers will forget to
use a versionchange event handler to protect their data. I think a
good design principal is that if sites do
Re: [IndexedDB] Atomic schema changes
On Sat, Jun 26, 2010 at 11:44 AM, Jeremy Orlow jor...@chromium.org wrote:
On Sat, Jun 26, 2010 at 10:09 AM, Jonas Sicking jo...@sicking.cc wrote:
On Fri, Jun 25, 2010 at 2:43 AM, Jeremy Orlow jor...@chromium.org wrote:
I'm OK with making createObjectStore/createIndex synchronous. It
would
definitely make such code cleaner and I don't see a major downside,
but
at
the same time I feel like this API is starting to get kind of ad-hoc
and
unintuitive to a new user. Having the create and the remove
functions
completely different and in different places seems weird.
So I agree with either A or leaving things as originally proposed by
Jonas
in [IndexDB] Proposal for async API changes.
Well, there's no reason not to make the same changes to
removeObjectStore/removeIndex. Though the more I think with this
design createObjectStore and removeObjectStore can stay on IDBDatabase
and simply throw if called outside the appropriate transaction
callback.
Here is the revised proposal:
interface IDBDatabase {
...
IDBObjectStore createObjectStore (in DOMString name, in optional
DOMString keyPath, in optional boolean autoIncrement);
void removeObjectStore (in DOMString storeName);
...
};
interface IDBObjectStore {
...
IDBIndex createIndex (in DOMString name, in DOMString keyPath, in
optional boolean unique);
IDBRequest removeIndex (in DOMString indexName);
...
};
Where createObjecStore/createIndex throws if a objectStore or index of
the given name already exists, if the keyPath has an invalid syntax,
or if the function is called when not inside a version-change
transaction callback. And removeObjectStore/removeIndex throws if the
objectStore or index doesn't exist or if the function is called when
not inside a version-change transaction callback.
Throwing if not inside the right type of callback isn't a super clean
solution, but is really not that different from how add/put throws if
called when not inside a READ_WRITE transaction.
Just to be clear, no async function (one that returns an IDBRequest)
should
_ever_ throw. (They should only call onerror.) I assume you didn't
mean
add/put throws literally?
Well, there's no reason we couldn't make add/put throw if called from
inside the wrong type of transaction. I guess I don't feel strongly
about it, but it seems to me that calling put from inside a READ_ONLY
transaction is a much bigger mistake than writing a duplicate key or
running out of disc space. Additionally it's a situation where we
synchronously know that there is a bug in the program.
When this came up before, the thought was that it'd be confusing to web
developers which errors would raise immediately and which would call the
error callback. My feeling is that consistency is more useful than raising
right away for errors like this. (Plus, if we find it hard to get web
developers to do any error checking, it seems even harder to get them to do
it in 2 ways. :-)
Well, I wouldn't expect them to check for put-in-wrong transaction,
and thus have that error bubble up and become a top level error, which
is handled through the usual error reporting mechanisms which should
alert the developer of a bug in their code. So it's actually
intentional that this error is handled different from other errors.
But if everyone else feels otherwise I'll shut up and accept that strategy.
But, I feel pretty strongly that a setVersion/schema change
transaction
should not simply kill off anything else currently running. The
reason
is
that it's not hard for apps to recover from a connection failing, but
it
is
hard to handle a transaction failing in an unpredictable way.
Especially
static transactions (which should rarely fail committing since
serialization
can be guaranteed before the transaction starts).
That might be a good idea for v1. I was planning on doing a separate
thread for setVersion, but maybe it's tied enough to the topic of
schema changes that it makes sense to bring up here.
What I suggest is that when setVersion is called, we fire
'versionchange' event on all other open IDBDatabase objects. This
event contains information of what the desired new version number is.
If no other IDBDatabase objects are open for the specific database, no
'versionchange' events are fired. This allows pages using the old
schema version to automatically save any pending data (for example any
draft emails) and display UI to the user suggesting that the tab be
closed. If possible without dataloss, the tab could even reload itself
to automatically load an updated version of the page which uses the
new schema version.
The 'versionchange' event would use an interface like
interface IDBVersionEvent : IDBEvent {
readonly attribute string version;
};
First of all, what I was originally advocating (sorry
Re: [IndexedDB] Atomic schema changes
On Sun, Jun 27, 2010 at 8:27 AM, Jonas Sicking jo...@sicking.cc wrote:
On Sat, Jun 26, 2010 at 11:44 AM, Jeremy Orlow jor...@chromium.org
wrote:
On Sat, Jun 26, 2010 at 10:09 AM, Jonas Sicking jo...@sicking.cc
wrote:
On Fri, Jun 25, 2010 at 2:43 AM, Jeremy Orlow jor...@chromium.org
wrote:
I'm OK with making createObjectStore/createIndex synchronous. It
would
definitely make such code cleaner and I don't see a major downside,
but
at
the same time I feel like this API is starting to get kind of
ad-hoc
and
unintuitive to a new user. Having the create and the remove
functions
completely different and in different places seems weird.
So I agree with either A or leaving things as originally proposed
by
Jonas
in [IndexDB] Proposal for async API changes.
Well, there's no reason not to make the same changes to
removeObjectStore/removeIndex. Though the more I think with this
design createObjectStore and removeObjectStore can stay on
IDBDatabase
and simply throw if called outside the appropriate transaction
callback.
Here is the revised proposal:
interface IDBDatabase {
...
IDBObjectStore createObjectStore (in DOMString name, in optional
DOMString keyPath, in optional boolean autoIncrement);
void removeObjectStore (in DOMString storeName);
...
};
interface IDBObjectStore {
...
IDBIndex createIndex (in DOMString name, in DOMString keyPath, in
optional boolean unique);
IDBRequest removeIndex (in DOMString indexName);
...
};
Where createObjecStore/createIndex throws if a objectStore or index
of
the given name already exists, if the keyPath has an invalid syntax,
or if the function is called when not inside a version-change
transaction callback. And removeObjectStore/removeIndex throws if the
objectStore or index doesn't exist or if the function is called when
not inside a version-change transaction callback.
Throwing if not inside the right type of callback isn't a super clean
solution, but is really not that different from how add/put throws if
called when not inside a READ_WRITE transaction.
Just to be clear, no async function (one that returns an IDBRequest)
should
_ever_ throw. (They should only call onerror.) I assume you didn't
mean
add/put throws literally?
Well, there's no reason we couldn't make add/put throw if called from
inside the wrong type of transaction. I guess I don't feel strongly
about it, but it seems to me that calling put from inside a READ_ONLY
transaction is a much bigger mistake than writing a duplicate key or
running out of disc space. Additionally it's a situation where we
synchronously know that there is a bug in the program.
When this came up before, the thought was that it'd be confusing to web
developers which errors would raise immediately and which would call the
error callback. My feeling is that consistency is more useful than
raising
right away for errors like this. (Plus, if we find it hard to get web
developers to do any error checking, it seems even harder to get them to
do
it in 2 ways. :-)
Well, I wouldn't expect them to check for put-in-wrong transaction,
and thus have that error bubble up and become a top level error, which
is handled through the usual error reporting mechanisms which should
alert the developer of a bug in their code. So it's actually
intentional that this error is handled different from other errors.
But if everyone else feels otherwise I'll shut up and accept that strategy.
Lets start another thread and examine this. If all the errors can pretty
clearly/intuitively be broken into two groups, then maybe this does make
sense. (I'll try to do it later today unless you beat me to it.)
But, I feel pretty strongly that a setVersion/schema change
transaction
should not simply kill off anything else currently running. The
reason
is
that it's not hard for apps to recover from a connection failing,
but
it
is
hard to handle a transaction failing in an unpredictable way.
Especially
static transactions (which should rarely fail committing since
serialization
can be guaranteed before the transaction starts).
That might be a good idea for v1. I was planning on doing a separate
thread for setVersion, but maybe it's tied enough to the topic of
schema changes that it makes sense to bring up here.
What I suggest is that when setVersion is called, we fire
'versionchange' event on all other open IDBDatabase objects. This
event contains information of what the desired new version number is.
If no other IDBDatabase objects are open for the specific database,
no
'versionchange' events are fired. This allows pages using the old
schema version to automatically save any pending data (for example
any
draft emails) and display UI to
Re: [IndexedDB] Atomic schema changes
On Fri, Jun 25, 2010 at 2:43 AM, Jeremy Orlow jor...@chromium.org wrote:
I'm OK with making createObjectStore/createIndex synchronous. It would
definitely make such code cleaner and I don't see a major downside, but
at
the same time I feel like this API is starting to get kind of ad-hoc and
unintuitive to a new user. Having the create and the remove functions
completely different and in different places seems weird.
So I agree with either A or leaving things as originally proposed by
Jonas
in [IndexDB] Proposal for async API changes.
Well, there's no reason not to make the same changes to
removeObjectStore/removeIndex. Though the more I think with this
design createObjectStore and removeObjectStore can stay on IDBDatabase
and simply throw if called outside the appropriate transaction
callback.
Here is the revised proposal:
interface IDBDatabase {
...
IDBObjectStore createObjectStore (in DOMString name, in optional
DOMString keyPath, in optional boolean autoIncrement);
void removeObjectStore (in DOMString storeName);
...
};
interface IDBObjectStore {
...
IDBIndex createIndex (in DOMString name, in DOMString keyPath, in
optional boolean unique);
IDBRequest removeIndex (in DOMString indexName);
...
};
Where createObjecStore/createIndex throws if a objectStore or index of
the given name already exists, if the keyPath has an invalid syntax,
or if the function is called when not inside a version-change
transaction callback. And removeObjectStore/removeIndex throws if the
objectStore or index doesn't exist or if the function is called when
not inside a version-change transaction callback.
Throwing if not inside the right type of callback isn't a super clean
solution, but is really not that different from how add/put throws if
called when not inside a READ_WRITE transaction.
Just to be clear, no async function (one that returns an IDBRequest) should
_ever_ throw. (They should only call onerror.) I assume you didn't mean
add/put throws literally?
Well, there's no reason we couldn't make add/put throw if called from
inside the wrong type of transaction. I guess I don't feel strongly
about it, but it seems to me that calling put from inside a READ_ONLY
transaction is a much bigger mistake than writing a duplicate key or
running out of disc space. Additionally it's a situation where we
synchronously know that there is a bug in the program.
But, I feel pretty strongly that a setVersion/schema change transaction
should not simply kill off anything else currently running. The reason
is
that it's not hard for apps to recover from a connection failing, but it
is
hard to handle a transaction failing in an unpredictable way.
Especially
static transactions (which should rarely fail committing since
serialization
can be guaranteed before the transaction starts).
That might be a good idea for v1. I was planning on doing a separate
thread for setVersion, but maybe it's tied enough to the topic of
schema changes that it makes sense to bring up here.
What I suggest is that when setVersion is called, we fire
'versionchange' event on all other open IDBDatabase objects. This
event contains information of what the desired new version number is.
If no other IDBDatabase objects are open for the specific database, no
'versionchange' events are fired. This allows pages using the old
schema version to automatically save any pending data (for example any
draft emails) and display UI to the user suggesting that the tab be
closed. If possible without dataloss, the tab could even reload itself
to automatically load an updated version of the page which uses the
new schema version.
The 'versionchange' event would use an interface like
interface IDBVersionEvent : IDBEvent {
readonly attribute string version;
};
First of all, what I was originally advocating (sorry for not being clear)
is that we should kill the database connection but not until all active
transactions are complete. Though we should probably block new transactions
from starting once setVersion is called.
But I really like your versionchange event idea regardless. I agree that
letting the app sync any data that might be in memory (for example, a draft
email) is important. And the idea that the web app could refresh itself (or
download new application code or something) seems pretty cool and useful.
I'm fine with it firing on all frames except the one that initiated (like
storage events). If we go with the kill the connection once all active
transactions are done and block new ones from starting, we'd want to start
the blocking only after all versionchange events have finished.
The main reason that I like the idea of not stating the version change until
all active connections have closed is that not all apps will handle
versionchange. My original idea was that we should just break such web apps
and let the user refresh, but now
Re: [IndexedDB] Atomic schema changes
On Sat, Jun 26, 2010 at 10:09 AM, Jonas Sicking jo...@sicking.cc wrote:
On Fri, Jun 25, 2010 at 2:43 AM, Jeremy Orlow jor...@chromium.org wrote:
I'm OK with making createObjectStore/createIndex synchronous. It
would
definitely make such code cleaner and I don't see a major downside,
but
at
the same time I feel like this API is starting to get kind of ad-hoc
and
unintuitive to a new user. Having the create and the remove functions
completely different and in different places seems weird.
So I agree with either A or leaving things as originally proposed by
Jonas
in [IndexDB] Proposal for async API changes.
Well, there's no reason not to make the same changes to
removeObjectStore/removeIndex. Though the more I think with this
design createObjectStore and removeObjectStore can stay on IDBDatabase
and simply throw if called outside the appropriate transaction
callback.
Here is the revised proposal:
interface IDBDatabase {
...
IDBObjectStore createObjectStore (in DOMString name, in optional
DOMString keyPath, in optional boolean autoIncrement);
void removeObjectStore (in DOMString storeName);
...
};
interface IDBObjectStore {
...
IDBIndex createIndex (in DOMString name, in DOMString keyPath, in
optional boolean unique);
IDBRequest removeIndex (in DOMString indexName);
...
};
Where createObjecStore/createIndex throws if a objectStore or index of
the given name already exists, if the keyPath has an invalid syntax,
or if the function is called when not inside a version-change
transaction callback. And removeObjectStore/removeIndex throws if the
objectStore or index doesn't exist or if the function is called when
not inside a version-change transaction callback.
Throwing if not inside the right type of callback isn't a super clean
solution, but is really not that different from how add/put throws if
called when not inside a READ_WRITE transaction.
Just to be clear, no async function (one that returns an IDBRequest)
should
_ever_ throw. (They should only call onerror.) I assume you didn't mean
add/put throws literally?
Well, there's no reason we couldn't make add/put throw if called from
inside the wrong type of transaction. I guess I don't feel strongly
about it, but it seems to me that calling put from inside a READ_ONLY
transaction is a much bigger mistake than writing a duplicate key or
running out of disc space. Additionally it's a situation where we
synchronously know that there is a bug in the program.
When this came up before, the thought was that it'd be confusing to web
developers which errors would raise immediately and which would call the
error callback. My feeling is that consistency is more useful than raising
right away for errors like this. (Plus, if we find it hard to get web
developers to do any error checking, it seems even harder to get them to do
it in 2 ways. :-)
But, I feel pretty strongly that a setVersion/schema change
transaction
should not simply kill off anything else currently running. The
reason
is
that it's not hard for apps to recover from a connection failing, but
it
is
hard to handle a transaction failing in an unpredictable way.
Especially
static transactions (which should rarely fail committing since
serialization
can be guaranteed before the transaction starts).
That might be a good idea for v1. I was planning on doing a separate
thread for setVersion, but maybe it's tied enough to the topic of
schema changes that it makes sense to bring up here.
What I suggest is that when setVersion is called, we fire
'versionchange' event on all other open IDBDatabase objects. This
event contains information of what the desired new version number is.
If no other IDBDatabase objects are open for the specific database, no
'versionchange' events are fired. This allows pages using the old
schema version to automatically save any pending data (for example any
draft emails) and display UI to the user suggesting that the tab be
closed. If possible without dataloss, the tab could even reload itself
to automatically load an updated version of the page which uses the
new schema version.
The 'versionchange' event would use an interface like
interface IDBVersionEvent : IDBEvent {
readonly attribute string version;
};
First of all, what I was originally advocating (sorry for not being
clear)
is that we should kill the database connection but not until all active
transactions are complete. Though we should probably block new
transactions
from starting once setVersion is called.
But I really like your versionchange event idea regardless. I agree that
letting the app sync any data that might be in memory (for example, a
draft
email) is important. And the idea that the web app could refresh itself
(or
download new application code or something) seems pretty
Re: [IndexedDB] Atomic schema changes
On Thu, Jun 24, 2010 at 4:32 AM, Jeremy Orlow jor...@chromium.org wrote:
On Thu, Jun 24, 2010 at 1:48 AM, Jonas Sicking jo...@sicking.cc wrote:
Hi All,
In bug 9975 comment 1 [1] Nikunj pointed out that it is unclear how to
make atomic changes to the schema of a database. For example adding an
objectStore and a couple of indexes.
While it actually currently is possible, it is quite quirky and so I
think we need to find a better solution.
One way this is already possible is by calling setVersion. When the
success event fires for this request, it contains an implicitly
created transaction which, while it is alive, holds a lock on the
whole database an prevents any other interactions with the database.
However setVersion is a fairly heavy operation. We have discussed a
couple of different ways it can work, but it seems like there is
agreement that any other open database connections will either have to
be close manually (by for example the user leaving the page), or they
will be close forcefully (by making any requests on them fail). I
intend on sending a starting a separate thread on defining the details
of setVersion.
We might want to allow making smaller schema changes, such as adding a
new objectStore or a new index, without requiring all other database
connections to be closed. Further, it would be nice if atomicness
was a default behavior as to avoid people accidentally creating race
conditions.
We've talked a bit about this at mozilla and have three alternative
proposals. In all three proposals we suggest moving the
createObjectStore to the Transaction interface (or possibly a new
SchemaTransaction interface). The createIndex function remains on
objectStore, but is defined to throw an exception if called outside a
transaction which allows schema changes.
Proposal A:
Always require calls to setVersion for changes to the database schema.
The success event fired on the setVersion request is a
IDBTransactionEvent. The author can use the createObjectStore method
on the transaction available on the event to create new object stores.
Additionally, since we know that no one else currently has an open
database connection, we can make creating objectStores synchronous.
The implementation can probably still asynchronously fail to create an
objectStore, due to diskspace or other hardware issues. This failure
will likely only be detected asynchronously, but can be raised as a
failure to commit the transaction as it is extremely rare.
The code would look something like:
if (db.version == 2.0) {
weAreDoneFunction();
}
db.setVersion(2.0).onsuccess = function(event) {
trans = event.transaction;
store1 = trans.createObjectStore(myStore1, ...);
store2 = trans.createObjectStore(myStore2, ...);
store1.createIndex(...);
store1.createIndex(...);
store2.createIndex(...);
trans.oncomplete = weAreDoneFunction;
}
Proposal B:
Add a new type of transaction SCHEMA_CHANGE, in addition to READ and
READ_WRITE. This transaction is required for any schema changes. As
long as the transaction is open, no other schema changes can be done.
The transaction is opened asynchronously using a new
'startSchemaTransaction' function. This ensures that no other
modifications are attempted at the same time.
Additionally, since we know that no one else currently is inside a
SCHEMA_CHANGE transaction we can make creating objectStores
synchronous. The implementation can probably still asynchronously fail
to create an objectStore, due to diskspace or other hardware issues.
This failure will likely only be detected asynchronously, but can be
raised as a failure to commit the transaction as it is extremely rare.
Code example:
if (db.objectStoreNames.contains(myStore1)) {
weAreDoneFunction();
return;
}
db.startSchemaTransaction().onsuccess = function(event) {
// Have to check again as the objectStore could have been created
before the callback fired
if (db.objectStoreNames.contains(myStore1)) {
weAreDoneFunction();
return;
}
trans = event.transaction;
store1 = trans.createObjectStore(myStore1, ...);
store2 = trans.createObjectStore(myStore2, ...);
store1.createIndex(...);
store1.createIndex(...);
store2.createIndex(...);
trans.oncomplete = weAreDoneFunction;
}
Proposal C:
This is like proposal B, however the schema change transaction is
started synchronously, same as other transactions (we could even reuse
the existing transaction() function, however we'd have to ignore the
storeNames argument).
Since two different pages, running in different processes, could now
start a schema-change transaction at the same time, and thus could
call createObjectStore at the same time and attempt to create the same
store, we have to keep createObjectStore asynchronous.
Code example:
if (db.objectStoreNames.contains(myStore1)) {
weAreDoneFunction();
return;
}
trans = db.startSchemaTransaction();
Re: [IndexedDB] Atomic schema changes
On Fri, Jun 25, 2010 at 9:04 AM, Jonas Sicking jo...@sicking.cc wrote:
On Thu, Jun 24, 2010 at 4:32 AM, Jeremy Orlow jor...@chromium.org wrote:
On Thu, Jun 24, 2010 at 1:48 AM, Jonas Sicking jo...@sicking.cc wrote:
Hi All,
In bug 9975 comment 1 [1] Nikunj pointed out that it is unclear how to
make atomic changes to the schema of a database. For example adding an
objectStore and a couple of indexes.
While it actually currently is possible, it is quite quirky and so I
think we need to find a better solution.
One way this is already possible is by calling setVersion. When the
success event fires for this request, it contains an implicitly
created transaction which, while it is alive, holds a lock on the
whole database an prevents any other interactions with the database.
However setVersion is a fairly heavy operation. We have discussed a
couple of different ways it can work, but it seems like there is
agreement that any other open database connections will either have to
be close manually (by for example the user leaving the page), or they
will be close forcefully (by making any requests on them fail). I
intend on sending a starting a separate thread on defining the details
of setVersion.
We might want to allow making smaller schema changes, such as adding a
new objectStore or a new index, without requiring all other database
connections to be closed. Further, it would be nice if atomicness
was a default behavior as to avoid people accidentally creating race
conditions.
We've talked a bit about this at mozilla and have three alternative
proposals. In all three proposals we suggest moving the
createObjectStore to the Transaction interface (or possibly a new
SchemaTransaction interface). The createIndex function remains on
objectStore, but is defined to throw an exception if called outside a
transaction which allows schema changes.
Proposal A:
Always require calls to setVersion for changes to the database schema.
The success event fired on the setVersion request is a
IDBTransactionEvent. The author can use the createObjectStore method
on the transaction available on the event to create new object stores.
Additionally, since we know that no one else currently has an open
database connection, we can make creating objectStores synchronous.
The implementation can probably still asynchronously fail to create an
objectStore, due to diskspace or other hardware issues. This failure
will likely only be detected asynchronously, but can be raised as a
failure to commit the transaction as it is extremely rare.
The code would look something like:
if (db.version == 2.0) {
weAreDoneFunction();
}
db.setVersion(2.0).onsuccess = function(event) {
trans = event.transaction;
store1 = trans.createObjectStore(myStore1, ...);
store2 = trans.createObjectStore(myStore2, ...);
store1.createIndex(...);
store1.createIndex(...);
store2.createIndex(...);
trans.oncomplete = weAreDoneFunction;
}
Proposal B:
Add a new type of transaction SCHEMA_CHANGE, in addition to READ and
READ_WRITE. This transaction is required for any schema changes. As
long as the transaction is open, no other schema changes can be done.
The transaction is opened asynchronously using a new
'startSchemaTransaction' function. This ensures that no other
modifications are attempted at the same time.
Additionally, since we know that no one else currently is inside a
SCHEMA_CHANGE transaction we can make creating objectStores
synchronous. The implementation can probably still asynchronously fail
to create an objectStore, due to diskspace or other hardware issues.
This failure will likely only be detected asynchronously, but can be
raised as a failure to commit the transaction as it is extremely rare.
Code example:
if (db.objectStoreNames.contains(myStore1)) {
weAreDoneFunction();
return;
}
db.startSchemaTransaction().onsuccess = function(event) {
// Have to check again as the objectStore could have been created
before the callback fired
if (db.objectStoreNames.contains(myStore1)) {
weAreDoneFunction();
return;
}
trans = event.transaction;
store1 = trans.createObjectStore(myStore1, ...);
store2 = trans.createObjectStore(myStore2, ...);
store1.createIndex(...);
store1.createIndex(...);
store2.createIndex(...);
trans.oncomplete = weAreDoneFunction;
}
Proposal C:
This is like proposal B, however the schema change transaction is
started synchronously, same as other transactions (we could even reuse
the existing transaction() function, however we'd have to ignore the
storeNames argument).
Since two different pages, running in different processes, could now
start a schema-change transaction at the same time, and thus could
call createObjectStore at the same time and attempt to create the same
store, we have to keep createObjectStore
Re: [IndexedDB] Atomic schema changes
In IDBCouch I don't have a schema but I do have to maintain
consistency of the by-sequence index which is a similar problem to
validating schema state before these kinds of operations.
What I'm currently doing is just starting each write transaction with
a lookup to the end of the by-sequence index to make sure the
lastSequence I have is, in fact, the current one and another
tab/window hasn't updated it.
My plan for view generation is a similar problem and I plan to solve
it with a an objectStore of meta information about all of the views.
Storing the last known sequence and conflict resolution information
about replicas is also a similar problem and I'll solve it the same
way with a meta objectStore.
I don't see why schema information couldn't also be stored in a meta
objectStore at the end transactions that modify it and all of these
higher level APIs could just start their transaction with a validation
of the meta info. Rather than trying to keep the information globally
and updating it with an event you can just validate it at the
beginning of each transaction. The overhead is minimal and it seems,
to me at least, to be a little less error prone.
-Mikeal
On Fri, Jun 25, 2010 at 2:43 AM, Jeremy Orlow jor...@chromium.org wrote:
On Fri, Jun 25, 2010 at 9:04 AM, Jonas Sicking jo...@sicking.cc wrote:
On Thu, Jun 24, 2010 at 4:32 AM, Jeremy Orlow jor...@chromium.org wrote:
On Thu, Jun 24, 2010 at 1:48 AM, Jonas Sicking jo...@sicking.cc wrote:
Hi All,
In bug 9975 comment 1 [1] Nikunj pointed out that it is unclear how to
make atomic changes to the schema of a database. For example adding an
objectStore and a couple of indexes.
While it actually currently is possible, it is quite quirky and so I
think we need to find a better solution.
One way this is already possible is by calling setVersion. When the
success event fires for this request, it contains an implicitly
created transaction which, while it is alive, holds a lock on the
whole database an prevents any other interactions with the database.
However setVersion is a fairly heavy operation. We have discussed a
couple of different ways it can work, but it seems like there is
agreement that any other open database connections will either have to
be close manually (by for example the user leaving the page), or they
will be close forcefully (by making any requests on them fail). I
intend on sending a starting a separate thread on defining the details
of setVersion.
We might want to allow making smaller schema changes, such as adding a
new objectStore or a new index, without requiring all other database
connections to be closed. Further, it would be nice if atomicness
was a default behavior as to avoid people accidentally creating race
conditions.
We've talked a bit about this at mozilla and have three alternative
proposals. In all three proposals we suggest moving the
createObjectStore to the Transaction interface (or possibly a new
SchemaTransaction interface). The createIndex function remains on
objectStore, but is defined to throw an exception if called outside a
transaction which allows schema changes.
Proposal A:
Always require calls to setVersion for changes to the database schema.
The success event fired on the setVersion request is a
IDBTransactionEvent. The author can use the createObjectStore method
on the transaction available on the event to create new object stores.
Additionally, since we know that no one else currently has an open
database connection, we can make creating objectStores synchronous.
The implementation can probably still asynchronously fail to create an
objectStore, due to diskspace or other hardware issues. This failure
will likely only be detected asynchronously, but can be raised as a
failure to commit the transaction as it is extremely rare.
The code would look something like:
if (db.version == 2.0) {
weAreDoneFunction();
}
db.setVersion(2.0).onsuccess = function(event) {
trans = event.transaction;
store1 = trans.createObjectStore(myStore1, ...);
store2 = trans.createObjectStore(myStore2, ...);
store1.createIndex(...);
store1.createIndex(...);
store2.createIndex(...);
trans.oncomplete = weAreDoneFunction;
}
Proposal B:
Add a new type of transaction SCHEMA_CHANGE, in addition to READ and
READ_WRITE. This transaction is required for any schema changes. As
long as the transaction is open, no other schema changes can be done.
The transaction is opened asynchronously using a new
'startSchemaTransaction' function. This ensures that no other
modifications are attempted at the same time.
Additionally, since we know that no one else currently is inside a
SCHEMA_CHANGE transaction we can make creating objectStores
synchronous. The implementation can probably still asynchronously fail
to create an objectStore, due to diskspace or other hardware issues.
This failure will
Re: [IndexedDB] Atomic schema changes
On Thu, Jun 24, 2010 at 1:48 AM, Jonas Sicking jo...@sicking.cc wrote:
Hi All,
In bug 9975 comment 1 [1] Nikunj pointed out that it is unclear how to
make atomic changes to the schema of a database. For example adding an
objectStore and a couple of indexes.
While it actually currently is possible, it is quite quirky and so I
think we need to find a better solution.
One way this is already possible is by calling setVersion. When the
success event fires for this request, it contains an implicitly
created transaction which, while it is alive, holds a lock on the
whole database an prevents any other interactions with the database.
However setVersion is a fairly heavy operation. We have discussed a
couple of different ways it can work, but it seems like there is
agreement that any other open database connections will either have to
be close manually (by for example the user leaving the page), or they
will be close forcefully (by making any requests on them fail). I
intend on sending a starting a separate thread on defining the details
of setVersion.
We might want to allow making smaller schema changes, such as adding a
new objectStore or a new index, without requiring all other database
connections to be closed. Further, it would be nice if atomicness
was a default behavior as to avoid people accidentally creating race
conditions.
We've talked a bit about this at mozilla and have three alternative
proposals. In all three proposals we suggest moving the
createObjectStore to the Transaction interface (or possibly a new
SchemaTransaction interface). The createIndex function remains on
objectStore, but is defined to throw an exception if called outside a
transaction which allows schema changes.
Proposal A:
Always require calls to setVersion for changes to the database schema.
The success event fired on the setVersion request is a
IDBTransactionEvent. The author can use the createObjectStore method
on the transaction available on the event to create new object stores.
Additionally, since we know that no one else currently has an open
database connection, we can make creating objectStores synchronous.
The implementation can probably still asynchronously fail to create an
objectStore, due to diskspace or other hardware issues. This failure
will likely only be detected asynchronously, but can be raised as a
failure to commit the transaction as it is extremely rare.
The code would look something like:
if (db.version == 2.0) {
weAreDoneFunction();
}
db.setVersion(2.0).onsuccess = function(event) {
trans = event.transaction;
store1 = trans.createObjectStore(myStore1, ...);
store2 = trans.createObjectStore(myStore2, ...);
store1.createIndex(...);
store1.createIndex(...);
store2.createIndex(...);
trans.oncomplete = weAreDoneFunction;
}
Proposal B:
Add a new type of transaction SCHEMA_CHANGE, in addition to READ and
READ_WRITE. This transaction is required for any schema changes. As
long as the transaction is open, no other schema changes can be done.
The transaction is opened asynchronously using a new
'startSchemaTransaction' function. This ensures that no other
modifications are attempted at the same time.
Additionally, since we know that no one else currently is inside a
SCHEMA_CHANGE transaction we can make creating objectStores
synchronous. The implementation can probably still asynchronously fail
to create an objectStore, due to diskspace or other hardware issues.
This failure will likely only be detected asynchronously, but can be
raised as a failure to commit the transaction as it is extremely rare.
Code example:
if (db.objectStoreNames.contains(myStore1)) {
weAreDoneFunction();
return;
}
db.startSchemaTransaction().onsuccess = function(event) {
// Have to check again as the objectStore could have been created
before the callback fired
if (db.objectStoreNames.contains(myStore1)) {
weAreDoneFunction();
return;
}
trans = event.transaction;
store1 = trans.createObjectStore(myStore1, ...);
store2 = trans.createObjectStore(myStore2, ...);
store1.createIndex(...);
store1.createIndex(...);
store2.createIndex(...);
trans.oncomplete = weAreDoneFunction;
}
Proposal C:
This is like proposal B, however the schema change transaction is
started synchronously, same as other transactions (we could even reuse
the existing transaction() function, however we'd have to ignore the
storeNames argument).
Since two different pages, running in different processes, could now
start a schema-change transaction at the same time, and thus could
call createObjectStore at the same time and attempt to create the same
store, we have to keep createObjectStore asynchronous.
Code example:
if (db.objectStoreNames.contains(myStore1)) {
weAreDoneFunction();
return;
}
trans = db.startSchemaTransaction();
trans.createObjectStore(myStore1, ...);
trans.createObjectStore(myStore2, ...).onsuccess =
[IndexedDB] Atomic schema changes
Hi All,
In bug 9975 comment 1 [1] Nikunj pointed out that it is unclear how to
make atomic changes to the schema of a database. For example adding an
objectStore and a couple of indexes.
While it actually currently is possible, it is quite quirky and so I
think we need to find a better solution.
One way this is already possible is by calling setVersion. When the
success event fires for this request, it contains an implicitly
created transaction which, while it is alive, holds a lock on the
whole database an prevents any other interactions with the database.
However setVersion is a fairly heavy operation. We have discussed a
couple of different ways it can work, but it seems like there is
agreement that any other open database connections will either have to
be close manually (by for example the user leaving the page), or they
will be close forcefully (by making any requests on them fail). I
intend on sending a starting a separate thread on defining the details
of setVersion.
We might want to allow making smaller schema changes, such as adding a
new objectStore or a new index, without requiring all other database
connections to be closed. Further, it would be nice if atomicness
was a default behavior as to avoid people accidentally creating race
conditions.
We've talked a bit about this at mozilla and have three alternative
proposals. In all three proposals we suggest moving the
createObjectStore to the Transaction interface (or possibly a new
SchemaTransaction interface). The createIndex function remains on
objectStore, but is defined to throw an exception if called outside a
transaction which allows schema changes.
Proposal A:
Always require calls to setVersion for changes to the database schema.
The success event fired on the setVersion request is a
IDBTransactionEvent. The author can use the createObjectStore method
on the transaction available on the event to create new object stores.
Additionally, since we know that no one else currently has an open
database connection, we can make creating objectStores synchronous.
The implementation can probably still asynchronously fail to create an
objectStore, due to diskspace or other hardware issues. This failure
will likely only be detected asynchronously, but can be raised as a
failure to commit the transaction as it is extremely rare.
The code would look something like:
if (db.version == 2.0) {
weAreDoneFunction();
}
db.setVersion(2.0).onsuccess = function(event) {
trans = event.transaction;
store1 = trans.createObjectStore(myStore1, ...);
store2 = trans.createObjectStore(myStore2, ...);
store1.createIndex(...);
store1.createIndex(...);
store2.createIndex(...);
trans.oncomplete = weAreDoneFunction;
}
Proposal B:
Add a new type of transaction SCHEMA_CHANGE, in addition to READ and
READ_WRITE. This transaction is required for any schema changes. As
long as the transaction is open, no other schema changes can be done.
The transaction is opened asynchronously using a new
'startSchemaTransaction' function. This ensures that no other
modifications are attempted at the same time.
Additionally, since we know that no one else currently is inside a
SCHEMA_CHANGE transaction we can make creating objectStores
synchronous. The implementation can probably still asynchronously fail
to create an objectStore, due to diskspace or other hardware issues.
This failure will likely only be detected asynchronously, but can be
raised as a failure to commit the transaction as it is extremely rare.
Code example:
if (db.objectStoreNames.contains(myStore1)) {
weAreDoneFunction();
return;
}
db.startSchemaTransaction().onsuccess = function(event) {
// Have to check again as the objectStore could have been created
before the callback fired
if (db.objectStoreNames.contains(myStore1)) {
weAreDoneFunction();
return;
}
trans = event.transaction;
store1 = trans.createObjectStore(myStore1, ...);
store2 = trans.createObjectStore(myStore2, ...);
store1.createIndex(...);
store1.createIndex(...);
store2.createIndex(...);
trans.oncomplete = weAreDoneFunction;
}
Proposal C:
This is like proposal B, however the schema change transaction is
started synchronously, same as other transactions (we could even reuse
the existing transaction() function, however we'd have to ignore the
storeNames argument).
Since two different pages, running in different processes, could now
start a schema-change transaction at the same time, and thus could
call createObjectStore at the same time and attempt to create the same
store, we have to keep createObjectStore asynchronous.
Code example:
if (db.objectStoreNames.contains(myStore1)) {
weAreDoneFunction();
return;
}
trans = db.startSchemaTransaction();
trans.createObjectStore(myStore1, ...);
trans.createObjectStore(myStore2, ...).onsuccess = function(event) {
store1 = trans.objectStore(myStore1);
store2 = trans.objectStore(myStore2);
store2 = trans.createObjectStore(myStore2, ...);
