Re: [IndexedDB] Interaction between transactions and objects that allow multiple operations
On Thu, May 6, 2010 at 8:25 PM, ben turner bent.mozi...@gmail.com wrote:
Hey folks,
I'm working with Shawn on the Firefox implementation. Here's our idea
as of now, would you all please comment about things you like or
dislike? Hopefully this follows the gist of the comments shared
already.
interface IndexedDatabaseRequest {
IDBRequest open(in DOMString name,
in DOMString description,
in optional boolean modifyDatabase);
};
interface IDBDatabaseRequest : IDBDatabase {
IDBRequest openTransaction(in optional DOMStringList storeNames,
in optional unsigned long timeout);
};
interface IDBTransactionRequest : IDBTransaction {
IDBRequest abort();
IDBRequest commit();
IDBRequest createObjectStore(in DOMString name,
in DOMString keyPath,
in optional boolean autoIncrement);
IDBRequest openObjectStore(in DOMString name,
in optional unsigned short mode);
IDBRequest createIndex(in DOMString name,
in DOMString storeName,
in DOMString keyPath,
in optional boolean unique);
IDBRequest openIndex(in DOMString name);
IDBRequest removeObjectStore(in DOMString storeName);
IDBRequest removeIndex(in DOMString indexName);
IDBRequest setVersion(in DOMString version);
};
We've made some additional changes to IDBRequest and actually
specified the success/error events, but they're not really relevant
here and I'll post about them later.
Note that if we go this route then the mode parameter of the
openObjectStore method becomes nearly meaningless since transactions
are currently supposed to exclusively lock the stores and indexes they
access.
As Shawn has said previously we're looking to make the async API as
close to the synchronous API as possible (never allowing anything to
block, of course) to avoid tons of nested functions that will make web
developers' lives harder. To that end we're wondering how much of the
IDBTransactionRequest interface (as written above) can be made
synchronous-like. For instance, we could simply wait to fire the
success callback until we've read some metadata back from the database
file. Then we could make openObjectStore simply return an
IDBObjectStoreRequest instead of another IDBRequest as we would
already know if the object store exists. It would be even simpler if
the caller used the storeNames parameter in the openTransaction call
since we wouldn't call the success callback unless those stores
existed. We could do similar cheats for the other methods.
What do you guys think?
I don't see any major problems with making the open and create methods
synchronous. It doesn't seem like this would limit UAs or future
improvements to the spec too much. Intuitively, setVersion and the remove
methods seem more dangerous though. And since they're a lot less on the
common path for web developers, I'd lean towards leaving them as is.
Btw, if we go with dynamic transactions only (as Pablo was thinking about
proposing [1]) then we could also make starting a transaction asynchronous.
Unfortunately, none of this makes fetching entries (whether via .get or
a cursor) any easier. And, since any fetch could possibly need to go to
disk, I don't see any easy way around that. Which is a big deal since
reading data is probably one of the most common tasks for a database.
Especially one without joins built in. :-) Any ideas on how to make this
better?
J
[1] http://lists.w3.org/Archives/Public/public-webapps/2010AprJun/0272.html
Re: [IndexedDB] Interaction between transactions and objects that allow multiple operations
On May 4, 2010, at 7:17 PM, Pablo Castro wrote: The interaction between transactions and objects that allow multiple operations is giving us trouble. I need to elaborate a little to explain the problem. You can perform operations in IndexedDB with or without an explicitly started transaction. When no transaction is present, you get an implicit one that is there for the duration of the operation and is committed and the end (or rolled-back if an error occurs). To provide context to those who might be missing it, an explicit transaction is active in an IndexedDB Database as long as it has not been explicitly committed or aborted. An implicit transaction's life time is under the control of the implementation and spans no more than the operation requested. There are a number of operations in IndexedDB that are a single step. For example, store.put() occurs either entirely in the current transaction (if the user started one explicitly) or in an implicit transaction if there isn't one active at the time the operation starts. The interaction between the operation and transactions is straightforward in this case. On the other hand, other operations in IndexedDB return an object that then allows multiple operations on it. For example, when you open a cursor over a store, you can then move to the next row, update a row, delete a row, etc. The question is, what is the interaction between these operations and transactions? Are all interactions with a given cursor supposed to happen within the transaction that was active (implicit or explicit) when the cursor was opened? Or should each interaction happen in its own transaction (unless there is a long-lived active transaction, of course)? The transactional context of a series of operations is the transaction that was created in the database. Each and every operation from that point on till one of the following happens is performed in that transaction: 1. The transaction is committed 2. The transaction is aborted 3. The database object goes out of scope. We have a few options: a) make multi-step objects bound to the transaction that was present when the object is first created (or an implicit one if none was present). This requires new APIs to mark cursors and such as done so implicit transactions can commit/abort, and has issues around use of the database object while a cursor with an implicit transaction is open. b) make each interaction happen in its own transaction (explicit or implicit). This is quite unusual and means you'll get inconsistent reads from row to row while scanning unless you wrap cursor/index scans on transactions. It also probably poses interesting implementation challenges depending on what you're using as your storage engine. c) require an explicit transaction always, along the lines Nikunj's original proposal had it. We would move most methods from database to transaction (except a few properties such as version and such, which it may still be ok to handle implicitly from the transactions perspective). This eliminates this whole problem altogether at the cost of an extra step required always. We would prefer to go with option c) and always require explicit transactions. Thoughts? The current specification allows using an explicit transaction and once initiated, the explicitly created transaction is applicable for its life time as described above. IOW a) is the same as c) If you intend to perform multiple steps, then an explicit transaction appears to be in order unless the application can tolerate inconsistent results. Therefore b) is not a good idea for multi-step operations. In addition, it is not a good idea to create/commit explicit transactions for each operation. There has been some discussion for nested transactions and the original proposal had support for those, but recall that some implementors were not convinced of the cost/benefit tradeoff on that one. Nikunj
Re: [IndexedDB] Interaction between transactions and objects that allow multiple operations
On May 5, 2010, at 1:56 PM, Shawn Wilsher wrote: On 5/5/2010 1:09 PM, Jeremy Orlow wrote: I'd also worry that if creating the transaction were completely transparent to the user that they might not think to close it either. (I'm mainly thinking about copy-and-paste coders here.) I should have been more clear. That statement goes along with the suggestion to make everything work off of a transaction - object stores, indexes, cursors, etc. They'd have to know about the transaction because they'd have to use it. I feel that auto transaction creation is syntactic sugar and should be left to libraries. On the other hand, I'd be worried if we were developing for complex multi-tab applications and not explicitly managing transactions. Nikunj
Re: [IndexedDB] Interaction between transactions and objects that allow multiple operations
On Thu, May 6, 2010 at 9:14 AM, Nikunj Mehta nik...@o-micron.com wrote: On May 4, 2010, at 7:17 PM, Pablo Castro wrote: The interaction between transactions and objects that allow multiple operations is giving us trouble. I need to elaborate a little to explain the problem. You can perform operations in IndexedDB with or without an explicitly started transaction. When no transaction is present, you get an implicit one that is there for the duration of the operation and is committed and the end (or rolled-back if an error occurs). To provide context to those who might be missing it, an explicit transaction is active in an IndexedDB Database as long as it has not been explicitly committed or aborted. An implicit transaction's life time is under the control of the implementation and spans no more than the operation requested. There are a number of operations in IndexedDB that are a single step. For example, store.put() occurs either entirely in the current transaction (if the user started one explicitly) or in an implicit transaction if there isn't one active at the time the operation starts. The interaction between the operation and transactions is straightforward in this case. On the other hand, other operations in IndexedDB return an object that then allows multiple operations on it. For example, when you open a cursor over a store, you can then move to the next row, update a row, delete a row, etc. The question is, what is the interaction between these operations and transactions? Are all interactions with a given cursor supposed to happen within the transaction that was active (implicit or explicit) when the cursor was opened? Or should each interaction happen in its own transaction (unless there is a long-lived active transaction, of course)? The transactional context of a series of operations is the transaction that was created in the database. Each and every operation from that point on till one of the following happens is performed in that transaction: 1. The transaction is committed 2. The transaction is aborted 3. The database object goes out of scope. We have a few options: a) make multi-step objects bound to the transaction that was present when the object is first created (or an implicit one if none was present). This requires new APIs to mark cursors and such as done so implicit transactions can commit/abort, and has issues around use of the database object while a cursor with an implicit transaction is open. b) make each interaction happen in its own transaction (explicit or implicit). This is quite unusual and means you'll get inconsistent reads from row to row while scanning unless you wrap cursor/index scans on transactions. It also probably poses interesting implementation challenges depending on what you're using as your storage engine. c) require an explicit transaction always, along the lines Nikunj's original proposal had it. We would move most methods from database to transaction (except a few properties such as version and such, which it may still be ok to handle implicitly from the transactions perspective). This eliminates this whole problem altogether at the cost of an extra step required always. We would prefer to go with option c) and always require explicit transactions. Thoughts? The current specification allows using an explicit transaction and once initiated, the explicitly created transaction is applicable for its life time as described above. IOW a) is the same as c) If you intend to perform multiple steps, then an explicit transaction appears to be in order unless the application can tolerate inconsistent results. Therefore b) is not a good idea for multi-step operations. In addition, it is not a good idea to create/commit explicit transactions for each operation. Nikunj, I don't really understand your responses. I'm pretty sure Pablo's whole question revolved around implicit transactions and whether we should get rid of them (option c), make cursors only available within transactions, etc. It looks like most of your response was clarifying how explicit transactions should work? There has been some discussion for nested transactions and the original proposal had support for those, but recall that some implementors were not convinced of the cost/benefit tradeoff on that one. One of the strongest pieces of feedback I've gotten from Google developers who have worked with gears/SQLDatabase is in support of open nested transactions. This is because it allows multiple layers of abstraction to all use the database without needing a lot of code to coordinate between them. Implementing them should be fairly straight forward: just keep a count of the nesting level and only commit the whole transaction if that count becomes 0. Aborts of course require a little extra work, but I don't believe it'll be a major burden on implementors. On Thu, May 6, 2010 at 9:18 AM, Nikunj Mehta
Re: [IndexedDB] Interaction between transactions and objects that allow multiple operations
Hey folks,
I'm working with Shawn on the Firefox implementation. Here's our idea
as of now, would you all please comment about things you like or
dislike? Hopefully this follows the gist of the comments shared
already.
interface IndexedDatabaseRequest {
IDBRequest open(in DOMString name,
in DOMString description,
in optional boolean modifyDatabase);
};
interface IDBDatabaseRequest : IDBDatabase {
IDBRequest openTransaction(in optional DOMStringList storeNames,
in optional unsigned long timeout);
};
interface IDBTransactionRequest : IDBTransaction {
IDBRequest abort();
IDBRequest commit();
IDBRequest createObjectStore(in DOMString name,
in DOMString keyPath,
in optional boolean autoIncrement);
IDBRequest openObjectStore(in DOMString name,
in optional unsigned short mode);
IDBRequest createIndex(in DOMString name,
in DOMString storeName,
in DOMString keyPath,
in optional boolean unique);
IDBRequest openIndex(in DOMString name);
IDBRequest removeObjectStore(in DOMString storeName);
IDBRequest removeIndex(in DOMString indexName);
IDBRequest setVersion(in DOMString version);
};
We've made some additional changes to IDBRequest and actually
specified the success/error events, but they're not really relevant
here and I'll post about them later.
Note that if we go this route then the mode parameter of the
openObjectStore method becomes nearly meaningless since transactions
are currently supposed to exclusively lock the stores and indexes they
access.
As Shawn has said previously we're looking to make the async API as
close to the synchronous API as possible (never allowing anything to
block, of course) to avoid tons of nested functions that will make web
developers' lives harder. To that end we're wondering how much of the
IDBTransactionRequest interface (as written above) can be made
synchronous-like. For instance, we could simply wait to fire the
success callback until we've read some metadata back from the database
file. Then we could make openObjectStore simply return an
IDBObjectStoreRequest instead of another IDBRequest as we would
already know if the object store exists. It would be even simpler if
the caller used the storeNames parameter in the openTransaction call
since we wouldn't call the success callback unless those stores
existed. We could do similar cheats for the other methods.
What do you guys think?
-Ben
On Thu, May 6, 2010 at 3:17 AM, Jeremy Orlow jor...@chromium.org wrote:
On Thu, May 6, 2010 at 9:14 AM, Nikunj Mehta nik...@o-micron.com wrote:
On May 4, 2010, at 7:17 PM, Pablo Castro wrote:
The interaction between transactions and objects that allow multiple
operations is giving us trouble. I need to elaborate a little to explain
the
problem.
You can perform operations in IndexedDB with or without an explicitly
started transaction. When no transaction is present, you get an implicit
one
that is there for the duration of the operation and is committed and the
end
(or rolled-back if an error occurs).
To provide context to those who might be missing it, an explicit
transaction is active in an IndexedDB Database as long as it has not been
explicitly committed or aborted. An implicit transaction's life time is
under the control of the implementation and spans no more than the operation
requested.
There are a number of operations in IndexedDB that are a single step.
For example, store.put() occurs either entirely in the current transaction
(if the user started one explicitly) or in an implicit transaction if there
isn't one active at the time the operation starts. The interaction between
the operation and transactions is straightforward in this case.
On the other hand, other operations in IndexedDB return an object that
then allows multiple operations on it. For example, when you open a cursor
over a store, you can then move to the next row, update a row, delete a
row,
etc. The question is, what is the interaction between these operations and
transactions? Are all interactions with a given cursor supposed to happen
within the transaction that was active (implicit or explicit) when the
cursor was opened? Or should each interaction happen in its own transaction
(unless there is a long-lived active transaction, of course)?
The transactional context of a series of operations is the transaction
that was created in the database. Each and every operation from that point
on till one of the following happens is performed in that transaction:
1. The transaction is committed
2. The transaction is aborted
3. The database object goes out of scope.
We have a few options:
a) make multi-step objects bound to the
Re: [IndexedDB] Interaction between transactions and objects that allow multiple operations
Hey folks,
I'm working with Shawn on the Firefox implementation. Here's our idea
as of now, would you all please comment about things you like or
dislike? Hopefully this follows the gist of the comments shared
already.
interface IndexedDatabaseRequest {
IDBRequest open(in DOMString name,
in DOMString description,
in optional boolean modifyDatabase);
};
interface IDBDatabaseRequest : IDBDatabase {
IDBRequest openTransaction(in optional DOMStringList storeNames,
in optional unsigned long timeout);
};
interface IDBTransactionRequest : IDBTransaction {
IDBRequest abort();
IDBRequest commit();
IDBRequest createObjectStore(in DOMString name,
in DOMString keyPath,
in optional boolean autoIncrement);
IDBRequest openObjectStore(in DOMString name,
in optional unsigned short mode);
IDBRequest createIndex(in DOMString name,
in DOMString storeName,
in DOMString keyPath,
in optional boolean unique);
IDBRequest openIndex(in DOMString name);
IDBRequest removeObjectStore(in DOMString storeName);
IDBRequest removeIndex(in DOMString indexName);
IDBRequest setVersion(in DOMString version);
};
We've made some additional changes to IDBRequest and actually
specified the success/error events, but they're not really relevant
here and I'll post about them later.
Note that if we go this route then the mode parameter of the
openObjectStore method becomes nearly meaningless since transactions
are currently supposed to exclusively lock the stores and indexes they
access.
As Shawn has said previously we're looking to make the async API as
close to the synchronous API as possible (never allowing anything to
block, of course) to avoid tons of nested functions that will make web
developers' lives harder. To that end we're wondering how much of the
IDBTransactionRequest interface (as written above) can be made
synchronous-like. For instance, we could simply wait to fire the
success callback until we've read some metadata back from the database
file. Then we could make openObjectStore simply return an
IDBObjectStoreRequest instead of another IDBRequest as we would
already know if the object store exists. It would be even simpler if
the caller used the storeNames parameter in the openTransaction call
since we wouldn't call the success callback unless those stores
existed. We could do similar cheats for the other methods.
What do you guys think?
-Ben
Re: [IndexedDB] Interaction between transactions and objects that allow multiple operations
On 5/5/2010 11:44 AM, Jeremy Orlow wrote: On the other hand, a lot of even the most basic tasks probably should be done within a transaction. But if the easiest way to do something is to just run it outside of a transaction, I'm guessing a good portion of users (including tutorial websites, people giving advice on forums, etc) will just do it that way. Mandating transactions forces users to be cognizant of them. Maybe that's a good thing... To be clear, I'm not disagreeing with that. Mozilla would really love to reduce the amount of async steps to do anything with a database. Right now, to get something from an object store, we have three steps: 1) open database; wait for callback 2) open object store; wait for callback 3) get from object store; wait for callback Adding the transaction stuff would result in four steps: 1) open database; wait for callback 2) open transaction; wait for callback 3) open object store; wait for callback 4) get from object store; wait for callback Maybe we can get rid of this additional step by giving a transaction to the consumer immediately available after the call to open, and have a property on the connection to access the current transaction. But that might complicate things more too. This is complicated by the fact that each step blocks similar operations (can only open a database when another one is not in the process of opening, same with object stores, etc). This problem goes away with my proposal in the past [1], but I'm not sure we have consensus on it (or the event based approach for that matter). Cheers, Shawn [1] http://lists.w3.org/Archives/Public/public-webapps/2010JanMar/0961.html smime.p7s Description: S/MIME Cryptographic Signature
Re: [IndexedDB] Interaction between transactions and objects that allow multiple operations
On Wed, May 5, 2010 at 8:56 PM, Shawn Wilsher sdwi...@mozilla.com wrote: On 5/5/2010 11:44 AM, Jeremy Orlow wrote: On the other hand, a lot of even the most basic tasks probably should be done within a transaction. But if the easiest way to do something is to just run it outside of a transaction, I'm guessing a good portion of users (including tutorial websites, people giving advice on forums, etc) will just do it that way. Mandating transactions forces users to be cognizant of them. Maybe that's a good thing... To be clear, I'm not disagreeing with that. Mozilla would really love to reduce the amount of async steps to do anything with a database. Right now, to get something from an object store, we have three steps: 1) open database; wait for callback 2) open object store; wait for callback 3) get from object store; wait for callback Adding the transaction stuff would result in four steps: 1) open database; wait for callback 2) open transaction; wait for callback 3) open object store; wait for callback 4) get from object store; wait for callback Maybe we can get rid of this additional step by giving a transaction to the consumer immediately available after the call to open, and have a property on the connection to access the current transaction. But that might complicate things more too. I'd also worry that if creating the transaction were completely transparent to the user that they might not think to close it either. (I'm mainly thinking about copy-and-paste coders here.) To be honest, while gross, I don't think 4 steps is _that_ much worse then 3 steps and I still think Pablo's option c is the best one. This is complicated by the fact that each step blocks similar operations (can only open a database when another one is not in the process of opening, same with object stores, etc). This problem goes away with my proposal in the past [1], but I'm not sure we have consensus on it (or the event based approach for that matter). Not sure about Microsoft/Pablo, but you've got my support. I'm definitely convinced that what's currently specced is not adequate. I was leaning towards a callback based interface, but after discussing the pros and cons with several developers and the fact that pretty much all the other new APIs are going with an event based model, it seemed like the best way to go. So I'm actually in the process of landing a patch to switch (my half working implementation in) WebKit over to an event based model. Cheers, Shawn [1] http://lists.w3.org/Archives/Public/public-webapps/2010JanMar/0961.html
Re: [IndexedDB] Interaction between transactions and objects that allow multiple operations
On 5/5/2010 1:09 PM, Jeremy Orlow wrote: I'd also worry that if creating the transaction were completely transparent to the user that they might not think to close it either. (I'm mainly thinking about copy-and-paste coders here.) I should have been more clear. That statement goes along with the suggestion to make everything work off of a transaction - object stores, indexes, cursors, etc. They'd have to know about the transaction because they'd have to use it. Cheers, Shawn smime.p7s Description: S/MIME Cryptographic Signature
