Re: [IndexedDB] Proposal for async API changes
On Thu, May 20, 2010 at 8:32 PM, Jeremy Orlow wrote: > On Thu, May 20, 2010 at 8:19 PM, Jonas Sicking wrote: >> >> On Thu, May 20, 2010 at 11:55 AM, Shawn Wilsher >> wrote: >> > On 5/20/2010 11:30 AM, Andrei Popescu wrote: >> >> >> >> As someone new to this API, I thought the naming used in the current >> >> draft is somewhat confusing. Consider the following interfaces: >> >> >> >> IndexedDatabase >> >> IndexedDatabaseRequest, >> >> IDBDatabaseRequest, >> >> IDBDatabase, >> >> IDBRequest >> >> >> >> Just by looking at this, it is pretty hard to understand what the >> >> relationship between these interfaces really is and what role do they >> >> play in the API. For instance, I thought that the IDBDatabaseRequest >> >> is some type of Request when, in fact, it isn't a Request at all. It >> >> also isn't immediately obvious what the difference between >> >> IndexedDatabase and IDBDatabase really is, etc. >> > >> > It should be noted that we did not want to rock the boat too much with >> > our >> > proposal, so we stuck with the existing names. I think the current spec >> > as >> > written has the same issues. >> >> We kept the existing names specifically to avoid tying bikeshed naming >> discussions to technical discussion about how these interfaces should >> behave :) > > Totally agree with both of you! But I think now is as good of a time as any > to discuss these issues (since they apply to both specs). In this case, I > actually don't think this is really bike shedding since we're all agreeing > the current names are confusing, but I guess the line is fine. :-) > Ok, so it looks like we all agree that these changes are needed. I think it would be good to update the spec to reflect this so we all can see the changes in context. How do we go about it? One option would be to edit the draft that Jonas sent but I think it would be nicer to edit the real draft. I volunteer to help but I'd need CVS access All the best, Andrei
RE: [IndexedDB] Proposal for async API changes
(still catching up on the rest of the long thread of API changes, will get back to that a bit later) From: public-webapps-requ...@w3.org [mailto:public-webapps-requ...@w3.org] On Behalf Of Jeremy Orlow Sent: Thursday, May 20, 2010 3:34 PM >> >> On Thu, May 20, 2010 at 11:25 PM, Shawn Wilsher >> >> wrote: >> >> On 5/20/2010 7:34 AM, Shawn Wilsher wrote: >> >> So far it's really just that joins are painful in IndexedDB. I'm working >> >> on a blog post on this very topic though, and I'll be sure to point >> >> everyone in this thread to it (I figure this is useful stuff to get out >> >> to a wider audience). >> >> And honestly, I thought that we had discussed joins on this list, but I >> >> only see a thread from Pablo mentioning it, but no real discussions. >> >> Should we start that? >> Joins were actually in the original spec but taken out during the effort to >> simply the API greatly. IIRC, the main reason why Nikunj took them out is >> that we believed you could fairly efficiently join yourself if you had 2 >> sorted lists and because we didn't see a simple way to do them without >> introducing a lot of API surface area or creating (or borrowing) some sort >> of syntax for the joins. (Now that I think about it, though, maybe doing >> this is not that big of a leap from what we're going to need to do to spec >> keyPaths. I'm starting to wonder if we need to rethink that as well) >> Anyway, the decision was made so long ago that maybe it's worth re-opening >> the discussion. I'll hunt through my mail archives tomorrow and start a new >> thread with references to any original bits of info I can find. My main concern with joins, besides API surface, was that in order to implement joins you need to choose an actual strategy. Depending on whether you have indexes or not and other circumstances you could choose to do range scans/lookups, a merge join, etc. So at least for fancier libraries this would only be of partial help, as they would probably want to do their own joins sometimes. I'm happy to explore again though. It's certainly the case that for simpler cases it might help users pull off tasks without depending on a library. I do wonder if we should try and land the async API first. -pablo J
Re: [IndexedDB] Proposal for async API changes
On Thu, May 20, 2010 at 11:25 PM, Shawn Wilsher wrote: > On 5/20/2010 7:34 AM, Shawn Wilsher wrote: > >> So far it's really just that joins are painful in IndexedDB. I'm working >> on a blog post on this very topic though, and I'll be sure to point >> everyone in this thread to it (I figure this is useful stuff to get out >> to a wider audience). >> > And honestly, I thought that we had discussed joins on this list, but I > only see a thread from Pablo mentioning it, but no real discussions. Should > we start that? > Joins were actually in the original spec but taken out during the effort to simply the API greatly. IIRC, the main reason why Nikunj took them out is that we believed you could fairly efficiently join yourself if you had 2 sorted lists and because we didn't see a simple way to do them without introducing a lot of API surface area or creating (or borrowing) some sort of syntax for the joins. (Now that I think about it, though, maybe doing this is not that big of a leap from what we're going to need to do to spec keyPaths. I'm starting to wonder if we need to rethink that as well) Anyway, the decision was made so long ago that maybe it's worth re-opening the discussion. I'll hunt through my mail archives tomorrow and start a new thread with references to any original bits of info I can find. J
Re: [IndexedDB] Proposal for async API changes
On 5/20/2010 7:34 AM, Shawn Wilsher wrote: So far it's really just that joins are painful in IndexedDB. I'm working on a blog post on this very topic though, and I'll be sure to point everyone in this thread to it (I figure this is useful stuff to get out to a wider audience). And honestly, I thought that we had discussed joins on this list, but I only see a thread from Pablo mentioning it, but no real discussions. Should we start that? Cheers, Shawn smime.p7s Description: S/MIME Cryptographic Signature
Re: [IndexedDB] Proposal for async API changes
On Thu, May 20, 2010 at 9:37 PM, Shawn Wilsher wrote: > On 5/20/2010 12:19 PM, Jonas Sicking wrote: > >> I additionally like the naming convention. The async interfaces is >> probably the interface that people will use first. Additionally that >> interface is available both to workers and to the main thread. So it >> makes sense to give the async interface the simpler name. >> > That is not how it is currently specified (and our proposal doesn't > indicate either way). The asynchronous versions are not available to worker > threads. I do recall discussion on this, however... > There was discussion, and I'm pretty sure the consensus was that it should be added. The main reason is that we want code that works in the page to work in a worker and that it really isn't much of a burden to implementors. I'll open an issue on this (tomorrow). J
Re: [IndexedDB] Proposal for async API changes
On 5/20/2010 12:19 PM, Jonas Sicking wrote: I additionally like the naming convention. The async interfaces is probably the interface that people will use first. Additionally that interface is available both to workers and to the main thread. So it makes sense to give the async interface the simpler name. That is not how it is currently specified (and our proposal doesn't indicate either way). The asynchronous versions are not available to worker threads. I do recall discussion on this, however... Cheers, Shawn smime.p7s Description: S/MIME Cryptographic Signature
Re: [IndexedDB] Proposal for async API changes
On Thu, May 20, 2010 at 8:19 PM, Jonas Sicking wrote: > On Thu, May 20, 2010 at 11:55 AM, Shawn Wilsher > wrote: > > On 5/20/2010 11:30 AM, Andrei Popescu wrote: > >> > >> As someone new to this API, I thought the naming used in the current > >> draft is somewhat confusing. Consider the following interfaces: > >> > >> IndexedDatabase > >> IndexedDatabaseRequest, > >> IDBDatabaseRequest, > >> IDBDatabase, > >> IDBRequest > >> > >> Just by looking at this, it is pretty hard to understand what the > >> relationship between these interfaces really is and what role do they > >> play in the API. For instance, I thought that the IDBDatabaseRequest > >> is some type of Request when, in fact, it isn't a Request at all. It > >> also isn't immediately obvious what the difference between > >> IndexedDatabase and IDBDatabase really is, etc. > > > > It should be noted that we did not want to rock the boat too much with > our > > proposal, so we stuck with the existing names. I think the current spec > as > > written has the same issues. > > We kept the existing names specifically to avoid tying bikeshed naming > discussions to technical discussion about how these interfaces should > behave :) > Totally agree with both of you! But I think now is as good of a time as any to discuss these issues (since they apply to both specs). In this case, I actually don't think this is really bike shedding since we're all agreeing the current names are confusing, but I guess the line is fine. :-) > >> - I know we need to keep the "IDB" prefix in order to avoid collisions > >> with other APIs. I would therefore think we should keep the IDB prefix > >> and make sure all the interfaces start with it (right now they don't). > > > > I agree with this. We should probably file a bug about this. > > > >> - The "Request" suffix is now used to denote the asynchronous versions > >> of the API interfaces. These interfaces aren't actually Requests of > >> any kind, so I would like to suggest changing this suffix. In fact, if > >> the primary usage of this API is via its async version, we could even > >> drop this suffix altogether and just add "Sync" to the synchronous > >> versions? > > > > I agree that Request seems confusing and seems to be contrary to what > other > > specs use. We should try to follow what other specs do here. > > Agreed on both accounts. There unfortunately isn't much in the way of > precedence here. There are three other specs to look at here, which > specify API for both workers and main thread. > > * Web Workers spec > http://www.whatwg.org/specs/web-workers/current-work/ This spec > doesn't actually use different interfaces for workers and main thread. > * File API http://dev.w3.org/2006/webapi/FileAPI/ Specifies FileReader > and FileReaderSync. The two interfaces are separate and doesn't > inherit from a common base > * WebSQLDatabase http://dev.w3.org/html5/webdatabase/ Specifies > separate interfaces, like Database and DatabaseSync. The two > interfaces are separate and doesn't inherit from a common base. > > I think we should follow the same convention as File API and > WebSQLDatabase. There really isn't anything to be gained by having a > common base interface, it just makes the spec harder to read as > functionality is distributed between the base interface and the > sync/async interface. > > I additionally like the naming convention. The async interfaces is > probably the interface that people will use first. Additionally that > interface is available both to workers and to the main thread. So it > makes sense to give the async interface the simpler name. > Agreed on all counts. I would add that, if we did decide to keep base interfaces, we could always suffix them with Base (which I think makes it more clear they're base interfaces)...but it sounds like that might not be necessary. > >> - Some of the interfaces could have names that would more closely > >> reflect their roles in the API. For instance, IDBDatabase could be > >> renamed to IDBConnection, since in the spec it is described as "a > >> connection to the database". > > Not really sold on this. I've always hated the "connection" > abstraction. With our suggested API IDBDatabase really just holds > metadata information about the database, and doesn't need to represent > a connection at all. > Well, if we don't make these proposed changes, then I think it's pretty important to rename to connection since the current spec does distinguish between each connection. With your proposal, this is not currently true, but I think it's hard to say whether or not these concepts will creep back in in later versions. In addition, the entire spec is related to databases, so I think naming it a "connection" rather than a "database" helps distinguish it from the rest. That said, if IndexedDatabase* becomes IDBFactory* then there certainly is a lot less confusion already, and this change is less important. J
Re: [IndexedDB] Proposal for async API changes
On Thu, May 20, 2010 at 11:55 AM, Shawn Wilsher wrote: > On 5/20/2010 11:30 AM, Andrei Popescu wrote: >> >> As someone new to this API, I thought the naming used in the current >> draft is somewhat confusing. Consider the following interfaces: >> >> IndexedDatabase >> IndexedDatabaseRequest, >> IDBDatabaseRequest, >> IDBDatabase, >> IDBRequest >> >> Just by looking at this, it is pretty hard to understand what the >> relationship between these interfaces really is and what role do they >> play in the API. For instance, I thought that the IDBDatabaseRequest >> is some type of Request when, in fact, it isn't a Request at all. It >> also isn't immediately obvious what the difference between >> IndexedDatabase and IDBDatabase really is, etc. > > It should be noted that we did not want to rock the boat too much with our > proposal, so we stuck with the existing names. I think the current spec as > written has the same issues. We kept the existing names specifically to avoid tying bikeshed naming discussions to technical discussion about how these interfaces should behave :) >> - I know we need to keep the "IDB" prefix in order to avoid collisions >> with other APIs. I would therefore think we should keep the IDB prefix >> and make sure all the interfaces start with it (right now they don't). > > I agree with this. We should probably file a bug about this. > >> - The "Request" suffix is now used to denote the asynchronous versions >> of the API interfaces. These interfaces aren't actually Requests of >> any kind, so I would like to suggest changing this suffix. In fact, if >> the primary usage of this API is via its async version, we could even >> drop this suffix altogether and just add "Sync" to the synchronous >> versions? > > I agree that Request seems confusing and seems to be contrary to what other > specs use. We should try to follow what other specs do here. Agreed on both accounts. There unfortunately isn't much in the way of precedence here. There are three other specs to look at here, which specify API for both workers and main thread. * Web Workers spec http://www.whatwg.org/specs/web-workers/current-work/ This spec doesn't actually use different interfaces for workers and main thread. * File API http://dev.w3.org/2006/webapi/FileAPI/ Specifies FileReader and FileReaderSync. The two interfaces are separate and doesn't inherit from a common base * WebSQLDatabase http://dev.w3.org/html5/webdatabase/ Specifies separate interfaces, like Database and DatabaseSync. The two interfaces are separate and doesn't inherit from a common base. I think we should follow the same convention as File API and WebSQLDatabase. There really isn't anything to be gained by having a common base interface, it just makes the spec harder to read as functionality is distributed between the base interface and the sync/async interface. I additionally like the naming convention. The async interfaces is probably the interface that people will use first. Additionally that interface is available both to workers and to the main thread. So it makes sense to give the async interface the simpler name. >> - Some of the interfaces could have names that would more closely >> reflect their roles in the API. For instance, IDBDatabase could be >> renamed to IDBConnection, since in the spec it is described as "a >> connection to the database". Not really sold on this. I've always hated the "connection" abstraction. With our suggested API IDBDatabase really just holds metadata information about the database, and doesn't need to represent a connection at all. >> Likewise, IndexedDatabase could become >> IDBFactory since it is used to create database connections or key >> ranges. This I agree with though. >> In any case, I want to make it clear that the current naming works >> once one takes the time to understand it. On the other hand, if we >> make it easier for people to understand the API, we could hopefully >> get feedback from more developers. > > Making it easier for someone to look at the method names and just know how > to use the API seems like a good goal in my book. Agreed. The 'Request' suffix in particular took me quite a while to get used to. / Jonas
Re: [IndexedDB] Proposal for async API changes
On 5/20/2010 11:30 AM, Andrei Popescu wrote: As someone new to this API, I thought the naming used in the current draft is somewhat confusing. Consider the following interfaces: IndexedDatabase IndexedDatabaseRequest, IDBDatabaseRequest, IDBDatabase, IDBRequest Just by looking at this, it is pretty hard to understand what the relationship between these interfaces really is and what role do they play in the API. For instance, I thought that the IDBDatabaseRequest is some type of Request when, in fact, it isn't a Request at all. It also isn't immediately obvious what the difference between IndexedDatabase and IDBDatabase really is, etc. It should be noted that we did not want to rock the boat too much with our proposal, so we stuck with the existing names. I think the current spec as written has the same issues. - I know we need to keep the "IDB" prefix in order to avoid collisions with other APIs. I would therefore think we should keep the IDB prefix and make sure all the interfaces start with it (right now they don't). I agree with this. We should probably file a bug about this. - The "Request" suffix is now used to denote the asynchronous versions of the API interfaces. These interfaces aren't actually Requests of any kind, so I would like to suggest changing this suffix. In fact, if the primary usage of this API is via its async version, we could even drop this suffix altogether and just add "Sync" to the synchronous versions? I agree that Request seems confusing and seems to be contrary to what other specs use. We should try to follow what other specs do here. - Some of the interfaces could have names that would more closely reflect their roles in the API. For instance, IDBDatabase could be renamed to IDBConnection, since in the spec it is described as "a connection to the database". Likewise, IndexedDatabase could become IDBFactory since it is used to create database connections or key ranges. This sounds good as well. In any case, I want to make it clear that the current naming works once one takes the time to understand it. On the other hand, if we make it easier for people to understand the API, we could hopefully get feedback from more developers. Making it easier for someone to look at the method names and just know how to use the API seems like a good goal in my book. Cheers, Shawn smime.p7s Description: S/MIME Cryptographic Signature
Re: [IndexedDB] Proposal for async API changes
On 5/20/2010 9:03 AM, Jonas Sicking wrote:
For what it's worth, one of the ideas behind object stores, rather
than rows+column stores, is to reduce the need for joins. I.e. in our
candy store example you could just as well store objects like:
{ id: 1, name: "Adam", sales: [{candyId: 1, date: "2010-01-02"}, ...]}
Well, if people care about the size of data on disk, they will likely
want to normalize their object stores. We had talked about doing joins
before in indexedDB, but I believe we decided to punt on the issue for now.
Cheers,
Shawn
smime.p7s
Description: S/MIME Cryptographic Signature
Re: [IndexedDB] Proposal for async API changes
On Thu, May 20, 2010 at 7:34 AM, Shawn Wilsher wrote:
> On 5/20/2010 2:55 AM, Jeremy Orlow wrote:
>>
>> Thanks for taking the time to do this!
>>
>> Can you maybe discuss the pros and cons you found in terms of implementing
>> something in WebSQLDatabase vs. IndexedDB? I'm mainly interested in
>> seeing
>> if there's any thing we can improve in IndexedDB that WebSQLDatabase
>> already
>> does well.
>
> So far it's really just that joins are painful in IndexedDB. I'm working on
> a blog post on this very topic though, and I'll be sure to point everyone in
> this thread to it (I figure this is useful stuff to get out to a wider
> audience).
For what it's worth, one of the ideas behind object stores, rather
than rows+column stores, is to reduce the need for joins. I.e. in our
candy store example you could just as well store objects like:
{ id: 1, name: "Adam", sales: [{candyId: 1, date: "2010-01-02"}, ...]}
However we wanted to use joins in our examples as they are a good way
to illustrate more complex queries and are sure to still happen, even
if less often.
/ Jonas
Re: [IndexedDB] Proposal for async API changes
On 5/20/2010 2:55 AM, Jeremy Orlow wrote: Thanks for taking the time to do this! Can you maybe discuss the pros and cons you found in terms of implementing something in WebSQLDatabase vs. IndexedDB? I'm mainly interested in seeing if there's any thing we can improve in IndexedDB that WebSQLDatabase already does well. So far it's really just that joins are painful in IndexedDB. I'm working on a blog post on this very topic though, and I'll be sure to point everyone in this thread to it (I figure this is useful stuff to get out to a wider audience). Cheers, Shawn smime.p7s Description: S/MIME Cryptographic Signature
Re: [IndexedDB] Proposal for async API changes
Thanks for taking the time to do this! Can you maybe discuss the pros and cons you found in terms of implementing something in WebSQLDatabase vs. IndexedDB? I'm mainly interested in seeing if there's any thing we can improve in IndexedDB that WebSQLDatabase already does well. J On Wed, May 19, 2010 at 10:03 PM, Shawn Wilsher wrote: > On 5/19/2010 1:50 PM, Shawn Wilsher wrote: > Er, and I managed to botch the SQL in the last two examples. Those should > be (if my non-tested SQL-fu is right): > SELECT name, COUNT(kids.id) > > FROM kids INNER JOIN candySales > ON kids.id = candySales.kidId > GROUP BY kids.id; > > and: > SELECT name, COUNT(kids.id) > > FROM kids LEFT JOIN candySales > ON kids.id = candySales.kidId > GROUP BY kids.id; > > respectively. > > Cheers, > > Shawn > >
Re: [IndexedDB] Proposal for async API changes
On 5/19/2010 1:50 PM, Shawn Wilsher wrote: Er, and I managed to botch the SQL in the last two examples. Those should be (if my non-tested SQL-fu is right): SELECT name, COUNT(kids.id) FROM kids INNER JOIN candySales ON kids.id = candySales.kidId GROUP BY kids.id; and: SELECT name, COUNT(kids.id) FROM kids LEFT JOIN candySales ON kids.id = candySales.kidId GROUP BY kids.id; respectively. Cheers, Shawn smime.p7s Description: S/MIME Cryptographic Signature
Re: [IndexedDB] Proposal for async API changes
On 5/17/2010 6:15 PM, Jonas Sicking wrote:
We've created some examples of what using this proposed API would look like:
http://docs.google.com/document/pub?id=1I__XnwvvSwyjvxi-FAAE0ecnUDhk5DF7L2GI6O31o18
we've also implemented the same examples using the currently drafted API:
http://docs.google.com/document/pub?id=1KKMAg_oHLeBvFUWND5km6FJtKi4jWxwKR0paKfZc8vU
And I've gone ahead and converted these samples to WebDatabase language
too for comparison purposes (I think, at least. I haven't used it
before, and these aren't actual fully runnable demos. If you spot an
error, let me know!)
It should be noted that WebDatabase fares much better when it comes to
joins, but I suspect we all knew that.
// Initialize database
var db = window.openDatabase("CandyDB", "", "My candy store database",
1024);
if (db.version != "1") {
db.changeVersion(db.version, "1", function(tx) {
// User's first visit. Initialize database.
var tables = [
{ name: "kids", columns: ["id INTEGER PRIMARY KEY",
"name TEXT"]},
{ name: "candy", columns: ["id INTEGER PRIMARY KEY",
"name TEXT"]},
{ name: "candySales", columns: ["kidId INTEGER",
"candyId INTEGER",
"date TEXT"]}
];
for (var index = 0; index < tables.length; index++) {
var table = tables[index];
tx.executeSql("CREATE TABLE " + table.name + "(" +
table.columns.join(", ") + ");",
null, tableCreated);
}
}, null, function() { loadData(db); });
}
else {
// User has been here before, no initialization required.
loadData(db);
}
function loadData(db) {
// Do stuff!
}
// List kids
var db = window.openDatabase("CandyDB", "1", "My candy store database",
1024);
db.readTransaction(function(tx) {
// Enumerate the entire table.
tx.executeSql("SELECT * FROM kids", function(tx, results) {
var rows = results.rows;
for (var index = 0; index < rows.length; index++) {
var item = rows.item(index);
var element = document.createElement("div");
element.textContent = item.name;
document.getElementById("kidList").appendChild(element);
}
});
});
// Store kids into database
var kids = [
{ name: "Anna" },
{ name: "Betty" },
{ name: "Christine" },
];
var db = window.openDatabase("CandyDB", "1", "My candy store database",
1024);
db.transaction(function(tx) {
for (var index = 0; index < kids.length; index++) {
var kid = kids[index];
tx.executeSql("INSERT INTO kids (name) VALUES (:name);", [kid],
function(tx, results) {
document.getElementById("display").textContent =
"Saved record for " + kid.name + " with id " + results.insertId;
});
}
});
// List kids who bought candy, and the number of purchases they made
var db = window.openDatabase("CandyDB", "1", "My candy store database",
1024);
db.readTransaction(function(tx) {
tx.executeSql("SELECT name, count " +
"FROM kids " +
"INNER JOIN candySales " +
"ON kids.id = candySales.kidId;",
function(tx, results) {
var rows = results.rows;
for (var index = 0; index < rows.length; index++) {
var item = rows.item(index);
display.textContent += ", " + item.name + "bought " +
item.count + "pieces";
}
});
});
// List kids who bought candy, and the number of purchases they made
// (some may have bought 0)
var db = window.openDatabase("CandyDB", "1", "My candy store database",
1024);
db.readTransaction(function(tx) {
tx.executeSql("SELECT name, count " +
"FROM kids " +
"LEFT JOIN candySales " +
"ON kids.id = candySales.kidId;",
function(tx, results) {
var rows = results.rows;
for (var index = 0; index < rows.length; index++) {
var item = rows.item(index);
display.textContent += ", " + item.name + "bought " +
item.count + "pieces";
}
});
});
Cheers,
Shawn
smime.p7s
Description: S/MIME Cryptographic Signature
Re: [IndexedDB] Proposal for async API changes
On Tue, May 18, 2010 at 12:10 PM, Jeremy Orlow wrote: 10. You are allowed to have multiple transactions per database connection. However if they use overlapping tables, only the first one will receive events until it is finished (with the usual exceptions of allowing multiple readers of the same table). >>> >>> Can you please clarify what you mean here? This seems like simply an >>> implementation detail to me, so maybe I'm missing something? >> >> What this is trying to say is that you can have an object store being used >> in more than one transaction, but they cannot access it at the same time. >> However, I think it's best for Jonas to chime in here because this doesn't >> quite seem right to me like it did yesterday. > > Oh, I see. The problem is that if you open an entity store and start > multiple transactions, it's not clear which it's associated with. I guess I > feel like what Jonas described would be pretty confusing. The objectStore accessor lives off of the IDBTransactionRequest API, so they are always associated with the transaction they were retrieved using. I agree it's a little confusing that you can have several references to the "same" objectStore through multiple transactions. However if you look at the examples the code works out quite nicely and is quite understandable IMHO. > What about creating a IDBSuccessEvents.transaction that's the transaction > the request is associated with (or null)? That's already there. Though it's on IDBTransactionEvent, which is the event type fired for successful operations that involve transactions (almost all of them except for the success event for opening the database). > Another option is to only allow > one (top level) transaction per connection. (I still think we should > support open nested transactions.) This is what the spec says right now. I don't really see what the advantage is though since you can create several connections to the same database which means that you basically have the exact same situation. Only difference is that you now have several database connections floating around as well. >>> 8) We can't leave deciding whether a cursor is pre-loaded up to UAs >>> since people will code for their favorite UA and then access >>> IDBCursorPreloadedRequest.count when some other UA does it as a >>> non-preloaded request. Even in the same UA this will cause problems when >>> users have different datasets than developers are testing with. >> >> I think that you might have been confused by our wording there. Sorry >> about that! IDBCursorPreloadedRequest is what you get if you pass sync=true >> into openCursor or openObjectCursor. Basically, sync cursors will give you >> a count, whereas async ones will not. > > > Oh. I missed that parameter and I guess let my imagination run wild. > :-) > I'm not sure I like the idea of offering sync cursors either since the UA > will either need to load everything into memory before starting or risk > blocking on disk IO for large data sets. Thus I'm not sure I support the > idea of synchronous cursors. But, at the same time, I'm concerned about the > overhead of firing one event per value with async cursors. Which is why I > was suggesting an interface where the common case (the data is in memory) is > done synchronously but the uncommon case (we'd block if we had to respond > synchronously) has to be handled since we guarantee that the first time will > be forced to be asynchronous. > Like I said, I'm not super happy with what I proposed, but I think some > hybrid async/sync interface is really what we need. Have you guys spent any > time thinking about something like this? How dead-set are you on > synchronous cursors? The idea is that synchronous cursors load all the required data into memory, yes. I think it would help authors a lot to be able to load small chunks of data into memory and read and write to it synchronously. Dealing with asynchronous operations constantly is certainly possible, but a bit of a pain for authors. I don't think we should obsess too much about not keeping things in memory, we already have things like canvas and the DOM which adds up to non-trivial amounts of memory. Just because data is loaded from a database doesn't mean it's huge. I do note that you're not as concerned about getAll(), which actually have worse memory characteristics than synchronous cursors since you need to create the full JS object graph in memory. / Jonas
Re: [IndexedDB] Proposal for async API changes
On Tue, May 18, 2010 at 6:32 PM, Shawn Wilsher wrote: > On 5/18/2010 7:20 AM, Jeremy Orlow wrote: > >> 1. Once a database has been opened (a database connection has been >>> established) read access to meta-data, such as objectStore and index >>> names, is synchronous. Changes to such meta data, such as creating >>> objectStores and indexes, is still asynchronous. >>> >> I believe this is already how it's specced. The IDBDatabase interface >> already gives you synchronous access to all of this. >> > Mostly that is the same, with the exception of getting an object store or > index which is now synchronous. > > > 9. IDBKeyRanges are created using functions on IndexedDatabaseRequest. >>> We couldn't figure out how the old API allowed you to create a range >>> object without first having a range object. >>> >> In the spec, I see the following in examples: >> var range = new IDBKeyRange.bound(2, 4); >> and >> var range = IDBKeyRange.leftBound(key); >> >> I'm not particularly happy with hanging functions off of >> IndexedDatabaseRequest for this. Can it work something like what I listed >> above? If not, maybe we can find a better place to put them? Or just >> create multiple openCursor functions for each case? >> > I think one concern with the above syntax is that it's adding another > object to the global scope. I recall Ben Turner and I discussing the > possibility of hanging it off of indexedDB, so: > var range = new indexedDB.KeyRange.bound(2, 4); > Good point re the global scope. > My concern with making multiple openCursor functions is that there is more > API complexity there. With that said, I don't have any strong opinions here > either way. How is there more API complexity to have several openCursor functions rather than several factories for KeyRange objects? 10. You are allowed to have multiple transactions per database >>> connection. However if they use overlapping tables, only the first one >>> will receive events until it is finished (with the usual exceptions of >>> allowing multiple readers of the same table). >>> >> Can you please clarify what you mean here? This seems like simply an >> implementation detail to me, so maybe I'm missing something? >> > What this is trying to say is that you can have an object store being used > in more than one transaction, but they cannot access it at the same time. > However, I think it's best for Jonas to chime in here because this doesn't > quite seem right to me like it did yesterday. Oh, I see. The problem is that if you open an entity store and start multiple transactions, it's not clear which it's associated with. I guess I feel like what Jonas described would be pretty confusing. What about creating a IDBSuccessEvents.transaction that's the transaction the request is associated with (or null)? Another option is to only allow one (top level) transaction per connection. (I still think we should support open nested transactions.) 5) You have two IDBTransactionRequest.onaborts. I think one is supposed >> to >> be an ontimeout. >> > Whoops, I thought we fixed that before he sent this out :) > > > 6) What is the default limit for the getAll functions? Should we make it >> 0 >> (and define any<=0 amount to mean infinity)? >> > I believe we intended the default to be infinity (as in, if you don't > specify, you get it all). Ahh, so not specifying would be the only way to get infinity? Yeah, I guess that works. What if someone passes 0? Just return a request with null? > 7) I expect "add or modify" to be more used than the add or the modify >> methods. As such, I wonder if it makes sense to optimize the naming for >> that. For example, addOrModify=>set, add=>add/insert, >> modify=>modify/update/replace maybe? >> > We had a lot of internal debate over this very thing. The problem we kept > running into was that set could easily be read as doing just updating so it > wouldn't be clear that you can also insert a new record there (without > reading the spec or some tutorial). The benefit with addOrModify is that it > is very explicit in what it does. We don't like how long it is, but we > couldn't come up with a shorter name that doesn't have a fair amount of > ambiguity. I see. Well, I'm OK starting with addOrModify and seeing if developers scream. > 8) We can't leave deciding whether a cursor is pre-loaded up to UAs >> since people will code for their favorite UA and then access >> IDBCursorPreloadedRequest.count when some other UA does it as a >> non-preloaded request. Even in the same UA this will cause problems when >> users have different datasets than developers are testing with. >> > I think that you might have been confused by our wording there. Sorry > about that! IDBCursorPreloadedRequest is what you get if you pass sync=true > into openCursor or openObjectCursor. Basically, sync cursors will give you > a count, whereas async ones will not. Oh. I missed that parameter and I guess let my i
Re: [IndexedDB] Proposal for async API changes
On 5/18/2010 7:20 AM, Jeremy Orlow wrote:
1. Once a database has been opened (a database connection has been
established) read access to meta-data, such as objectStore and index
names, is synchronous. Changes to such meta data, such as creating
objectStores and indexes, is still asynchronous.
I believe this is already how it's specced. The IDBDatabase interface
already gives you synchronous access to all of this.
Mostly that is the same, with the exception of getting an object store
or index which is now synchronous.
9. IDBKeyRanges are created using functions on IndexedDatabaseRequest.
We couldn't figure out how the old API allowed you to create a range
object without first having a range object.
In the spec, I see the following in examples:
var range = new IDBKeyRange.bound(2, 4);
and
var range = IDBKeyRange.leftBound(key);
I'm not particularly happy with hanging functions off of
IndexedDatabaseRequest for this. Can it work something like what I listed
above? If not, maybe we can find a better place to put them? Or just
create multiple openCursor functions for each case?
I think one concern with the above syntax is that it's adding another
object to the global scope. I recall Ben Turner and I discussing the
possibility of hanging it off of indexedDB, so:
var range = new indexedDB.KeyRange.bound(2, 4);
My concern with making multiple openCursor functions is that there is
more API complexity there. With that said, I don't have any strong
opinions here either way.
10. You are allowed to have multiple transactions per database
connection. However if they use overlapping tables, only the first one
will receive events until it is finished (with the usual exceptions of
allowing multiple readers of the same table).
Can you please clarify what you mean here? This seems like simply an
implementation detail to me, so maybe I'm missing something?
What this is trying to say is that you can have an object store being
used in more than one transaction, but they cannot access it at the same
time. However, I think it's best for Jonas to chime in here because
this doesn't quite seem right to me like it did yesterday.
5) You have two IDBTransactionRequest.onaborts. I think one is supposed to
be an ontimeout.
Whoops, I thought we fixed that before he sent this out :)
6) What is the default limit for the getAll functions? Should we make it 0
(and define any<=0 amount to mean infinity)?
I believe we intended the default to be infinity (as in, if you don't
specify, you get it all).
7) I expect "add or modify" to be more used than the add or the modify
methods. As such, I wonder if it makes sense to optimize the naming for
that. For example, addOrModify=>set, add=>add/insert,
modify=>modify/update/replace maybe?
We had a lot of internal debate over this very thing. The problem we
kept running into was that set could easily be read as doing just
updating so it wouldn't be clear that you can also insert a new record
there (without reading the spec or some tutorial). The benefit with
addOrModify is that it is very explicit in what it does. We don't like
how long it is, but we couldn't come up with a shorter name that doesn't
have a fair amount of ambiguity.
8) We can't leave deciding whether a cursor is pre-loaded up to UAs
since people will code for their favorite UA and then access
IDBCursorPreloadedRequest.count when some other UA does it as a
non-preloaded request. Even in the same UA this will cause problems when
users have different datasets than developers are testing with.
I think that you might have been confused by our wording there. Sorry
about that! IDBCursorPreloadedRequest is what you get if you pass
sync=true into openCursor or openObjectCursor. Basically, sync cursors
will give you a count, whereas async ones will not.
interface IBDCursorRequest : IDBCursor {
readonly attribute any key;
readonly attribute any value;
*readonly attribute unsigned long long estimatedCount;*
* // Returns null if the cursor was updated synchronously. Otherwise*
* // will return an IDBRequest object whose result will be set to this*
* // cursor on success. Until onsuccess is called, any access of key*
* // or value will raise an exception. The first request MUST cause*
* // an asynchronous request but the behavior of subsequent calls is*
* // up to the UA.*
* IDBRequest continue(in optional any key /* null */);*
// Success fires IDBTransactionEvent, result == key
IDBRequest update(in any value);
// Success fires IDBTransactionEvent, result == null
IDBRequest remove();
};
I'm not super happy with this interface, but I think it's a lot better for a
few reasons:
1) It's not all or nothing. Even if a result can't be 100% loaded into
memory, it doesn't mean that we'll have to deal with the overhead of every
fetch causing an asynchronous firing of an event.
2) There's an estimated count even if it's not pre-loaded (which has been
re
Re: [IndexedDB] Proposal for async API changes
It has been pointed out to me that I used the wrong subject "marker".
Fixed here in case people have filters etc.
On Mon, May 17, 2010 at 6:15 PM, Jonas Sicking wrote:
> Hi All,
>
> I, together with Ben Turner and Shawn Wilsher have been looking at the
> asynchronous API defined in the IndexDB specification and have a set
> of changes to propose. The main goal of these changes is to simplify
> the API that we expose to authors, making it easier for them to work
> with. Another goal has been to reduce the risk that authors misuse the
> API and use long running transactions. Finally, it has been a goal to
> reduce the risk of situations that can race.
>
> It has explicitly not been a goal to simplify the implementation. In
> some cases it is definitely harder to implement the proposed API.
> However, we believe that the extra complexity in implementation is
> outweighed by simplicity for users of the API.
>
> The main changes are:
>
> 1. Once a database has been opened (a database connection has been
> established) read access to meta-data, such as objectStore and index
> names, is synchronous. Changes to such meta data, such as creating
> objectStores and indexes, is still asynchronous.
> 2. You can only add "requests" to read and write data to a transaction
> during a transaction callback. There is one exception to this rule
> (more below).
> 3. Transactions are automatically committed. Once a request in a
> transaction finishes and there are no more requests queued against the
> transaction, the transaction is committed.
> 4. Cursors do not fire error events if a request to open a cursor
> yields zero results or when iterating using a cursor reaches the end
> of the found results. Instead, a success event is fired which
> indicates that no more results are available.
> 5. All reads and writes are done through transactions. However in some
> places the transaction is implicit (but defined).
> 6. Access to index objects are done through API on objectStore objects.
> 7. Separate functions for add/modify/add-or-modify.
> 8. Calling abort() on read request always cancels the request, even if
> the implementation has already read the data and is ready to fire a
> success event. The error event is always fired if abort() is called,
> and the success event is suppressed.
> 9. IDBKeyRanges are created using functions on IndexedDatabaseRequest.
> We couldn't figure out how the old API allowed you to create a range
> object without first having a range object.
> 10. You are allowed to have multiple transactions per database
> connection. However if they use overlapping tables, only the first one
> will receive events until it is finished (with the usual exceptions of
> allowing multiple readers of the same table).
>
> A draft of the proposed API is here:
>
> http://docs.google.com/View?id=dfs2skx2_4g3s5f857
>
>
> You get a IDBDatabaseRequest as before, using:
>
> var request = indexedDB.open("School", "My school database");
> request.onsuccess = function(event) {
> var db = event.result;
> ...
> }
>
> Once you have a IDBDatabaseRequest object, things are however
> different. You can read data using:
>
> request = db.objectStore("students").get("Benny");
> request.onsuccess = function(event) {
> displayStudent(event.result);
> }
>
> And write using:
>
> request = db.objectStore("students").add({ name: "Benny", year: 8 });
> request.onerror = function(event) {
> displayError("Writing Benny failed");
> }
>
>
> If you need to operate on multiple stores stores, you can use an
> explicit transaction:
>
> trans = db.transaction(["students", "classes"]);
> trans.get("Benny").onsuccess = function(event) {
> trans.objectStore("classes").get(event.result.year).onsuccess = ...
> }
>
> This also shows the exception for when you are allowed to add requests
> to a transaction outside of a callback. When the transaction()
> function is called, this synchronously returns a transaction object.
> You are allowed to immediately start making requests on this object
> despite not being in a callback. In fact, no callbacks will happen
> until you start making requests. However no reads or writes will be
> performed until the implementation has managed to grab the correct
> (read vs. write) lock on the specified tables, and thus no callbacks
> will happen until that time.
>
>
> Reading using an index is similar to reading from an objectStore directly.
>
> request = db.objectStore("students").index("year").get(...);
> request.onsuccess = ...
> and
> request = db.objectStore("students").index("year").getObject(...);
> request.onsuccess = ...
>
> Since indexes can return multiple entries for a given key, the above
> functions use the first matching entry.
>
> Cursors are, as before, available both on objectStores and indexes.
> However using them is simpler since you don't have to listen for error
> events for normal iteration. In the current spec draft, you need to
> register error event handlers if you didn't know which was the last
