[widgets] rename View Mode Media Feature spec
Hi, Can we please rename the View Mode Media Feature to The 'view-mode' media feature? The current name of the spec is confusing [me]. Kind regards, Marcos -- Marcos Caceres Opera Software
Re: [IndexedDB] Proposal for async API changes
It has been pointed out to me that I used the wrong subject marker.
Fixed here in case people have filters etc.
On Mon, May 17, 2010 at 6:15 PM, Jonas Sicking jo...@sicking.cc wrote:
Hi All,
I, together with Ben Turner and Shawn Wilsher have been looking at the
asynchronous API defined in the IndexDB specification and have a set
of changes to propose. The main goal of these changes is to simplify
the API that we expose to authors, making it easier for them to work
with. Another goal has been to reduce the risk that authors misuse the
API and use long running transactions. Finally, it has been a goal to
reduce the risk of situations that can race.
It has explicitly not been a goal to simplify the implementation. In
some cases it is definitely harder to implement the proposed API.
However, we believe that the extra complexity in implementation is
outweighed by simplicity for users of the API.
The main changes are:
1. Once a database has been opened (a database connection has been
established) read access to meta-data, such as objectStore and index
names, is synchronous. Changes to such meta data, such as creating
objectStores and indexes, is still asynchronous.
2. You can only add requests to read and write data to a transaction
during a transaction callback. There is one exception to this rule
(more below).
3. Transactions are automatically committed. Once a request in a
transaction finishes and there are no more requests queued against the
transaction, the transaction is committed.
4. Cursors do not fire error events if a request to open a cursor
yields zero results or when iterating using a cursor reaches the end
of the found results. Instead, a success event is fired which
indicates that no more results are available.
5. All reads and writes are done through transactions. However in some
places the transaction is implicit (but defined).
6. Access to index objects are done through API on objectStore objects.
7. Separate functions for add/modify/add-or-modify.
8. Calling abort() on read request always cancels the request, even if
the implementation has already read the data and is ready to fire a
success event. The error event is always fired if abort() is called,
and the success event is suppressed.
9. IDBKeyRanges are created using functions on IndexedDatabaseRequest.
We couldn't figure out how the old API allowed you to create a range
object without first having a range object.
10. You are allowed to have multiple transactions per database
connection. However if they use overlapping tables, only the first one
will receive events until it is finished (with the usual exceptions of
allowing multiple readers of the same table).
A draft of the proposed API is here:
http://docs.google.com/View?id=dfs2skx2_4g3s5f857
You get a IDBDatabaseRequest as before, using:
var request = indexedDB.open(School, My school database);
request.onsuccess = function(event) {
var db = event.result;
...
}
Once you have a IDBDatabaseRequest object, things are however
different. You can read data using:
request = db.objectStore(students).get(Benny);
request.onsuccess = function(event) {
displayStudent(event.result);
}
And write using:
request = db.objectStore(students).add({ name: Benny, year: 8 });
request.onerror = function(event) {
displayError(Writing Benny failed);
}
If you need to operate on multiple stores stores, you can use an
explicit transaction:
trans = db.transaction([students, classes]);
trans.get(Benny).onsuccess = function(event) {
trans.objectStore(classes).get(event.result.year).onsuccess = ...
}
This also shows the exception for when you are allowed to add requests
to a transaction outside of a callback. When the transaction()
function is called, this synchronously returns a transaction object.
You are allowed to immediately start making requests on this object
despite not being in a callback. In fact, no callbacks will happen
until you start making requests. However no reads or writes will be
performed until the implementation has managed to grab the correct
(read vs. write) lock on the specified tables, and thus no callbacks
will happen until that time.
Reading using an index is similar to reading from an objectStore directly.
request = db.objectStore(students).index(year).get(...);
request.onsuccess = ...
and
request = db.objectStore(students).index(year).getObject(...);
request.onsuccess = ...
Since indexes can return multiple entries for a given key, the above
functions use the first matching entry.
Cursors are, as before, available both on objectStores and indexes.
However using them is simpler since you don't have to listen for error
events for normal iteration. In the current spec draft, you need to
register error event handlers if you didn't know which was the last
result in a search, or if there was a risk that a search would result
in zero results. With our proposal you'll get a normal
Re: [IndexedDB] What happens when the version changes?
On Tue, May 18, 2010 at 2:42 AM, Jeremy Orlow jor...@chromium.org wrote: What happens to existing connections that were opened against the original database version (once the DB has been updated)? Once a call to setVersion() has happened, any existing connections will forever fail all requests with a WRONG_VERSION_ERR. I guess you could argue that if the upgrade transaction is rolled back then we could reenable existing connections. However this just seems to be an optimization for an edge case and just adds needless complexity, so I don't think we should do it. Are we sure that having versions is really a v1 feature for IndexedDB? Nothing is set in stone, that's why we're still debating :) Though what would you propose that a website does if it needs to update its database format? You could always hope that the user only has one tab open, but that seems scary and risks data corruption. / Jonas On Tue, May 18, 2010 at 9:54 AM, Jonas Sicking jo...@sicking.cc wrote: On Thu, May 13, 2010 at 10:25 AM, Shawn Wilsher sdwi...@mozilla.com wrote: On 5/13/2010 7:51 AM, Nikunj Mehta wrote: If you search archives you will find a discussion on versioning and that we gave up on doing version management inside the browser and instead leave it to applications to do their own versioning and upgrades. Right, I'm not saying we should manage it, but I want to make sure we don't end up making it very easy for apps to break themselves. For example: 1) Site A has two different tabs (T1 and T2) open that were loaded such that one got a script (T1) with a newer indexedDB version than the other (T2). 2) T1 upgrades the database in a way that T2 now gets a constraint violation on every operation (or some other error due to the database changing). This could easily happen any time a site changes the version number on their database. As the spec is written right now, there is no way for a site to know when the version changes without reopening the database since the version property is a sync getter, implementations would have to load it on open and cache it, or come up with some way to update all open databases (not so fun). I think what we should do is to make it so that a database connection is version specific. When you open the database connection (A) the implementation remembers what version the database had when it was opened. If another database connection (B) changes the version of the database, then any requests made to connection A will fail with a WRONG_VERSION_ERR error. The implementation must of course wait for any currently executing transactions in any database connection to finish before changing the version. Further the success-callback should likely receive a transaction that locks the whole database in order to allow the success callback to actually upgrade the database to the new version format. Not until this transaction finishes and is committed should new connections be allowed to be established. These new connections would see the new database version. / Jonas
Re: [IndexedDB] What happens when the version changes?
On Tue, May 18, 2010 at 11:04 AM, Jonas Sicking jo...@sicking.cc wrote: On Tue, May 18, 2010 at 2:42 AM, Jeremy Orlow jor...@chromium.org wrote: What happens to existing connections that were opened against the original database version (once the DB has been updated)? Once a call to setVersion() has happened, any existing connections will forever fail all requests with a WRONG_VERSION_ERR. Ahh! Got it. I guess you could argue that if the upgrade transaction is rolled back then we could reenable existing connections. However this just seems to be an optimization for an edge case and just adds needless complexity, so I don't think we should do it. I don't care all that much, but I don't really see this as adding that much complexity. I'd be weakly in favor of an aborted upgrade not requiring re-connection. Are we sure that having versions is really a v1 feature for IndexedDB? Nothing is set in stone, that's why we're still debating :) Of course. I was just bringing up the question because this was a feature in the original spec that has already had a lot of nice to have stuff pruned from it and this seems somewhat borderline for whether it should be pruned. Though what would you propose that a website does if it needs to update its database format? You could always hope that the user only has one tab open, but that seems scary and risks data corruption. Well, you could leave it up to apps themselves. But then I suppose they'd have to check whether the version has been changed every single time they access the databasewhich I guess is sub-optimal. They post a message to all open windows, use a shared worker, or something else like that to communicate when a database is being updatedbut it'd be complicated to do this without it being racy OK, I'm convinced. :-) / Jonas On Tue, May 18, 2010 at 9:54 AM, Jonas Sicking jo...@sicking.cc wrote: On Thu, May 13, 2010 at 10:25 AM, Shawn Wilsher sdwi...@mozilla.com wrote: On 5/13/2010 7:51 AM, Nikunj Mehta wrote: If you search archives you will find a discussion on versioning and that we gave up on doing version management inside the browser and instead leave it to applications to do their own versioning and upgrades. Right, I'm not saying we should manage it, but I want to make sure we don't end up making it very easy for apps to break themselves. For example: 1) Site A has two different tabs (T1 and T2) open that were loaded such that one got a script (T1) with a newer indexedDB version than the other (T2). 2) T1 upgrades the database in a way that T2 now gets a constraint violation on every operation (or some other error due to the database changing). This could easily happen any time a site changes the version number on their database. As the spec is written right now, there is no way for a site to know when the version changes without reopening the database since the version property is a sync getter, implementations would have to load it on open and cache it, or come up with some way to update all open databases (not so fun). I think what we should do is to make it so that a database connection is version specific. When you open the database connection (A) the implementation remembers what version the database had when it was opened. If another database connection (B) changes the version of the database, then any requests made to connection A will fail with a WRONG_VERSION_ERR error. The implementation must of course wait for any currently executing transactions in any database connection to finish before changing the version. Further the success-callback should likely receive a transaction that locks the whole database in order to allow the success callback to actually upgrade the database to the new version format. Not until this transaction finishes and is committed should new connections be allowed to be established. These new connections would see the new database version. / Jonas
Re: [widgets] rename View Mode Media Feature spec
I'm OK with that. Looking at the naming of other specs we have names such as CSSOM Views maybe we should use something along the lines of Widgets Views: Media Feature? Just a thought though Cheers, Kenneth On Tue, May 18, 2010 at 5:45 AM, Marcos Caceres marc...@opera.com wrote: Hi, Can we please rename the View Mode Media Feature to The 'view-mode' media feature? The current name of the spec is confusing [me]. Kind regards, Marcos -- Marcos Caceres Opera Software -- Kenneth Rohde Christiansen Technical Lead / Senior Software Engineer Qt Labs Americas, Nokia Technology Institute, INdT Phone +55 81 8895 6002 / E-mail kenneth.christiansen at openbossa.org
Re: [IndexedDB] What happens when the version changes?
On Tue, May 18, 2010 at 3:54 AM, Jeremy Orlow jor...@chromium.org wrote: On Tue, May 18, 2010 at 11:04 AM, Jonas Sicking jo...@sicking.cc wrote: On Tue, May 18, 2010 at 2:42 AM, Jeremy Orlow jor...@chromium.org wrote: What happens to existing connections that were opened against the original database version (once the DB has been updated)? Once a call to setVersion() has happened, any existing connections will forever fail all requests with a WRONG_VERSION_ERR. Ahh! Got it. I guess you could argue that if the upgrade transaction is rolled back then we could reenable existing connections. However this just seems to be an optimization for an edge case and just adds needless complexity, so I don't think we should do it. I don't care all that much, but I don't really see this as adding that much complexity. I'd be weakly in favor of an aborted upgrade not requiring re-connection. The complexity I'm concerned about is for the application developers. I bet most of them will not properly add error handlers to all of their requests checking for this error. The result is that I suspect that any tab that gets shut out due to a version upgrade will likely behave strangely. (However this is better than data corruption). But if we then start allowing requests to go through after having denyed a bunch of them, then I'd be worried that the tab will corrupt its own data. That is why I was thinking that once we've shut someone out, it might be better to keep it that way. In general, I tend to operate under the assumption that people are likely to not add error handlers in most cases, but instead assume that their requests will always go through. / Jonas
Re: [IndexedDB] Proposal for async API changes
On 5/18/2010 7:20 AM, Jeremy Orlow wrote:
1. Once a database has been opened (a database connection has been
established) read access to meta-data, such as objectStore and index
names, is synchronous. Changes to such meta data, such as creating
objectStores and indexes, is still asynchronous.
I believe this is already how it's specced. The IDBDatabase interface
already gives you synchronous access to all of this.
Mostly that is the same, with the exception of getting an object store
or index which is now synchronous.
9. IDBKeyRanges are created using functions on IndexedDatabaseRequest.
We couldn't figure out how the old API allowed you to create a range
object without first having a range object.
In the spec, I see the following in examples:
var range = new IDBKeyRange.bound(2, 4);
and
var range = IDBKeyRange.leftBound(key);
I'm not particularly happy with hanging functions off of
IndexedDatabaseRequest for this. Can it work something like what I listed
above? If not, maybe we can find a better place to put them? Or just
create multiple openCursor functions for each case?
I think one concern with the above syntax is that it's adding another
object to the global scope. I recall Ben Turner and I discussing the
possibility of hanging it off of indexedDB, so:
var range = new indexedDB.KeyRange.bound(2, 4);
My concern with making multiple openCursor functions is that there is
more API complexity there. With that said, I don't have any strong
opinions here either way.
10. You are allowed to have multiple transactions per database
connection. However if they use overlapping tables, only the first one
will receive events until it is finished (with the usual exceptions of
allowing multiple readers of the same table).
Can you please clarify what you mean here? This seems like simply an
implementation detail to me, so maybe I'm missing something?
What this is trying to say is that you can have an object store being
used in more than one transaction, but they cannot access it at the same
time. However, I think it's best for Jonas to chime in here because
this doesn't quite seem right to me like it did yesterday.
5) You have two IDBTransactionRequest.onaborts. I think one is supposed to
be an ontimeout.
Whoops, I thought we fixed that before he sent this out :)
6) What is the default limit for the getAll functions? Should we make it 0
(and define any=0 amount to mean infinity)?
I believe we intended the default to be infinity (as in, if you don't
specify, you get it all).
7) I expect add or modify to be more used than the add or the modify
methods. As such, I wonder if it makes sense to optimize the naming for
that. For example, addOrModify=set, add=add/insert,
modify=modify/update/replace maybe?
We had a lot of internal debate over this very thing. The problem we
kept running into was that set could easily be read as doing just
updating so it wouldn't be clear that you can also insert a new record
there (without reading the spec or some tutorial). The benefit with
addOrModify is that it is very explicit in what it does. We don't like
how long it is, but we couldn't come up with a shorter name that doesn't
have a fair amount of ambiguity.
8) We can't leave deciding whether a cursor is pre-loaded up to UAs
since people will code for their favorite UA and then access
IDBCursorPreloadedRequest.count when some other UA does it as a
non-preloaded request. Even in the same UA this will cause problems when
users have different datasets than developers are testing with.
I think that you might have been confused by our wording there. Sorry
about that! IDBCursorPreloadedRequest is what you get if you pass
sync=true into openCursor or openObjectCursor. Basically, sync cursors
will give you a count, whereas async ones will not.
interface IBDCursorRequest : IDBCursor {
readonly attribute any key;
readonly attribute any value;
*readonly attribute unsigned long long estimatedCount;*
* // Returns null if the cursor was updated synchronously. Otherwise*
* // will return an IDBRequest object whose result will be set to this*
* // cursor on success. Until onsuccess is called, any access of key*
* // or value will raise an exception. The first request MUST cause*
* // an asynchronous request but the behavior of subsequent calls is*
* // up to the UA.*
* IDBRequest continue(in optional any key /* null */);*
// Success fires IDBTransactionEvent, result == key
IDBRequest update(in any value);
// Success fires IDBTransactionEvent, result == null
IDBRequest remove();
};
I'm not super happy with this interface, but I think it's a lot better for a
few reasons:
1) It's not all or nothing. Even if a result can't be 100% loaded into
memory, it doesn't mean that we'll have to deal with the overhead of every
fetch causing an asynchronous firing of an event.
2) There's an estimated count even if it's not pre-loaded (which has been
Re: [IndexedDB] What happens when the version changes?
This sounds very similar to the strategy used for WebSQLDatabase. On Tue, May 18, 2010 at 1:54 AM, Jonas Sicking jo...@sicking.cc wrote: On Thu, May 13, 2010 at 10:25 AM, Shawn Wilsher sdwi...@mozilla.com wrote: On 5/13/2010 7:51 AM, Nikunj Mehta wrote: If you search archives you will find a discussion on versioning and that we gave up on doing version management inside the browser and instead leave it to applications to do their own versioning and upgrades. Right, I'm not saying we should manage it, but I want to make sure we don't end up making it very easy for apps to break themselves. For example: 1) Site A has two different tabs (T1 and T2) open that were loaded such that one got a script (T1) with a newer indexedDB version than the other (T2). 2) T1 upgrades the database in a way that T2 now gets a constraint violation on every operation (or some other error due to the database changing). This could easily happen any time a site changes the version number on their database. As the spec is written right now, there is no way for a site to know when the version changes without reopening the database since the version property is a sync getter, implementations would have to load it on open and cache it, or come up with some way to update all open databases (not so fun). I think what we should do is to make it so that a database connection is version specific. When you open the database connection (A) the implementation remembers what version the database had when it was opened. If another database connection (B) changes the version of the database, then any requests made to connection A will fail with a WRONG_VERSION_ERR error. The implementation must of course wait for any currently executing transactions in any database connection to finish before changing the version. Further the success-callback should likely receive a transaction that locks the whole database in order to allow the success callback to actually upgrade the database to the new version format. Not until this transaction finishes and is committed should new connections be allowed to be established. These new connections would see the new database version. / Jonas
Re: [IndexedDB] Proposal for async API changes
On Tue, May 18, 2010 at 6:32 PM, Shawn Wilsher sdwi...@mozilla.com wrote: On 5/18/2010 7:20 AM, Jeremy Orlow wrote: 1. Once a database has been opened (a database connection has been established) read access to meta-data, such as objectStore and index names, is synchronous. Changes to such meta data, such as creating objectStores and indexes, is still asynchronous. I believe this is already how it's specced. The IDBDatabase interface already gives you synchronous access to all of this. Mostly that is the same, with the exception of getting an object store or index which is now synchronous. 9. IDBKeyRanges are created using functions on IndexedDatabaseRequest. We couldn't figure out how the old API allowed you to create a range object without first having a range object. In the spec, I see the following in examples: var range = new IDBKeyRange.bound(2, 4); and var range = IDBKeyRange.leftBound(key); I'm not particularly happy with hanging functions off of IndexedDatabaseRequest for this. Can it work something like what I listed above? If not, maybe we can find a better place to put them? Or just create multiple openCursor functions for each case? I think one concern with the above syntax is that it's adding another object to the global scope. I recall Ben Turner and I discussing the possibility of hanging it off of indexedDB, so: var range = new indexedDB.KeyRange.bound(2, 4); Good point re the global scope. My concern with making multiple openCursor functions is that there is more API complexity there. With that said, I don't have any strong opinions here either way. How is there more API complexity to have several openCursor functions rather than several factories for KeyRange objects? 10. You are allowed to have multiple transactions per database connection. However if they use overlapping tables, only the first one will receive events until it is finished (with the usual exceptions of allowing multiple readers of the same table). Can you please clarify what you mean here? This seems like simply an implementation detail to me, so maybe I'm missing something? What this is trying to say is that you can have an object store being used in more than one transaction, but they cannot access it at the same time. However, I think it's best for Jonas to chime in here because this doesn't quite seem right to me like it did yesterday. Oh, I see. The problem is that if you open an entity store and start multiple transactions, it's not clear which it's associated with. I guess I feel like what Jonas described would be pretty confusing. What about creating a IDBSuccessEvents.transaction that's the transaction the request is associated with (or null)? Another option is to only allow one (top level) transaction per connection. (I still think we should support open nested transactions.) 5) You have two IDBTransactionRequest.onaborts. I think one is supposed to be an ontimeout. Whoops, I thought we fixed that before he sent this out :) 6) What is the default limit for the getAll functions? Should we make it 0 (and define any=0 amount to mean infinity)? I believe we intended the default to be infinity (as in, if you don't specify, you get it all). Ahh, so not specifying would be the only way to get infinity? Yeah, I guess that works. What if someone passes 0? Just return a request with null? 7) I expect add or modify to be more used than the add or the modify methods. As such, I wonder if it makes sense to optimize the naming for that. For example, addOrModify=set, add=add/insert, modify=modify/update/replace maybe? We had a lot of internal debate over this very thing. The problem we kept running into was that set could easily be read as doing just updating so it wouldn't be clear that you can also insert a new record there (without reading the spec or some tutorial). The benefit with addOrModify is that it is very explicit in what it does. We don't like how long it is, but we couldn't come up with a shorter name that doesn't have a fair amount of ambiguity. I see. Well, I'm OK starting with addOrModify and seeing if developers scream. 8) We can't leave deciding whether a cursor is pre-loaded up to UAs since people will code for their favorite UA and then access IDBCursorPreloadedRequest.count when some other UA does it as a non-preloaded request. Even in the same UA this will cause problems when users have different datasets than developers are testing with. I think that you might have been confused by our wording there. Sorry about that! IDBCursorPreloadedRequest is what you get if you pass sync=true into openCursor or openObjectCursor. Basically, sync cursors will give you a count, whereas async ones will not. Oh. I missed that parameter and I guess let my imagination run wild. :-) I'm not sure I like the idea of offering sync cursors either since the UA will either need to
Re: [IndexDB] Proposal for async API changes
On Tue, May 18, 2010 at 7:20 AM, Jeremy Orlow jor...@chromium.org wrote:
Overall, I'm pretty happy with these changes. I support making these
changes to the spec. Additional comments inline...
On Tue, May 18, 2010 at 2:15 AM, Jonas Sicking jo...@sicking.cc wrote:
Hi All,
I, together with Ben Turner and Shawn Wilsher have been looking at the
asynchronous API defined in the IndexDB specification and have a set
of changes to propose. The main goal of these changes is to simplify
the API that we expose to authors, making it easier for them to work
with. Another goal has been to reduce the risk that authors misuse the
API and use long running transactions. Finally, it has been a goal to
reduce the risk of situations that can race.
It has explicitly not been a goal to simplify the implementation. In
some cases it is definitely harder to implement the proposed API.
However, we believe that the extra complexity in implementation is
outweighed by simplicity for users of the API.
The main changes are:
1. Once a database has been opened (a database connection has been
established) read access to meta-data, such as objectStore and index
names, is synchronous. Changes to such meta data, such as creating
objectStores and indexes, is still asynchronous.
I believe this is already how it's specced. The IDBDatabase interface
already gives you synchronous access to all of this.
The big difference is that the current spec makes openObjectStore()
and openIndex() asynchronous. Our proposal makes openObjectStore() and
openIndex() (renamed objectStore() and index()) synchronous. So
opening an objectstore, or even starting a transaction, only
synchronously accesses metadata. But any requests you make on the
transaction will be held until the transaction has managed to grab the
requested tables.
So when you, in our proposal, call:
db.objectStore(foo, READ_WRITE).put(...);
the objectStore function synchronously creates a transaction object
representing a transaction which only contains the foo objectStore.
The implementation then fires off a asynchronous request to lock the
foo objectStore with a write lock. It then returns the synchronously
created transaction object.
When the put() function is called, the implementation notices that the
lock is not yet acquired. So it simply records what information should
be written to the object store.
Later, when the write lock is successfully acquired, the
implementation executes the recorded operations and once they finish
we call their callbacks.
9. IDBKeyRanges are created using functions on IndexedDatabaseRequest.
We couldn't figure out how the old API allowed you to create a range
object without first having a range object.
In the spec, I see the following in examples:
var range = new IDBKeyRange.bound(2, 4);
and
var range = IDBKeyRange.leftBound(key);
I'm not particularly happy with hanging functions off of
IndexedDatabaseRequest for this. Can it work something like what I listed
above? If not, maybe we can find a better place to put them? Or just
create multiple openCursor functions for each case?
Mostly we were just confused as to what syntax was actually proposed.
You are listing two syntaxes (with and without 'new'), neither of
which match the WebIDL in the spec. I personally think that most
proposed syntaxes are ok and don't care much which one we choose, as
long as it's clearly defined.
10. You are allowed to have multiple transactions per database
connection. However if they use overlapping tables, only the first one
will receive events until it is finished (with the usual exceptions of
allowing multiple readers of the same table).
Can you please clarify what you mean here? This seems like simply an
implementation detail to me, so maybe I'm missing something?
The spec currently does explicitly forbid having multiple transactions
per database connection. The syntax doesn't even support it since
there is a .currentTransaction property on IDBDatabase. I.e. the
following code seems forbidden (though I'm not sure if forbidden means
that an exception will be thrown somewhere, or if it means that the
code will just silently fail to work).
request = indexedDB.openDatabase(myDB, ...);
request.onsuccess = function() {
db = request.result;
r1 = db.openTransaction([foo]);
r1.onsuccess = function() { ... };
r2 = db.openTransaction([bar]);
r2.onsuccess = function() { ... };
};
The spec says that the above is forbidden. In our new proposal the
following would be allowed:
request = indexedDB.openDatabase(myDB, ...);
request.onsuccess = function() {
db = request.result;
t1 = db.transaction([foo], READ_WRITE);
t2 = db.transaction([bar], READ_WRITE);
};
And would allow the two transactions to run concurrently.
A draft of the proposed API is here:
http://docs.google.com/View?id=dfs2skx2_4g3s5f857
Comments:
1) IDBRequest.abort() in IDBRequest needs to be able to raise.
I think in general we haven't indicated
Re: [IndexedDB] Proposal for async API changes
On Tue, May 18, 2010 at 12:10 PM, Jeremy Orlow jor...@chromium.org wrote: 10. You are allowed to have multiple transactions per database connection. However if they use overlapping tables, only the first one will receive events until it is finished (with the usual exceptions of allowing multiple readers of the same table). Can you please clarify what you mean here? This seems like simply an implementation detail to me, so maybe I'm missing something? What this is trying to say is that you can have an object store being used in more than one transaction, but they cannot access it at the same time. However, I think it's best for Jonas to chime in here because this doesn't quite seem right to me like it did yesterday. Oh, I see. The problem is that if you open an entity store and start multiple transactions, it's not clear which it's associated with. I guess I feel like what Jonas described would be pretty confusing. The objectStore accessor lives off of the IDBTransactionRequest API, so they are always associated with the transaction they were retrieved using. I agree it's a little confusing that you can have several references to the same objectStore through multiple transactions. However if you look at the examples the code works out quite nicely and is quite understandable IMHO. What about creating a IDBSuccessEvents.transaction that's the transaction the request is associated with (or null)? That's already there. Though it's on IDBTransactionEvent, which is the event type fired for successful operations that involve transactions (almost all of them except for the success event for opening the database). Another option is to only allow one (top level) transaction per connection. (I still think we should support open nested transactions.) This is what the spec says right now. I don't really see what the advantage is though since you can create several connections to the same database which means that you basically have the exact same situation. Only difference is that you now have several database connections floating around as well. 8) We can't leave deciding whether a cursor is pre-loaded up to UAs since people will code for their favorite UA and then access IDBCursorPreloadedRequest.count when some other UA does it as a non-preloaded request. Even in the same UA this will cause problems when users have different datasets than developers are testing with. I think that you might have been confused by our wording there. Sorry about that! IDBCursorPreloadedRequest is what you get if you pass sync=true into openCursor or openObjectCursor. Basically, sync cursors will give you a count, whereas async ones will not. Oh. I missed that parameter and I guess let my imagination run wild. :-) I'm not sure I like the idea of offering sync cursors either since the UA will either need to load everything into memory before starting or risk blocking on disk IO for large data sets. Thus I'm not sure I support the idea of synchronous cursors. But, at the same time, I'm concerned about the overhead of firing one event per value with async cursors. Which is why I was suggesting an interface where the common case (the data is in memory) is done synchronously but the uncommon case (we'd block if we had to respond synchronously) has to be handled since we guarantee that the first time will be forced to be asynchronous. Like I said, I'm not super happy with what I proposed, but I think some hybrid async/sync interface is really what we need. Have you guys spent any time thinking about something like this? How dead-set are you on synchronous cursors? The idea is that synchronous cursors load all the required data into memory, yes. I think it would help authors a lot to be able to load small chunks of data into memory and read and write to it synchronously. Dealing with asynchronous operations constantly is certainly possible, but a bit of a pain for authors. I don't think we should obsess too much about not keeping things in memory, we already have things like canvas and the DOM which adds up to non-trivial amounts of memory. Just because data is loaded from a database doesn't mean it's huge. I do note that you're not as concerned about getAll(), which actually have worse memory characteristics than synchronous cursors since you need to create the full JS object graph in memory. / Jonas
Re: [IndexedDB] What happens when the version changes?
If the use case here is to avoid tripping up on schema changes, then: 1. Lock the database when starting a database connection. This is the non-sharing access mode defined in 3.2.9 as the first option under step 2. 2. Produce events when an application changes the version so that other tabs of the application are alerted 3. Through a library develop sophisticated means of performing incompatible changes without breaking applications using an earlier version of the indexedDB. More below. On May 18, 2010, at 1:54 AM, Jonas Sicking wrote: On Thu, May 13, 2010 at 10:25 AM, Shawn Wilsher sdwi...@mozilla.com wrote: On 5/13/2010 7:51 AM, Nikunj Mehta wrote: If you search archives you will find a discussion on versioning and that we gave up on doing version management inside the browser and instead leave it to applications to do their own versioning and upgrades. Right, I'm not saying we should manage it, but I want to make sure we don't end up making it very easy for apps to break themselves. For example: 1) Site A has two different tabs (T1 and T2) open that were loaded such that one got a script (T1) with a newer indexedDB version than the other (T2). 2) T1 upgrades the database in a way that T2 now gets a constraint violation on every operation (or some other error due to the database changing). This could easily happen any time a site changes the version number on their database. As the spec is written right now, there is no way for a site to know when the version changes without reopening the database since the version property is a sync getter, implementations would have to load it on open and cache it, or come up with some way to update all open databases (not so fun). I think what we should do is to make it so that a database connection is version specific. This is draconian and does not permit compatible schema upgrades, which a perfectly normal application is willing to make. When you open the database connection (A) the implementation remembers what version the database had when it was opened. If another database connection (B) changes the version of the database, then any requests made to connection A will fail with a WRONG_VERSION_ERR error. I would not support this change. The implementation must of course wait for any currently executing transactions in any database connection to finish before changing the version. Further the success-callback should likely receive a transaction that locks the whole database in order to allow the success callback to actually upgrade the database to the new version format. Not until this transaction finishes and is committed should new connections be allowed to be established. These new connections would see the new database version. In case a database is being upgraded, the application is advised to begin a transaction with the entire database locked thereby preventing problems with partial schema upgrades being visible to applications thus breaking them. This behavior is already specced. Is there anything missing in the text about it?
Re: [IndexedDB] What happens when the version changes?
On Tue, May 18, 2010 at 12:37 PM, Nikunj Mehta nik...@o-micron.com wrote: If the use case here is to avoid tripping up on schema changes, then: 1. Lock the database when starting a database connection. This is the non-sharing access mode defined in 3.2.9 as the first option under step 2. Will locking the database prevent others from creating new objectStores? Step 2 only talks about acquiring locks on the existing database objects. 2. Produce events when an application changes the version so that other tabs of the application are alerted How? 3. Through a library develop sophisticated means of performing incompatible changes without breaking applications using an earlier version of the indexedDB. I don't understand, performing incompatible changes seems contrary to without breaking applications using an earlier version. I don't see how you could to both at the same time in the general case. On May 18, 2010, at 1:54 AM, Jonas Sicking wrote: On Thu, May 13, 2010 at 10:25 AM, Shawn Wilsher sdwi...@mozilla.com wrote: On 5/13/2010 7:51 AM, Nikunj Mehta wrote: If you search archives you will find a discussion on versioning and that we gave up on doing version management inside the browser and instead leave it to applications to do their own versioning and upgrades. Right, I'm not saying we should manage it, but I want to make sure we don't end up making it very easy for apps to break themselves. For example: 1) Site A has two different tabs (T1 and T2) open that were loaded such that one got a script (T1) with a newer indexedDB version than the other (T2). 2) T1 upgrades the database in a way that T2 now gets a constraint violation on every operation (or some other error due to the database changing). This could easily happen any time a site changes the version number on their database. As the spec is written right now, there is no way for a site to know when the version changes without reopening the database since the version property is a sync getter, implementations would have to load it on open and cache it, or come up with some way to update all open databases (not so fun). I think what we should do is to make it so that a database connection is version specific. This is draconian and does not permit compatible schema upgrades, which a perfectly normal application is willing to make. If the schema upgrade is compatible with the existing one, then no need to update the version. I thought the whole point of the version identifier was to assist in making incompatible changes. / Jonas
Re: [IndexedDB] What happens when the version changes?
On Tue, May 18, 2010 at 12:46 PM, Jonas Sicking jo...@sicking.cc wrote: On Tue, May 18, 2010 at 12:37 PM, Nikunj Mehta nik...@o-micron.com wrote: If the use case here is to avoid tripping up on schema changes, then: 1. Lock the database when starting a database connection. This is the non-sharing access mode defined in 3.2.9 as the first option under step 2. Will locking the database prevent others from creating new objectStores? Step 2 only talks about acquiring locks on the existing database objects. Also, what happens to already existing database connections? Do you have to wait until the user has closed all other tabs which use the same database before making the upgrade? / Jonas
Re: [IndexedDB] What happens when the version changes?
On May 18, 2010, at 12:50 PM, Jonas Sicking wrote: On Tue, May 18, 2010 at 12:46 PM, Jonas Sicking jo...@sicking.cc wrote: On Tue, May 18, 2010 at 12:37 PM, Nikunj Mehta nik...@o-micron.com wrote: If the use case here is to avoid tripping up on schema changes, then: 1. Lock the database when starting a database connection. This is the non-sharing access mode defined in 3.2.9 as the first option under step 2. Will locking the database prevent others from creating new objectStores? Step 2 only talks about acquiring locks on the existing database objects. Also, what happens to already existing database connections? Do you have to wait until the user has closed all other tabs which use the same database before making the upgrade? I won't talk about tabs and such. Let's make clarification questions be related to spec text. A database connection that locks the entire database cannot be opened if there is another database connection that locks at least one database object, e.g., an index or object store.
Re: [IndexedDB] What happens when the version changes?
On May 18, 2010, at 12:46 PM, Jonas Sicking wrote: On Tue, May 18, 2010 at 12:37 PM, Nikunj Mehta nik...@o-micron.com wrote: If the use case here is to avoid tripping up on schema changes, then: 1. Lock the database when starting a database connection. This is the non-sharing access mode defined in 3.2.9 as the first option under step 2. Will locking the database prevent others from creating new objectStores? Step 2 only talks about acquiring locks on the existing database objects. Here's some text from the existing spec: If these steps are not called with a list of database objects Acquire a lock on the entire database. Is there confusion about the meaning of acquiring a lock on the entire database? 2. Produce events when an application changes the version so that other tabs of the application are alerted How? Spec text could be written if this serves a purpose. 3. Through a library develop sophisticated means of performing incompatible changes without breaking applications using an earlier version of the indexedDB. I don't understand, performing incompatible changes seems contrary to without breaking applications using an earlier version. I don't see how you could to both at the same time in the general case. Since there is no restriction on what libraries can do, one could do seemingly contrary things. Take an example. Say we want to change the content of a compound index, e.g., the order of attributes in the index. This requires changing the contents of the index. It also requires translation of the request to match the sequence of properties being stored in the index. A library can keep extra data to perform translation where it is required. This sort of stuff is done in many applications, so it is not unimaginable that someone would want to do it with IndexedDB. On May 18, 2010, at 1:54 AM, Jonas Sicking wrote: On Thu, May 13, 2010 at 10:25 AM, Shawn Wilsher sdwi...@mozilla.com wrote: On 5/13/2010 7:51 AM, Nikunj Mehta wrote: If you search archives you will find a discussion on versioning and that we gave up on doing version management inside the browser and instead leave it to applications to do their own versioning and upgrades. Right, I'm not saying we should manage it, but I want to make sure we don't end up making it very easy for apps to break themselves. For example: 1) Site A has two different tabs (T1 and T2) open that were loaded such that one got a script (T1) with a newer indexedDB version than the other (T2). 2) T1 upgrades the database in a way that T2 now gets a constraint violation on every operation (or some other error due to the database changing). This could easily happen any time a site changes the version number on their database. As the spec is written right now, there is no way for a site to know when the version changes without reopening the database since the version property is a sync getter, implementations would have to load it on open and cache it, or come up with some way to update all open databases (not so fun). I think what we should do is to make it so that a database connection is version specific. This is draconian and does not permit compatible schema upgrades, which a perfectly normal application is willing to make. If the schema upgrade is compatible with the existing one, then no need to update the version. I thought the whole point of the version identifier was to assist in making incompatible changes. The version is book-keeping support in IndexedDB for an application. I don't see why an application should be forced to keep the version ID the same after schema upgrades. As a use case, an application may queue up schema upgrades based on version numbers.
Re: [IndexedDB] What happens when the version changes?
I have pointed out three options when dealing with upgrades and concurrency [1] in a thread started by Pablo and Shawn 6 months ago [2]: # Allow special DDL like operations at connection time in a special transaction with spec-based versioning of schema # Combine DDL and DML in ordinary transactions, and app-managed versioning of schema # Allow DDL like operations in a special transaction at any time We went with the middle option after some amount of analysis and discussion. On May 13, 2010, at 10:25 AM, Shawn Wilsher wrote: On 5/13/2010 7:51 AM, Nikunj Mehta wrote: If you search archives you will find a discussion on versioning and that we gave up on doing version management inside the browser and instead leave it to applications to do their own versioning and upgrades. Right, I'm not saying we should manage it, but I want to make sure we don't end up making it very easy for apps to break themselves. For example: 1) Site A has two different tabs (T1 and T2) open that were loaded such that one got a script (T1) with a newer indexedDB version than the other (T2). 2) T1 upgrades the database in a way that T2 now gets a constraint violation on every operation (or some other error due to the database changing). In other words, the schema changed in an incompatible way. This could easily happen any time a site changes the version number on their database. As the spec is written right now, there is no way for a site to know when the version changes without reopening the database since the version property is a sync getter, implementations would have to load it on open and cache it, or come up with some way to update all open databases (not so fun). The spec merely asks that implementations provide the version number that is in play when the application requests it. The spec does not mandate that the version stay the same after the database is opened. Perhaps we could produce an event if the version number changes so that an application has an opportunity to deal with that. What would the problem be with this approach? [1] http://lists.w3.org/Archives/Public/public-webapps/2009OctDec/1180.html [2] http://lists.w3.org/Archives/Public/public-webapps/2009OctDec/0927.html
[Bug 9766] New: We should expose the subprotocol for the case of the client not specifying one but the server specifying one
http://www.w3.org/Bugs/Public/show_bug.cgi?id=9766 Summary: We should expose the subprotocol for the case of the client not specifying one but the server specifying one Product: WebAppsWG Version: unspecified Platform: Other URL: http://www.whatwg.org/specs/web-apps/current-work/#web socket OS/Version: other Status: NEW Severity: normal Priority: P3 Component: WebSocket API (editor: Ian Hickson) AssignedTo: i...@hixie.ch ReportedBy: contribu...@whatwg.org QAContact: member-webapi-...@w3.org CC: i...@hixie.ch, m...@w3.org, public-webapps@w3.org Section: http://www.whatwg.org/specs/web-apps/current-work/complete.html#websocket Comment: We should expose the subprotocol for the case of the client not specifying one but the server specifying one Posted from: 98.248.33.53 by i...@hixie.ch -- Configure bugmail: http://www.w3.org/Bugs/Public/userprefs.cgi?tab=email --- You are receiving this mail because: --- You are on the CC list for the bug.
Re: [IndexedDB] What happens when the version changes?
On May 18, 2010, at 1:36 PM, Shawn Wilsher wrote: On 5/18/2010 1:02 PM, Nikunj Mehta wrote: I won't talk about tabs and such. Let's make clarification questions be related to spec text. Simply replace any instance of tabs with database connections. A database connection that locks the entire database cannot be opened if there is another database connection that locks at least one database object, e.g., an index or object store. So basically, as long as some connection holds a database lock, you won't be able to do any upgrade. Correct.
Re: [IndexedDB] What happens when the version changes?
On Tue, May 18, 2010 at 9:36 PM, Shawn Wilsher sdwi...@mozilla.com wrote: On 5/18/2010 1:02 PM, Nikunj Mehta wrote: A database connection that locks the entire database cannot be opened if there is another database connection that locks at least one database object, e.g., an index or object store. So basically, as long as some connection holds a database lock, you won't be able to do any upgrade. Sure, but this is true of Jonas' proposal as well. On Tue, May 18, 2010 at 8:30 PM, Nikunj Mehta nik...@o-micron.com wrote: Perhaps we could produce an event if the version number changes so that an application has an opportunity to deal with that. What would the problem be with this approach? The spec would need to mandate that the event would be received by every open connection. For the async interface, this would be easy. But in the sync interface, we'd need to either run the event synchronously in a nested fashion (which isn't unheard-of, but I believe we avoid whenever possible for good reason) or throw an exception (which most developers probably won't handle correctly). Neither of which seem like great options. Nikunj: in your vision of the world, what is the point of the version parameter? It seems like it could be easily emulated by a value in an objectStore. And, if you're assuming that developers will through a library develop sophisticated means of performing incompatible changes without breaking applications using an earlier version of the indexedDB then it seems pretty easy for such a library to also keep track of versioning data in a special objectStore. Of course, that position is also basically saying that any user of this API will either need to roll complex coordination code themselves, use a complex (under the hood) library, or else their code will probably be racy. Although we expect many users to use libraries to simplify things like joins, I think essentially requiring _any_ user to use a library is a bit overboard. So basically I think I'm with Jonas on this one: setVersion should be a convenient (but maybe overly-simplistic for some) way to manage schema changes. If someone thinks it's too draconian, then they can simply implement things themselves and leave the version alone (as Nikunj explained in the thread). But if we're not going to do what Jonas proposed, then it seems like worthless API surface area that should be completely removed. J
Re: Updates to File API
On Mon, May 17, 2010 at 3:37 PM, Dmitry Titov dim...@chromium.org wrote: I have couple of questions, mostly clarifications I think: 1. FileReader takes Blob but there are multiple hints that the blob should be actually a 'file'. As we see Blob concept grows in popularity with such specs as FileWriter that defines BlobBuilder. Other proposals include Image Resizing that returns a Blob with a compressed image data. Can all types of Blobs be 'read' using FileReader? If not, then it would be logical to only accept File parameter. If any type of Blob can be read (as I think the spirit of the spec assumes) then would it be less confusing to cange the name to BlobReader? I'd support that. I think we always want to allow reading of any type of Blob--it's the interchange container--so calling it BlobReader makes sense. Arun, how do you feel about that? Would FileWriter ever be used to write anything other than a File? I think not, so it should probably stay as it is, despite the lack of symmetry. 2. The FileReader.result is a string. There could be useful cases where it could be useful to read the data as ArrayBuffer. For example, if a page tries to crack the JPG file to extract the EXIF metadata. Maybe returning a Blob that can later be asked for ArrayBuffer would be as good. You're going to give a Blob to a FileReader, and get the same Blob back? Dmitry On Fri, May 14, 2010 at 11:52 AM, Arun Ranganathan a...@mozilla.com wrote: On 5/13/10 9:32 PM, Darin Fisher wrote: Glad to hear that you didn't intend sync access :-) I have thoughts on Blob and how it should behave (and about the inheritance relationship between Blob and File), which is why I left the unfortunate error in the editor's draft for now (commented out and caveated). This is the subject of a separate email thread (but don't worry -- while my thoughts on Blob and ArrayBuffer may be in some flux, sync access to File objects is *always* going to be a no-no, I promise :-) ). Now aside from the Blob - ArrayBuffer relationship, which I introduced, the rest of the changes are in keeping with threads discussing the File API. Can you define the contentType parameter to slice better? Is that intended to correspond to the value of a HTTP Content-Type response header? For example, can the contentType value include a charset attribute? It might be useful to indicate that a slice of a file should be treated as text/html with a specific encoding. I'm happy to define it better in terms of what it *should* be, but web developers are likely to use it in ways that we can't predict, which is why forcing Content-Types is useful, but weird. Why exactly do you mean when you say that a slice of a file should be treated as text/html with a specific encoding? Can you give me a use case that illustrates why this is a good way to define this? I'm also a fan of providing a way to specify optional Content-Disposition parameters in the slice call. So I'm really not a Content-Disposition fan, since all the use cases I've seen so far seem to be to force download behavior (or trigger Download Manager). Is there something I'm missing -- e.g. is there something here that FileWriter or BlobBuilder do *not* address, that putting Content-Disposition on Blob URLs *does* address? Sorry if I'm missing something obvious. -- A*
Re: Updates to File API
On Fri, May 14, 2010 at 11:52 AM, Arun Ranganathan a...@mozilla.com wrote: On 5/13/10 9:32 PM, Darin Fisher wrote: Glad to hear that you didn't intend sync access :-) I have thoughts on Blob and how it should behave (and about the inheritance relationship between Blob and File), which is why I left the unfortunate error in the editor's draft for now (commented out and caveated). This is the subject of a separate email thread (but don't worry -- while my thoughts on Blob and ArrayBuffer may be in some flux, sync access to File objects is *always* going to be a no-no, I promise :-) ). Now aside from the Blob - ArrayBuffer relationship, which I introduced, the rest of the changes are in keeping with threads discussing the File API. Can you define the contentType parameter to slice better? Is that intended to correspond to the value of a HTTP Content-Type response header? For example, can the contentType value include a charset attribute? It might be useful to indicate that a slice of a file should be treated as text/html with a specific encoding. I'm happy to define it better in terms of what it *should* be, but web developers are likely to use it in ways that we can't predict, which is why forcing Content-Types is useful, but weird. Why exactly do you mean when you say that a slice of a file should be treated as text/html with a specific encoding? Can you give me a use case that illustrates why this is a good way to define this? I can't speak for Darin, but I'd think the same reasoning that applies whenever a server adds those headers via HTTP should apply whenever a client-side app wants to add them to a Blob.url. I'm also a fan of providing a way to specify optional Content-Disposition parameters in the slice call. So I'm really not a Content-Disposition fan, since all the use cases I've seen so far seem to be to force download behavior (or trigger Download Manager). Is there something I'm missing -- e.g. is there something here that FileWriter or BlobBuilder do *not* address, that putting Content-Disposition on Blob URLs *does* address? Sorry if I'm missing something obvious. It is indeed generally intended to trigger Download Manager. If you take a look at my use case at [1], the idea is to give web developers a facility that's just like the one they're already using, so that anything they do with URLs for files online they can also do with URLs for Blobs offline/client-side. The FileWriter spec's a bit up in the air over the same issue; I haven't yet specced a good way for FileWriter to solve this problem, so it's hard to say it's going to handle it better. Eric [1] http://lists.w3.org/Archives/Public/public-webapps/2010AprJun/0412.html
Re: [IndexedDB] What happens when the version changes?
On Tue, May 18, 2010 at 1:00 PM, Nikunj Mehta nik...@o-micron.com wrote: On May 18, 2010, at 12:46 PM, Jonas Sicking wrote: On Tue, May 18, 2010 at 12:37 PM, Nikunj Mehta nik...@o-micron.com wrote: If the use case here is to avoid tripping up on schema changes, then: 1. Lock the database when starting a database connection. This is the non-sharing access mode defined in 3.2.9 as the first option under step 2. Will locking the database prevent others from creating new objectStores? Step 2 only talks about acquiring locks on the existing database objects. Here's some text from the existing spec: If these steps are not called with a list of database objects Acquire a lock on the entire database. Is there confusion about the meaning of acquiring a lock on the entire database? Yes. Especially given the language in step 2. However note that the proposal we made, the locking level is moved from the database-open call to the transaction-open call. This in order to allow the page to just open the database at the start of the page (and potentially upgrade it to its required version). The database can then be left open for as long as the user is on the page. This means that for any given interaction with the database only one level of asynchronous call is needed. 2. Produce events when an application changes the version so that other tabs of the application are alerted How? Spec text could be written if this serves a purpose. What functionality are you proposing? 3. Through a library develop sophisticated means of performing incompatible changes without breaking applications using an earlier version of the indexedDB. I don't understand, performing incompatible changes seems contrary to without breaking applications using an earlier version. I don't see how you could to both at the same time in the general case. Since there is no restriction on what libraries can do, one could do seemingly contrary things. Take an example. Say we want to change the content of a compound index, e.g., the order of attributes in the index. This requires changing the contents of the index. It also requires translation of the request to match the sequence of properties being stored in the index. A library can keep extra data to perform translation where it is required. This sort of stuff is done in many applications, so it is not unimaginable that someone would want to do it with IndexedDB. I still don't understand why this is considered performing an incompatible change. It seems to me that if the user has a version of the application loaded which is able to keep extra data to perform translation where it is required, then it isn't an incompatible change. So in this case I would say that the application should just leave the version number unchanged. Granted, it's a little hard to follow the example given that we don't have compound indexes in the spec, so maybe I'm missing something? On May 18, 2010, at 1:54 AM, Jonas Sicking wrote: On Thu, May 13, 2010 at 10:25 AM, Shawn Wilsher sdwi...@mozilla.com wrote: On 5/13/2010 7:51 AM, Nikunj Mehta wrote: If you search archives you will find a discussion on versioning and that we gave up on doing version management inside the browser and instead leave it to applications to do their own versioning and upgrades. Right, I'm not saying we should manage it, but I want to make sure we don't end up making it very easy for apps to break themselves. For example: 1) Site A has two different tabs (T1 and T2) open that were loaded such that one got a script (T1) with a newer indexedDB version than the other (T2). 2) T1 upgrades the database in a way that T2 now gets a constraint violation on every operation (or some other error due to the database changing). This could easily happen any time a site changes the version number on their database. As the spec is written right now, there is no way for a site to know when the version changes without reopening the database since the version property is a sync getter, implementations would have to load it on open and cache it, or come up with some way to update all open databases (not so fun). I think what we should do is to make it so that a database connection is version specific. This is draconian and does not permit compatible schema upgrades, which a perfectly normal application is willing to make. If the schema upgrade is compatible with the existing one, then no need to update the version. I thought the whole point of the version identifier was to assist in making incompatible changes. The version is book-keeping support in IndexedDB for an application. I don't see why an application should be forced to keep the version ID the same after schema upgrades. As a use case, an application may queue up schema upgrades based on version numbers. I think asking [Why should the application] be forced to keep the version ID the
Re: Updates to File API
On 5/18/10 2:35 PM, Eric Uhrhane wrote: On Mon, May 17, 2010 at 3:37 PM, Dmitry Titovdim...@chromium.org wrote: I have couple of questions, mostly clarifications I think: 1. FileReader takes Blob but there are multiple hints that the blob should be actually a 'file'. As we see Blob concept grows in popularity with such specs as FileWriter that defines BlobBuilder. Other proposals include Image Resizing that returns a Blob with a compressed image data. Can all types of Blobs be 'read' using FileReader? If not, then it would be logical to only accept File parameter. If any type of Blob can be read (as I think the spirit of the spec assumes) then would it be less confusing to cange the name to BlobReader? I'd support that. I think we always want to allow reading of any type of Blob--it's the interchange container--so calling it BlobReader makes sense. Arun, how do you feel about that? The FileReader object accepts File objects for DataURL-reads, and Blob objects for binary string, text, and binary reads. I agree that having a name like FileReader is generally a bit confusing, given that we do allow Blobs to be read, including Blobs which aren't directly coined from files. Blob itself isn't a great name, though it's a stand-in for Binary Large Object. Aside from the slight bikeshed-ish nature of this discussion, there are implementations in circulation that already use the name FileReader (e.g. Firefox 3.6.3). This doesn't mean I'm against changing it, but I do wish the name change suggestion came earlier. Also, I'm keen that the main object name addresses the initial use case -- reading file objects. Perhaps in the future Blobs that are not files will be the norm; maybe then, Blob APIs will evolve, including implementations with ArrayBuffer and potential streaming use cases getting addressed better. Perhaps it is late to have a name change, and we've added to less-than-adequate naming on the Web (example: XMLHttpRequest). Would FileWriter ever be used to write anything other than a File? I think not, so it should probably stay as it is, despite the lack of symmetry. 2. The FileReader.result is a string. Actually, in my next draft, I will have FileReader.result be of type 'any' (WebILD's 'any') since it could also be an ArrayBuffer (using the readAsBinary method, which will function like the other asynchrous read methods, but read into ArrayBuffers across the ProgressEvent spectrum. -- A*
Re: [IndexedDB] What happens when the version changes?
On May 18, 2010, at 2:33 PM, Jeremy Orlow wrote: On Tue, May 18, 2010 at 9:36 PM, Shawn Wilsher sdwi...@mozilla.com wrote: On 5/18/2010 1:02 PM, Nikunj Mehta wrote: A database connection that locks the entire database cannot be opened if there is another database connection that locks at least one database object, e.g., an index or object store. So basically, as long as some connection holds a database lock, you won't be able to do any upgrade. Sure, but this is true of Jonas' proposal as well. But to me it is a no-op. The spec already does what Jonas is proposing - lock out users when an upgrade is in progress and wait to start an upgrade if a user is using the application. On Tue, May 18, 2010 at 8:30 PM, Nikunj Mehta nik...@o-micron.com wrote: Perhaps we could produce an event if the version number changes so that an application has an opportunity to deal with that. What would the problem be with this approach? The spec would need to mandate that the event would be received by every open connection. For the async interface, this would be easy. But in the sync interface, we'd need to either run the event synchronously in a nested fashion (which isn't unheard-of, but I believe we avoid whenever possible for good reason) or throw an exception (which most developers probably won't handle correctly). Neither of which seem like great options. I am not pretending that this is cheap, but then I am not asking that this be in the spec either. I am checking to see if this is useful and worth the cost. Nikunj: in your vision of the world, what is the point of the version parameter? I stated the use case for it in my email that is snipped out in this one: As a use case, an application may queue up schema upgrades based on version numbers. It seems like it could be easily emulated by a value in an objectStore. And, if you're assuming that developers will through a library develop sophisticated means of performing incompatible changes without breaking applications using an earlier version of the indexedDB then it seems pretty easy for such a library to also keep track of versioning data in a special objectStore. Since upgrades are typically performed when an application is started, as opposed to the middle of an application's execution, there needs to be a way for an application to test the database for some abstract representation of what is in the database. Ergo, version. Applications manage the content of the version property (which intentionally is not a number) and use it to the best of their (in)ability. Of course, that position is also basically saying that any user of this API will either need to roll complex coordination code themselves, use a complex (under the hood) library, or else their code will probably be racy. Although we expect many users to use libraries to simplify things like joins, I think essentially requiring _any_ user to use a library is a bit overboard. So basically I think I'm with Jonas on this one: setVersion should be a convenient (but maybe overly-simplistic for some) way to manage schema changes. If someone thinks it's too draconian, then they can simply implement things themselves and leave the version alone (as Nikunj explained in the thread). But if we're not going to do what Jonas proposed, then it seems like worthless API surface area that should be completely removed. J
Re: Updates to File API
On Tue, May 18, 2010 at 2:56 PM, Arun Ranganathan a...@mozilla.com wrote: On 5/18/10 2:35 PM, Eric Uhrhane wrote: On Mon, May 17, 2010 at 3:37 PM, Dmitry Titovdim...@chromium.org wrote: I have couple of questions, mostly clarifications I think: 1. FileReader takes Blob but there are multiple hints that the blob should be actually a 'file'. As we see Blob concept grows in popularity with such specs as FileWriter that defines BlobBuilder. Other proposals include Image Resizing that returns a Blob with a compressed image data. Can all types of Blobs be 'read' using FileReader? If not, then it would be logical to only accept File parameter. If any type of Blob can be read (as I think the spirit of the spec assumes) then would it be less confusing to cange the name to BlobReader? I'd support that. I think we always want to allow reading of any type of Blob--it's the interchange container--so calling it BlobReader makes sense. Arun, how do you feel about that? The FileReader object accepts File objects for DataURL-reads, and Blob objects for binary string, text, and binary reads. I agree that having a name like FileReader is generally a bit confusing, given that we do allow Blobs to be read, including Blobs which aren't directly coined from files. Blob itself isn't a great name, though it's a stand-in for Binary Large Object. Aside from the slight bikeshed-ish nature of this discussion, there are implementations in circulation that already use the name FileReader (e.g. Firefox 3.6.3). This doesn't mean I'm against changing it, but I do wish the name change suggestion came earlier. Also, I'm keen that the main object name addresses the initial use case -- reading file objects. Perhaps in the future Blobs that are not files will be the norm; maybe then, Blob APIs will evolve, including implementations with ArrayBuffer and potential streaming use cases getting addressed better. Perhaps it is late to have a name change, and we've added to less-than-adequate naming on the Web (example: XMLHttpRequest). Ok, I can see how it can be late if FF already shipped it... Perhaps the spec could at least avoid using 'fileBlob' as names of arguments, since the naming currently may be interpreted as if only file-backed blobs are welcome :-) Would FileWriter ever be used to write anything other than a File? I think not, so it should probably stay as it is, despite the lack of symmetry. 2. The FileReader.result is a string. Actually, in my next draft, I will have FileReader.result be of type 'any' (WebILD's 'any') since it could also be an ArrayBuffer (using the readAsBinary method, which will function like the other asynchrous read methods, but read into ArrayBuffers across the ProgressEvent spectrum. Getting an ArrayBuffer on each ProgressEvent could be a cool idea indeed. I guess when we have ArrayBuffers we'll be able to use them in BlobBuilder as well. -- A*
Re: [IndexedDB] What happens when the version changes?
On May 18, 2010, at 2:48 PM, Jonas Sicking wrote: On Tue, May 18, 2010 at 1:00 PM, Nikunj Mehta nik...@o-micron.com wrote: On May 18, 2010, at 12:46 PM, Jonas Sicking wrote: On Tue, May 18, 2010 at 12:37 PM, Nikunj Mehta nik...@o-micron.com wrote: If the use case here is to avoid tripping up on schema changes, then: 1. Lock the database when starting a database connection. This is the non-sharing access mode defined in 3.2.9 as the first option under step 2. Will locking the database prevent others from creating new objectStores? Step 2 only talks about acquiring locks on the existing database objects. Here's some text from the existing spec: If these steps are not called with a list of database objects Acquire a lock on the entire database. Is there confusion about the meaning of acquiring a lock on the entire database? Yes. Especially given the language in step 2. Would you mind proposing better spec text? However note that the proposal we made, the locking level is moved from the database-open call to the transaction-open call. I will respond separately to that. Locking is performed in the spec at transaction open time, not at database open time. I am not proposing to change that. This in order to allow the page to just open the database at the start of the page (and potentially upgrade it to its required version). The database can then be left open for as long as the user is on the page. This means that for any given interaction with the database only one level of asynchronous call is needed. 2. Produce events when an application changes the version so that other tabs of the application are alerted How? Spec text could be written if this serves a purpose. What functionality are you proposing? I answered this in other parts of this thread. 3. Through a library develop sophisticated means of performing incompatible changes without breaking applications using an earlier version of the indexedDB. I don't understand, performing incompatible changes seems contrary to without breaking applications using an earlier version. I don't see how you could to both at the same time in the general case. Since there is no restriction on what libraries can do, one could do seemingly contrary things. Take an example. Say we want to change the content of a compound index, e.g., the order of attributes in the index. This requires changing the contents of the index. It also requires translation of the request to match the sequence of properties being stored in the index. A library can keep extra data to perform translation where it is required. This sort of stuff is done in many applications, so it is not unimaginable that someone would want to do it with IndexedDB. I still don't understand why this is considered performing an incompatible change. It is an incompatible change at some level, isn't it? It seems to me that if the user has a version of the application loaded which is able to keep extra data to perform translation where it is required, then it isn't an incompatible change. So in this case I would say that the application should just leave the version number unchanged. Your approach can work too. Nevertheless, I don't want to constrain an application's behavior in this respect. Granted, it's a little hard to follow the example given that we don't have compound indexes in the spec, so maybe I'm missing something? In my understanding the spec supports compound indexes and has since at least October last year. On May 18, 2010, at 1:54 AM, Jonas Sicking wrote: On Thu, May 13, 2010 at 10:25 AM, Shawn Wilsher sdwi...@mozilla.com wrote: On 5/13/2010 7:51 AM, Nikunj Mehta wrote: If you search archives you will find a discussion on versioning and that we gave up on doing version management inside the browser and instead leave it to applications to do their own versioning and upgrades. Right, I'm not saying we should manage it, but I want to make sure we don't end up making it very easy for apps to break themselves. For example: 1) Site A has two different tabs (T1 and T2) open that were loaded such that one got a script (T1) with a newer indexedDB version than the other (T2). 2) T1 upgrades the database in a way that T2 now gets a constraint violation on every operation (or some other error due to the database changing). This could easily happen any time a site changes the version number on their database. As the spec is written right now, there is no way for a site to know when the version changes without reopening the database since the version property is a sync getter, implementations would have to load it on open and cache it, or come up with some way to update all open databases (not so fun). I think what we should do is to make it so that a database connection is version specific. This is draconian and does not permit
Re: [IndexedDB] What happens when the version changes?
On Tue, May 18, 2010 at 2:57 PM, Nikunj Mehta nik...@o-micron.com wrote: On May 18, 2010, at 2:33 PM, Jeremy Orlow wrote: On Tue, May 18, 2010 at 9:36 PM, Shawn Wilsher sdwi...@mozilla.com wrote: On 5/18/2010 1:02 PM, Nikunj Mehta wrote: A database connection that locks the entire database cannot be opened if there is another database connection that locks at least one database object, e.g., an index or object store. So basically, as long as some connection holds a database lock, you won't be able to do any upgrade. Sure, but this is true of Jonas' proposal as well. But to me it is a no-op. The spec already does what Jonas is proposing - lock out users when an upgrade is in progress and wait to start an upgrade if a user is using the application. Note that in our proposal, having a database open does not hold any locks. It simply holds metadata (such as objectStore and index names) in memory. It is totally ok for the application to open the database when the user initially loads the page, and never close it until the user leaves the page. In fact, I would encourage users to do this since it allows for reading data using a single level asynchronous callbacks, and so makes for very readable code. Because our proposal is incompatible with the versioning mechanism that the spec uses, I would propose that we first decide on if mozillas proposal should be accepted before we decide on what versioning mechanism we should use. / Jonas
[IndexedDB] KeyPaths and missing properties.
Hi IndexedDB fans!
So another issue that's come up here. This question is orthogonal to
the other discussed issues and applies equally to both the existing
API, and the mozilla proposed API, so I figured it was safe to raise
right away.
What should happen if you insert a object into an objectStore which
has an index attached to it, but the property defined by the keyPath
in the index does not exist? Consider a database with an objectStore
called people with keyPath name. The objectStore has an
auto-populated index named phone numbers with keyPath phone. The
objectStore is initially empty.
What should happen if you attempt to store an object with value {
name: Benny, email: be...@sweden.com } ?
There are three possible behaviors I can think of:
1. The storing operation should fail and an error event should be fired.
2. The storing operation should succeed, but no records should be
added to the phone index.
3. The storing operation should succeed and a record should be added
to the phone index using null as key.
Which one is correct behavior? What if the phone numbers index is
flagged as unique?
I definitely think there is a use case for allowing records to be
stored even if there are indexes on missing properties. Consider for
example a objectStore containing people where not everyone has a known
phone number. It still seems useful to be able to search based on
phone number in this objectStore. To satisfy this use case either
solution 2 or 3 would work. 3 would additionally allow searching for
everyone without a phone number, though I'm not sure how important
that is. On the other hand 2 would be more space effective in that
only relevant records would be stored.
Possibly we'll want to add a 'required' flag to the createIndex
function specifying if the index should cause behavior 1 or behavior
2. So far I'm not a big fan of 3, but I'm all ears for input.
The same question arises when an index is added to an objectStore
which has existing values that lack the property that the index uses
as keyPath. Whichever solution we go with above I think should be used
here too.
There is a similar issue if an objectStore uses in-line keys. Consider
a database with a objectStore called people with keyPath name and
no key generator. What if someone inserts an object with the value {
email: be...@sweden.com }? I would think that this should be an
error, but it doesn't appear that any of the steps in Object Store
Storage steps says to create an error.
Finally, there is the issue of what should happen if someone tries to
insert a number as a value into an objectStore with a keyPath and a
key generator. Consider the objectStore albums with keyPath id and
a key generator enabled. What should happen if someone tries to insert
the value 9 into this table? If an object was inserted, then this
would result in that objects 'id' property being set to the next
generated value from the generator, however it doesn't really make
sense to set a property on a number. Same question obviously applies
if the stored value is a boolean, a string, or any other primitive JS
type, or types such as regexps and File objects. I think that in all
these cases the operation should fail.
The question also applies if the stored value is an array. Technically
Javascript allows setting values on arrays, however the structured
clone algorithm drops these values, and so it seems to add needless
complexity for the implementation if the only time you need to
serialize/deserialize arrays with non-integer properties is in this
one case. Additionally, the id would be lost when the value is read as
that is done using the structured clone algorithm too.
/ Jonas
