Re: [indexeddb] transaction commit failure
On 8/15/2011 3:31 PM, Israel Hilerio wrote: When the db is doing a commit after processing all records on the transaction, if for some reason it fails, should we produce an error event first and let the bubbling produce a transaction abort event or should we only produce a transaction abort event. It seems that doing the first approach would be more complete. I agree; the first approach seems better and I can't think of any reason why it would be difficult to implement. The catch is that calling `preventDefault` will not prevent the abort, which is (I think) different from how we handle other errors, right? Cheers, Shawn
Re: [indexeddb] Using WebIDL Dictionary in IDBObjectStore.createIndex for optionalParameters
On 6/6/2011 12:03 PM, Israel Hilerio wrote: Have you considered using the WebIDL dictionary definition as a mechanism to define the optional parameters in the IDBDatabase.createObjectStore method? I don't believe this was available when we added this. Changing this seems sensible to me. Cheers, Shawn smime.p7s Description: S/MIME Cryptographic Signature
Re: [IndexedDB] IDBDatabase.transaction needs to specify exception for invalid mode parameter (Bug# 11406)
On 5/31/2011 10:56 AM, Israel Hilerio wrote: Should I interpret the silence to mean we agree? I believe that's a safe thing to do :) Cheers, Shawn smime.p7s Description: S/MIME Cryptographic Signature
Re: [IndexedDB] Closing on bug 9903 (collations)
On 5/6/2011 7:07 AM, timeless wrote: I think that a stored procedure could be considered as a compiled version of a serialized function. i.e. something which loses its scope chain, and which loses access to its parent object. If it loses access to its scope chain which includes the interesting globals, it will no longer have access to fun things like DOM objects, roughly like DOMWorkers but with even less exciting objects available. I'd hope that a jit should be able to do a fairly reasonable job of optimizing such a function given these constraints. This may be what we go with, but not in version 1. Cheers, Shawn smime.p7s Description: S/MIME Cryptographic Signature
Re: [IndexedDB] Spec Question on IDBFactory open method
On 4/20/2011 1:33 PM, Israel Hilerio wrote: The open method description in the IDBFactory talks about setting the source of the IDBRequest to no source. What does no source means (undefined, null, other)? In addition, what should be the value of the transaction property in the IDBRequest object returned from the open method? It seems this should be either undefined or null. I think undefined makes sense for both of these. Cheers, Shawn smime.p7s Description: S/MIME Cryptographic Signature
Re: [WebSQL] Any future plans, or has IndexedDB replaced WebSQL?
On 4/4/2011 8:07 AM, Joran Greef wrote: SQLite has a fantastic track record of maintaining backwards compatibility. Sort of. They didn't between SQLite 2 and SQLite 3. There also have been some (albeit minor) backwards compatibility issues with SQLite 3.x releases. The most serious of which deal with performance characteristics changing because they changed how the optimizer works. These type of things are acceptable to deal with in browser code because you can change your code unlike on the web (unless you want to have different code for each browser, and then each browser version). It's that, or browsers can ship one version of SQLite for all eternity. Cheers, Shawn smime.p7s Description: S/MIME Cryptographic Signature
Re: [WebSQL] Any future plans, or has IndexedDB replaced WebSQL?
On 4/4/2011 10:28 AM, Joran Greef wrote: Do you think it would be wise then to advocate doing away with SQLite before IndexedDB has had a chance to prove itself? Surely two competing APIs would be the fastest way to bring IndexedDB up to speed? Who is advocating doing away with it? Note that I would not consider vendors not implementing it as doing away with it. Cheers, Shawn smime.p7s Description: S/MIME Cryptographic Signature
Re: [WebSQL] Any future plans, or has IndexedDB replaced WebSQL?
On 4/4/2011 10:18 AM, Joran Greef wrote: How would you create an index on an existing object store in IndexedDB containing more than 50,000 objects on an iPad, without incurring any object deserialization/serialization overhead, without being an order of magnitude slower than SQLite, and without bringing the iPad to its knees? If you can do it with even one IndexedDB implementation out there then kudos and hats off to you. :) You keep bringing this point up, but only a naive implementation of IndexedDB would bring a device to it's knees (or a poorly implemented thread scheduler, which I don't expect the iPad to have). The API is asynchronous, which means it doesn't need to (nor should it) happen on any thread that the UI is being drawn on. You still have a point about it possibly taking longer, but even then, that will be implementation dependent. Cheers, Shawn smime.p7s Description: S/MIME Cryptographic Signature
Re: [WebSQL] Any future plans, or has IndexedDB replaced WebSQL?
On 4/6/2011 9:44 AM, Joran Greef wrote: We only need one fixed version of SQLite to be shipped across Chrome, Safari, Opera, Firefox and IE. That in itself would represent a tremendous goal for IndexedDB to target and to try and achieve. When it actually does, and surpasses the fixed version of SQLite, those developers requiring the raw performance and reliability of SQLite could then switch over. I don't believe any browser vendor would be interested in shipping two different version of SQLite (one for internal use, and one for the web). I can say, with certainty, that Mozilla is not. Cheers, Shawn smime.p7s Description: S/MIME Cryptographic Signature
Re: [WebSQL] Any future plans, or has IndexedDB replaced WebSQL?
On 4/6/2011 10:06 AM, Joran Greef wrote: I bring up the iPad example because I had experience with a LocalStorage implementation (I think it was Safari) loading the contents of LocalStorage into memory synchronously on first access, blocking the UI thread. I am probably wrong on this one but I think I remember reading on Web Apps that this was one of the motivations behind limiting LocalStorage quota to around 10mb. At the time I was one of those who believed that LocalStorage would support storage of at least 10 GB as a matter of course. I hope you can understand my slight distrust of subsequent storage APIs (other than those of proven track record) in this light. localStorage had a major design flaw that shipped and was then used before it was identified (at least that's my understanding of the timeline). That flaw is that it requires disk I/O to happen before (in webkit's case onload) or during the call to the API. IndexedDB (and WebSQL) were specifically designed to avoid this by doing the work asynchronously. Cheers, Shawn smime.p7s Description: S/MIME Cryptographic Signature
Re: [WebSQL] Any future plans, or has IndexedDB replaced WebSQL?
On 4/1/2011 5:40 AM, Nathan Kitchen wrote: Are there any browser vendor representatives on the mailing list who would care to comment on the criteria for implementing something akin to Keean's RelationalDBhttps://github.com/keean/RelationalDB idea? What would need to be in place to start work on such an implementation? It wouldn't be terribly difficult to prototype this as an add-on for Firefox, I don't think (and I'd be happy to provide technical assistance to anyone wishing to do so). Doing this would allow web developers to install the add-on and play with it, which can give us useful feedback. I'm not saying we'd move it into the tree at that point, but it's a good first step to building a case to take it. 1. Opportunity to explore more solutions to offline data than *just * IndexedDB. There is also http://dev.w3.org/html5/spec/offline.html and http://dev.w3.org/html5/webstorage/ (even if you don't like them, they are other solutions to the offline problem). Browser vendors are not just looking at IndexedDB. 2. Many web developers have a working knowledge of SQL, so the concepts of a relational database may be more familiar. If adoption could be considered a proxy for the success of a standard, I'd suggest that aiming for something the web development community understands would be a large factor in adoption. I don't really think IndexedDB is that dissimilar to a relational database. There are a lot of one-to-one mappings of concepts of one to the other. 3. It's probably (!) easier to implement RelationalDB than IndexedDB, as it maps fairly cleanly to existing relational database technologies. This would allow vendors to implement it using Sqlite, Access, etc independent of the spec. Given that most vendors already have working implementations of IndexedDB, I don't think this is a good argument ;) Cheers, Shawn smime.p7s Description: S/MIME Cryptographic Signature
Re: [WebSQL] Any future plans, or has IndexedDB replaced WebSQL?
On 4/1/2011 9:39 AM, Aryeh Gregor wrote: IE6 is closed-source software written for a single platform. SQLite is in the public domain, works for all major operating systems and lots of minor ones, and is already used (I think?) by every major browser except IE. That makes all the difference. There's some benefit to having multiple interoperable implementations even if the reference implementation is public-domain, but enormously less than when the only implementations are controlled by particular parties. How, exactly, does it make all the difference? I sure hope you aren't suggesting that the spec say do what this code does. So if the only objection to WebSQL is there's no way we're going to get a formal spec or two interoperable implementations, I'd really encourage objectors to step back and ask themselves why they *want* a formal spec and two interoperable implementations. Those requirements are not axiomatic, they're means to obtain practical ends like allowing competitions and avoiding user lock-in. How many of those ends are really contrary to using SQLite as a de facto standard, and do the remaining ones really outweigh the practical advantages? That's not the only reason. Mozilla laid out others ten months ago: https://hacks.mozilla.org/2010/06/beyond-html5-database-apis-and-the-road-to-indexeddb/ Cheers, Shawn smime.p7s Description: S/MIME Cryptographic Signature
Re: [IndexedDB] Design Flaws: Not Stateless, Not Treating Objects As Opaque
On 3/31/2011 11:47 AM, Joran Greef wrote: Let those who introduced these design flaws be among the first to take responsibility and fix them. You aren't being constructive, and that's a surefire way to be ignored. You have yet to convince the working group that these are design flaws in the first place. /sdwilsh smime.p7s Description: S/MIME Cryptographic Signature
Re: [IndexedDB] setVersion blocked on uncollected garbage IDBDatabases
On 2/8/2011 11:51 AM, ben turner wrote: I'm actually fine with keeping the setVersion from proceeding until the old database is collected. First, this is probably a bug in the web page, and the page should be fixed. Second, the new database that is waiting for setVersion to proceed will get an onblocked event, so the page should know that something is wrong. I really don't think this is that big of a deal, and certainly not worth changing the opt-in vs. opt-out behavior that we've settled on. Agreed. Cheers, Shawn smime.p7s Description: S/MIME Cryptographic Signature
Re: [Bug 11351] New: [IndexedDB] Should we have a maximum key size (or something like that)?
On 2/7/2011 12:32 AM, Glenn Maynard wrote: Is that a safe assumption to design around? The API might later be bound to other languages fortunate enough not to be stuck in UTF-16. As I recall, we've already made design decisions based on the fact that the primary consumer of this API is going to be JavaScript on the web. (What those decisions were about, I don't recall offhand, however.) Cheers, Shawn smime.p7s Description: S/MIME Cryptographic Signature
Re: [Bug 11351] New: [IndexedDB] Should we have a maximum key size (or something like that)?
On 2/6/2011 12:42 PM, Jeremy Orlow wrote: My current thinking is that we should have some relatively large limitmaybe on the order of 64k? It seems like it'd be very difficult to hit such a limit with any sort of legitimate use case, and the chances of some subtle data-dependent error would be much less. But a 1GB key is just not going to work well in any implementation (if it doesn't simply oom the process!). So despite what I said earlier, I guess I think we should have some limit...but keep it an order of magnitude or two larger than what we expect any legitimate usage to hit just to keep the system as flexible as possible. Does that sound reasonable to people? Are we thinking about making this a MUST requirement, or a SHOULD? I'm hesitant to spec an exact size as a MUST given how technology has a way of changing in unexpected ways that makes old constraints obsolete. But then, I may just be overly concerned about this too. Cheers, Shawn smime.p7s Description: S/MIME Cryptographic Signature
Re: [Bug 11948] New: index.openCursor's cursor should have a way to access the index's value (in addition to the index's key and objectStore's value)
On 2/1/2011 11:00 AM, bugzi...@jessica.w3.org wrote: As discussed in the mailing list thread from bug 11257, we should add some way for index.openCursor cursors to access the primary key for the objectStore. .indexValue, .objectStoreKey, or .primaryKey might be good names to use for it. .objectStoreKey seems to be the most clear way to express this to me. Cheers, Shawn smime.p7s Description: S/MIME Cryptographic Signature
Re: Quota API to query/request quota for offline storages (e.g. IndexedDB, FileSystem)
On 2/3/2011 3:59 PM, João Eiras wrote: Because the user agent needs to differentiate which api will use each quota. But why does a user agent need to do that? It seems like that is adding unnecessary complication to the API. Cheers, Shawn smime.p7s Description: S/MIME Cryptographic Signature
Re: Quota API to query/request quota for offline storages (e.g. IndexedDB, FileSystem)
On 2/3/2011 4:35 PM, João Eiras wrote: Or adding unnecessary complication to the implementation. I'm not looking to make my job easier (as an implementer); I'm looking to make it easy to use. At least with IndexedDB, we generally choose the option that is easier for the consumer as long as it isn't extremely difficult to implement (which I do not see this being the case for quotas). Cheers, Shawn smime.p7s Description: S/MIME Cryptographic Signature
Re: [Bug 11348] New: [IndexedDB] Overhaul of the event model
On 1/28/2011 1:15 AM, Axel Rauschmayer wrote: All API invocations that I have seen relied on run-to-completion semantics and add a listener after the initial invocation. These now have to check the flag? No, all that works just like it did before. The flag just allows for some additional flexibility for authors. Cheers, Shawn smime.p7s Description: S/MIME Cryptographic Signature
Re: [Bug 11348] New: [IndexedDB] Overhaul of the event model
On 1/28/2011 1:07 AM, Axel Rauschmayer wrote: Agreed. My only aside would be that for API design, it’s usually not a good idea to listen to web developers, but to someone who has experience with designing DB APIs (= not me, but possibly anyone of you or anyone at Mozilla, MS, Google). It sounds like you are saying we aren't listening to people who have designed database APIs. We certainly have (and have borrowed from models of existing APIs for other databases too). It seems a bit disingenuous to not listen to feedback of web developers who are the primary target audience of this API. Cheers, Shawn smime.p7s Description: S/MIME Cryptographic Signature
Re: [IndexedDB] Why rely on run-to-completion?
On 12/29/2010 2:44 PM, Axel Rauschmayer wrote: The pattern of assigning the success continuation after invoking the operation seems to be to closely tied to JavaScript’s current run-to-completion event handling. But what about future JavaScript environments, e.g. a multi-threaded Node.js with IndexedDB built in or Rhino with IndexedDB running in parallel? Wouldn’t a reliance on run-to-completion unnecessarily limit future developments? Could you elaborate on how the current spec would fall-apart without run-to-completion semantics? Preferably with specific examples. Maybe it is just me, but I would like it better if the last argument was an object with the error and the success continuations (they could also be individual arguments). That is also how current JavaScript RPC APIs are designed, resulting in a familiar look. Are there any arguments *against* this approach? Right now we support having multiple callbacks by registering multiple listeners for the event (with addEventListener). Using an object would limit us to one callback. There is a thread somewhere in this group where we decided to go with an event-based API instead of a callback one (I cannot recall the reasons off the top of my head). Whatever the reasoning behind the design, I think it should be explained in the spec, because the current API is a bit tricky to understand for newbies. I do not think that a spec is the right place to justify design decisions. Cheers, Shawn smime.p7s Description: S/MIME Cryptographic Signature
Re: [Bug 11375] New: [IndexedDB] Error codes need to be assigned new numbers
On 12/14/2010 4:16 PM, Jeremy Orlow wrote: Shawn said NOT_FOUND_ERR. NOT_ALLOWED_ERR seems slightly better to me. Shawn, what do you think? I don't have a strong opinion either way. Cheers, Shawn smime.p7s Description: S/MIME Cryptographic Signature
Re: [Bug 11375] New: [IndexedDB] Error codes need to be assigned new numbers
On 12/10/2010 5:03 AM, Jeremy Orlow wrote: Speaking of which, we use UNKNOWN_ERR for a bunch of other internal consistency issues. Is this OK by everyone, should we use another, or should we create a new one? (Ideally these issues will be few and far between as we make things more robust.) Would a CONSTRAINT_ERR make more sense? What error code should we use for IDBCursor.update/delete when the cursor is not currently on an item (or that item has been deleted)? I think NOT_FOUND_ERR makes sense there. TRANSIENT_ERR doesn't seem to be used anywhere in the spec. Should it be removed? I can't think of anything that we'd actually want to use it for. As for the numbering: does anyone object to me just starting from 1 and going sequentially? I.e. does anyone have a problem with them all getting new numbers, or should I keep the numbers the same when possible. (i.e. only UNKNOWN_ERR, RECOVERABLE_ERR, TRANSIENT_ERR, TIMEOUT_ERR, DEADLOCK_ERR would change number, but the ordering of those on the page would change.) I think it is fine to just renumber. If anyone is relying on the numbers being a certain thing now, I think it's probably best just to have a clean break instead of sometimes being right still. Cheers, Shawn smime.p7s Description: S/MIME Cryptographic Signature
Re: [Bug 11398] New: [IndexedDB] Methods that take multiple optional parameters should instead take an options object
On 12/10/2010 7:27 AM, Jeremy Orlow wrote: We did all of these two weeks ago in Chromium and have gotten some feedback. The main downside is that typos are silently ignored by JavaScript. We considered throwing if someone passed in an option we didn't recognize, but this would make it impossible to add more options later (which is one of the main reasons for doing this change). I think what we might do is just log something in the console with this happens. (Should the spec actually make a recommendation to this effect?) Besides that, I think overall we're happy with the change. I think logging should maybe be recommended by the spec, but probably not required. Cheers, Shawn smime.p7s Description: S/MIME Cryptographic Signature
Re: [IndexedDB] Behavior of IDBObjectStore.get() and IDBObjectStore.delete() when record doesn't exist
On 11/11/2010 11:44 AM, Jeremy Orlow wrote: The email I responded to: It would make sense if you make setting a key to undefined semantically equivalent to deleting the value (and no error if it does not exist), and return undefined on a get when no such key exists. That way 'undefined' cannot exist as a value in the object store, and is a safe marker for the key not existing in that index. undefined should be symmetric. If something not existing returns undefined then passing in undefined should make it not exist. Overloading the meaning of a get returning undefined is ugly. And simply disallowing a value also seems a bit odd. But I think this is pretty elegant semantically. Sorry, but I disagree. I feel that calling put results in a deletion to be highly counter-intuitive, even if it makes sense when you think about it. Cheers, Shawn smime.p7s Description: S/MIME Cryptographic Signature
Re: [IndexedDB] Events and requests
On 11/9/2010 12:35 AM, Jonas Sicking wrote:
One thing we could do is to move
.source
.transaction
.result
.error
to IDBRequest. Then make success and error events be simple events
which only implement the Event interface. I.e. we could get rid of the
IDBEvent, IDBSuccessEvent, IDBTransactionEvent and IDBErrorEvent
interfaces.
We'd still have to keep IDBVersionChangeEvent, but it can inherit
Event directly.
The request created from IDBFactory.open would return a IDBRequest
where .transaction and .source is null. We already fire a IDBEvent
where .source is null (actually, the spec currently doesn't define
what the source should be I see now).
This seems fine to me.
db.transaction([foo]).objectStore(foo).get(mykey).onsuccess = function(e) {
alert(e.result);
}
would turn into the slightly more verbose
db.transaction([foo]).objectStore(foo).get(mykey).onsuccess = function(e) {
alert(e.target.result);
}
The only difference is e.result vs e.target.result, right? I'm not sure
we should worry about that little bit (and consumers can always use a
local variable for that if it bothers them).
Cheers,
Shawn
smime.p7s
Description: S/MIME Cryptographic Signature
Re: [Bug 10430] New: [IndexedDB] We need to make it more clear IDBRequests can be reused and spec readyState's behavior
On 11/1/2010 5:29 AM, Jeremy Orlow wrote: If not, I think we should avoid adding surface area for something we don't really understand very well. I agree with this. Less is better at this point I think (when appropriate, of course). Cheers, Shawn smime.p7s Description: S/MIME Cryptographic Signature
Re: [IndexedDB] Calling setVersion while already in a setVersion transaction
On 9/30/2010 2:16 AM, Jeremy Orlow wrote: Hm. Actually, I think I like Jonas' proposal better than 1 or 2 (and Shawn, I see your point about why 2 is better than 1). I also think that Shawn's example of doing multiple schema upgrades should still work 3 since we fire onsuccesses in the order that items were queued up. I agree. Three seems safer/better still! Cheers, Shawn smime.p7s Description: S/MIME Cryptographic Signature
Re: [IndexedDB] Calling setVersion while already in a setVersion transaction
On 9/29/2010 6:12 AM, Jeremy Orlow wrote: Off the top of my head, I can think of two behaviors we might want to spec: 1) Have the subsequent setVersion simply throw an error. 2) Have the subsequent setVersion adopt the existing setVersion transaction and change the version. (i.e. whatever the last setVersion call sets as the version string will win.) Any others? What do you guys think is the most sane behavior? I think (2) is most consistent with how transactions work in other places. I think it's sane too. I can't come up with a good reason why we'd throw for subsequent setVersion calls either. Thinking about it some more, this might make sense for someone who has had more than one version change. They could first update the the version from 1 to 2, and then from 2 to 3. Since it's possible that clients visiting the site could be on any of these three versions, it makes sense to have functions that only go from the previous version to the next, otherwise you'll end up with lots of logic to deal with every version combination. Doing this all in the same transaction seems to make sense to me. Cheers, Shawn smime.p7s Description: S/MIME Cryptographic Signature
Re: [IndexedDB] setVersion with multiple IDBDatabase objects
On 9/28/2010 7:42 AM, Jeremy Orlow wrote: What do we want to do if the user calls window.indexedDB.open(myDB, some description) twice and stores the result as db1 and db2, calls db1.setVersion(), and then in the success handler does db2.createObjectStore? In other words, is the setVersion transaction tied to the database instance that setVersion was called on, or is it available to any instance/object that points to that particular database for that particular origin in that particular page/worker? While I can see an argument either way, I think I'd prefer it to be tied to the instance. I like this because then it works the same as it does being in different windows. I don't have a strong opinion on this though. Cheers, Shawn smime.p7s Description: S/MIME Cryptographic Signature
Re: [IndexedDB] IDBCursor.update for cursors returned from IDBIndex.openCursor
On 9/17/2010 3:14 PM, Jonas Sicking wrote: How do you then guarantee that a transaction that spans multiple objectStores either fully succeeds or is fully rolled back? Especially in the event of a crash during commit. If you don't use write ahead logging, and connect to each database with ATTACH, SQLite makes sure it is atomic for you (I don't recall how offhand though). Not using write ahead logging does mean that writers will block readers though. Cheers, Shawn smime.p7s Description: S/MIME Cryptographic Signature
Re: [IndexedDB] Constants and interfaces
On 8/31/2010 12:03 PM, Jonas Sicking wrote: So IMHO we should heavily optimize for javascript, while ensuring that other languages are possible. Yes, please. Cheers, Shawn smime.p7s Description: S/MIME Cryptographic Signature
Re: [IndexedDB] question about description argument of IDBFactory::open()
On 8/9/2010 1:13 PM, Andrei Popescu wrote: While implementing IDBFactory::open(), we thought that the description argument is optional but we were surprised to find out it's actually mandatory. Is there any reason not to make this argument optional? And, assuming it is optional, should the default value be the empty string? Also, how should the null and undefined values be treated? My suggestion would be to treat them as if the argument wasn't specified, so the description of the database would not change. I think we want something there, because that is likely what the user agent is going to tell the user when it has to ask for space (if the user agent limits) or display if the user wants to see what a site is storing locally. I think we should probably enforce a non-null string too because of this, although a null string for future connections meaning do not change seem fine to me. I started a bug [1] a while back about what to do if it changes, but it went nowhere. Cheers, Shawn [1] http://www.w3.org/Bugs/Public/show_bug.cgi?id=9562 smime.p7s Description: S/MIME Cryptographic Signature
Re: [IndexedDB] Implicit transactions
On 8/4/2010 10:24 AM, Jeremy Orlow wrote: Jonas/Shawn: Since it seems you've been getting some feedback on your implementation, do you have any data to suggest that implicit transactions are being used and considered helpful in the wild? I have not yet seen any specific feedback about it as of yet. Cheers, Shawn smime.p7s Description: S/MIME Cryptographic Signature
Re: [IndexedDB] Implicit transactions
On 8/4/2010 8:42 AM, Jeremy Orlow wrote: In the IndexedDB spec, there are two ways to create a transaction. One is explicit (by calling IDBDatabase.transaction()) and one is implicit (for example, by calling IDBDatabase.objectStore.get(someKey)). I have questions about the latter, but before bringing these up, I think it might be best to give a bit of background (as I understand it) to make sure we're all on the same page: If I recall correctly, the original proposal from us basically said that IDBDatabase.objectStore(foo, [mode]) was just a shortcut for IDBDatabase.transaction(foo, [mode]).getObjectStore(foo); I'm not sure if putting some text like that would help make things clearer though. Cheers, Shawn smime.p7s Description: S/MIME Cryptographic Signature
[IndexedDB] IndexedDB Sample Code Documentation Tool
Hey guys, Someone just posted a link to this on one of our blog posts about IndexedDB and I figured I'd share it with the list since it's pretty darn neat: http://tinyurl.com/ff-idxdb It appears to be sample code + documentation mixed together. It does seem to be a bit specific to our implementation, but I'm sure that can be improved. Cheers, Shawn smime.p7s Description: S/MIME Cryptographic Signature
Re: [IndexedDB] Implicit transactions
On 8/4/2010 10:53 AM, Jeremy Orlow wrote: Whoatransaction() is synchronous?!? Ok, so I guess the entire premise of my question was super confused. :-) It is certainly spec'd that way [1]. The locks do not get acquired until the first actual bit of work is done though. Cheers, Shawn [1] http://dvcs.w3.org/hg/IndexedDB/raw-file/tip/Overview.html#database-interface smime.p7s Description: S/MIME Cryptographic Signature
[IndexedDB] Need a method to remove a database
Hey all, Some of the feedback I've been seeing on the web is that there is no way to remove a database. Examples seem to be web page wants to allow the user to remove the data they stored. A site can almost accomplish this now by removing all object stores, but we still end up storing some meta data (version number). Does this seem like a legit request to everyone? Cheers, Shawn smime.p7s Description: S/MIME Cryptographic Signature
Re: [IndexedDB] Current editor's draft
On 7/9/2010 11:05 AM, Nikunj Mehta wrote: We would not make dynamic transactions be the default since they would produce more concurrency than static scoped transactions, correct? I'm still of the opinion that dynamic transactions are a bad idea because it's too easy to hold a transaction open for a long period of time, making it easy for programmers to write incorrect/buggy code. All along our primary objective with IndexedDB is to assist programmers who are not well versed with database programming to be able to write simple programs without errors. By that token, reducing the effort required for their use of IndexedDB seems to be the primary criteria and not great concurrency. Er, I thought we were targeting libraries but still wanted this to be simple to use. Regardless, allowing for more concurrency doesn't hurt ease of use in any of what we've discussed so far (as far as I can tell). Another downside is that authors should specify lock-type more often, for optimal performance, if we think that READ_ONLY is more common. You haven't provided any evidence about this yet. I can assert that all the demos I've written (admittedly not many), simply reading from the database has been far more common than writing to it. It's pretty much write data in but then do all operations on the local database. It is quite common in various languages to specify as a performance or safety hint when someone desires a shared lock and use a read-write version by default. You seem to have contradictory statements here. Earlier you argued that reducing the effort required for their [programmers] use of IndexedDB seems to be the primary criteria, but having them to read and know performance or safety hints seems to me like we are making the API more complicated. Having simple clear API calls with sensible error messages on misuse is far better than having an API that you can use one way (the most common way) but would be more efficient if you use it another way. For all we know, programmers would lock the entire database when they create a transaction. If dynamic transactions appear to be a non-v1 feature, then READ_ONLY being default appears out of place. You asked Jonas for data backing up his claims, and now I'm going to ask the same of you. It's been my experience and Ben Turner's experience that defaulting to READ_ONLY results in less code being written using this API. Reads are far more common than writes (in databases in general, I think although edge cases certainly exist), so it makes sense to make that the default. Cheers, Shawn smime.p7s Description: S/MIME Cryptographic Signature
Re: [IndexedDB] Current editor's draft
On 7/9/2010 12:50 PM, Nikunj Mehta wrote: The point is that we are talking of leaving out dynamic scope in v1, while, in the same vein, talking of making READ_ONLY the default _because_ it produces good performance. That is, IMHO, contradictory. Dynamic scope == dynamic transactions, correct? Can you please elaborate on how dynamic transactions improve performance? This seems to be conveniently justified. A strict interpretation of the objective would not require the programmer to specify READ_WRITE even though that involves less mental (cognitive) and physical (typing) effort. FWIW, this isn't constructive. Your argument also seems to assume that writes are more common than reads, which hasn't been my experience with this API. It should have worked right the first time. Why wait for a programmer to find out why their code didn't work? Why make writing code with good performance characteristics the uncommon case? We clearly have two different schools of thoughts here, and I'm not sure we are going to find a consensus... There are many ways to get performance improvement, including dynamic transactions, which you seem not to be favorable towards. I don't see why READ_ONLY should be given special treatment. It's unclear to me why you think READ_ONLY is getting special treatment (or what that even means in this context). Various hints are used in SQL syntax, e.g., [1], to manage locks, a certain kind of B-tree behavior, or a level of isolation. These are all aimed at improving performance, but they are set as default behavior. My point is that expecting good performance from a single variable in database systems out of possibly hundreds is not all that helpful. It is also a slippery slope because it confuses performance with options. The database's job is to be fast at what it does, not play performance tricks using default values. While I think this argument makes sense in some cases, I really don't feel like it applies at all in this situation. Even the page you linked to says Because the SQL Server query optimizer typically selects the best execution plan for a query... which in most cases would be READ_ONLY (based on evidence the Mozilla team has from demos written). The default should represent the commonly used case. I don't know that we can make the right performance decisions for people. What we can do is make things perform well and provide tools to improve performance. We aren't making performance decisions though; we are just picking the default to be the most commonly used option. It just happens to be the one to allow more concurrency. Cheers, Shawn smime.p7s Description: S/MIME Cryptographic Signature
Re: [IndexedDB] Current editor's draft
On 7/6/2010 6:31 PM, Nikunj Mehta wrote: To begin with, 10052 shuts down the users of the database completely when only one is changing its structure, i.e., adding or removing an object store. How can we make it less draconian? Secondly, I don't see how that approach can produce atomic changes to the database. Thirdly, we shouldn't need to change version in order to perform database changes. Finally, I am not sure why you consider the syntax proposal simpler. Note that I am not averse to the version change event notification. In what use case would you want to change the database structure without modifying the version? That almost seems like a footgun for consumers. Cheers, Shawn smime.p7s Description: S/MIME Cryptographic Signature
Re: [IndexedDB] Current editor's draft
On 7/7/2010 12:27 AM, Jonas Sicking wrote: This interface allows asynchronously requesting more objectStores to be locked. The author must take care whenever calling openObjectStores that the request might fail due to deadlocks. But as previously stated, I think this adds too much complexity and too much racyness to the API. And so I'd prefer to not add this. I feel like we should not be creating an API that allows for deadlocks to happen. Especially with an API that allows for races to happen (which we have) such that it will be hard for web developers to test to ensure they do not have deadlocks. Cheers, Shawn smime.p7s Description: S/MIME Cryptographic Signature
Re: [IndexedDB] Existence checking
On 7/5/2010 3:19 AM, Nathan Kitchen wrote: There are a couple of ways to do this: I think you missed one. IDBDatabase has a property objectStores [1] that is an array of object stores. Additionally, IDBObjectStore has a property called indexNames [2] that is an array of indexes for the object store. We should probably look into having the same naming convention for both of those (objectStores and indexes or objectStoreNames and indexNames probably?). This accomplishes what you want, right? Cheers, Shawn [1] http://dvcs.w3.org/hg/IndexedDB/raw-file/tip/Overview.html#widl-IDBDatabase-objectStores [2] http://dvcs.w3.org/hg/IndexedDB/raw-file/tip/Overview.html#widl-IDBObjectStore-indexNames smime.p7s Description: S/MIME Cryptographic Signature
Re: [IndexedDB] IDBEvent and Event
On 6/30/2010 8:25 AM, Andrei Popescu wrote: Agreed. In WebKit, Jeremy already made it inherit from Event. I filed bug 10056 [1] on this. Cheers, Shawn [1] http://www.w3.org/Bugs/Public/show_bug.cgi?id=10056 smime.p7s Description: S/MIME Cryptographic Signature
Re: [IndexedDB] Computed indexes
On 6/28/2010 4:59 PM, Jeremy Orlow wrote: Gotcha. Yeah, this technique can be used to replace any use case for explicitly managed indexes. And, for that reason, I think we should just get rid of them now. Agreed. As for whether we should have a feature to compute index keys via a function: I think we should put it on it on the back burner for now, but use your proposal as the starting when we re-examine it. I'd like to do this after there's a working implementation in the wild that developers can play with so we can address the biggest pain points--rather than just going on our intuition. If this does get punted past v1, I'd definitely like to see it addressed in v2 though. Once Firefox ships a beta for 4.0 (likely late this week or early next) we'll have something in the wild. Nightlies currently have it (although we haven't been pushing it much). Cheers, Shawn smime.p7s Description: S/MIME Cryptographic Signature
[IndexedDB] IDBEvent and Event
Hey all, I think that IDBEvent needs to inherit from Event [1] in order for us to properly inherit from EventTarget in IDBRequest. Specifically, EventTarget takes an EventListener [2] which has a method, handleEvent, that takes an Event object. I'm not sure this makes sense for us though, so I figured I'd start a discussion before filing the bug. Cheers, Shawn [1] http://www.w3.org/TR/DOM-Level-3-Events/#interface-Event [2] http://www.w3.org/TR/DOM-Level-3-Events/#interface-EventListener smime.p7s Description: S/MIME Cryptographic Signature
Re: [IndexedDB] Computed indexes
On 6/24/2010 7:01 AM, Jeremy Orlow wrote: So what your proposing is that the keyPath would essentially be a string of the body of a function which runs for every index (on that objectStore) for every value inserted into that object store? This seems like half way between the eval-like idea I mentioned earlier. It certainly seems to have advantages for complex keyPaths, but I'm still not so hot on having boilerplate/assumptions (like needing return and assuming value is present) present in every single keyPath. Especially when the use cases (while important) don't seem to be the common case. (In fact, can you even do this in SQL? If not, I think it's pretty strong evidence against needing to do arbitrary calculations in a keyPath.) You can do something like this with triggers I think. Cheers, Shawn smime.p7s Description: S/MIME Cryptographic Signature
Re: [IndexedDB] Changing the default overwrite behavior of Put
On 6/16/2010 9:43 AM, Nikunj Mehta wrote: There are three theoretical modes as you say. However, the second mode does not exist in practice. If you must overwrite, then you know that the record exists and hence don't need to specify that option. To be clear, you are saying that there are only two modes in practice: 1) add 2) add or modify But you don't believe that modify doesn't exist in practice? In terms of SQL, these three concepts exists and get used all the time. add maps to INSERT INTO, add or modify maps to INSERT OR REPLACE INTO, and modify maps to UPDATE. So, in summary, I agree to splitting the put method in to two - put and putNoOverwrite. I am also in favor of retaining the name as put (contrasted with get). I would like to avoid bikeshedding on names even though there have been ample opportunities on this list lately with that. I think you are completely ignoring the arguments in this thread about the issues with naming it put. I don't think it is bikeshedding; these seem like legitimate concerns. Cheers, Shawn smime.p7s Description: S/MIME Cryptographic Signature
Re: [IndexDB] Proposal for async API changes
On 6/9/2010 3:48 PM, Kris Zyp wrote:
Another option would be to have cursors essentially implement a JS
array-like API:
db.objectStore(foo).openCursor(range).forEach(function(object){
// do something with each object
}).onsuccess = function(){
// all done
});
(Or perhaps the cursor with a forEach would be nested inside a
callback, not sure).
The standard some function is also useful if you know you probably
won't need to iterate through everything
db.objectStore(foo).openCursor(range).some(function(object){
return object.name == John;
}).onsuccess = function(johnIsInDatabase){
if(johnIsInDatabase){
...
}
});
This allows us to have an async interface (the callbacks can be called
at any time) and still follows normal JS array patterns, for
programmer convenience (so programmers wouldn't need to iterate over a
cursor and push the results into another array). I don't think anyone
would miss getAll() with this design, since cursors would already be
array-like.
To me, this feels like we are basically doing what we expect a library
to do: make the syntactic sugar work. I don't see why a library
couldn't provide a some or forEach method with the currently proposed API.
Cheers,
Shawn
smime.p7s
Description: S/MIME Cryptographic Signature
Re: [IndexDB] Proposal for async API changes
On 6/9/2010 3:36 PM, Tab Atkins Jr. wrote: At the very least, explicitly loading things into an honest-to-god array can make it more obvious that you're eating memory in the form of a big array, as opposed to just a magically transform my blob of data into something more convenient. I'm sorry, but if a developer can't figure out that if they are given a big array (that is a proper Array in JavaScript) that it is the cause of large amounts of memory usage, I don't see how them populating it themselves is going to raise any additional flags. Cheers, Shawn smime.p7s Description: S/MIME Cryptographic Signature
[IndexedDB] Re: [Bug 9882] New: Behavior when attempting to write data to an objectStore opened as read only unspecified
On 6/8/2010 10:22 AM, bugzi...@jessica.w3.org wrote: When you open an objectStore with a read only mode, any subsequent write operation should probably result in an error. I don't see anywhere in the spec that specifically says this or mentions what error code should be used. Currently in the spec, writing with the noOverwrite flag set causes a CONSTRAINT_ERR, so I'm guessing that's the best one to go with for this condition as well? What about just creating a new error code for this? I don't like overloading error codes here for different meanings (while I can see how someone might view this as a constraint error, I think we could do better). New error codes are cheap, right? Cheers, Shawn smime.p7s Description: S/MIME Cryptographic Signature
Re: [IndexedDB] [Bug 9562] New: Opening a database with a different description is underspecified
On 4/20/2010 11:46 AM, bugzi...@jessica.w3.org wrote: The spec is unspecified as to what we should do when a database is opened with a different description than it was previously opened. I'd assume we'd want to update the description. Does anybody else have thoughts on what the right behavior should be here? Cheers, Shawn smime.p7s Description: S/MIME Cryptographic Signature
Re: [IndexedDB] Proposal for async API changes
On 5/20/2010 2:55 AM, Jeremy Orlow wrote: Thanks for taking the time to do this! Can you maybe discuss the pros and cons you found in terms of implementing something in WebSQLDatabase vs. IndexedDB? I'm mainly interested in seeing if there's any thing we can improve in IndexedDB that WebSQLDatabase already does well. So far it's really just that joins are painful in IndexedDB. I'm working on a blog post on this very topic though, and I'll be sure to point everyone in this thread to it (I figure this is useful stuff to get out to a wider audience). Cheers, Shawn smime.p7s Description: S/MIME Cryptographic Signature
Re: [IndexedDB] Proposal for async API changes
On 5/20/2010 9:03 AM, Jonas Sicking wrote:
For what it's worth, one of the ideas behind object stores, rather
than rows+column stores, is to reduce the need for joins. I.e. in our
candy store example you could just as well store objects like:
{ id: 1, name: Adam, sales: [{candyId: 1, date: 2010-01-02}, ...]}
Well, if people care about the size of data on disk, they will likely
want to normalize their object stores. We had talked about doing joins
before in indexedDB, but I believe we decided to punt on the issue for now.
Cheers,
Shawn
smime.p7s
Description: S/MIME Cryptographic Signature
Re: [IndexedDB] Proposal for async API changes
On 5/20/2010 11:30 AM, Andrei Popescu wrote: As someone new to this API, I thought the naming used in the current draft is somewhat confusing. Consider the following interfaces: IndexedDatabase IndexedDatabaseRequest, IDBDatabaseRequest, IDBDatabase, IDBRequest Just by looking at this, it is pretty hard to understand what the relationship between these interfaces really is and what role do they play in the API. For instance, I thought that the IDBDatabaseRequest is some type of Request when, in fact, it isn't a Request at all. It also isn't immediately obvious what the difference between IndexedDatabase and IDBDatabase really is, etc. It should be noted that we did not want to rock the boat too much with our proposal, so we stuck with the existing names. I think the current spec as written has the same issues. - I know we need to keep the IDB prefix in order to avoid collisions with other APIs. I would therefore think we should keep the IDB prefix and make sure all the interfaces start with it (right now they don't). I agree with this. We should probably file a bug about this. - The Request suffix is now used to denote the asynchronous versions of the API interfaces. These interfaces aren't actually Requests of any kind, so I would like to suggest changing this suffix. In fact, if the primary usage of this API is via its async version, we could even drop this suffix altogether and just add Sync to the synchronous versions? I agree that Request seems confusing and seems to be contrary to what other specs use. We should try to follow what other specs do here. - Some of the interfaces could have names that would more closely reflect their roles in the API. For instance, IDBDatabase could be renamed to IDBConnection, since in the spec it is described as a connection to the database. Likewise, IndexedDatabase could become IDBFactory since it is used to create database connections or key ranges. This sounds good as well. In any case, I want to make it clear that the current naming works once one takes the time to understand it. On the other hand, if we make it easier for people to understand the API, we could hopefully get feedback from more developers. Making it easier for someone to look at the method names and just know how to use the API seems like a good goal in my book. Cheers, Shawn smime.p7s Description: S/MIME Cryptographic Signature
Re: [IndexedDB] Proposal for async API changes
On 5/20/2010 12:19 PM, Jonas Sicking wrote: I additionally like the naming convention. The async interfaces is probably the interface that people will use first. Additionally that interface is available both to workers and to the main thread. So it makes sense to give the async interface the simpler name. That is not how it is currently specified (and our proposal doesn't indicate either way). The asynchronous versions are not available to worker threads. I do recall discussion on this, however... Cheers, Shawn smime.p7s Description: S/MIME Cryptographic Signature
Re: [IndexedDB] Proposal for async API changes
On 5/20/2010 7:34 AM, Shawn Wilsher wrote: So far it's really just that joins are painful in IndexedDB. I'm working on a blog post on this very topic though, and I'll be sure to point everyone in this thread to it (I figure this is useful stuff to get out to a wider audience). And honestly, I thought that we had discussed joins on this list, but I only see a thread from Pablo mentioning it, but no real discussions. Should we start that? Cheers, Shawn smime.p7s Description: S/MIME Cryptographic Signature
Re: [IndexedDB] Proposal for async API changes
On 5/17/2010 6:15 PM, Jonas Sicking wrote:
We've created some examples of what using this proposed API would look like:
http://docs.google.com/document/pub?id=1I__XnwvvSwyjvxi-FAAE0ecnUDhk5DF7L2GI6O31o18
we've also implemented the same examples using the currently drafted API:
http://docs.google.com/document/pub?id=1KKMAg_oHLeBvFUWND5km6FJtKi4jWxwKR0paKfZc8vU
And I've gone ahead and converted these samples to WebDatabase language
too for comparison purposes (I think, at least. I haven't used it
before, and these aren't actual fully runnable demos. If you spot an
error, let me know!)
It should be noted that WebDatabase fares much better when it comes to
joins, but I suspect we all knew that.
// Initialize database
var db = window.openDatabase(CandyDB, , My candy store database,
1024);
if (db.version != 1) {
db.changeVersion(db.version, 1, function(tx) {
// User's first visit. Initialize database.
var tables = [
{ name: kids, columns: [id INTEGER PRIMARY KEY,
name TEXT]},
{ name: candy, columns: [id INTEGER PRIMARY KEY,
name TEXT]},
{ name: candySales, columns: [kidId INTEGER,
candyId INTEGER,
date TEXT]}
];
for (var index = 0; index tables.length; index++) {
var table = tables[index];
tx.executeSql(CREATE TABLE + table.name + ( +
table.columns.join(, ) + );,
null, tableCreated);
}
}, null, function() { loadData(db); });
}
else {
// User has been here before, no initialization required.
loadData(db);
}
function loadData(db) {
// Do stuff!
}
// List kids
var db = window.openDatabase(CandyDB, 1, My candy store database,
1024);
db.readTransaction(function(tx) {
// Enumerate the entire table.
tx.executeSql(SELECT * FROM kids, function(tx, results) {
var rows = results.rows;
for (var index = 0; index rows.length; index++) {
var item = rows.item(index);
var element = document.createElement(div);
element.textContent = item.name;
document.getElementById(kidList).appendChild(element);
}
});
});
// Store kids into database
var kids = [
{ name: Anna },
{ name: Betty },
{ name: Christine },
];
var db = window.openDatabase(CandyDB, 1, My candy store database,
1024);
db.transaction(function(tx) {
for (var index = 0; index kids.length; index++) {
var kid = kids[index];
tx.executeSql(INSERT INTO kids (name) VALUES (:name);, [kid],
function(tx, results) {
document.getElementById(display).textContent =
Saved record for + kid.name + with id + results.insertId;
});
}
});
// List kids who bought candy, and the number of purchases they made
var db = window.openDatabase(CandyDB, 1, My candy store database,
1024);
db.readTransaction(function(tx) {
tx.executeSql(SELECT name, count +
FROM kids +
INNER JOIN candySales +
ON kids.id = candySales.kidId;,
function(tx, results) {
var rows = results.rows;
for (var index = 0; index rows.length; index++) {
var item = rows.item(index);
display.textContent += , + item.name + bought +
item.count + pieces;
}
});
});
// List kids who bought candy, and the number of purchases they made
// (some may have bought 0)
var db = window.openDatabase(CandyDB, 1, My candy store database,
1024);
db.readTransaction(function(tx) {
tx.executeSql(SELECT name, count +
FROM kids +
LEFT JOIN candySales +
ON kids.id = candySales.kidId;,
function(tx, results) {
var rows = results.rows;
for (var index = 0; index rows.length; index++) {
var item = rows.item(index);
display.textContent += , + item.name + bought +
item.count + pieces;
}
});
});
Cheers,
Shawn
smime.p7s
Description: S/MIME Cryptographic Signature
Re: [IndexedDB] Proposal for async API changes
On 5/19/2010 1:50 PM, Shawn Wilsher wrote: Er, and I managed to botch the SQL in the last two examples. Those should be (if my non-tested SQL-fu is right): SELECT name, COUNT(kids.id) FROM kids INNER JOIN candySales ON kids.id = candySales.kidId GROUP BY kids.id; and: SELECT name, COUNT(kids.id) FROM kids LEFT JOIN candySales ON kids.id = candySales.kidId GROUP BY kids.id; respectively. Cheers, Shawn smime.p7s Description: S/MIME Cryptographic Signature
Re: [IndexedDB] Proposal for async API changes
On 5/18/2010 7:20 AM, Jeremy Orlow wrote:
1. Once a database has been opened (a database connection has been
established) read access to meta-data, such as objectStore and index
names, is synchronous. Changes to such meta data, such as creating
objectStores and indexes, is still asynchronous.
I believe this is already how it's specced. The IDBDatabase interface
already gives you synchronous access to all of this.
Mostly that is the same, with the exception of getting an object store
or index which is now synchronous.
9. IDBKeyRanges are created using functions on IndexedDatabaseRequest.
We couldn't figure out how the old API allowed you to create a range
object without first having a range object.
In the spec, I see the following in examples:
var range = new IDBKeyRange.bound(2, 4);
and
var range = IDBKeyRange.leftBound(key);
I'm not particularly happy with hanging functions off of
IndexedDatabaseRequest for this. Can it work something like what I listed
above? If not, maybe we can find a better place to put them? Or just
create multiple openCursor functions for each case?
I think one concern with the above syntax is that it's adding another
object to the global scope. I recall Ben Turner and I discussing the
possibility of hanging it off of indexedDB, so:
var range = new indexedDB.KeyRange.bound(2, 4);
My concern with making multiple openCursor functions is that there is
more API complexity there. With that said, I don't have any strong
opinions here either way.
10. You are allowed to have multiple transactions per database
connection. However if they use overlapping tables, only the first one
will receive events until it is finished (with the usual exceptions of
allowing multiple readers of the same table).
Can you please clarify what you mean here? This seems like simply an
implementation detail to me, so maybe I'm missing something?
What this is trying to say is that you can have an object store being
used in more than one transaction, but they cannot access it at the same
time. However, I think it's best for Jonas to chime in here because
this doesn't quite seem right to me like it did yesterday.
5) You have two IDBTransactionRequest.onaborts. I think one is supposed to
be an ontimeout.
Whoops, I thought we fixed that before he sent this out :)
6) What is the default limit for the getAll functions? Should we make it 0
(and define any=0 amount to mean infinity)?
I believe we intended the default to be infinity (as in, if you don't
specify, you get it all).
7) I expect add or modify to be more used than the add or the modify
methods. As such, I wonder if it makes sense to optimize the naming for
that. For example, addOrModify=set, add=add/insert,
modify=modify/update/replace maybe?
We had a lot of internal debate over this very thing. The problem we
kept running into was that set could easily be read as doing just
updating so it wouldn't be clear that you can also insert a new record
there (without reading the spec or some tutorial). The benefit with
addOrModify is that it is very explicit in what it does. We don't like
how long it is, but we couldn't come up with a shorter name that doesn't
have a fair amount of ambiguity.
8) We can't leave deciding whether a cursor is pre-loaded up to UAs
since people will code for their favorite UA and then access
IDBCursorPreloadedRequest.count when some other UA does it as a
non-preloaded request. Even in the same UA this will cause problems when
users have different datasets than developers are testing with.
I think that you might have been confused by our wording there. Sorry
about that! IDBCursorPreloadedRequest is what you get if you pass
sync=true into openCursor or openObjectCursor. Basically, sync cursors
will give you a count, whereas async ones will not.
interface IBDCursorRequest : IDBCursor {
readonly attribute any key;
readonly attribute any value;
*readonly attribute unsigned long long estimatedCount;*
* // Returns null if the cursor was updated synchronously. Otherwise*
* // will return an IDBRequest object whose result will be set to this*
* // cursor on success. Until onsuccess is called, any access of key*
* // or value will raise an exception. The first request MUST cause*
* // an asynchronous request but the behavior of subsequent calls is*
* // up to the UA.*
* IDBRequest continue(in optional any key /* null */);*
// Success fires IDBTransactionEvent, result == key
IDBRequest update(in any value);
// Success fires IDBTransactionEvent, result == null
IDBRequest remove();
};
I'm not super happy with this interface, but I think it's a lot better for a
few reasons:
1) It's not all or nothing. Even if a result can't be 100% loaded into
memory, it doesn't mean that we'll have to deal with the overhead of every
fetch causing an asynchronous firing of an event.
2) There's an estimated count even if it's not pre-loaded (which has been
Re: [IndexedDB] What happens when the version changes?
On 5/13/2010 7:51 AM, Nikunj Mehta wrote: If you search archives you will find a discussion on versioning and that we gave up on doing version management inside the browser and instead leave it to applications to do their own versioning and upgrades. Right, I'm not saying we should manage it, but I want to make sure we don't end up making it very easy for apps to break themselves. For example: 1) Site A has two different tabs (T1 and T2) open that were loaded such that one got a script (T1) with a newer indexedDB version than the other (T2). 2) T1 upgrades the database in a way that T2 now gets a constraint violation on every operation (or some other error due to the database changing). This could easily happen any time a site changes the version number on their database. As the spec is written right now, there is no way for a site to know when the version changes without reopening the database since the version property is a sync getter, implementations would have to load it on open and cache it, or come up with some way to update all open databases (not so fun). Cheers, Shawn smime.p7s Description: S/MIME Cryptographic Signature
[IndexedDB] What happens when the version changes?
Hey all, A recent concern that we have come across at Mozilla is what happens when the version changes? Do we silently continue to work and hope for the best? Do we throw an error every time saying that the version this database was opened with is no longer the version of the database? It's not at all clear on what we should be doing in the spec, so we'd love to hear thoughts on this. (We don't have a solution we are happy with yet to this problem either, so other options would be great). Cheers, Shawn smime.p7s Description: S/MIME Cryptographic Signature
Re: [IndexedDB] Changing the default overwrite behavior of Put
On 5/10/2010 1:33 AM, Jeremy Orlow wrote: Either sounds fine to me. Please open something in the bug tracker? Filed bug 9698 [1] on changing noOverwrite to overwrite. I'm going to wait to file a bug about changing the default until we get more feedback. Cheers, Shawn [1] http://www.w3.org/Bugs/Public/show_bug.cgi?id=9698 smime.p7s Description: S/MIME Cryptographic Signature
Re: [IndexedDB] Changing the default overwrite behavior of Put
On 5/10/2010 10:36 AM, Kris Zyp wrote: I believe there are three useful modes: overwrite: false - Must create a new record overwrite: true - Must overwrite/update an existing record (something else) - Create a new record or overwrite/update an existing (depending on the key of course). FWIW, right now the last two cases are the same (at least, how I'm reading the spec). Cheers, Shawn smime.p7s Description: S/MIME Cryptographic Signature
[IndexedDB] Changing the default overwrite behavior of Put
Hey all, Per the current spec [1], noOverwrite defaults to false for put operations on an object store. Ben Turner and I have been discussing changing the default of put to not allow overwriting by default. We feel this is better behavior because simply omitting the flag should not result in destroying data. Putting my application developer hat on, I'd much rather have to be explicit about destroying existing data instead of having it happen on accident. We could also change the parameter to just overwrite and have it default to false. What is everyone's thoughts on this? Cheers, Shawn smime.p7s Description: S/MIME Cryptographic Signature
Re: [IndexedDB] Interaction between transactions and objects that allow multiple operations
On 5/5/2010 11:44 AM, Jeremy Orlow wrote: On the other hand, a lot of even the most basic tasks probably should be done within a transaction. But if the easiest way to do something is to just run it outside of a transaction, I'm guessing a good portion of users (including tutorial websites, people giving advice on forums, etc) will just do it that way. Mandating transactions forces users to be cognizant of them. Maybe that's a good thing... To be clear, I'm not disagreeing with that. Mozilla would really love to reduce the amount of async steps to do anything with a database. Right now, to get something from an object store, we have three steps: 1) open database; wait for callback 2) open object store; wait for callback 3) get from object store; wait for callback Adding the transaction stuff would result in four steps: 1) open database; wait for callback 2) open transaction; wait for callback 3) open object store; wait for callback 4) get from object store; wait for callback Maybe we can get rid of this additional step by giving a transaction to the consumer immediately available after the call to open, and have a property on the connection to access the current transaction. But that might complicate things more too. This is complicated by the fact that each step blocks similar operations (can only open a database when another one is not in the process of opening, same with object stores, etc). This problem goes away with my proposal in the past [1], but I'm not sure we have consensus on it (or the event based approach for that matter). Cheers, Shawn [1] http://lists.w3.org/Archives/Public/public-webapps/2010JanMar/0961.html smime.p7s Description: S/MIME Cryptographic Signature
Re: [IndexedDB] Interaction between transactions and objects that allow multiple operations
On 5/5/2010 1:09 PM, Jeremy Orlow wrote: I'd also worry that if creating the transaction were completely transparent to the user that they might not think to close it either. (I'm mainly thinking about copy-and-paste coders here.) I should have been more clear. That statement goes along with the suggestion to make everything work off of a transaction - object stores, indexes, cursors, etc. They'd have to know about the transaction because they'd have to use it. Cheers, Shawn smime.p7s Description: S/MIME Cryptographic Signature
Re: [IndexedDB] Granting storage quotas
On 4/29/2010 1:08 PM, Tab Atkins Jr. wrote: When you say per site do you mean per subdomain, or per domain? The former is too permissive, the latter is too restrictive. I believe he means per origin. At least that's what I took from our discussion. Cheers, Shawn smime.p7s Description: S/MIME Cryptographic Signature
Re: [IndexedDB] Granting storage quotas
On 4/28/2010 2:54 PM, Dumitru Daniliuc wrote: shawn, did you have a chance to give this some thought? how would mozilla like to handle cases like the ones jeremy and robin mentioned? how would you like to manage quotas? We chatted yesterday, but I haven't had a chance to get it down into writing. Sorry about the delay, but I've had some other things come up. Should be able to get it down in writing to this list this week. Cheers, Shawn smime.p7s Description: S/MIME Cryptographic Signature
Re: [IndexedDB] Granting storage quotas
On 4/23/2010 7:39 AM, Nikunj Mehta wrote: Could we create an additional optional parameter for an open request with the type of permanence required? Or is it not a good idea? I haven't talked to anyone at Mozilla that thinks that having permanent and non-permanent-but-possibly-long-lasting data to be a good idea. There does seem to be support for a session-only version of indexedDB. Cheers, Shawn smime.p7s Description: S/MIME Cryptographic Signature
Re: [IndexedDB] Explaining Asynchronous event-style interface
On 3/30/2010 1:37 PM, Shawn Wilsher wrote: Having open return an IDBRequest and not having the request property on IndexedDatabaseRequest would solve this problem. I'm going to prototype an implementation in Mozilla this way to see if any issues come up, but I don't currently foresee any. Does anybody object to having all the asynchronous methods return an IDBRequest instead of having to access the property on the parent object? This change greatly reduces the code complexity (Ben Turner has converted the sample code to it and it's much easier to read/follow). If not, I'll file a bug to have the spec updated. Cheers, Shawn smime.p7s Description: S/MIME Cryptographic Signature
Re: [IndexedDB] Granting storage quotas
On 4/20/2010 4:11 AM, Mark Seaborn wrote: 1) It doesn't allow a web app to ask for a storage allocation up front, before it starts to consume the storage. Why does that matter? 2) In Opera, the quota can only be increased in multiples of about 15, so it takes three prompts to get up into the range of gigabytes. But there is an unlimited option, yeah? 3) The web app can't choose when the question is put to the user. 4) The web app doesn't know how much storage has been allocated, so it doesn't know when a question will be asked. 5) In Opera, if the user chooses Reject, they don't get prompted again. This means that asking the user at an appropriate time is important for the continued functioning of the web app. Prompting the user at the wrong time will interrupt them with a page-modal dialog which they might want to get rid of with Reject, which would potentially break the web app by leaving it unable to get more storage. These all feel like user-agent specific worries on how the user agent wants to bring this to the attention of the user. Cheers, Shawn smime.p7s Description: S/MIME Cryptographic Signature
Re: [IndexedDB] Granting storage quotas
On 4/19/2010 10:08 PM, Jeremy Orlow wrote: On Tue, Apr 13, 2010 at 3:09 AM, Mark Seabornmseab...@chromium.org wrote: 2) It is too permissive because it enforces no limit on the amount of space a web app can use: A web app from example.com can create an unlimited number of puppet subdomains: aaa.example.com, bbb.example.com, etc. It can use aaa.example.com's 5Mb allocation by loading a script from aaa.example.com in an iframe and communicating with it using postMessage(). As far as I'm aware, no one (including Chromium) has a solution to at the moment. Which probably should be a cause for concern. :-) In the thread you linked to earlier, it was suggested that user agents could also factor in the top level domain into the cap (e.g. example.com only gets 50MB of space). But really, I don't see how any of this is testable in a deterministic matter, so I don't see the point in adding it to the spec. Cheers, Shawn smime.p7s Description: S/MIME Cryptographic Signature
Re: [IndexedDB] Granting storage quotas
On 4/13/2010 8:53 AM, João Eiras wrote: Not really. The user agent can ask for quota from the user when the limit is being hit without the webpage even having to worry about it. Opera 10.50 does that. I agree with this, and do see the benefit of adding more to the spec. Cheers, Shawn smime.p7s Description: S/MIME Cryptographic Signature
Re: [IndexedDB] Exceptions on IndexedDatabaseRequest.open
On 3/31/2010 6:42 AM, Jeremy Orlow wrote: I don't see any reason to special case this one method with synchronous properties. If the name parameter is not valid or any other error, I think it should be returned via an error callback just like everywhere else. I agree. Cheers, Shawn smime.p7s Description: S/MIME Cryptographic Signature
Re: [IndexedDB] Explaining Asynchronous event-style interface
On 3/22/2010 10:49 AM, Shawn Wilsher wrote: On 3/13/2010 1:43 AM, Nikunj Mehta wrote: As specced, it is possible to have multiple concurrent requests at various API entry points, except for the IndexedDatabaseRequest interface. In this particular case, you can only have one request to open a database in a Window object. Given that this is setup kind of work, it does not appear that this limitation amounts to much. If multiple connections to a single DB are required, or if different DBs have to be opened, that would have to be done sequentially, i.e., start a new request only after the previous one completed. This seems like an artificial constraint to me. Is there any reason why we couldn't open up more than one database connection at a time (or at least allow the possibility for a UA to do so). In fact, I realized a use case today that makes this constraint difficult to work around. Say you have two different libraries that use indexedDB. You'll now have to be careful when you initialize each library so they don't try to open a database at the same time. Even then, if they open up databases on the fly, you'd have to be very careful to make sure they don't stomp on each other. Having open return an IDBRequest and not having the request property on IndexedDatabaseRequest would solve this problem. I'm going to prototype an implementation in Mozilla this way to see if any issues come up, but I don't currently foresee any. Cheers, Shawn smime.p7s Description: S/MIME Cryptographic Signature
[IndexedDB] Opening a database with a different description
Hey all, The spec is unspecified as to what we should do when a database is opened with a different description than it was previously opened. I'd assume we'd want to update it, but maybe folks have other ideas. Cheers, Shawn smime.p7s Description: S/MIME Cryptographic Signature
Re: [IndexedDB] Detailed comments for the current draft
On 1/31/2010 11:33 PM, Nikunj Mehta wrote: a. 3.1.3: do we really need in-line + out-of-line keys? Besides the concept-count increase, we wonder whether out-of-line keys would cause trouble to generic libraries, as the values for the keys wouldn't be part of the values iterated when doing a foreach over the table. Certainly it is a matter of prioritizing among various requirements. Out-of-line keys enable people to store simple persistent hash maps. I think it would be wrong to require that data be always stored as objects. A library can always elide the availability of out-of-line keys if that poses a problem to its users. What about just supporting out-of-line keys? If somebody wanted the key to be part of what they were iterating, they could still have the object contain the key. With that said, wouldn't a persistent hash map be better done in local or session storage? I really think we should drop one of these concepts. I don't presently have a strong opinion on which. Cheers, Shawn smime.p7s Description: S/MIME Cryptographic Signature
Re: [IndexedDB] IDBRequest Interface Questions
On 3/22/2010 10:05 AM, Jeremy Orlow wrote: I also can't seem to figure out what the success event is supposed to be for just about anything. Am I just missing something, or is this not yet specified? When onsuccess fires, you can then start the next request. Sorry, I guess I'm not being totally clear in what I'm asking. In the Events [1] section, only IDBDatabaseError is specified. I'm not sure what a success message should look like, and that's what I'm asking to be clarified. Do you have anything to say regarding a callback based API vs. event one that inherits from EventTarget? By callback you mean passing a callback function into the API calls, correct? Assuming that, I prefer the event based API, but I think the current spec could you some modifications to address some concerns that were brought up. As I recall, the issue with the current event-based model is that you can only have one request in flight at a time. I think we could solve this problem by having the asynchronous methods return an IDBRequest object instead of having a global one for a given context (not completely sure how to describe all the places where we have a request attribute but that is what I am talking about). I'm not 100% sure yet if we'd need the request attribute on all of these things after doing this, or what it would be in each instance. Cheers, Shawn smime.p7s Description: S/MIME Cryptographic Signature
[IndexedDB] IDBRequest Interface Questions
Hey all, I'm starting to work on a prototype of the IndexedDB spec to get a better understanding of it. While working with the IDBRequest I think I understand why nobody likes the current event-based model. I'm also seeing that as it is currently specified, it doesn't meet the requirements of the web developers we had talked to were looking for in an event based API (per the latest editors draft on March 16). Essentially, what they wanted was a way to add more than one listener for success or error, and it doesn't look like that is currently possible unless you roll your own callback that manages this (which they wanted to avoid doing). As they described it, they wanted to be able to use something like addEventListener with the option of also just setting onsuccess and onerror (much like how XHR works). I also can't seem to figure out what the success event is supposed to be for just about anything. Am I just missing something, or is this not yet specified? Cheers, Shawn smime.p7s Description: S/MIME Cryptographic Signature
Re: [IndexedDB] Promises (WAS: Seeking pre-LCWD comments for Indexed Database API; deadline February 2)
On 3/5/2010 4:54 AM, Jeremy Orlow wrote: For what it's worth, regardless of the answers to the above questions, I think we should switch to a callback based model. It's great to use events when natural to do so, but this is a very unnatural use. It provides artificial limitations (only one request in flight at a time, per request object). It's ugly and confusing syntax wise (hard to keep track of which request object is associated with which request method, requires multiple statements to do each request, requires the handlers to be placed prior to the actual call...which is why the async example in http://www.w3.org/TR/IndexedDB/#introduction is so difficult to read, etc). And there really isn't any precedent (that I'm aware of) for using events like this. And the web developers I've spoken to have all been confused by the async API. For what it is worth, all the web developers we've talked to have pushed for an event based API, which is why we've been pushing for it. This happened with the file reader API as I understand it (Jonas or Arun would be able to say more). Note that we didn't show them this exact API. Cheers, Shawn smime.p7s Description: S/MIME Cryptographic Signature
Re: [WebSimpleDB] Allowing schema operations anywhere
On 11/22/09 3:14 PM, Pablo Castro wrote: Does this seem reasonable? This seems reasonable to me. The use case makes sense, and is something I've seen many times in real life. Cheers, Shawn
Re: Web Storage SQL
On 4/10/09 1:53 PM, Maciej Stachowiak wrote: I don't think this one point should be decisive by itself. But I don't think it should be given zero weight either. I do think the existence of an implementation of the current spec and Web content using it somewhat raises the burden of proof on anyone proposing a redesign. I'm not so sure of the usefulness of that argument. Just because company X implements and proposes feature Y, and convinces company Z to implement it on their site, it doesn't somehow get extra credibility (strictly a hypothetical situation). If there are serious issues with the proposal, they should be looked at and not glossed over. For what I've seen of this discussion, serious issues that are being raised are being glossed over or being dismissed as essentially not our problem. Note that one of the clients in question is the offline-enabled mobile version of GMail. I think this demonstrates that the SQL-based Database Storage can serve the needs of an advanced and polished Web application. In addition, we have a rough demonstration that it is practically implementable in a modern Web engine. Just because an API serves the needs of an advanced and polished web application doesn't mean it's the right API. We could let web authors write some form of assembly to do task X, and they could make it serve their needs, but it most certainly would not be the right API. An exaggeration, sure, but this is an API we are going to have to live with for a long time. We should take care to ensure that it's the right API for the web. My biggest problem with this draft is that I still don't see an answer for why SQL was chosen, especially given the drawbacks that have already been highlighted. Cheers, Shawn begin:vcard fn:Shawn Wilsher n:Wilsher;Shawn email;internet:sdwi...@mozilla.com title:Platform Engineer version:2.1 end:vcard
