Re: IndexedDB, what were the issues? How do we stop it from happening again?
On Mon, Mar 18, 2013 at 5:42 PM, Alec Flett alecfl...@google.com wrote:
transactions - Also should be optional. Vital to complex apps, but totally
not necessary for many.. there should be a default transaction, like
db.objectStore(foo).get(bar)
I disagree. This would have made it too trivial to create pages that
have race conditions. I.e. people would write code like:
db.objectStore(foo).get(bar).onsuccess = function(e) {
db.objectStore(foo).set(bar, e.target.result + 1)
}
without realizing that that contains a race condition.
Its always possible for users to hang themselves - the web platform has lots
of rope. I could name a dozen gotchas like that in the JavaScript language
alone. The fact that we introduced shared workers introduces a whole mess of
issues like that. Not to criticize either - I think its just something that
happens as you introduce more flexible capabilities into the platform.
In the above example, you could approach this with automatic transactions -
all operations that run in the callback of another IDB operation run in the
same transaction. So the set() and the get() are in the same transaction.
That stops working pretty quickly. If you simply want to add property
'a' to property 'b' you'd get something like:
var a;
db.objectStore(foo).get(a).onsuccess = function(e) {
a = e.target.result;
}
db.objectStore(foo).get(b).onsuccess = function(e) {
db.objectStore(foo).set(b, e.target.result + a);
}
And yes, it's easy to write that code correctly, but I don't think it
passes the following design goal:
* Make it easy to create pages that are race-free even if opened in
multiple tabs at the same time. Ideally it should be easier to create
a race-free page than a page that has race hazards.
When you need explicit transactional control then you use the transaction()
API.
My point is that it's very common to want transactions, even when you
don't think you want to.
I'd much rather spend effort on making it so easy to create
transactions that it's something people don't mind doing, than to
create a transaction-less syntax.
If we enable the use of a default objectStore then your example code
would become
var trans = db.transaction();
trans.get(bar).onsuccess = function(e) {
trans.set(bar, e.target.result + 1)
}
And my example code
var a;
trans = db.transaction();
trans.get(a).onsuccess = function(e) {
a = e.target.result;
}
trans.get(b).onsuccess = function(e) {
trans.set(b, e.target.result + a);
}
Which in both cases is just one additional line of code, but fewer
total number of characters typed. I think that's a win compared to
something that will result in more race conditions.
named object stores - frankly, for many use cases, a single objectStore
is
all you need. a simple db.get(foo) would be sufficient. Simply naming
a
default isn't bad - whats bad is all the onupgradeneeded scaffolding
required to create the objectstore in the first place.
I think we should do this as part of a simple API. Similar to something
like
https://github.com/slightlyoff/async-local-storage
Yes! I mean that's kind of where this conversation took off... I just don't
think there should be an obvious distinction between the API with the
transactions and versions and the one without. - if anything presenting
them in a unified fashion allows for developers to migrate as they need
individual features (Transactions, versions, etc)
I definitely agree that it would be cool if indexedDB could have a
smooth transition curve from a simple API up to the full feature set
that it currently has. That's definitely something that I think we
failed on.
But I don't think that requires that we get rid of transactions from
the simple API. And I suspect that that doesn't need to meaningfully
need to make the simple API that much more complicated.
/ Jonas
Re: IndexedDB, what were the issues? How do we stop it from happening again?
One option that will probably never happen: Stop making jank-inducing API's available on the main thread. Do them all as synchronous APIs only, and available only to workers. Your APIs stay simple, and you don't have to make double the surface area. Building a library that marshals sync APIs to the main thread over async MessagePorts is tractable, and may the best implementation win. With enough standardization across APIs you may even be able to make a generalized broker. Rob. On Thu, Mar 14, 2013 at 6:58 PM, Tab Atkins Jr. jackalm...@gmail.comwrote: On Thu, Mar 14, 2013 at 6:36 PM, Glenn Maynard gl...@zewt.org wrote: On Thu, Mar 14, 2013 at 1:54 PM, Alex Russell slightly...@google.com wrote: I don't understand why that's true. Workers have a message-oriented API that's inherently async. They can get back to their caller whenevs. What's the motivator for needing this? Being able to write synchronous code is one of the basic uses for Workers in the first place. Synchronously creating streams is useful in the same way that other synchronous APIs are useful, such as FileReaderSync. That doesn't necessarily mean having a synchronous API for a complex interface like this is the ideal approach (there are other ways to do it), but that's the end goal. Yes, this seems to be missing the point of Workers entirely. If all you have are async apis, you don't need Workers in the first place, as you can just use them in the main thread without jank. Workers exist explicitly to allow you to do expensive synchronous stuff without janking the main thread. (Often, the expensive synchronous stuff will just be a bunch of calculations, so you don't have to explicitly break it up into setTimeout-able chunks.) The entire reason for most async (all?) APIs is thus irrelevant in a Worker, and it may be a good idea to provide sync versions, or do something else that negates the annoyance of dealing with async code. (FYI, the messaging in Workers isn't inherently async; it just happens to only have an async interface. There's been discussion about adding a synchronous interface to messaging.) Specifically, this was for workers to be able to synchronously wait for messages from their sub-workers. Again, the whole point for async worker messaging is to prevent the main thread from janking, which is irrelevant inside of a worker. ~TJ
Re: IndexedDB, what were the issues? How do we stop it from happening again?
Sorry in advance for the long reply here..
TL;DR is:
1) I still don't think transactions are a requirement
2) I think fixing the platform's motley crew of async apis, and giving
developers better control over transaction commits (When you do use them)
is probably most important.
On Tue, Mar 19, 2013 at 1:52 PM, Jonas Sicking jo...@sicking.cc wrote:
var a;
trans = db.transaction();
trans.get(a).onsuccess = function(e) {
a = e.target.result;
}
trans.get(b).onsuccess = function(e) {
trans.set(b, e.target.result + a);
}
I'll be honest: as a web developer who has been bitten numerous times with
coordinating setTimeout and XHR callbacks, I'd never trust that the above
worked at all, even with explicit transactions.
I just don't think you'd ever write code like that because you're relying
with the subtleties of how scoping works in JS, in addition to the order
guarantees of IndexedDB.
you'd write this instead:
db.objectStore(foo).get(a).onsuccess = function(e) {
var a = e.target.result;
db.objectStore(foo).get(b).onsuccess = function(e) {
db.objectStore(foo).set(b, e.target.result + a);
}
}
But this is still a degenerate contrived example that I just don't believe
is representative of real-world code. We'd be optimizing for a fairly
specific pattern and I think people are far more often bit by
auto-committing transactions than they are by races like this. If anything,
XHR and setTimeout (and all the new-fangled HTML5 APIs) have taught people
to be careful about races in async apis.
But I don't think that requires that we get rid of transactions from
the simple API. And I suspect that that doesn't need to meaningfully
need to make the simple API that much more complicated.
It seems like there are two kinds of races that we're talking about here:
database races (i.e. read/writes not being atomic, the A in ACID) and
event races (i.e. any two arbitrary operations not having guaranteed order,
the I in ACID) - I think the latter is often solved with a better
asynchronous API abstraction like Futures/Promises - i.e. an async pattern
that lets you be explicit about your ordering rather than relying on a
containing abstraction like transactions.
I mean imagine your same identical code with XHR (drastically simplified,
but hopefully you get the idea)
xhr1.open(/url?key=key1);
xhr1.onsuccess = function(e) {
a = xhr1.responseText;
}
xhr2.open(/url?key=key2);
xhr2.onsuccess = function(e) {
xhr3.open(/update?first= + a + second= + xhr2.responseText);
}
in the context of XHR, it's now obvious to everyone watching that this is a
race condition.. just a very crude one. I guess I'm doing this to
demonstrate why no developer worth their salt would purposefully write the
race condition that you're afraid may happen without transactions.
No other Async api has a notion of transactions to work around races due
to async responses. If that's our concern, then we should be focusing on
getting to Futures/Promises. Having transactions doesn't solve races
between async subsystems, like when using XHR + IDB together.
The following pattern going to be far more common:
var key = ...
xhr1.open(/url?key= + key);
xhr1.onsuccess = function(e) {
var xhrValue = xhr1.responseText;
indexedDB.get(key).onsuccess = function(e) {
if (keyValue.value != e.target.result) {
// update my cache...
}
}
}
but ultimately this is still ugly because you're serializing your
operations and it's complicated to write code that runs them both in
parallel and only compares them when both callbacks have fired. (Nevermind
the fact that if we were dealing with our current auto-committing
transactions, any open transaction would have committed while we were
waiting for the XHR response)
but with futures/promises and nice libraries like q.js you can imagine
stuff like:
Q.all([
xhr1.open(/get?key= + key)
indexedDB.get(key)
])
.spread(function(responseText, idbValue) {
if (responseText != idbValue) {
// update my cache...
}
});
Bam. Ordering races are gone.
Alec
Re: IndexedDB, what were the issues? How do we stop it from happening again?
On Tue, Mar 19, 2013 at 2:25 PM, Robert Ginda rgi...@chromium.org wrote: One option that will probably never happen: Stop making jank-inducing API's available on the main thread. I think we're already doing this. As far as I know all browser vendors are on board with the idea of not having APIs which run on the main thread and which are slow enough that they can introduce jank. Including anything that does IO, but also things like encryption and image decoding/encoding. Unfortunately we already have a pretty large debt in form of janky APIs :( Do them all as synchronous APIs only, and available only to workers. This part is more controversial I think. Forcing people to use workers even in simple cases might not be popular. There's also the issue that synchronous APIs generally have worse performance. But it would need to be measured. / Jonas / Jonas
Re: IndexedDB, what were the issues? How do we stop it from happening again?
On Tue, Mar 19, 2013 at 3:25 PM, Alec Flett alecfl...@google.com wrote:
Sorry in advance for the long reply here..
TL;DR is:
1) I still don't think transactions are a requirement
2) I think fixing the platform's motley crew of async apis, and giving
developers better control over transaction commits (When you do use them) is
probably most important.
I agree with 2, at least mostly. But I don't think that means that we
need to give up on transactions. But I suspect we need to agree to
disagree on the meta-point and see if we can reach agreement on the
API level instead.
On Tue, Mar 19, 2013 at 1:52 PM, Jonas Sicking jo...@sicking.cc wrote:
var a;
trans = db.transaction();
trans.get(a).onsuccess = function(e) {
a = e.target.result;
}
trans.get(b).onsuccess = function(e) {
trans.set(b, e.target.result + a);
}
I'll be honest: as a web developer who has been bitten numerous times with
coordinating setTimeout and XHR callbacks, I'd never trust that the above
worked at all, even with explicit transactions.
If we don't trust implementations to follow the specs, then I think
we're hosed no matter what :(
I just don't think you'd ever write code like that because you're relying
with the subtleties of how scoping works in JS, in addition to the order
guarantees of IndexedDB.
you'd write this instead:
db.objectStore(foo).get(a).onsuccess = function(e) {
var a = e.target.result;
db.objectStore(foo).get(b).onsuccess = function(e) {
db.objectStore(foo).set(b, e.target.result + a);
}
}
But this is still a degenerate contrived example that I just don't believe
is representative of real-world code. We'd be optimizing for a fairly
specific pattern and I think people are far more often bit by
auto-committing transactions than they are by races like this. If anything,
XHR and setTimeout (and all the new-fangled HTML5 APIs) have taught people
to be careful about races in async apis.
I agree it's somewhat of an edge case. But my point is that it's very
easy and subtle to slide from the ok code patterns to the racy
code patterns.
But like I said, I think the explicit transaction doesn't need to be
meaningfully more complex than the implicit one.
But I don't think that requires that we get rid of transactions from
the simple API. And I suspect that that doesn't need to meaningfully
need to make the simple API that much more complicated.
It seems like there are two kinds of races that we're talking about here:
database races (i.e. read/writes not being atomic, the A in ACID) and
event races (i.e. any two arbitrary operations not having guaranteed order,
the I in ACID) - I think the latter is often solved with a better
asynchronous API abstraction like Futures/Promises - i.e. an async pattern
that lets you be explicit about your ordering rather than relying on a
containing abstraction like transactions.
I'm generally talking about the A part.
I agree that we should try to solve the I part using promises. But
even with promises I think we should guarantee a callback order.
People will come to rely on callback order unintentionally, even if we
use Futures. And in many cases it will enable simpler code patterns.
People can choose not to rely on it if they want to. I'm all for code clarity.
Also, I'm worried that solving the I part using promises will force
us to sacrifice the A part. I hope we don't.
The following pattern going to be far more common:
var key = ...
xhr1.open(/url?key= + key);
xhr1.onsuccess = function(e) {
var xhrValue = xhr1.responseText;
indexedDB.get(key).onsuccess = function(e) {
if (keyValue.value != e.target.result) {
// update my cache...
}
}
}
but ultimately this is still ugly because you're serializing your operations
and it's complicated to write code that runs them both in parallel and only
compares them when both callbacks have fired. (Nevermind the fact that if we
were dealing with our current auto-committing transactions, any open
transaction would have committed while we were waiting for the XHR response)
but with futures/promises and nice libraries like q.js you can imagine stuff
like:
Q.all([
xhr1.open(/get?key= + key)
indexedDB.get(key)
])
.spread(function(responseText, idbValue) {
if (responseText != idbValue) {
// update my cache...
}
});
Bam. Ordering races are gone.
Like I said above, the ordering races is not what I'm trying to solve.
As far as I can tell IDB handles those just fine and I've never heard
complaints from people that are having IDB code with ordering races.
You'll note that the original complaint in this thread was that code
that *looks* like it's suffering from I races actually isn't. That's
a problem that I can live with :)
/ Jonas
Re: IndexedDB, what were the issues? How do we stop it from happening again?
transactions - Also should be optional. Vital to complex apps, but totally
not necessary for many.. there should be a default transaction, like
db.objectStore(foo).get(bar)
I disagree. This would have made it too trivial to create pages that
have race conditions. I.e. people would write code like:
db.objectStore(foo).get(bar).onsuccess = function(e) {
db.objectStore(foo).set(bar, e.target.result + 1)
}
without realizing that that contains a race condition.
Its always possible for users to hang themselves - the web platform has
lots of rope. I could name a dozen gotchas like that in the JavaScript
language alone. The fact that we introduced shared workers introduces a
whole mess of issues like that. Not to criticize either - I think its just
something that happens as you introduce more flexible capabilities into the
platform.
In the above example, you could approach this with automatic transactions -
all operations that run in the callback of another IDB operation run in the
same transaction. So the set() and the get() are in the same transaction.
When you need explicit transactional control then you use the transaction()
API.
transaction scoping - even when you do want transactions, the api is just
too verbose and repetitive for get one key from one object store -
db.transaction(foo).objectStore(foo).get(bar) - there should be
implicit (lightweight) transactions like db.objectStore(foo).get(bar)
We used to have this exact syntax, but it got everyone confused about
what how the implicit transaction actually worked.
This is surprising to me - *shrug* - I assume it was like the automatic
transactions I mentioned above.
named object stores - frankly, for many use cases, a single objectStore
is
all you need. a simple db.get(foo) would be sufficient. Simply naming a
default isn't bad - whats bad is all the onupgradeneeded scaffolding
required to create the objectstore in the first place.
I think we should do this as part of a simple API. Similar to something
like
https://github.com/slightlyoff/async-local-storage
Yes! I mean that's kind of where this conversation took off... I just don't
think there should be an obvious distinction between the API with the
transactions and versions and the one without. - if anything presenting
them in a unified fashion allows for developers to migrate as they need
individual features (Transactions, versions, etc)
Alec
Re: IndexedDB, what were the issues? How do we stop it from happening again?
On Wed, Mar 6, 2013 at 6:01 AM, Alex Russell slightly...@google.com wrote: I've avoided weighing in on this thread until I had more IDB experience. I've been wrestling with it on two fronts of late: A re-interpretation of the API based on Futures: https://github.com/slightlyoff/DOMFuture/tree/master/reworked_APIs/IndexedDB A new async LocalStorage design + p(r)olyfill that's bootstrapped on IDB: https://github.com/slightlyoff/async-local-storage While you might be right that it's unlikely that the API can be simplified, I think it's trivial to extend it in ways that make it easier to reason about and use. This thread started out with a discussion of what might be done to keep IDB's perceived mistakes from reoccurring. Here's a quick stab at both an outline of the mistakes and what can be done to avoid them: Abuse of events The current IDB design models one-time operations using events. This can make sense insofar as events can occur zero or more times in the future, but it's not a natural fit. What does it mean for oncomplete to happen more than once? Is that an error? Are onsuccess and onerror exclusive? Can they both be dispatched for an operation? The API isn't clear. Events don't lead to good design here as they don't encapsulate these concerns. Similarly, event handlers don't chain. This is natural, as they could be invoked multiple times (conceptually), but it's not a good fit for data access. It's great that IDB as async, and events are the existing DOM model for this, but IDB's IDBRequest object is calling out for a different kind of abstraction. I'll submit Futures for the job, but others might work (explicit callback, whatever) so long as they maintain chainability + async. Whether it's an abuse of events or not I guess is a matter of opinion. DOM Events have always been used in situations when the Event fired either 0 or 1 time. They've even been used in situations when an eventual success/error has been signaled. In particular the load and error events for Documents were among some of the first Events created. That said, I agree that Events are generally a better fit for situations when you have a reoccurring thing that happen. And yes, if we had had Futures when we designed IDB it might have lead to a much easier to use API. We had a hunch that that was the case when we designed the API. However there weren't then, and there still aren't, a standardized promise API. We felt then, and I still feel that way now, that it would have been a mistake to standardize a promise library as part of a database API. This is why I've been pushing for someone to step up and take on creating a standardized promise library that we can rely on for future APIs. https://twitter.com/SickingJ/status/20215262974322 Implicitness IDB is implicit in a number of places that cause confusion for folks not intimately familiar with the contract(s) that IDB expects you to enter into. First, the use of events for delivery of notifications means that sequential-looking code that you might expect to have timing issues doesn't. Why not? Because IDB operates in some vaguely async way; you can't reason at all about events that have occurred in the past (they're not values, they're points in time). You seem to be under the impression that Events are only used/intended for situations when something will happen zero or more times at some point in the future. I agree that this is the scenario when Events really shine. Especially when that something is connected to DOM Nodes. However the way they are actually used in the DOM platform is as a generic way of doing callbacks. So despite the fact that IDB uses Events, it still has quite strict requirements in which order they fire. Having a strict order of delivering results was a quite intentional design decision. I can't find anywhere in the spec that the explicit gaurantees about delivery timing are noted (http://www.w3.org/TR/IndexedDB/#async-api), https://dvcs.w3.org/hg/IndexedDB/raw-file/tip/Overview.html#steps-for-asynchronously-executing-a-request See in particular step 4 which guarantees that requests are run and deliver their result in the order they were scheduled. so one could read IDB code that registers two callbacks as having a temporal dead-zone: a space in code where something might have happened but which your code might not have a chance to hear about. I realize that in practice this isn't the case; event delivery for these is asynchronous, but the soonest timing isn't defined: end of turn? next turn? end-of-microtask? This means that it's possible to have implementations the differ on delivery timing, astonishing those who register event handlers at the wrong time. This is part DOM-ish use of events for things they're not suited to and a lack of specificity in the spec. Both can be fixed. I agree that this is something that is vague right now. The intent is that all results are delivered as new tasks.
Re: IndexedDB, what were the issues? How do we stop it from happening again?
On Wed, Mar 6, 2013 at 10:14 AM, Alec Flett alecfl...@chromium.org wrote:
My primary takeaway from both working on IDB and working with IDB for some
demo apps is that IDB has just the right amount of complexity for really
large, robust database use.. but for a welcome to noSQL in the browser it
is way too complicated.
Specifically:
versioning - The reason this exists in IDB is to guarantee a schema (read: a
fixed set of objectStores + indexes) for a given set of operations.
Versioning should be optional. And if versioning is optional, so should
opening - the only reason you need to open a database is so that you have
a handle to a versioned database. You can almost implement versioning in JS
if you really care about it...(either keep an explicit key, or auto-detect
the state of the schema) its one of those cases where 80% of versioning is
dirt simple and the complicated stuff is really about maintaining version
changes across multiply-opened windows. (i.e. one window opens an idb, the
next window opens it and changes the schema, the first window may need to
know that and be able to adapt without breaking any in-flight transactions)
Yeah, the versioning is the part of the API that I'm least happy with.
The reason we need it is because we're synchronously exposing the
schema of the database. Instead we could have made the set of object
stores and indexes entirely dynamic and created on-demand.
However that would have made it tricky to do schema which has
constraints (like the unique index-flag) or that affects how data is
stored (for example the ability to set a collation on an objectStore
or index as has been discussed).
transactions - Also should be optional. Vital to complex apps, but totally
not necessary for many.. there should be a default transaction, like
db.objectStore(foo).get(bar)
I disagree. This would have made it too trivial to create pages that
have race conditions. I.e. people would write code like:
db.objectStore(foo).get(bar).onsuccess = function(e) {
db.objectStore(foo).set(bar, e.target.result + 1)
}
without realizing that that contains a race condition.
transaction scoping - even when you do want transactions, the api is just
too verbose and repetitive for get one key from one object store -
db.transaction(foo).objectStore(foo).get(bar) - there should be
implicit (lightweight) transactions like db.objectStore(foo).get(bar)
We used to have this exact syntax, but it got everyone confused about
what how the implicit transaction actually worked.
forced versioning - when versioning is optional, it should be then possible
to change the schema during a regular transaction. Yes, this is a lot of
rope but this is actually for much more complex apps, rather than simple
ones. In particular, it's not uncommon for more complex database systems to
dynamically create indexes based on observed behavior of the API, or
observed data (i.e. when data with a particular key becomes prevalent,
generate an index for it) and then dynamically use them if present. At the
moment you have to do a manual close/open/version change to dynamically bump
up the version - effectively rendering fixed-value versions moot (i.e. the
schema for version 23 in my browser may look totally different than the
schema for version 23 in your browser) and drastically complicating all your
code (Because if you try to close/open while transactions are in flight,
they will be aborted - so you have to temporarily pause all new
transactions, wait for all in-flight transactions to finish, do a
close/open, then start running all pending/paused transactions.) This last
case MIGHT be as simple as adding db.reopen(newVersion) to the existing
spec.
Yeah, this is something that I've run into as well. It's tricky to
solve this without removing the ability to synchronously access the
list of objectStores and indexes, or without introducing race
conditions if a page is open in multiple tabs.
named object stores - frankly, for many use cases, a single objectStore is
all you need. a simple db.get(foo) would be sufficient. Simply naming a
default isn't bad - whats bad is all the onupgradeneeded scaffolding
required to create the objectstore in the first place.
I think we should do this as part of a simple API. Similar to something like
https://github.com/slightlyoff/async-local-storage
I do think that the IDBRequest model needs tweaking, and Futures seem like
the obvious direction to head in.
Yup. Though see the list of constraints in my previous email in this thread.
FWIW, the sync version of the API is more or less dead - nobody has
actually implemented it.
We're actually working on an implementation and hope to have it ready mid year.
/ Jonas
Re: IndexedDB, what were the issues? How do we stop it from happening again?
On Friday, March 15, 2013 at 4:11 AM, Jarred Nicholls wrote: On Thu, Mar 14, 2013 at 10:19 PM, Alex Russell slightly...@google.com (mailto:slightly...@google.com) wrote: On Thu, Mar 14, 2013 at 6:36 PM, Glenn Maynard gl...@zewt.org wrote: Workers exist explicitly to allow you to do expensive synchronous stuff without janking the main thread. (Often, the expensive synchronous stuff will just be a bunch of calculations, so you don't have to explicitly break it up into setTimeout-able chunks.) The entire reason for most async (all?) APIs is thus irrelevant in a Worker, and it may be a good idea to provide sync versions, or do something else that negates the annoyance of dealing with async code. My *first* approach to this annoyance would be to start adding some async primitives to the platform that don't suck so hard; e.g., Futures/Promises. +1. Libraries cover that fairly well; albeit I think we all would enjoy such things to be first-class citizens of the platform. I've seen some good looking implementations and some decent control flow libraries. I use https://github.com/caolan/async a lot in node projects. Saying that you should do something does not imply that doubling up on API surface area for a corner-case is the right solution. I agree. It may have seemed like a good and simple idea at first - well intentioned for sure - but upon reflection we have to admit it's sloppy, a greater surface area to maintain, and the antithesis of DRY. It's not what I personally would expect from a modern, quality JS api, and I'm probably not the only web dev to share that feeling. At the risk of making a blanketed statement using anecdotal evidence, I would claim that overindulgence from modern libraries in existence today has raised the expectations of web devs in how the web platform architects new apis. Node.js comes with sync and async APIs (for good reasons) and I haven't heard anyone complain that this wasn't DRY. --tobie
Re: IndexedDB, what were the issues? How do we stop it from happening again?
On Wednesday, March 6, 2013, Tobie Langel wrote: On Wednesday, March 6, 2013 at 5:51 PM, Jarred Nicholls wrote: This is an entirely different conversation though. I don't know the answer to why sync interfaces are there and expected, except that some would argue that it makes the code easier to read/write for some devs. Since this is mirrored throughout other platform APIs, I wouldn't count this as a fault in IDB specifically. Sync APIs are useful to do I/O inside of a Worker. I don't understand why that's true. Workers have a message-oriented API that's inherently async. They can get back to their caller whenevs. What's the motivator for needing this? They're also critical for data consistency in some scenarios, e.g. updating the database after a successful xhr request when a worker is about to be terminated. Unload-catching is a known bug in much o the web platform. Why would we enable it here?
Re: IndexedDB, what were the issues? How do we stop it from happening again?
On Thursday, March 14, 2013 at 7:54 PM, Alex Russell wrote: On Wednesday, March 6, 2013, Tobie Langel wrote: Sync APIs are useful to do I/O inside of a Worker. I don't understand why that's true. Workers have a message-oriented API that's inherently async. They can get back to their caller whenevs. What's the motivator for needing this? There's no need per se. Sync API are easier to handle, and given you're already out of the UI thread, blocking in that context isn't much of an issue. They're also critical for data consistency in some scenarios, e.g. updating the database after a successful xhr request when a worker is about to be terminated. Unload-catching is a known bug in much o the web platform. Why would we enable it here? Nevermind. The Web Worker termination process (now?) says scripts get aborted anyway. --tobie
Re: IndexedDB, what were the issues? How do we stop it from happening again?
On Thu, Mar 14, 2013 at 1:54 PM, Alex Russell slightly...@google.comwrote: On Wednesday, March 6, 2013, Tobie Langel wrote: On Wednesday, March 6, 2013 at 5:51 PM, Jarred Nicholls wrote: This is an entirely different conversation though. I don't know the answer to why sync interfaces are there and expected, except that some would argue that it makes the code easier to read/write for some devs. Since this is mirrored throughout other platform APIs, I wouldn't count this as a fault in IDB specifically. Sync APIs are useful to do I/O inside of a Worker. I don't understand why that's true. Workers have a message-oriented API that's inherently async. They can get back to their caller whenevs. What's the motivator for needing this? Being able to write synchronous code is one of the basic uses for Workers in the first place. Synchronously creating streams is useful in the same way that other synchronous APIs are useful, such as FileReaderSync. That doesn't necessarily mean having a synchronous API for a complex interface like this is the ideal approach (there are other ways to do it), but that's the end goal. (FYI, the messaging in Workers isn't inherently async; it just happens to only have an async interface. There's been discussion about adding a synchronous interface to messaging.) -- Glenn Maynard
Re: IndexedDB, what were the issues? How do we stop it from happening again?
On Thu, Mar 14, 2013 at 6:36 PM, Glenn Maynard gl...@zewt.org wrote: On Thu, Mar 14, 2013 at 1:54 PM, Alex Russell slightly...@google.com wrote: I don't understand why that's true. Workers have a message-oriented API that's inherently async. They can get back to their caller whenevs. What's the motivator for needing this? Being able to write synchronous code is one of the basic uses for Workers in the first place. Synchronously creating streams is useful in the same way that other synchronous APIs are useful, such as FileReaderSync. That doesn't necessarily mean having a synchronous API for a complex interface like this is the ideal approach (there are other ways to do it), but that's the end goal. Yes, this seems to be missing the point of Workers entirely. If all you have are async apis, you don't need Workers in the first place, as you can just use them in the main thread without jank. Workers exist explicitly to allow you to do expensive synchronous stuff without janking the main thread. (Often, the expensive synchronous stuff will just be a bunch of calculations, so you don't have to explicitly break it up into setTimeout-able chunks.) The entire reason for most async (all?) APIs is thus irrelevant in a Worker, and it may be a good idea to provide sync versions, or do something else that negates the annoyance of dealing with async code. (FYI, the messaging in Workers isn't inherently async; it just happens to only have an async interface. There's been discussion about adding a synchronous interface to messaging.) Specifically, this was for workers to be able to synchronously wait for messages from their sub-workers. Again, the whole point for async worker messaging is to prevent the main thread from janking, which is irrelevant inside of a worker. ~TJ
Re: IndexedDB, what were the issues? How do we stop it from happening again?
On Thursday, March 14, 2013, Tab Atkins Jr. wrote: On Thu, Mar 14, 2013 at 6:36 PM, Glenn Maynard gl...@zewt.orgjavascript:; wrote: On Thu, Mar 14, 2013 at 1:54 PM, Alex Russell slightly...@google.comjavascript:; wrote: I don't understand why that's true. Workers have a message-oriented API that's inherently async. They can get back to their caller whenevs. What's the motivator for needing this? Being able to write synchronous code is one of the basic uses for Workers in the first place. Synchronously creating streams is useful in the same way that other synchronous APIs are useful, such as FileReaderSync. That doesn't necessarily mean having a synchronous API for a complex interface like this is the ideal approach (there are other ways to do it), but that's the end goal. Yes, this seems to be missing the point of Workers entirely. If all you have are async apis, you don't need Workers in the first place, as you can just use them in the main thread without jank. Workers exist explicitly to allow you to do expensive synchronous stuff without janking the main thread. (Often, the expensive synchronous stuff will just be a bunch of calculations, so you don't have to explicitly break it up into setTimeout-able chunks.) The entire reason for most async (all?) APIs is thus irrelevant in a Worker, and it may be a good idea to provide sync versions, or do something else that negates the annoyance of dealing with async code. My *first* approach to this annoyance would be to start adding some async primitives to the platform that don't suck so hard; e.g., Futures/Promises. Saying that you should do something does not imply that doubling up on API surface area for a corner-case is the right solution. (FYI, the messaging in Workers isn't inherently async; it just happens to only have an async interface. There's been discussion about adding a synchronous interface to messaging.) Specifically, this was for workers to be able to synchronously wait for messages from their sub-workers. Again, the whole point for async worker messaging is to prevent the main thread from janking, which is irrelevant inside of a worker. ~TJ
Re: IndexedDB, what were the issues? How do we stop it from happening again?
On Thu, Mar 14, 2013 at 8:58 PM, Tab Atkins Jr. jackalm...@gmail.comwrote: The entire reason for most async (all?) APIs is thus irrelevant in a Worker, and it may be a good idea to provide sync versions, or do something else that negates the annoyance of dealing with async code. I agree, except that async APIs are also useful and relevant in workers. Sometimes you want synchronous code and sometimes you want asynchronous code, depending on what you're doing. On Thu, Mar 14, 2013 at 9:19 PM, Alex Russell slightly...@google.com wrote: My *first* approach to this annoyance would be to start adding some async primitives to the platform that don't suck so hard; e.g., Futures/Promises. Saying that you should do something does not imply that doubling up on API surface area for a corner-case is the right solution. Futures are nothing but a different async API. They're in no way comparable to synchronous code. But, as I said, it's true that a second synchronous interface isn't necessarily the best solution for complex APIs like IndexedDB. At least in this particular case, if we have a synchronous messaging API I might call the synchronous IDB interface unnecessary. -- Glenn Maynard
Re: IndexedDB, what were the issues? How do we stop it from happening again?
On Thursday, March 14, 2013, Glenn Maynard wrote:
On Thu, Mar 14, 2013 at 8:58 PM, Tab Atkins Jr.
jackalm...@gmail.comjavascript:_e({}, 'cvml', 'jackalm...@gmail.com');
wrote:
The entire reason for most async (all?) APIs is thus irrelevant in a
Worker, and it may be a good idea to provide sync versions, or do
something else that negates the annoyance of dealing with async code.
I agree, except that async APIs are also useful and relevant in workers.
Sometimes you want synchronous code and sometimes you want asynchronous
code, depending on what you're doing.
On Thu, Mar 14, 2013 at 9:19 PM, Alex Russell
slightly...@google.comjavascript:_e({}, 'cvml', 'slightly...@google.com');
wrote:
My *first* approach to this annoyance would be to start adding some async
primitives to the platform that don't suck so hard; e.g., Futures/Promises.
Saying that you should do something does not imply that doubling up on API
surface area for a corner-case is the right solution.
Futures are nothing but a different async API. They're in no way
comparable to synchronous code.
I didn't imply they were. But addressing the pain point of asynchronous
code that's hard to use doesn't imply that the only answer is a synchronous
version. This is not a particularly hard or subtle point.
But, as I said, it's true that a second synchronous interface isn't
necessarily the best solution for complex APIs like IndexedDB. At least in
this particular case, if we have a synchronous messaging API I might call
the synchronous IDB interface unnecessary.
--
Glenn Maynard
Re: IndexedDB, what were the issues? How do we stop it from happening again?
On Thu, Mar 14, 2013 at 9:41 PM, Alex Russell slightly...@google.comwrote: I didn't imply they were. But addressing the pain point of asynchronous code that's hard to use doesn't imply that the only answer is a synchronous version. The asynchronous programming model is often inherently inconvenient compared to synchronous code, and synchronous APIs (whether at the individual API level, or at another level such as synchronous messaging or yieldable coroutines) are indeed the only practical solution that's been proposed so far. I believe this is a fundamental issue, but if you have a concrete alternative proposal to make then by all means do so (in another thread). Otherwise, this just isn't helpful. This is not a particularly hard or subtle point. (Let's try to remain civil.) -- Glenn Maynard
Re: IndexedDB, what were the issues? How do we stop it from happening again?
* Alex Russell wrote: My *first* approach to this annoyance would be to start adding some async primitives to the platform that don't suck so hard; e.g., Futures/Promises. Saying that you should do something does not imply that doubling up on API surface area for a corner-case is the right solution. http://lists.w3.org/Archives/Public/www-archive/2008Jul/0009.html was my first approach. Workers of course have it much easier, they just need a single waiting primitive to make an asynchronous API synchronous. I've http://lists.w3.org/Archives/Public/public-webapps/2011OctDec/1016.html argued against duplicating APIs for workers as proposed here, but so far without much success, it would seem... -- Björn Höhrmann · mailto:bjo...@hoehrmann.de · http://bjoern.hoehrmann.de Am Badedeich 7 · Telefon: +49(0)160/4415681 · http://www.bjoernsworld.de 25899 Dagebüll · PGP Pub. KeyID: 0xA4357E78 · http://www.websitedev.de/
Re: IndexedDB, what were the issues? How do we stop it from happening again?
On Thu, Mar 14, 2013 at 10:19 PM, Alex Russell slightly...@google.comwrote: On Thursday, March 14, 2013, Tab Atkins Jr. wrote: On Thu, Mar 14, 2013 at 6:36 PM, Glenn Maynard gl...@zewt.org wrote: On Thu, Mar 14, 2013 at 1:54 PM, Alex Russell slightly...@google.com wrote: I don't understand why that's true. Workers have a message-oriented API that's inherently async. They can get back to their caller whenevs. What's the motivator for needing this? Being able to write synchronous code is one of the basic uses for Workers in the first place. Synchronously creating streams is useful in the same way that other synchronous APIs are useful, such as FileReaderSync. That doesn't necessarily mean having a synchronous API for a complex interface like this is the ideal approach (there are other ways to do it), but that's the end goal. Yes, this seems to be missing the point of Workers entirely. If all you have are async apis, you don't need Workers in the first place, as you can just use them in the main thread without jank. I wouldn't say that. Async code will eventually be scheduled to execute on the UI thread, which can cause contention with other tasks that must run on that same UI thread. Using a worker thread to perform (async) XHR requests and JSON decoding while the UI thread focused on other rendering tasks was one of the methods we (Sencha) used to increase News Feed performance for Fastbook. I agree with a lot of what you're saying re: workers, but I don't agree that they wouldn't be needed if all we had were async apis. Workers exist explicitly to allow you to do expensive synchronous stuff without janking the main thread. (Often, the expensive synchronous stuff will just be a bunch of calculations, so you don't have to explicitly break it up into setTimeout-able chunks.) The entire reason for most async (all?) APIs is thus irrelevant in a Worker, and it may be a good idea to provide sync versions, or do something else that negates the annoyance of dealing with async code. My *first* approach to this annoyance would be to start adding some async primitives to the platform that don't suck so hard; e.g., Futures/Promises. +1. Libraries cover that fairly well; albeit I think we all would enjoy such things to be first-class citizens of the platform. I've seen some good looking implementations and some decent control flow libraries. I use https://github.com/caolan/async a lot in node projects. Saying that you should do something does not imply that doubling up on API surface area for a corner-case is the right solution. I agree. It may have seemed like a good and simple idea at first - well intentioned for sure - but upon reflection we have to admit it's sloppy, a greater surface area to maintain, and the antithesis of DRY. It's not what I personally would expect from a modern, quality JS api, and I'm probably not the only web dev to share that feeling. At the risk of making a blanketed statement using anecdotal evidence, I would claim that overindulgence from modern libraries in existence today has raised the expectations of web devs in how the web platform architects new apis. (FYI, the messaging in Workers isn't inherently async; it just happens to only have an async interface. There's been discussion about adding a synchronous interface to messaging.) Specifically, this was for workers to be able to synchronously wait for messages from their sub-workers. Again, the whole point for async worker messaging is to prevent the main thread from janking, which is irrelevant inside of a worker. ~TJ
Re: IndexedDB, what were the issues? How do we stop it from happening again?
On Wednesday, March 6, 2013, Ian Fette (イアンフェッティ) wrote: I seem to recall we contemplated people writing libraries on top of IDB from the beginning. I'm not sure why this is a bad thing. It's not bad as an assumption, but it can quickly turn into an excuse for API design malpractice because it often leads to the (mistaken) assumption that user-provided code is as cheap as browser-provided API. Given that users pull their libraries from the network more often than from disk (and must parse/compile, etc.), the incentives of these two API-providers could not be more different. That's why it's critical that API designers try to forestall the need for libraries for as long as possible when it comes to web features. We originally shipped web sql / sqlite, which was a familiar interface for many and relatively easy to use, but had a sufficiently large API surface area that no one felt they wanted to document the whole thing such that we could have an inter-operable standard. (Yes, I'm simplifying a bit.) Yeah, I recall that the SQLite semantics were the big obstacle. As a result, we came up with an approach of What are the fundamental primitives that we need?, spec'd that out, and shipped it. We had discussions at the time that we expected library authors to produce abstraction layers that made IDB easier to use, as the fundamental primitives approach was not necessarily intended to produce an API that was as straightforward and easy to use as what we were trying to replace. If that's now what is happening, that seems like a good thing, not a failure. It's fine in the short run to provide just the low-level stuff and work up to the high-level things -- but only when you can't predict what the high-level needs will be. Assuming that's what the WG's view was, you're right; feature not bug, although there's now more work to do. Anyhow, IDB is incredibly high-level in many places and primitive in others. ISTM that it's not easy to get a handle on it's intended level of abstraction. On Wed, Mar 6, 2013 at 10:14 AM, Alec Flett alecfl...@chromium.orgwrote: My primary takeaway from both working on IDB and working with IDB for some demo apps is that IDB has just the right amount of complexity for really large, robust database use.. but for a welcome to noSQL in the browser it is way too complicated. Specifically: 1. *versioning* - The reason this exists in IDB is to guarantee a schema (read: a fixed set of objectStores + indexes) for a given set of operations. Versioning should be optional. And if versioning is optional, so should *opening* - the only reason you need to open a database is so that you have a handle to a versioned database. You can *almost* implement versioning in JS if you really care about it...(either keep an explicit key, or auto-detect the state of the schema) its one of those cases where 80% of versioning is dirt simple and the complicated stuff is really about maintaining version changes across multiply-opened windows. (i.e. one window opens an idb, the next window opens it and changes the schema, the first window *may* need to know that and be able to adapt without breaking any in-flight transactions) - 2. *transactions* - Also should be optional. Vital to complex apps, but totally not necessary for many.. there should be a default transaction, like db.objectStore(foo).get(bar) 3. *transaction scoping* - even when you do want transactions, the api is just too verbose and repetitive for get one key from one object store - db.transaction(foo).objectStore(foo).get(bar) - there should be implicit (lightweight) transactions like db.objectStore(foo).get(bar) 4. *forced versioning* - when versioning is optional, it should be then possible to change the schema during a regular transaction. Yes, this is a lot of rope but this is actually for much more complex apps, rather than simple ones. In particular, it's not uncommon for more complex database systems to dynamically create indexes based on observed behavior of the API, or observed data (i.e. when data with a particular key becomes prevalent, generate an index for it) and then dynamically use them if present. At the moment you have to do a manual close/open/version change to dynamically bump up the version - effectively rendering fixed-value versions moot (i.e. the schema for version 23 in my browser may look totally different than the schema for version 23 in your browser) and drastically complicating all your code (Because if you try to close/open while transactions are in flight, they will be aborted - so you have to temporarily pause all new transactions, wait for all in-flight transactions to finish, do a close/open, then start running all pending/paused transactions.) This last case MIGHT be as simple as adding db.reopen(newVersion) to the existing spec. 5.
Re: IndexedDB, what were the issues? How do we stop it from happening again?
On Thu, 07 Mar 2013 13:01:20 +0100, Alex Russell slightly...@google.com wrote: On Wednesday, March 6, 2013, Ian Fette (イアンフェッティ) wrote: I seem to recall we contemplated people writing libraries on top of IDB from the beginning. I'm not sure why this is a bad thing. It's not bad as an assumption, but it can quickly turn into an excuse for API design malpractice because it often leads to the (mistaken) assumption that user-provided code is as cheap as browser-provided API. Given that users pull their libraries from the network more often than from disk (and must parse/compile, etc.), the incentives of these two API-providers could not be more different. That's why it's critical that API designers try to forestall the need for libraries for as long as possible when it comes to web features. +1 Libraries are important in many areas, but the goal should be to have a spec which doesn't *require* it. It should be easy to understand and implement without it. I would rather learn the spec and write a few lines of code and have it run - rather than learn the spec, then learn a library, and then use that library in every required page (increasing my bandwidth costs and the costs to my users who are accessing my site on mobile, often on limited data plans). The former option should be the design goal whenever possible. Also, Alec's points were spot on. We originally shipped web sql / sqlite, which was a familiar interface for many and relatively easy to use, but had a sufficiently large API surface area that no one felt they wanted to document the whole thing such that we could have an inter-operable standard. (Yes, I'm simplifying a bit.) Yeah, I recall that the SQLite semantics were the big obstacle. As a result, we came up with an approach of What are the fundamental primitives that we need?, spec'd that out, and shipped it. We had discussions at the time that we expected library authors to produce abstraction layers that made IDB easier to use, as the fundamental primitives approach was not necessarily intended to produce an API that was as straightforward and easy to use as what we were trying to replace. If that's now what is happening, that seems like a good thing, not a failure. It's fine in the short run to provide just the low-level stuff and work up to the high-level things -- but only when you can't predict what the high-level needs will be. Assuming that's what the WG's view was, you're right; feature not bug, although there's now more work to do. Anyhow, IDB is incredibly high-level in many places and primitive in others. ISTM that it's not easy to get a handle on it's intended level of abstraction. On Wed, Mar 6, 2013 at 10:14 AM, Alec Flett alecfl...@chromium.orgwrote: My primary takeaway from both working on IDB and working with IDB for some demo apps is that IDB has just the right amount of complexity for really large, robust database use.. but for a welcome to noSQL in the browser it is way too complicated. Specifically: 1. *versioning* - The reason this exists in IDB is to guarantee a schema (read: a fixed set of objectStores + indexes) for a given set of operations. Versioning should be optional. And if versioning is optional, so should *opening* - the only reason you need to open a database is so that you have a handle to a versioned database. You can *almost* implement versioning in JS if you really care about it...(either keep an explicit key, or auto-detect the state of the schema) its one of those cases where 80% of versioning is dirt simple and the complicated stuff is really about maintaining version changes across multiply-opened windows. (i.e. one window opens an idb, the next window opens it and changes the schema, the first window *may* need to know that and be able to adapt without breaking any in-flight transactions) - 2. *transactions* - Also should be optional. Vital to complex apps, but totally not necessary for many.. there should be a default transaction, like db.objectStore(foo).get(bar) 3. *transaction scoping* - even when you do want transactions, the api is just too verbose and repetitive for get one key from one object store - db.transaction(foo).objectStore(foo).get(bar) - there should be implicit (lightweight) transactions like db.objectStore(foo).get(bar) 4. *forced versioning* - when versioning is optional, it should be then possible to change the schema during a regular transaction. Yes, this is a lot of rope but this is actually for much more complex apps, rather than simple ones. In particular, it's not uncommon for more complex database systems to dynamically create indexes based on observed behavior of the API, or observed data (i.e. when data with a particular key becomes prevalent, generate an index for it) and then dynamically use them if present. At the moment you
Re: IndexedDB, what were the issues? How do we stop it from happening again?
+1 to be able to use easily the default API without requiring third party libraries. Sent from my Android cell phone, please forgive the lack of format on the text, and my fat thumbs :-P El 07/03/2013 21:30, Shwetank Dixit shweta...@opera.com escribió: On Thu, 07 Mar 2013 13:01:20 +0100, Alex Russell slightly...@google.com wrote: On Wednesday, March 6, 2013, Ian Fette (イアンフェッティ) wrote: I seem to recall we contemplated people writing libraries on top of IDB from the beginning. I'm not sure why this is a bad thing. It's not bad as an assumption, but it can quickly turn into an excuse for API design malpractice because it often leads to the (mistaken) assumption that user-provided code is as cheap as browser-provided API. Given that users pull their libraries from the network more often than from disk (and must parse/compile, etc.), the incentives of these two API-providers could not be more different. That's why it's critical that API designers try to forestall the need for libraries for as long as possible when it comes to web features. +1 Libraries are important in many areas, but the goal should be to have a spec which doesn't *require* it. It should be easy to understand and implement without it. I would rather learn the spec and write a few lines of code and have it run - rather than learn the spec, then learn a library, and then use that library in every required page (increasing my bandwidth costs and the costs to my users who are accessing my site on mobile, often on limited data plans). The former option should be the design goal whenever possible. Also, Alec's points were spot on. We originally shipped web sql / sqlite, which was a familiar interface for many and relatively easy to use, but had a sufficiently large API surface area that no one felt they wanted to document the whole thing such that we could have an inter-operable standard. (Yes, I'm simplifying a bit.) Yeah, I recall that the SQLite semantics were the big obstacle. As a result, we came up with an approach of What are the fundamental primitives that we need?, spec'd that out, and shipped it. We had discussions at the time that we expected library authors to produce abstraction layers that made IDB easier to use, as the fundamental primitives approach was not necessarily intended to produce an API that was as straightforward and easy to use as what we were trying to replace. If that's now what is happening, that seems like a good thing, not a failure. It's fine in the short run to provide just the low-level stuff and work up to the high-level things -- but only when you can't predict what the high-level needs will be. Assuming that's what the WG's view was, you're right; feature not bug, although there's now more work to do. Anyhow, IDB is incredibly high-level in many places and primitive in others. ISTM that it's not easy to get a handle on it's intended level of abstraction. On Wed, Mar 6, 2013 at 10:14 AM, Alec Flett alecfl...@chromium.org wrote: My primary takeaway from both working on IDB and working with IDB for some demo apps is that IDB has just the right amount of complexity for really large, robust database use.. but for a welcome to noSQL in the browser it is way too complicated. Specifically: 1. *versioning* - The reason this exists in IDB is to guarantee a schema (read: a fixed set of objectStores + indexes) for a given set of operations. Versioning should be optional. And if versioning is optional, so should *opening* - the only reason you need to open a database is so that you have a handle to a versioned database. You can *almost* implement versioning in JS if you really care about it...(either keep an explicit key, or auto-detect the state of the schema) its one of those cases where 80% of versioning is dirt simple and the complicated stuff is really about maintaining version changes across multiply-opened windows. (i.e. one window opens an idb, the next window opens it and changes the schema, the first window *may* need to know that and be able to adapt without breaking any in-flight transactions) - 2. *transactions* - Also should be optional. Vital to complex apps, but totally not necessary for many.. there should be a default transaction, like db.objectStore(foo).get(**bar) 3. *transaction scoping* - even when you do want transactions, the api is just too verbose and repetitive for get one key from one object store - db.transaction(foo).**objectStore(foo).get(bar) - there should be implicit (lightweight) transactions like db.objectStore(foo).get(* *bar) 4. *forced versioning* - when versioning is optional, it should be then possible to change the schema during a regular transaction. Yes, this is a lot of rope but this is actually for much more complex apps, rather than simple ones. In particular, it's not uncommon for more
Re: IndexedDB, what were the issues? How do we stop it from happening again?
On Wed, Mar 6, 2013 at 1:02 PM, Ian Fette (イアンフェッティ) ife...@google.comwrote: I seem to recall we contemplated people writing libraries on top of IDB from the beginning. I'm not sure why this is a bad thing. Expecting libraries providing higher-level abstractions is fine, but it's bad if an API is inconvenient to use directly for common cases. For example, it's natural to expect people to use a game engine library wrapping Canvas to write a game, but Canvas itself is easy to use directly most of the time, for lots of use cases. The only API on the platform that I regularly use which I honestly find unreasonable to use without a wrapper of some kind is cookies, which is one of the worst APIs we've got. Other than that, I can't think of any web API that I actually need a wrapper for. This is very good, since it means everyone else reading my code already understands the APIs I'm using. We originally shipped web sql / sqlite, which was a familiar interface for many and relatively easy to use, but had a sufficiently large API surface area that no one felt they wanted to document the whole thing such that we could have an inter-operable standard. (Yes, I'm simplifying a bit.) (Not to get sidetracked on this, but this seems oversimplified to the point of being confusing. http://lists.w3.org/Archives/Public/public-webapps/2011AprJun/0025.html) As a result, we came up with an approach of What are the fundamental primitives that we need?, spec'd that out, and shipped it. We had discussions at the time that we expected library authors to produce abstraction layers that made IDB easier to use, as the fundamental primitives approach was not necessarily intended to produce an API that was as straightforward and easy to use as what we were trying to replace. If that's now what is happening, that seems like a good thing, not a failure. It's fine to not try to be as simple to use as localStorage. That's not an attainable goal; it's not a database in any practical sense and never tried to be. But if we've added a new API to the platform that typical developers wouldn't want to use directly without any wrapper library, we've made an error. -- Glenn Maynard
Re: IndexedDB, what were the issues? How do we stop it from happening again?
I wrote a quick overview of the issues I have had using the indexedDB API http://arandomurl.com/2013/02/21/thoughts-on-indexeddb.html Most of them are just implementation details, however I still havent met a webdev who understands the transaction model without having one of the indexeddb implementors explain it to them, and past being confusing it also makes things much harder to implement due to being tied to the event loop. and I agree with pretty much everything that Alex wrote, at least the parts that I understood. Cheers Dale On 6 March 2013 15:01, Alex Russell slightly...@google.com wrote: Comments inline. Adding some folks from the IDB team at Google to the thread as well as public-webapps. On Sunday, February 17, 2013, Miko Nieminen wrote: 2013/2/15 Shwetank Dixit shweta...@opera.com Why did you feel it was necessary to write a layer on top of IndexedDB? I think this is the main issue here. As it stands, IDB is great in terms of features and power it offers, but the feedback I recieved from other devs was that writing raw IndexedDB requires an uncomfortable amount of verbosity even for some simple tasks (This can be disputed, but that is the views I got from some of the developers I interacted with). Adding that much amount of code (once again, im talking of raw IndexedDB) makes it less readable and understandable. For beginners, this all seemed very intimidating, and for some people more experienced, it was a bit frustrating. After my experiments with IDB, I don't feel that it is particularly verbose. I have to admit that often I prefer slightly verbose syntax over shorter one when it makes reading the code easier. In IDB's case, I think this is the case. For the latter bit, I reckon it would be a good practice for groups working on low-level APIs to more or less systematically produce a library that operates at a higher level. This would not only help developers in that they could pick that up instead of the lower-level stuff, but more importantly (at least in terms of goals) it would serve to validate that the lower-level design is indeed appropriate for librarification. I think that would be a good idea. Also, people making those low level APIs should still keep in mind that the resulting code should not be too verbose or complex. Librarification should be an advantage, but not a de facto requirement for developers when it comes to such APIs. It should still be feasable for them to write code in the raw low level API without writing uncomfortably verbose or complex code for simple tasks. Spec designers of low level APIs should not take this as a license to make things so complex that only they and a few others understand it, and then hope that some others will go ahead and make it simple for the 'common folk' through an abstraction library. I quite don't see how to simplify IDB syntax much more. I've avoided weighing in on this thread until I had more IDB experience. I've been wrestling with it on two fronts of late: - A re-interpretation of the API based on Futures: https://github.com/slightlyoff/DOMFuture/tree/master/reworked_APIs/IndexedDB - A new async LocalStorage design + p(r)olyfill that's bootstrapped on IDB: https://github.com/slightlyoff/async-local-storage While you might be right that it's unlikely that the API can be simplified, I think it's trivial to extend it in ways that make it easier to reason about and use. This thread started out with a discussion of what might be done to keep IDB's perceived mistakes from reoccurring. Here's a quick stab at both an outline of the mistakes and what can be done to avoid them: - *Abuse of events* The current IDB design models one-time operations using events. This * can* make sense insofar as events can occur zero or more times in the future, but it's not a natural fit. What does it mean for oncomplete to happen more than once? Is that an error? Are onsuccess and onerror exclusive? Can they both be dispatched for an operation? The API isn't clear. Events don't lead to good design here as they don't encapsulate these concerns. Similarly, event handlers don't chain. This is natural, as they could be invoked multiple times (conceptually), but it's not a good fit for data access. It's great that IDB as async, and events are the existing DOM model for this, but IDB's IDBRequest object is calling out for a different kind of abstraction. I'll submit Futures for the job, but others might work (explicit callback, whatever) so long as they maintain chainability + async. - *Implicitness* IDB is implicit in a number of places that cause confusion for folks not intimately familiar with the contract(s) that IDB expects you to enter into. First, the use of events for delivery of notifications means that sequential-looking code that you might expect to have timing issues doesn't.
Re: IndexedDB, what were the issues? How do we stop it from happening again?
On Wed, Mar 6, 2013 at 8:01 AM, Alex Russell slightly...@google.com wrote: Comments inline. Adding some folks from the IDB team at Google to the thread as well as public-webapps. (I don't want to cold CC so many people, and anybody working on an IDB implementation should be on -webapps already, so I've trimmed the CC to that. I'm not subscribed to -tag, so a mail there would probably bounce anyway.) - *Abuse of events* The current IDB design models one-time operations using events. This * can* make sense insofar as events can occur zero or more times in the future, but it's not a natural fit. What does it mean for oncomplete to happen more than once? Is that an error? Are onsuccess and onerror exclusive? Can they both be dispatched for an operation? The API isn't clear. Events don't lead to good design here as they don't encapsulate these concerns. Similarly, event handlers don't chain. This is natural, as they could be invoked multiple times (conceptually), but it's not a good fit for data access. It's great that IDB as async, and events are the existing DOM model for this, but IDB's IDBRequest object is calling out for a different kind of abstraction. I'll submit Futures for the job, but others might work (explicit callback, whatever) so long as they maintain chainability + async. I disagree. DOM events are used this way across the entire platform. Everybody understands it, it works well, and coming up with something different can only add more complexity and inconsistency to the platform by having additional ways to model the same job. I disagree both that we need a new way of handling this, and that IDB made a mistake in using the standard mechanism in an ordinary, well-practiced way. - *Doubled API surface for sync version* I assume I just don't understand why this choice was made, but the explosion of API surface area combined with the conditional availability of this version of the API make it an odd beast (to be charitable). There's currently no other way to allow an API to be synchronous in workers but only async in the UI thread. There was some discussion about a generalized way to allow workers to block on a message from another thread, which would make it possible to implement a synchronous shim for any async API in JavaScript. In theory this could make it unnecessary for each API to have its own synchronous interface. It wouldn't be as convenient, and probably wouldn't be suitable for every API, but for big, complex interfaces like IDB it might make sense. There might also be other ways to express synchronous APIs based on their async interfaces without having a whole second interface (eg. maybe something like a method to block until an event is received). - *The idea that this is all going to be wrapped up by libraries anyway * I don't have an opinion about IDB specifically yet, but I agree that this is wrong. People have become so used to using wrappers around APIs that they've come to think of them as normal, and that we should design APIs assuming people will keep doing that. People wrap libraries when they're hard to use, and if they're hard to use then they're badly designed. Just because people wrap bad APIs isn't an excuse for designing more bad APIs. Wrappers for basic usage are always a bad thing: you always end up with lots of them, which means everyone is using different APIs. When everyone uses the provided APIs directly, we can all read each others' code and all of our code interoperates much more naturally. (As you said, this is only referring to wrappers at the same level of abstraction, of course, not libraries providing higher-level abstractions.) -- Glenn Maynard
Re: IndexedDB, what were the issues? How do we stop it from happening again?
On Wednesday, March 6, 2013, Glenn Maynard wrote:
On Wed, Mar 6, 2013 at 8:01 AM, Alex Russell
slightly...@google.comjavascript:_e({}, 'cvml', 'slightly...@google.com');
wrote:
Comments inline. Adding some folks from the IDB team at Google to the
thread as well as public-webapps.
(I don't want to cold CC so many people, and anybody working on an IDB
implementation should be on -webapps already, so I've trimmed the CC to
that. I'm not subscribed to -tag, so a mail there would probably bounce
anyway.)
- *Abuse of events*
The current IDB design models one-time operations using events. This *
can* make sense insofar as events can occur zero or more times in the
future, but it's not a natural fit. What does it mean for oncomplete to
happen more than once? Is that an error? Are onsuccess and onerror
exclusive? Can they both be dispatched for an operation? The API isn't
clear. Events don't lead to good design here as they don't encapsulate
these concerns. Similarly, event handlers don't chain. This is natural, as
they could be invoked multiple times (conceptually), but it's not a good
fit for data access. It's great that IDB as async, and events are the
existing DOM model for this, but IDB's IDBRequest object is calling out
for
a different kind of abstraction. I'll submit Futures for the job, but
others might work (explicit callback, whatever) so long as they maintain
chainability + async.
I disagree. DOM events are used this way across the entire platform.
So which part do you disagree with? That events are a bad model for a
one-time action? Or that it's not clear what the expected contract is?
Going by what you've written below, I have to assume the latter, so I'll
just say this: try sitting a non-webdev down with IDB or any other DOM API
that works this way and try to get them to figure it out from code samples.
Yes, yes, being a webdev means knowing the platform idioms, but if we can
agree they're confusing and difficult, we can start to do something about
it. And in any case, you haven't refuted the former; events are simply a
bad model here.
Everybody understands it, it works well, and coming up with something
different can only add more complexity and inconsistency to the platform by
having additional ways to model the same job. I disagree both that we need
a new way of handling this, and that IDB made a mistake in using the
standard mechanism in an ordinary, well-practiced way.
- *Doubled API surface for sync version*
I assume I just don't understand why this choice was made, but the
explosion of API surface area combined with the conditional availability
of
this version of the API make it an odd beast (to be charitable).
There's currently no other way to allow an API to be synchronous in
workers but only async in the UI thread.
Of course not...but what does that have to do with the price of fish? The
core question is what's motivating a sync API here in the first place.
I won't be responding to the rest of your message.
There was some discussion about a generalized way to allow workers to
block on a message from another thread, which would make it possible to
implement a synchronous shim for any async API in JavaScript. In theory
this could make it unnecessary for each API to have its own synchronous
interface. It wouldn't be as convenient, and probably wouldn't be suitable
for every API, but for big, complex interfaces like IDB it might make
sense. There might also be other ways to express synchronous APIs based on
their async interfaces without having a whole second interface (eg. maybe
something like a method to block until an event is received).
- *The idea that this is all going to be wrapped up by libraries
anyway*
I don't have an opinion about IDB specifically yet, but I agree that this
is wrong.
People have become so used to using wrappers around APIs that they've come
to think of them as normal, and that we should design APIs assuming people
will keep doing that.
People wrap libraries when they're hard to use, and if they're hard to use
then they're badly designed. Just because people wrap bad APIs isn't an
excuse for designing more bad APIs. Wrappers for basic usage are always a
bad thing: you always end up with lots of them, which means everyone is
using different APIs. When everyone uses the provided APIs directly, we
can all read each others' code and all of our code interoperates much more
naturally.
(As you said, this is only referring to wrappers at the same level of
abstraction, of course, not libraries providing higher-level abstractions.)
--
Glenn Maynard
Re: IndexedDB, what were the issues? How do we stop it from happening again?
On Wednesday, March 6, 2013, Glenn Maynard wrote:
On Wed, Mar 6, 2013 at 8:01 AM, Alex Russell
slightly...@google.comjavascript:_e({}, 'cvml', 'slightly...@google.com');
wrote:
Comments inline. Adding some folks from the IDB team at Google to the
thread as well as public-webapps.
(I don't want to cold CC so many people, and anybody working on an IDB
implementation should be on -webapps already, so I've trimmed the CC to
that. I'm not subscribed to -tag, so a mail there would probably bounce
anyway.)
- *Abuse of events*
The current IDB design models one-time operations using events. This *
can* make sense insofar as events can occur zero or more times in the
future, but it's not a natural fit. What does it mean for oncomplete to
happen more than once? Is that an error? Are onsuccess and onerror
exclusive? Can they both be dispatched for an operation? The API isn't
clear. Events don't lead to good design here as they don't encapsulate
these concerns. Similarly, event handlers don't chain. This is natural, as
they could be invoked multiple times (conceptually), but it's not a good
fit for data access. It's great that IDB as async, and events are the
existing DOM model for this, but IDB's IDBRequest object is calling out
for
a different kind of abstraction. I'll submit Futures for the job, but
others might work (explicit callback, whatever) so long as they maintain
chainability + async.
I disagree. DOM events are used this way across the entire platform.
Everybody understands it, it works well, and coming up with something
different can only add more complexity and inconsistency to the platform by
having additional ways to model the same job. I disagree both that we need
a new way of handling this, and that IDB made a mistake in using the
standard mechanism in an ordinary, well-practiced way.
I'm not understanding how this refutes the fact that single-fired events is
an odd model for the job. Sure it may be consistent, but it's consistently
bad. There are plenty of criteria for judging an API, one of which is
consistency amongst the rest of the platform idioms. But it should be no
wonder that solid API developers come along and create comprehensible
wrappers around the platform.
- *Doubled API surface for sync version*
I assume I just don't understand why this choice was made, but the
explosion of API surface area combined with the conditional availability
of
this version of the API make it an odd beast (to be charitable).
There's currently no other way to allow an API to be synchronous in
workers but only async in the UI thread.
There was some discussion about a generalized way to allow workers to
block on a message from another thread, which would make it possible to
implement a synchronous shim for any async API in JavaScript. In theory
this could make it unnecessary for each API to have its own synchronous
interface. It wouldn't be as convenient, and probably wouldn't be suitable
for every API, but for big, complex interfaces like IDB it might make
sense. There might also be other ways to express synchronous APIs based on
their async interfaces without having a whole second interface (eg. maybe
something like a method to block until an event is received).
I think this would be very desirable on all fronts to avoid duplication of
interfaces; maybe something like signal events on which a caller can wait:
var signal = someAsyncAction(),
result = signal.wait();
This is an entirely different conversation though. I don't know the answer
to why sync interfaces are there and expected, except that some would argue
that it makes the code easier to read/write for some devs. Since this is
mirrored throughout other platform APIs, I wouldn't count this as a fault
in IDB specifically.
- *The idea that this is all going to be wrapped up by libraries
anyway*
I don't have an opinion about IDB specifically yet, but I agree that this
is wrong.
People have become so used to using wrappers around APIs that they've come
to think of them as normal, and that we should design APIs assuming people
will keep doing that.
People wrap libraries when they're hard to use, and if they're hard to use
then they're badly designed. Just because people wrap bad APIs isn't an
excuse for designing more bad APIs. Wrappers for basic usage are always a
bad thing: you always end up with lots of them, which means everyone is
using different APIs. When everyone uses the provided APIs directly, we
can all read each others' code and all of our code interoperates much more
naturally.
(As you said, this is only referring to wrappers at the same level of
abstraction, of course, not libraries providing higher-level abstractions.)
--
Glenn Maynard
Re: IndexedDB, what were the issues? How do we stop it from happening again?
On Wednesday, March 6, 2013 at 5:51 PM, Jarred Nicholls wrote: This is an entirely different conversation though. I don't know the answer to why sync interfaces are there and expected, except that some would argue that it makes the code easier to read/write for some devs. Since this is mirrored throughout other platform APIs, I wouldn't count this as a fault in IDB specifically. Sync APIs are useful to do I/O inside of a Worker. They're also critical for data consistency in some scenarios, e.g. updating the database after a successful xhr request when a worker is about to be terminated. --tobie
Re: IndexedDB, what were the issues? How do we stop it from happening again?
My primary takeaway from both working on IDB and working with IDB for some
demo apps is that IDB has just the right amount of complexity for really
large, robust database use.. but for a welcome to noSQL in the browser it
is way too complicated.
Specifically:
1. *versioning* - The reason this exists in IDB is to guarantee a schema
(read: a fixed set of objectStores + indexes) for a given set of
operations. Versioning should be optional. And if versioning is optional,
so should *opening* - the only reason you need to open a database is
so that you have a handle to a versioned database. You can *almost*
implement
versioning in JS if you really care about it...(either keep an explicit
key, or auto-detect the state of the schema) its one of those cases where
80% of versioning is dirt simple and the complicated stuff is really about
maintaining version changes across multiply-opened windows. (i.e. one
window opens an idb, the next window opens it and changes the schema, the
first window *may* need to know that and be able to adapt without
breaking any in-flight transactions) -
2. *transactions* - Also should be optional. Vital to complex apps, but
totally not necessary for many.. there should be a default transaction,
like db.objectStore(foo).get(bar)
3. *transaction scoping* - even when you do want transactions, the api
is just too verbose and repetitive for get one key from one object store
- db.transaction(foo).objectStore(foo).get(bar) - there should be
implicit (lightweight) transactions like db.objectStore(foo).get(bar)
4. *forced versioning* - when versioning is optional, it should be then
possible to change the schema during a regular transaction. Yes, this is a
lot of rope but this is actually for much more complex apps, rather than
simple ones. In particular, it's not uncommon for more complex database
systems to dynamically create indexes based on observed behavior of the
API, or observed data (i.e. when data with a particular key becomes
prevalent, generate an index for it) and then dynamically use them if
present. At the moment you have to do a manual close/open/version change to
dynamically bump up the version - effectively rendering fixed-value
versions moot (i.e. the schema for version 23 in my browser may look
totally different than the schema for version 23 in your browser) and
drastically complicating all your code (Because if you try to close/open
while transactions are in flight, they will be aborted - so you have to
temporarily pause all new transactions, wait for all in-flight transactions
to finish, do a close/open, then start running all pending/paused
transactions.) This last case MIGHT be as simple as adding
db.reopen(newVersion) to the existing spec.
5. *named object stores* - frankly, for *many* use cases, a single
objectStore is all you need. a simple db.get(foo) would be sufficient.
Simply naming a default isn't bad - whats bad is all the onupgradeneeded
scaffolding required to create the objectstore in the first place.
I do think that the IDBRequest model needs tweaking, and Futures seem like
the obvious direction to head in.
FWIW, the sync version of the API is more or less dead - nobody has
actually implemented it.
I think there is a very specialized set of applications that absolutely
need the features that IDB has right now. Google Docs is a perfect example
- long lived complicated application that needs to keep absolute integrity
of schema across multiple tabs over a long period of time.. but for 99% of
usecases out there, I think they're unnecessary.
I think ultimately, a simplified IDB would allow progressive use of the api
as your application grows.
// basic interaction - some objectStore named 'default' gets crated under
the hood.
indexedDB.get(mykey);
// named database, auto-create the 'first' objectStore named 'default', no
need to 'close' anything
indexedDB.database(mydb).get(mykey)
// now we need multiple objectstores:
indexedDB.database(mydb).objectStore(default).get(mykey)
// time for versioning, but using 'default'
indexedDB.open(mydb, 12).onupgradeneeded(function (db) {...}).get(bar)
etc...
Alec
On Wed, Mar 6, 2013 at 6:01 AM, Alex Russell slightly...@google.com wrote:
Comments inline. Adding some folks from the IDB team at Google to the
thread as well as public-webapps.
On Sunday, February 17, 2013, Miko Nieminen wrote:
2013/2/15 Shwetank Dixit shweta...@opera.com
Why did you feel it was necessary to write a layer on top of IndexedDB?
I think this is the main issue here.
As it stands, IDB is great in terms of features and power it offers, but
the feedback I recieved from other devs was that writing raw IndexedDB
requires an uncomfortable amount of verbosity even for some simple tasks
(This can be disputed, but that is the views I got from some of the
developers I interacted with). Adding that much amount of
Re: IndexedDB, what were the issues? How do we stop it from happening again?
I seem to recall we contemplated people writing libraries on top of IDB from the beginning. I'm not sure why this is a bad thing. We originally shipped web sql / sqlite, which was a familiar interface for many and relatively easy to use, but had a sufficiently large API surface area that no one felt they wanted to document the whole thing such that we could have an inter-operable standard. (Yes, I'm simplifying a bit.) As a result, we came up with an approach of What are the fundamental primitives that we need?, spec'd that out, and shipped it. We had discussions at the time that we expected library authors to produce abstraction layers that made IDB easier to use, as the fundamental primitives approach was not necessarily intended to produce an API that was as straightforward and easy to use as what we were trying to replace. If that's now what is happening, that seems like a good thing, not a failure. -Ian On Wed, Mar 6, 2013 at 10:14 AM, Alec Flett alecfl...@chromium.org wrote: My primary takeaway from both working on IDB and working with IDB for some demo apps is that IDB has just the right amount of complexity for really large, robust database use.. but for a welcome to noSQL in the browser it is way too complicated. Specifically: 1. *versioning* - The reason this exists in IDB is to guarantee a schema (read: a fixed set of objectStores + indexes) for a given set of operations. Versioning should be optional. And if versioning is optional, so should *opening* - the only reason you need to open a database is so that you have a handle to a versioned database. You can *almost* implement versioning in JS if you really care about it...(either keep an explicit key, or auto-detect the state of the schema) its one of those cases where 80% of versioning is dirt simple and the complicated stuff is really about maintaining version changes across multiply-opened windows. (i.e. one window opens an idb, the next window opens it and changes the schema, the first window *may* need to know that and be able to adapt without breaking any in-flight transactions) - 2. *transactions* - Also should be optional. Vital to complex apps, but totally not necessary for many.. there should be a default transaction, like db.objectStore(foo).get(bar) 3. *transaction scoping* - even when you do want transactions, the api is just too verbose and repetitive for get one key from one object store - db.transaction(foo).objectStore(foo).get(bar) - there should be implicit (lightweight) transactions like db.objectStore(foo).get(bar) 4. *forced versioning* - when versioning is optional, it should be then possible to change the schema during a regular transaction. Yes, this is a lot of rope but this is actually for much more complex apps, rather than simple ones. In particular, it's not uncommon for more complex database systems to dynamically create indexes based on observed behavior of the API, or observed data (i.e. when data with a particular key becomes prevalent, generate an index for it) and then dynamically use them if present. At the moment you have to do a manual close/open/version change to dynamically bump up the version - effectively rendering fixed-value versions moot (i.e. the schema for version 23 in my browser may look totally different than the schema for version 23 in your browser) and drastically complicating all your code (Because if you try to close/open while transactions are in flight, they will be aborted - so you have to temporarily pause all new transactions, wait for all in-flight transactions to finish, do a close/open, then start running all pending/paused transactions.) This last case MIGHT be as simple as adding db.reopen(newVersion) to the existing spec. 5. *named object stores* - frankly, for *many* use cases, a single objectStore is all you need. a simple db.get(foo) would be sufficient. Simply naming a default isn't bad - whats bad is all the onupgradeneeded scaffolding required to create the objectstore in the first place. I do think that the IDBRequest model needs tweaking, and Futures seem like the obvious direction to head in. FWIW, the sync version of the API is more or less dead - nobody has actually implemented it. I think there is a very specialized set of applications that absolutely need the features that IDB has right now. Google Docs is a perfect example - long lived complicated application that needs to keep absolute integrity of schema across multiple tabs over a long period of time.. but for 99% of usecases out there, I think they're unnecessary. I think ultimately, a simplified IDB would allow progressive use of the api as your application grows. // basic interaction - some objectStore named 'default' gets crated under the hood. indexedDB.get(mykey); // named database, auto-create the 'first'
Re: IndexedDB, what were the issues? How do we stop it from happening again?
On Wed, Mar 6, 2013 at 11:02 AM, Ian Fette (イアンフェッティ) ife...@google.comwrote: I seem to recall we contemplated people writing libraries on top of IDB from the beginning. I'm not sure why this is a bad thing. We originally shipped web sql / sqlite, which was a familiar interface for many and relatively easy to use, but had a sufficiently large API surface area that no one felt they wanted to document the whole thing such that we could have an inter-operable standard. (Yes, I'm simplifying a bit.) As a result, we came up with an approach of What are the fundamental primitives that we need?, spec'd that out, and shipped it. We had discussions at the time that we expected library authors to produce abstraction layers that made IDB easier to use, as the fundamental primitives approach was not necessarily intended to produce an API that was as straightforward and easy to use as what we were trying to replace. If that's now what is happening, that seems like a good thing, not a failure. That's fine for building up, but I guess what I'm saying is that the primitives are too complicated to allow you to get started. There is an excellent HTML5rocks page on indexeddb describing a very simple usecase. But if I were a web developer, I'd say screw that, back to localStorage. Most of the html5 APIs seem almost too simple to be useful, and then people chain primitives together into useful APIs with libraries. For example, the DOM APIs are woefully verbose and primitive, but people have built much better APIs around them. But fundamentally they are primitive. IndexedDB seems like its build the other way - we looked at what would be hard for developers to implement themselves (transactions, versions) and built *primitives* that required them. At the same time, I'd argue that versioning and transactions *could* be built upon a transactionless, versionless keystore in JavaScript, using a library. Even the notion of multiple objectStores is an abstraction that could be implemented on a single keystore. To be fair, implementing transactions against a transactionless keystore would not be *performant*, but thats a separate issue. Now that we have transactions and versions, I wouldn't eliminate them from IDB by any means, but they're not required. Alec -Ian On Wed, Mar 6, 2013 at 10:14 AM, Alec Flett alecfl...@chromium.orgwrote: My primary takeaway from both working on IDB and working with IDB for some demo apps is that IDB has just the right amount of complexity for really large, robust database use.. but for a welcome to noSQL in the browser it is way too complicated. Specifically: 1. *versioning* - The reason this exists in IDB is to guarantee a schema (read: a fixed set of objectStores + indexes) for a given set of operations. Versioning should be optional. And if versioning is optional, so should *opening* - the only reason you need to open a database is so that you have a handle to a versioned database. You can *almost* implement versioning in JS if you really care about it...(either keep an explicit key, or auto-detect the state of the schema) its one of those cases where 80% of versioning is dirt simple and the complicated stuff is really about maintaining version changes across multiply-opened windows. (i.e. one window opens an idb, the next window opens it and changes the schema, the first window *may* need to know that and be able to adapt without breaking any in-flight transactions) - 2. *transactions* - Also should be optional. Vital to complex apps, but totally not necessary for many.. there should be a default transaction, like db.objectStore(foo).get(bar) 3. *transaction scoping* - even when you do want transactions, the api is just too verbose and repetitive for get one key from one object store - db.transaction(foo).objectStore(foo).get(bar) - there should be implicit (lightweight) transactions like db.objectStore(foo).get(bar) 4. *forced versioning* - when versioning is optional, it should be then possible to change the schema during a regular transaction. Yes, this is a lot of rope but this is actually for much more complex apps, rather than simple ones. In particular, it's not uncommon for more complex database systems to dynamically create indexes based on observed behavior of the API, or observed data (i.e. when data with a particular key becomes prevalent, generate an index for it) and then dynamically use them if present. At the moment you have to do a manual close/open/version change to dynamically bump up the version - effectively rendering fixed-value versions moot (i.e. the schema for version 23 in my browser may look totally different than the schema for version 23 in your browser) and drastically complicating all your code (Because if you try to close/open while transactions are in flight, they will be aborted - so you have to
