RE: [IndexedDB] Promises (WAS: Seeking pre-LCWD comments for Indexed Database API; deadline February 2)
On Fri, Mar 12, 2010 at 7:26 AM, Jeremy Orlow wrote: On Fri, Mar 12, 2010 at 3:23 PM, Jeremy Orlow jor...@chromium.org wrote: On Fri, Mar 12, 2010 at 3:04 PM, Kris Zyp k...@sitepen.com wrote: I believe computer science has clearly observed the fragility of passing callbacks to the initial function since it conflates the concerns of the operation with the asynchronous notifications and consequently greatly complicates composability. I don't understand this sentence. I'm pretty sure that you can wrap any callback based API in JavaScript with a promised, differed, etc based API. As Nikunj mentioned earlier, we're more concerned about creating a small API surface area and sticking with well understood API designs rather than eliminating the need for libraries that wrap IndexedDB. Trying to digest this thread, I think we've sort of gone full-circle with the whole promises thing. When looking at the code with the chained then pattern I just love the result, but it seems that we can't get all the way there (and nesting instead of chaining stuff kind of lacks the magic). My take is that either we get the really nice pattern by going all the way or we create a more traditional callback/events-based API and then we build promises on top. Things seem to indicate that frameworks are still cooking on promises, so it may be safe to stay with callbacks/events and just build libraries on top (I would have loved to have this be the thing that saved us from needing a library always...but it seems we'll fall just a bit short). As for callbacks versus events, while now I'm starting to get used to the events hooked up to the result object after the call, the callbacks may be a more natural mechanism for this particular usage. I'm not sure why this is fundamentally broken...would love to see examples or reference. If that's the case, then events are the obvious choice. Thanks -pablo
Re: [IndexedDB] Promises (WAS: Seeking pre-LCWD comments for Indexed Database API; deadline February 2)
On 3/5/2010 4:54 AM, Jeremy Orlow wrote: For what it's worth, regardless of the answers to the above questions, I think we should switch to a callback based model. It's great to use events when natural to do so, but this is a very unnatural use. It provides artificial limitations (only one request in flight at a time, per request object). It's ugly and confusing syntax wise (hard to keep track of which request object is associated with which request method, requires multiple statements to do each request, requires the handlers to be placed prior to the actual call...which is why the async example in http://www.w3.org/TR/IndexedDB/#introduction is so difficult to read, etc). And there really isn't any precedent (that I'm aware of) for using events like this. And the web developers I've spoken to have all been confused by the async API. For what it is worth, all the web developers we've talked to have pushed for an event based API, which is why we've been pushing for it. This happened with the file reader API as I understand it (Jonas or Arun would be able to say more). Note that we didn't show them this exact API. Cheers, Shawn smime.p7s Description: S/MIME Cryptographic Signature
Re: [IndexedDB] Promises (WAS: Seeking pre-LCWD comments for Indexed Database API; deadline February 2)
On Thu, Mar 4, 2010 at 7:44 PM, Nikunj Mehta nik...@o-micron.com wrote: On Mar 4, 2010, at 10:55 AM, Kris Zyp wrote: On 3/4/2010 11:46 AM, Nikunj Mehta wrote: On Mar 4, 2010, at 10:23 AM, Kris Zyp wrote: On 3/4/2010 11:08 AM, Aaron Boodman wrote: [snip] * There is nothing preventing JS authors from implementing a promise-style API on top of IndexedDB, if that is what they want to do. Yes, you can always make an API harder to use so that JS authors have more they can do with it ;). You will agree that we don't want to wait for one style of promises to win out over others before IndexedDB can be made available to programmers. Till the soil and let a thousand flowers bloom. The IndexedDB spec isn't and can't just sit back and not define the asynchronous interface. Like it or not, IndexedDB has defined a promise-like entity with the |DBRequest| interface. Why is inventing a new (and somewhat ugly) flower better than designing based on the many flowers that have already bloomed? I meant to say that the IndexedDB spec should be updated to use a model that supports promises. If the current one is not adequate then, by all means, let's make it. However, we don't need a full-fledged promises in IndexedDB. I hope you agree this time. FWIW, I agree. To get promises to work with the current event based implementation, it'd be somewhat complex. Since there can only be one request in flight at a time, the implementation would need to know which requests would be using the same request object and implement a queue for each one (or implement a global queue if that's not practical). Whenever an onsuccess or onerror callback is called, it'd need to check to see if there are any queued up requests (and would fire them if so). This seems complex and ugly, but certainly possible. If the interface were callback based, a library would simply create a promise object, create an onsuccess closure, create an onerror closure, and pass those into the callbacks. When the callbacks are called, they'd have a reference to the promise (since they were created with access to the promise object due to their scope) and could easily fulfill the promise. Of course, I'm probably re-inventing the wheel here; there are enough other callback based APIs that I assume this is a solved (and optimized problem). Are there any other APIs that use the request event based style like how IndexedDB is currently specced? If so, can anyone share any experience with the list? If not, does anyone foresee major problems and/or have an opinion on how easy it'll be to adapt promises to the API? For what it's worth, regardless of the answers to the above questions, I think we should switch to a callback based model. It's great to use events when natural to do so, but this is a very unnatural use. It provides artificial limitations (only one request in flight at a time, per request object). It's ugly and confusing syntax wise (hard to keep track of which request object is associated with which request method, requires multiple statements to do each request, requires the handlers to be placed prior to the actual call...which is why the async example in http://www.w3.org/TR/IndexedDB/#introduction is so difficult to read, etc). And there really isn't any precedent (that I'm aware of) for using events like this. And the web developers I've spoken to have all been confused by the async API. I believe the API itself won't need to change much at all in order to make callbacks work. I'm happy to take a shot at making the necessary edits if no one objects to changing the async API to a callback based one. J
Re: [IndexedDB] Promises (WAS: Seeking pre-LCWD comments for Indexed Database API; deadline February 2)
On Wed, Mar 3, 2010 at 8:48 PM, Kris Zyp k...@sitepen.com wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 3/3/2010 4:01 AM, Jeremy Orlow wrote: On Wed, Mar 3, 2010 at 4:49 AM, Kris Zyp k...@sitepen.com mailto:k...@sitepen.com k...@sitepen.com wrote: [snip] The promises would only have a then method which would take in an onsuccess and onerror callback. Both are optional. The onsuccess function should take in a single parameter which matches the return value of the synchronous counterpart. The onerror function should take in an IDBDatabaseError. If the callbacks are null, undefined, or omitted, they're ignored. If they're anything else, we should probably either raise an exception immediately or ignore them. Yes. Any thoughts on whether we'd raise or ignore improper inputs? I'm leaning towards raise since it would be deterministic and silently ignoring seems like a headache from a developer standpoint. Throwing an error on improper inputs is fine with me. If there's an error, all onerror callbacks would be called with the IDBDatabaseError. Yes. Exceptions within callbacks would be ignored. With CommonJS promises, the promise returned by the then() call goes into an error state if a callback throws an exception. For example, someAsyncOperation.then(successHandler, function(){ throw new Error(test) }) .then(null, function(error){ console.log(error); }); Would log the thrown error, effectively giving you a way of catching the error. Are you suggesting this as a simplification so that IndexedDB impls doesn't have to worry about recursive creation of promises? If so, I suppose that seems like a reasonable simplification to me. Although if promises are something that could be potentially reused in other specs, it would be nice to have a quality solution, and I don't think this is a big implementation burden, I've implemented the recursive capabilities in dozen or two lines of JS code. But if burden is too onerous, I am fine with the simplification. When you say recursive capabilities are you just talking about how to handle exceptions, or something more? In terms of exceptions: I don't think it's an enormous implementational burden and thus I think it's fine to ignore that part of the equation. So the question mainly comes down to whether the added complexity is worth it. Can you think of any real-world examples of when this capability is useful in promises? If so, that'd definitely help us understand the pro's and con's. Maybe I misunderstanding your suggestion. By recursive capability I meant having then() return a promise (that is fulfilled with the result of executing the callback), and I thought you were suggesting that instead, then() would not return a promise. If then() returns a promise, I think the returned promise should clearly go into an error state if the callback throws an error. The goal of promises is to asynchronously model computations, and if a computation throws, it should result in the associated promise entering error state. The promise returned by then() exists to represent the result of the execution of the callback, and so it should resolve to the value returned by the callback or an error if the callback throws. Silenty swallowing errors seems highly undesirable. Now if we are simplifying then() to not return a promise at all, than I would think callbacks would just behave like any other event listener in regards to uncaught errors. You are quite right! I misunderstood how this part of promises worked. Is there excitement about speccing promises in general? If not, it seems a little odd to spec such a powerful mechanism into just IndexedDBand it might be best to spec the simplified version of .then(): .then() will return undefined, onsuccess/onerror's return values will be swallowed, and any thrown exceptions will be thrown. This should make it easy to make IndexedDB support full blown promises if/whenever they're specced. (It's not clear to me whether UA support for them would offer enough advantages to warrant it.) It sounds like you're OK with such an approach, Kris? What do others think? J In terms of speccing, I'm not sure if we can get away with speccing one promise interface or whether we'd need to create one for each type of promise. Certainly the intent of promises is that there is exists only one generic promise interface that can be reused everywhere, at least from the JS perspective, not sure if the extra type constraints in IDL demand multiple interfaces to model promise's effectively parameterized generic type form. Unfortunately, I
Re: [IndexedDB] Promises (WAS: Seeking pre-LCWD comments for Indexed Database API; deadline February 2)
On Thu, Mar 4, 2010 at 2:37 PM, Jeremy Orlow jor...@chromium.org wrote: On Wed, Mar 3, 2010 at 8:48 PM, Kris Zyp k...@sitepen.com wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 3/3/2010 4:01 AM, Jeremy Orlow wrote: On Wed, Mar 3, 2010 at 4:49 AM, Kris Zyp k...@sitepen.com mailto:k...@sitepen.com k...@sitepen.com wrote: [snip] The promises would only have a then method which would take in an onsuccess and onerror callback. Both are optional. The onsuccess function should take in a single parameter which matches the return value of the synchronous counterpart. The onerror function should take in an IDBDatabaseError. If the callbacks are null, undefined, or omitted, they're ignored. If they're anything else, we should probably either raise an exception immediately or ignore them. Yes. Any thoughts on whether we'd raise or ignore improper inputs? I'm leaning towards raise since it would be deterministic and silently ignoring seems like a headache from a developer standpoint. Throwing an error on improper inputs is fine with me. If there's an error, all onerror callbacks would be called with the IDBDatabaseError. Yes. Exceptions within callbacks would be ignored. With CommonJS promises, the promise returned by the then() call goes into an error state if a callback throws an exception. For example, someAsyncOperation.then(successHandler, function(){ throw new Error(test) }) .then(null, function(error){ console.log(error); }); Would log the thrown error, effectively giving you a way of catching the error. Are you suggesting this as a simplification so that IndexedDB impls doesn't have to worry about recursive creation of promises? If so, I suppose that seems like a reasonable simplification to me. Although if promises are something that could be potentially reused in other specs, it would be nice to have a quality solution, and I don't think this is a big implementation burden, I've implemented the recursive capabilities in dozen or two lines of JS code. But if burden is too onerous, I am fine with the simplification. When you say recursive capabilities are you just talking about how to handle exceptions, or something more? In terms of exceptions: I don't think it's an enormous implementational burden and thus I think it's fine to ignore that part of the equation. So the question mainly comes down to whether the added complexity is worth it. Can you think of any real-world examples of when this capability is useful in promises? If so, that'd definitely help us understand the pro's and con's. Maybe I misunderstanding your suggestion. By recursive capability I meant having then() return a promise (that is fulfilled with the result of executing the callback), and I thought you were suggesting that instead, then() would not return a promise. If then() returns a promise, I think the returned promise should clearly go into an error state if the callback throws an error. The goal of promises is to asynchronously model computations, and if a computation throws, it should result in the associated promise entering error state. The promise returned by then() exists to represent the result of the execution of the callback, and so it should resolve to the value returned by the callback or an error if the callback throws. Silenty swallowing errors seems highly undesirable. Now if we are simplifying then() to not return a promise at all, than I would think callbacks would just behave like any other event listener in regards to uncaught errors. You are quite right! I misunderstood how this part of promises worked. Is there excitement about speccing promises in general? If not, it seems a little odd to spec such a powerful mechanism into just IndexedDBand it might be best to spec the simplified version of .then(): .then() will return undefined, onsuccess/onerror's return values will be swallowed, and any thrown exceptions will be thrown. Erthrown exceptions will be _swallowed_ (not thrown). This should make it easy to make IndexedDB support full blown promises if/whenever they're specced. (It's not clear to me whether UA support for them would offer enough advantages to warrant it.) It sounds like you're OK with such an approach, Kris? What do others think? J In terms of speccing, I'm not sure if we can get away with speccing one promise interface or whether we'd need to create one for each type of promise. Certainly the intent of promises is that there is exists only one generic promise interface that can be reused everywhere, at least from the JS perspective, not sure if the extra type
Re: [IndexedDB] Promises (WAS: Seeking pre-LCWD comments for Indexed Database API; deadline February 2)
On Thu, Mar 4, 2010 at 6:37 AM, Jeremy Orlow jor...@chromium.org wrote: You are quite right! I misunderstood how this part of promises worked. Is there excitement about speccing promises in general? Yes. The starting point for a lot of the commonjs promises work is Tyler's ref_send promise library, documented at http://waterken.sourceforge.net/web_send/#Q. The commonjs work got more complicated than this in order to try to accommodate legacy deferred-based usage patterns within the same framework. While it may have helped adoption within the commonjs community, IMO this extra complexity should not be in any standard promise spec. Caja implements Tyler's spec without the extra complexity, and we're quite happy with it. I hope to work with Tyler and others to propose this to the EcmaScript committee as part of a more general proposal for a communicating-event-loops concurrency and distribution framework for future EcmaScript. Don't hold your breath though, this is not yet even an EcmaScript strawman. Neither is there any general consensus on the EcmaScript committee that EcmaScript should be extended in these directions. In the meantime, I suggest just using Tyler's ref_send and web_send libraries. If not, it seems a little odd to spec such a powerful mechanism into just IndexedDBand it might be best to spec the simplified version of .then(): .then() will return undefined, onsuccess/onerror's return values will be swallowed, and any thrown exceptions will be thrown. This should make it easy to make IndexedDB support full blown promises if/whenever they're specced. (It's not clear to me whether UA support for them would offer enough advantages to warrant it.) Note that ref_send exposes the .then() style functionality as a static .when() method on Q rather than an instance .then() method on promises. This is important, as it 1) allows resolved values to be used where a promise is expected, and 2) it protects the caller from interleavings happening during their Q.when() call, even if the alleged promise they are operating on is something else. It sounds like you're OK with such an approach, Kris? What do others think? J In terms of speccing, I'm not sure if we can get away with speccing one promise interface or whether we'd need to create one for each type of promise. Certainly the intent of promises is that there is exists only one generic promise interface that can be reused everywhere, at least from the JS perspective, not sure if the extra type constraints in IDL demand multiple interfaces to model promise's effectively parameterized generic type form. Unfortunately, I don't really know. Before we try speccing it, I'll definitely see if any WebIDL experts have suggestions. Also, do we want to explicitly spec what happens in the following case? window.indexedDB.open(...).then( function(db) { db.openObjectStore(a).then( function(os) { alert(Opened a); } ) } ).then( function(db) { alert(Second db opened); } ); Clearly the first function(db) is called first. But the question is whether it'd be a race of which alert is called first or whether the Second db opened alert should always be shown first (since clearly if the first is called, the second _can_ be fired immediately afterwards). I'm on the fence about whether it'd be useful to spec that the entire chain needs to be called one after the other before calling any other callbacks. Does anyone have thoughts on whether this is useful or not? If we do spec it to call the entire chain, then what happens if inside one of the callbacks, something is added to the chain (via another .then() call). Specing the order of multiple events in the event loop seems like it would be excessive burden on implementors, IMO. I've been talking to a co-worker here who seems to know a decent amount about promises (as implemented in E) and some about differed (as implemented in Python's Twisted library). From talking to him, it seems that my original suggestion for not handling exceptions thrown inside a .then() callback is the way to go. It seems as though promises put a lot of weight on composability and making it so that the order of .then() calls not mattering. This means that you can then pass promises to other async interfaces and not have to worry about different timings leading to different results. It also means that if you pass a promise into multiple consumers (say, javascript libraries) you don't need to worry about one using a promise in a way that screws up another. Differed seems to be more expressive and flexible. For example, instead of doing this: window.indexedDB.open(...).then( function(db) { db.openObjectStore(a).then( function(os) { os.get(x).then( function(value) { alert(Value: + value); } ) }
Re: [IndexedDB] Promises (WAS: Seeking pre-LCWD comments for Indexed Database API; deadline February 2)
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 3/4/2010 10:35 AM, Mark S. Miller wrote: On Thu, Mar 4, 2010 at 6:37 AM, Jeremy Orlow jor...@chromium.org mailto:jor...@chromium.org wrote: You are quite right! I misunderstood how this part of promises worked. Is there excitement about speccing promises in general? Yes. The starting point for a lot of the commonjs promises work is Tyler's ref_send promise library, documented at http://waterken.sourceforge.net/web_send/#Q. The commonjs work got more complicated than this in order to try to accommodate legacy deferred-based usage patterns within the same framework. While it may have helped adoption within the commonjs community, IMO this extra complexity should not be in any standard promise spec. Caja implements Tyler's spec without the extra complexity, and we're quite happy with it. I hope to work with Tyler and others to propose this to the EcmaScript committee as part of a more general proposal for a communicating-event-loops concurrency and distribution framework for future EcmaScript. Don't hold your breath though, this is not yet even an EcmaScript strawman. Neither is there any general consensus on the EcmaScript committee that EcmaScript should be extended in these directions. In the meantime, I suggest just using Tyler's ref_send and web_send libraries. It would be great if promises become first class, but obviously the IndexedDB specification can't be dependent on someone's JS library. If not, it seems a little odd to spec such a powerful mechanism into just IndexedDBand it might be best to spec the simplified version of .then(): .then() will return undefined, onsuccess/onerror's return values will be swallowed, and any thrown exceptions will be thrown. This should make it easy to make IndexedDB support full blown promises if/whenever they're specced. (It's not clear to me whether UA support for them would offer enough advantages to warrant it.) Note that ref_send exposes the .then() style functionality as a static .when() method on Q rather than an instance .then() method on promises. This is important, as it 1) allows resolved values to be used where a promise is expected, and 2) it protects the caller from interleavings happening during their Q.when() call, even if the alleged promise they are operating on is something else. The .then() function is in no way intended to be a replacement for a static .when() function. In contrast to ref_send, having promises defined by having a .then() function is in lieu of ref_send's definition of a promise where the promise is a function that must be called: promise(WHEN, callback, errback); This group can consider it an API like this, but I don't think that IndexedDB or any other W3C API would want to define promises in that way, as it is pretty awkward. Using .then() based promises in no way precludes the use of Q.when() implementations which meet both your criteria for safe operation. However, these can easily be implemented in JS, and I don't think the IndexedDB API needs to worry about such promise libraries. - -- Kris Zyp SitePen (503) 806-1841 http://sitepen.com -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.9 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAkuP8f0ACgkQ9VpNnHc4zAwm9gCfajBUy0PZpaxvSctlorVeYIsK yQwAnAwtSd6BWPbpOOJTniZcojmNFQtw =GHjA -END PGP SIGNATURE-
Re: [IndexedDB] Promises (WAS: Seeking pre-LCWD comments for Indexed Database API; deadline February 2)
On Thu, Feb 18, 2010 at 4:31 AM, Jeremy Orlow jor...@google.com wrote: On Wed, Jan 27, 2010 at 9:46 PM, Kris Zyp k...@sitepen.com wrote: * Use promises for async interfaces - In server side JavaScript, most projects are moving towards using promises for asynchronous interfaces instead of trying to define the specific callback parameters for each interface. I believe the advantages of using promises over callbacks are pretty well understood in terms of decoupling async semantics from interface definitions, and improving encapsulation of concerns. For the indexed database API this would mean that sync and async interfaces could essentially look the same except sync would return completed values and async would return promises. I realize that defining a promise interface would have implications beyond the indexed database API, as the goal of promises is to provide a consistent interface for asynchronous interaction across components, but perhaps this would be a good time for the W3C to define such an API. It seems like the indexed database API would be a perfect interface to leverage promises. If you are interested in proposal, there is one from CommonJS here [1] (the get() and call() wouldn't apply here). With this interface, a promise.then(callback, errorHandler) function is the only function a promise would need to provide. [1] http://wiki.commonjs.org/wiki/Promises Very interesting. The general concept seems promising and fairly flexible. You can easily code in a similar style to normal async/callback semantics, but it seems like you have a lot more flexibility. I do have a few questions though. Are there any good examples of these used in the wild that you can point me towards? I used my imagination for prototyping up some examples, but it'd be great to see some real examples + be able to see the exact semantics used in those implementations. I see that you can supply an error handling callback to .then(), but does that only apply to the one operation? I could easily imagine emulating try/catch type semantics and have errors continue down the line of .then's until someone handles it. It might even make sense to allow the error handlers to re-raise (i.e. allow to bubble) errors so that later routines would get them as well. Maybe you'd even want it to bubble by default? What have other implementations done with this stuff? What is the most robust and least cumbersome for typical applications? (And, in te complete absence of real experience, are there any expert opinions on what might work?) Overall this seems fairly promising and not that hard to implement. Do others see pitfalls that I'm missing? J I disagree that IndexedDB should use promises, for several reasons: * Promises are only really useful when they are used ubiquitously throughout the platform, so that you can pass them around like references. In libraries like Dojo, MochiKit, and Twisted, this is exactly the situation. But in the web platform, this would be the first such API. Without places to pass a promise to, all you really have is a lot of additional complexity. * ISTM that the entire space is still evolving quite rapidly. Many JavaScript libraries have implemented a form of this, and this proposal is also slightly different from any of them. I think it is premature to have browsers implement this while library authors are still hashing out best practice. Once it is in browsers, it's forever. * There is nothing preventing JS authors from implementing a promise-style API on top of IndexedDB, if that is what they want to do. - a
Re: [IndexedDB] Promises (WAS: Seeking pre-LCWD comments for Indexed Database API; deadline February 2)
On Thu, Mar 4, 2010 at 5:46 PM, Kris Zyp k...@sitepen.com wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 3/4/2010 10:35 AM, Mark S. Miller wrote: On Thu, Mar 4, 2010 at 6:37 AM, Jeremy Orlow jor...@chromium.org mailto:jor...@chromium.org jor...@chromium.org wrote: You are quite right! I misunderstood how this part of promises worked. Is there excitement about speccing promises in general? Yes. The starting point for a lot of the commonjs promises work is Tyler's ref_send promise library, documented at http://waterken.sourceforge.net/web_send/#Qhttp://waterken.sourceforge.net/web_send/#Q. The commonjs work got more complicated than this in order to try to accommodate legacy deferred-based usage patterns within the same framework. While it may have helped adoption within the commonjs community, IMO this extra complexity should not be in any standard promise spec. Caja implements Tyler's spec without the extra complexity, and we're quite happy with it. I hope to work with Tyler and others to propose this to the EcmaScript committee as part of a more general proposal for a communicating-event-loops concurrency and distribution framework for future EcmaScript. Don't hold your breath though, this is not yet even an EcmaScript strawman. Neither is there any general consensus on the EcmaScript committee that EcmaScript should be extended in these directions. In the meantime, I suggest just using Tyler's ref_send and web_send libraries. It would be great if promises become first class, but obviously the IndexedDB specification can't be dependent on someone's JS library. If not, it seems a little odd to spec such a powerful mechanism into just IndexedDBand it might be best to spec the simplified version of .then(): .then() will return undefined, onsuccess/onerror's return values will be swallowed, and any thrown exceptions will be thrown. This should make it easy to make IndexedDB support full blown promises if/whenever they're specced. (It's not clear to me whether UA support for them would offer enough advantages to warrant it.) Note that ref_send exposes the .then() style functionality as a static .when() method on Q rather than an instance .then() method on promises. This is important, as it 1) allows resolved values to be used where a promise is expected, and 2) it protects the caller from interleavings happening during their Q.when() call, even if the alleged promise they are operating on is something else. Thanks a lot for your feedback! This is very valuable and definitely provided some food for thought. I started working on a rambly email about the pro's and cons of when when I saw Kris's response. The .then() function is in no way intended to be a replacement for a static .when() function. In contrast to ref_send, having promises defined by having a .then() function is in lieu of ref_send's definition of a promise where the promise is a function that must be called: promise(WHEN, callback, errback); This group can consider it an API like this, but I don't think that IndexedDB or any other W3C API would want to define promises in that way, as it is pretty awkward. Using .then() based promises in no way precludes the use of Q.when() implementations which meet both your criteria for safe operation. However, these can easily be implemented in JS, and I don't think the IndexedDB API needs to worry about such promise libraries. Which is basically what I had arrived at in my mind as well. It'll definitely be interesting to see how the EMCAScript side of promises shapes up. But in the mean time, I think the simpler version that we've been discussing will be a good balance of features but minimized API surface area...and keeping chances high that what ends up being standardized would fit in well with the API. At this point, I feel fairly confident that using a scaled down version of promises would work well in IndexedDB. But, at the same time, a callback based API would be much more standard and it wouldn't be that hard for someone to build a promise based library around IndexedDB. Nikunj, Pablo, Mozilla, etc...what do you think is the best way forward here? Should we give scaled back promises a shot? Or should we just go with a callback based approach? J Summary of what I'm currently thinking we should do, if we go with a Promises type async API: Each async function would return a promise. A promise has one method: .then(). Then takes up to two callbacks. The first is onsuccess. The second is onerror. You can call .then() before and after the async call has finished--in fact, there's no way to know for sure whether it has finished before you call .then() (but that's fine). If you pass in garbage to the callbacks, it'll throw an exception, but null/undefined and omitting them is fine. When the promise is ready to fire the callbacks, it'll always do it
Re: [IndexedDB] Promises (WAS: Seeking pre-LCWD comments for Indexed Database API; deadline February 2)
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 3/4/2010 11:08 AM, Aaron Boodman wrote: On Thu, Feb 18, 2010 at 4:31 AM, Jeremy Orlow jor...@google.com wrote: On Wed, Jan 27, 2010 at 9:46 PM, Kris Zyp k...@sitepen.com wrote: * Use promises for async interfaces - In server side JavaScript, most projects are moving towards using promises for asynchronous interfaces instead of trying to define the specific callback parameters for each interface. I believe the advantages of using promises over callbacks are pretty well understood in terms of decoupling async semantics from interface definitions, and improving encapsulation of concerns. For the indexed database API this would mean that sync and async interfaces could essentially look the same except sync would return completed values and async would return promises. I realize that defining a promise interface would have implications beyond the indexed database API, as the goal of promises is to provide a consistent interface for asynchronous interaction across components, but perhaps this would be a good time for the W3C to define such an API. It seems like the indexed database API would be a perfect interface to leverage promises. If you are interested in proposal, there is one from CommonJS here [1] (the get() and call() wouldn't apply here). With this interface, a promise.then(callback, errorHandler) function is the only function a promise would need to provide. [1] http://wiki.commonjs.org/wiki/Promises Very interesting. The general concept seems promising and fairly flexible. You can easily code in a similar style to normal async/callback semantics, but it seems like you have a lot more flexibility. I do have a few questions though. Are there any good examples of these used in the wild that you can point me towards? I used my imagination for prototyping up some examples, but it'd be great to see some real examples + be able to see the exact semantics used in those implementations. I see that you can supply an error handling callback to .then(), but does that only apply to the one operation? I could easily imagine emulating try/catch type semantics and have errors continue down the line of .then's until someone handles it. It might even make sense to allow the error handlers to re-raise (i.e. allow to bubble) errors so that later routines would get them as well. Maybe you'd even want it to bubble by default? What have other implementations done with this stuff? What is the most robust and least cumbersome for typical applications? (And, in te complete absence of real experience, are there any expert opinions on what might work?) Overall this seems fairly promising and not that hard to implement. Do others see pitfalls that I'm missing? J I disagree that IndexedDB should use promises, for several reasons: * Promises are only really useful when they are used ubiquitously throughout the platform, so that you can pass them around like references. In libraries like Dojo, MochiKit, and Twisted, this is exactly the situation. But in the web platform, this would be the first such API. Without places to pass a promise to, all you really have is a lot of additional complexity. I certainly agree that promises are more useful when used ubiquitously. However, promises have many advantages besides just being a common interface for asynchronous operations, including interface simplicity, composibility, and separation of concerns. But, your point about this being the first such API is really important. If we are going to use promises in the IndexedDB, I think they should the webapps group should be looking at them beyond the scope of just the IndexedDB API, and how they could be used in other APIs, such that common interface advantage could be realized. Looking at the broad perspective is key here. * ISTM that the entire space is still evolving quite rapidly. Many JavaScript libraries have implemented a form of this, and this proposal is also slightly different from any of them. I think it is premature to have browsers implement this while library authors are still hashing out best practice. Once it is in browsers, it's forever. Promises have been around for a number of years, we already have a lot of experience to draw from, this isn't exactly a brand new idea, promises are a well-established concept. The CommonJS proposal is nothing ground breaking, it is more based on the culmination of ideas of Dojo, ref_send and others. It is also worth noting that a number of JS libraries have expressed interest in moving towards the CommonJS promise proposal, and Dojo will probably support them in 1.5. * There is nothing preventing JS authors from implementing a promise-style API on top of IndexedDB, if that is what they want to do. Yes, you can always make an API harder to use so that JS authors have more they can do with it ;). But it is true, we can build promises on top of an plain event-based
Re: [IndexedDB] Promises (WAS: Seeking pre-LCWD comments for Indexed Database API; deadline February 2)
On Mar 4, 2010, at 10:23 AM, Kris Zyp wrote: On 3/4/2010 11:08 AM, Aaron Boodman wrote: On Thu, Feb 18, 2010 at 4:31 AM, Jeremy Orlow jor...@google.com wrote: On Wed, Jan 27, 2010 at 9:46 PM, Kris Zyp k...@sitepen.com wrote: * Use promises for async interfaces - In server side JavaScript, most projects are moving towards using promises for asynchronous interfaces instead of trying to define the specific callback parameters for each interface. I believe the advantages of using promises over callbacks are pretty well understood in terms of decoupling async semantics from interface definitions, and improving encapsulation of concerns. For the indexed database API this would mean that sync and async interfaces could essentially look the same except sync would return completed values and async would return promises. I realize that defining a promise interface would have implications beyond the indexed database API, as the goal of promises is to provide a consistent interface for asynchronous interaction across components, but perhaps this would be a good time for the W3C to define such an API. It seems like the indexed database API would be a perfect interface to leverage promises. If you are interested in proposal, there is one from CommonJS here [1] (the get() and call() wouldn't apply here). With this interface, a promise.then(callback, errorHandler) function is the only function a promise would need to provide. [1] http://wiki.commonjs.org/wiki/Promises Very interesting. The general concept seems promising and fairly flexible. You can easily code in a similar style to normal async/callback semantics, but it seems like you have a lot more flexibility. I do have a few questions though. Are there any good examples of these used in the wild that you can point me towards? I used my imagination for prototyping up some examples, but it'd be great to see some real examples + be able to see the exact semantics used in those implementations. I see that you can supply an error handling callback to .then(), but does that only apply to the one operation? I could easily imagine emulating try/catch type semantics and have errors continue down the line of .then's until someone handles it. It might even make sense to allow the error handlers to re-raise (i.e. allow to bubble) errors so that later routines would get them as well. Maybe you'd even want it to bubble by default? What have other implementations done with this stuff? What is the most robust and least cumbersome for typical applications? (And, in te complete absence of real experience, are there any expert opinions on what might work?) Overall this seems fairly promising and not that hard to implement. Do others see pitfalls that I'm missing? J I disagree that IndexedDB should use promises, for several reasons: * Promises are only really useful when they are used ubiquitously throughout the platform, so that you can pass them around like references. In libraries like Dojo, MochiKit, and Twisted, this is exactly the situation. But in the web platform, this would be the first such API. Without places to pass a promise to, all you really have is a lot of additional complexity. I certainly agree that promises are more useful when used ubiquitously. However, promises have many advantages besides just being a common interface for asynchronous operations, including interface simplicity, composibility, and separation of concerns. But, your point about this being the first such API is really important. If we are going to use promises in the IndexedDB, I think they should the webapps group should be looking at them beyond the scope of just the IndexedDB API, and how they could be used in other APIs, such that common interface advantage could be realized. Looking at the broad perspective is key here. In general, IndexedDB has taken an approach of leaving ease of programming to libraries. There seems to be a good case to build libraries to make asynchronous programming with IndexedDB easier through the use of such mechanisms as promises. In fact, IndexedDB might be yet another area for libraries to slug it out. * ISTM that the entire space is still evolving quite rapidly. Many JavaScript libraries have implemented a form of this, and this proposal is also slightly different from any of them. I think it is premature to have browsers implement this while library authors are still hashing out best practice. Once it is in browsers, it's forever. Promises have been around for a number of years, we already have a lot of experience to draw from, this isn't exactly a brand new idea, promises are a well-established concept. The CommonJS proposal is nothing ground breaking, it is more based on the culmination of ideas of Dojo, ref_send and others. It is also worth noting that a number of JS libraries have expressed interest in moving towards the CommonJS promise proposal, and Dojo will probably support them in 1.5. I feel that we should avoid
Re: [IndexedDB] Promises (WAS: Seeking pre-LCWD comments for Indexed Database API; deadline February 2)
On Thu, Mar 4, 2010 at 6:46 PM, Nikunj Mehta nik...@o-micron.com wrote: On Mar 4, 2010, at 10:23 AM, Kris Zyp wrote: On 3/4/2010 11:08 AM, Aaron Boodman wrote: On Thu, Feb 18, 2010 at 4:31 AM, Jeremy Orlow jor...@google.com wrote: On Wed, Jan 27, 2010 at 9:46 PM, Kris Zyp k...@sitepen.com wrote: * Use promises for async interfaces - In server side JavaScript, most projects are moving towards using promises for asynchronous interfaces instead of trying to define the specific callback parameters for each interface. I believe the advantages of using promises over callbacks are pretty well understood in terms of decoupling async semantics from interface definitions, and improving encapsulation of concerns. For the indexed database API this would mean that sync and async interfaces could essentially look the same except sync would return completed values and async would return promises. I realize that defining a promise interface would have implications beyond the indexed database API, as the goal of promises is to provide a consistent interface for asynchronous interaction across components, but perhaps this would be a good time for the W3C to define such an API. It seems like the indexed database API would be a perfect interface to leverage promises. If you are interested in proposal, there is one from CommonJS here [1] (the get() and call() wouldn't apply here). With this interface, a promise.then(callback, errorHandler) function is the only function a promise would need to provide. [1] http://wiki.commonjs.org/wiki/Promises Very interesting. The general concept seems promising and fairly flexible. You can easily code in a similar style to normal async/callback semantics, but it seems like you have a lot more flexibility. I do have a few questions though. Are there any good examples of these used in the wild that you can point me towards? I used my imagination for prototyping up some examples, but it'd be great to see some real examples + be able to see the exact semantics used in those implementations. I see that you can supply an error handling callback to .then(), but does that only apply to the one operation? I could easily imagine emulating try/catch type semantics and have errors continue down the line of .then's until someone handles it. It might even make sense to allow the error handlers to re-raise (i.e. allow to bubble) errors so that later routines would get them as well. Maybe you'd even want it to bubble by default? What have other implementations done with this stuff? What is the most robust and least cumbersome for typical applications? (And, in te complete absence of real experience, are there any expert opinions on what might work?) Overall this seems fairly promising and not that hard to implement. Do others see pitfalls that I'm missing? J I disagree that IndexedDB should use promises, for several reasons: * Promises are only really useful when they are used ubiquitously throughout the platform, so that you can pass them around like references. In libraries like Dojo, MochiKit, and Twisted, this is exactly the situation. But in the web platform, this would be the first such API. Without places to pass a promise to, all you really have is a lot of additional complexity. I certainly agree that promises are more useful when used ubiquitously. However, promises have many advantages besides just being a common interface for asynchronous operations, including interface simplicity, composibility, and separation of concerns. But, your point about this being the first such API is really important. If we are going to use promises in the IndexedDB, I think they should the webapps group should be looking at them beyond the scope of just the IndexedDB API, and how they could be used in other APIs, such that common interface advantage could be realized. Looking at the broad perspective is key here. In general, IndexedDB has taken an approach of leaving ease of programming to libraries. There seems to be a good case to build libraries to make asynchronous programming with IndexedDB easier through the use of such mechanisms as promises. In fact, IndexedDB might be yet another area for libraries to slug it out. * ISTM that the entire space is still evolving quite rapidly. Many JavaScript libraries have implemented a form of this, and this proposal is also slightly different from any of them. I think it is premature to have browsers implement this while library authors are still hashing out best practice. Once it is in browsers, it's forever. Promises have been around for a number of years, we already have a lot of experience to draw from, this isn't exactly a brand new idea, promises are a well-established concept. The CommonJS proposal is nothing ground breaking, it is more based on the culmination of ideas of Dojo, ref_send and others. It is also worth noting that a number of JS libraries
Re: [IndexedDB] Promises (WAS: Seeking pre-LCWD comments for Indexed Database API; deadline February 2)
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 3/4/2010 11:46 AM, Nikunj Mehta wrote: On Mar 4, 2010, at 10:23 AM, Kris Zyp wrote: On 3/4/2010 11:08 AM, Aaron Boodman wrote: [snip] * There is nothing preventing JS authors from implementing a promise-style API on top of IndexedDB, if that is what they want to do. Yes, you can always make an API harder to use so that JS authors have more they can do with it ;). You will agree that we don't want to wait for one style of promises to win out over others before IndexedDB can be made available to programmers. Till the soil and let a thousand flowers bloom. The IndexedDB spec isn't and can't just sit back and not define the asynchronous interface. Like it or not, IndexedDB has defined a promise-like entity with the |DBRequest| interface. Why is inventing a new (and somewhat ugly) flower better than designing based on the many flowers that have already bloomed? - -- Kris Zyp SitePen (503) 806-1841 http://sitepen.com -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.9 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAkuQAiUACgkQ9VpNnHc4zAzZkgCeIjAVz56S3sR5BeKt8lZPGMJo 6rYAoJ4x4WJN9W9LhdXkbbJaT94A8/om =oJbA -END PGP SIGNATURE-
Re: [IndexedDB] Promises (WAS: Seeking pre-LCWD comments for Indexed Database API; deadline February 2)
On Mar 4, 2010, at 10:55 AM, Kris Zyp wrote: On 3/4/2010 11:46 AM, Nikunj Mehta wrote: On Mar 4, 2010, at 10:23 AM, Kris Zyp wrote: On 3/4/2010 11:08 AM, Aaron Boodman wrote: [snip] * There is nothing preventing JS authors from implementing a promise-style API on top of IndexedDB, if that is what they want to do. Yes, you can always make an API harder to use so that JS authors have more they can do with it ;). You will agree that we don't want to wait for one style of promises to win out over others before IndexedDB can be made available to programmers. Till the soil and let a thousand flowers bloom. The IndexedDB spec isn't and can't just sit back and not define the asynchronous interface. Like it or not, IndexedDB has defined a promise-like entity with the |DBRequest| interface. Why is inventing a new (and somewhat ugly) flower better than designing based on the many flowers that have already bloomed? I meant to say that the IndexedDB spec should be updated to use a model that supports promises. If the current one is not adequate then, by all means, let's make it. However, we don't need a full-fledged promises in IndexedDB. I hope you agree this time.
Re: [IndexedDB] Promises (WAS: Seeking pre-LCWD comments for Indexed Database API; deadline February 2)
On Wed, Mar 3, 2010 at 4:49 AM, Kris Zyp k...@sitepen.com wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 3/1/2010 2:52 PM, Jeremy Orlow wrote: Thanks for the pointers. I'm actually pretty sold on the general idea of promises, and my intuition is that there won't be a very big resource penalty for using an API like this rather than callbacks or what's currently specced. At the same time, it seems as though there isn't much of a standard in terms of the precise semantics and some of the techniques (such as optionally taking callbacks and not returning a promise if they are supplied) seems like a decent answer for pure javascript APIs, but maybe not as good for IDL and a standard like this. Do you guys have any recommendations for the precise semantics we'd use, if we used promises in IndexedDB? To get started, let me list what I'd propose and maybe you can offer counter proposals or feedback on what would or wouldn't work? Each method on a Request interface (the async ones in the spec) whose counterpart returns something other than void would instead return a Promise. Asynchronous counterparts to void-returning synchronous functions can still return promises. The promise would just resolve to undefined, but it still fulfills the role of indicating when the operation is complete. Good point! Silly me. The promises would only have a then method which would take in an onsuccess and onerror callback. Both are optional. The onsuccess function should take in a single parameter which matches the return value of the synchronous counterpart. The onerror function should take in an IDBDatabaseError. If the callbacks are null, undefined, or omitted, they're ignored. If they're anything else, we should probably either raise an exception immediately or ignore them. Yes. Any thoughts on whether we'd raise or ignore improper inputs? I'm leaning towards raise since it would be deterministic and silently ignoring seems like a headache from a developer standpoint. If there's an error, all onerror callbacks would be called with the IDBDatabaseError. Yes. Exceptions within callbacks would be ignored. With CommonJS promises, the promise returned by the then() call goes into an error state if a callback throws an exception. For example, someAsyncOperation.then(successHandler, function(){ throw new Error(test) }) .then(null, function(error){ console.log(error); }); Would log the thrown error, effectively giving you a way of catching the error. Are you suggesting this as a simplification so that IndexedDB impls doesn't have to worry about recursive creation of promises? If so, I suppose that seems like a reasonable simplification to me. Although if promises are something that could be potentially reused in other specs, it would be nice to have a quality solution, and I don't think this is a big implementation burden, I've implemented the recursive capabilities in dozen or two lines of JS code. But if burden is too onerous, I am fine with the simplification. When you say recursive capabilities are you just talking about how to handle exceptions, or something more? In terms of exceptions: I don't think it's an enormous implementational burden and thus I think it's fine to ignore that part of the equation. So the question mainly comes down to whether the added complexity is worth it. Can you think of any real-world examples of when this capability is useful in promises? If so, that'd definitely help us understand the pro's and con's. In terms of speccing, I'm not sure if we can get away with speccing one promise interface or whether we'd need to create one for each type of promise. Certainly the intent of promises is that there is exists only one generic promise interface that can be reused everywhere, at least from the JS perspective, not sure if the extra type constraints in IDL demand multiple interfaces to model promise's effectively parameterized generic type form. Unfortunately, I don't really know. Before we try speccing it, I'll definitely see if any WebIDL experts have suggestions. Also, do we want to explicitly spec what happens in the following case? window.indexedDB.open(...).then( function(db) { db.openObjectStore(a).then( function(os) { alert(Opened a); } ) } ).then( function(db) { alert(Second db opened); } ); Clearly the first function(db) is called first. But the question is whether it'd be a race of which alert is called first or whether the Second db opened alert should always be shown first (since clearly if the first is called, the second _can_ be fired immediately afterwards). I'm on the fence about whether it'd be useful to spec that the entire chain needs to be called one after the other before calling any other callbacks. Does anyone have thoughts on whether this is useful or not? If we do spec it to call the entire chain, then what happens if
Re: [IndexedDB] Promises (WAS: Seeking pre-LCWD comments for Indexed Database API; deadline February 2)
On Wed, Mar 3, 2010 at 11:01 AM, Jeremy Orlow jor...@chromium.org wrote: On Wed, Mar 3, 2010 at 4:49 AM, Kris Zyp k...@sitepen.com wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 3/1/2010 2:52 PM, Jeremy Orlow wrote: Thanks for the pointers. I'm actually pretty sold on the general idea of promises, and my intuition is that there won't be a very big resource penalty for using an API like this rather than callbacks or what's currently specced. At the same time, it seems as though there isn't much of a standard in terms of the precise semantics and some of the techniques (such as optionally taking callbacks and not returning a promise if they are supplied) seems like a decent answer for pure javascript APIs, but maybe not as good for IDL and a standard like this. Do you guys have any recommendations for the precise semantics we'd use, if we used promises in IndexedDB? To get started, let me list what I'd propose and maybe you can offer counter proposals or feedback on what would or wouldn't work? Each method on a Request interface (the async ones in the spec) whose counterpart returns something other than void would instead return a Promise. Asynchronous counterparts to void-returning synchronous functions can still return promises. The promise would just resolve to undefined, but it still fulfills the role of indicating when the operation is complete. Good point! Silly me. The promises would only have a then method which would take in an onsuccess and onerror callback. Both are optional. The onsuccess function should take in a single parameter which matches the return value of the synchronous counterpart. The onerror function should take in an IDBDatabaseError. If the callbacks are null, undefined, or omitted, they're ignored. If they're anything else, we should probably either raise an exception immediately or ignore them. Yes. Any thoughts on whether we'd raise or ignore improper inputs? I'm leaning towards raise since it would be deterministic and silently ignoring seems like a headache from a developer standpoint. If there's an error, all onerror callbacks would be called with the IDBDatabaseError. Yes. Exceptions within callbacks would be ignored. With CommonJS promises, the promise returned by the then() call goes into an error state if a callback throws an exception. For example, someAsyncOperation.then(successHandler, function(){ throw new Error(test) }) .then(null, function(error){ console.log(error); }); Would log the thrown error, effectively giving you a way of catching the error. Are you suggesting this as a simplification so that IndexedDB impls doesn't have to worry about recursive creation of promises? If so, I suppose that seems like a reasonable simplification to me. Although if promises are something that could be potentially reused in other specs, it would be nice to have a quality solution, and I don't think this is a big implementation burden, I've implemented the recursive capabilities in dozen or two lines of JS code. But if burden is too onerous, I am fine with the simplification. When you say recursive capabilities are you just talking about how to handle exceptions, or something more? In terms of exceptions: I don't think it's an enormous implementational burden and thus I think it's fine to ignore that part of the equation. So the question mainly comes down to whether the added complexity is worth it. Can you think of any real-world examples of when this capability is useful in promises? If so, that'd definitely help us understand the pro's and con's. I've been talking to a co-worker here who seems to know a decent amount about promises (as implemented in E) and some about differed (as implemented in Python's Twisted library). From talking to him, it seems that my original suggestion for not handling exceptions thrown inside a .then() callback is the way to go. It seems as though promises put a lot of weight on composability and making it so that the order of .then() calls not mattering. This means that you can then pass promises to other async interfaces and not have to worry about different timings leading to different results. It also means that if you pass a promise into multiple consumers (say, javascript libraries) you don't need to worry about one using a promise in a way that screws up another. Differed seems to be more expressive and flexible. For example, instead of doing this: window.indexedDB.open(...).then( function(db) { db.openObjectStore(a).then( function(os) { os.get(x).then( function(value) { alert(Value: + value); } ) } ) } ); I could do this: window.indexedDB.open(...).then( function(db) { return db.openObjectStore(a); }// Note the return value is passed on to the next step. ).then( function(os) { return os.get(x); } ).then( function(value) {
Re: [IndexedDB] Promises (WAS: Seeking pre-LCWD comments for Indexed Database API; deadline February 2)
Erm... s/differed/deferred/g On Wed, Mar 3, 2010 at 4:58 PM, Jeremy Orlow jor...@chromium.org wrote: On Wed, Mar 3, 2010 at 11:01 AM, Jeremy Orlow jor...@chromium.org wrote: On Wed, Mar 3, 2010 at 4:49 AM, Kris Zyp k...@sitepen.com wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 3/1/2010 2:52 PM, Jeremy Orlow wrote: Thanks for the pointers. I'm actually pretty sold on the general idea of promises, and my intuition is that there won't be a very big resource penalty for using an API like this rather than callbacks or what's currently specced. At the same time, it seems as though there isn't much of a standard in terms of the precise semantics and some of the techniques (such as optionally taking callbacks and not returning a promise if they are supplied) seems like a decent answer for pure javascript APIs, but maybe not as good for IDL and a standard like this. Do you guys have any recommendations for the precise semantics we'd use, if we used promises in IndexedDB? To get started, let me list what I'd propose and maybe you can offer counter proposals or feedback on what would or wouldn't work? Each method on a Request interface (the async ones in the spec) whose counterpart returns something other than void would instead return a Promise. Asynchronous counterparts to void-returning synchronous functions can still return promises. The promise would just resolve to undefined, but it still fulfills the role of indicating when the operation is complete. Good point! Silly me. The promises would only have a then method which would take in an onsuccess and onerror callback. Both are optional. The onsuccess function should take in a single parameter which matches the return value of the synchronous counterpart. The onerror function should take in an IDBDatabaseError. If the callbacks are null, undefined, or omitted, they're ignored. If they're anything else, we should probably either raise an exception immediately or ignore them. Yes. Any thoughts on whether we'd raise or ignore improper inputs? I'm leaning towards raise since it would be deterministic and silently ignoring seems like a headache from a developer standpoint. If there's an error, all onerror callbacks would be called with the IDBDatabaseError. Yes. Exceptions within callbacks would be ignored. With CommonJS promises, the promise returned by the then() call goes into an error state if a callback throws an exception. For example, someAsyncOperation.then(successHandler, function(){ throw new Error(test) }) .then(null, function(error){ console.log(error); }); Would log the thrown error, effectively giving you a way of catching the error. Are you suggesting this as a simplification so that IndexedDB impls doesn't have to worry about recursive creation of promises? If so, I suppose that seems like a reasonable simplification to me. Although if promises are something that could be potentially reused in other specs, it would be nice to have a quality solution, and I don't think this is a big implementation burden, I've implemented the recursive capabilities in dozen or two lines of JS code. But if burden is too onerous, I am fine with the simplification. When you say recursive capabilities are you just talking about how to handle exceptions, or something more? In terms of exceptions: I don't think it's an enormous implementational burden and thus I think it's fine to ignore that part of the equation. So the question mainly comes down to whether the added complexity is worth it. Can you think of any real-world examples of when this capability is useful in promises? If so, that'd definitely help us understand the pro's and con's. I've been talking to a co-worker here who seems to know a decent amount about promises (as implemented in E) and some about differed (as implemented in Python's Twisted library). From talking to him, it seems that my original suggestion for not handling exceptions thrown inside a .then() callback is the way to go. It seems as though promises put a lot of weight on composability and making it so that the order of .then() calls not mattering. This means that you can then pass promises to other async interfaces and not have to worry about different timings leading to different results. It also means that if you pass a promise into multiple consumers (say, javascript libraries) you don't need to worry about one using a promise in a way that screws up another. Differed seems to be more expressive and flexible. For example, instead of doing this: window.indexedDB.open(...).then( function(db) { db.openObjectStore(a).then( function(os) { os.get(x).then( function(value) { alert(Value: + value); } ) } ) } ); I could do this: window.indexedDB.open(...).then( function(db) { return db.openObjectStore(a); }// Note
Re: [IndexedDB] Promises (WAS: Seeking pre-LCWD comments for Indexed Database API; deadline February 2)
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 3/3/2010 4:01 AM, Jeremy Orlow wrote: On Wed, Mar 3, 2010 at 4:49 AM, Kris Zyp k...@sitepen.com mailto:k...@sitepen.com wrote: [snip] The promises would only have a then method which would take in an onsuccess and onerror callback. Both are optional. The onsuccess function should take in a single parameter which matches the return value of the synchronous counterpart. The onerror function should take in an IDBDatabaseError. If the callbacks are null, undefined, or omitted, they're ignored. If they're anything else, we should probably either raise an exception immediately or ignore them. Yes. Any thoughts on whether we'd raise or ignore improper inputs? I'm leaning towards raise since it would be deterministic and silently ignoring seems like a headache from a developer standpoint. Throwing an error on improper inputs is fine with me. If there's an error, all onerror callbacks would be called with the IDBDatabaseError. Yes. Exceptions within callbacks would be ignored. With CommonJS promises, the promise returned by the then() call goes into an error state if a callback throws an exception. For example, someAsyncOperation.then(successHandler, function(){ throw new Error(test) }) .then(null, function(error){ console.log(error); }); Would log the thrown error, effectively giving you a way of catching the error. Are you suggesting this as a simplification so that IndexedDB impls doesn't have to worry about recursive creation of promises? If so, I suppose that seems like a reasonable simplification to me. Although if promises are something that could be potentially reused in other specs, it would be nice to have a quality solution, and I don't think this is a big implementation burden, I've implemented the recursive capabilities in dozen or two lines of JS code. But if burden is too onerous, I am fine with the simplification. When you say recursive capabilities are you just talking about how to handle exceptions, or something more? In terms of exceptions: I don't think it's an enormous implementational burden and thus I think it's fine to ignore that part of the equation. So the question mainly comes down to whether the added complexity is worth it. Can you think of any real-world examples of when this capability is useful in promises? If so, that'd definitely help us understand the pro's and con's. Maybe I misunderstanding your suggestion. By recursive capability I meant having then() return a promise (that is fulfilled with the result of executing the callback), and I thought you were suggesting that instead, then() would not return a promise. If then() returns a promise, I think the returned promise should clearly go into an error state if the callback throws an error. The goal of promises is to asynchronously model computations, and if a computation throws, it should result in the associated promise entering error state. The promise returned by then() exists to represent the result of the execution of the callback, and so it should resolve to the value returned by the callback or an error if the callback throws. Silenty swallowing errors seems highly undesirable. Now if we are simplifying then() to not return a promise at all, than I would think callbacks would just behave like any other event listener in regards to uncaught errors. In terms of speccing, I'm not sure if we can get away with speccing one promise interface or whether we'd need to create one for each type of promise. Certainly the intent of promises is that there is exists only one generic promise interface that can be reused everywhere, at least from the JS perspective, not sure if the extra type constraints in IDL demand multiple interfaces to model promise's effectively parameterized generic type form. Unfortunately, I don't really know. Before we try speccing it, I'll definitely see if any WebIDL experts have suggestions. Also, do we want to explicitly spec what happens in the following case? window.indexedDB.open(...).then( function(db) { db.openObjectStore(a).then( function(os) { alert(Opened a); } ) } ).then( function(db) { alert(Second db opened); } ); Clearly the first function(db) is called first. But the question is whether it'd be a race of which alert is called first or whether the Second db opened alert should always be shown first (since clearly if the first is called, the second _can_ be fired immediately afterwards). I'm on the fence about whether it'd be useful to spec that the entire chain needs to be called one after the other before calling any other callbacks. Does anyone have thoughts on whether this is useful or not? If we do spec it
Re: [IndexedDB] Promises (WAS: Seeking pre-LCWD comments for Indexed Database API; deadline February 2)
On Mon, Mar 1, 2010 at 9:52 PM, Jeremy Orlow jor...@chromium.org wrote: Thanks for the pointers. I'm actually pretty sold on the general idea of promises, and my intuition is that there won't be a very big resource penalty for using an API like this rather than callbacks or what's currently specced. At the same time, it seems as though there isn't much of a standard in terms of the precise semantics and some of the techniques (such as optionally taking callbacks and not returning a promise if they are supplied) seems like a decent answer for pure javascript APIs, but maybe not as good for IDL and a standard like this. Do you guys have any recommendations for the precise semantics we'd use, if we used promises in IndexedDB? To get started, let me list what I'd propose and maybe you can offer counter proposals or feedback on what would or wouldn't work? Each method on a Request interface (the async ones in the spec) whose counterpart returns something other than void would instead return a Promise. The promises would only have a then method which would take in an onsuccess and onerror callback. Both are optional. The onsuccess function should take in a single parameter which matches the return value of the synchronous counterpart. The onerror function should take in an IDBDatabaseError. If the callbacks are null, undefined, or omitted, they're ignored. If they're anything else, we should probably either raise an exception immediately or ignore them. If there's an error, all onerror callbacks would be called with the IDBDatabaseError. Exceptions within callbacks would be ignored. Oh. And the callbacks should probably be enqueued in the main event loop when the result/error is ready. This way, even if the result is already available when .then() is called, an implementation won't simply call the callback immediately (in a nested fashion). This will ensure consistent behavior despite implementational differences and races. In terms of speccing, I'm not sure if we can get away with speccing one promise interface or whether we'd need to create one for each type of promise. On Thu, Feb 18, 2010 at 4:20 PM, Kris Zyp k...@sitepen.com wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 2/18/2010 5:31 AM, Jeremy Orlow wrote: On Wed, Jan 27, 2010 at 9:46 PM, Kris Zyp k...@sitepen.com mailto:k...@sitepen.com k...@sitepen.com wrote: * Use promises for async interfaces - In server side JavaScript, most projects are moving towards using promises for asynchronous interfaces instead of trying to define the specific callback parameters for each interface. I believe the advantages of using promises over callbacks are pretty well understood in terms of decoupling async semantics from interface definitions, and improving encapsulation of concerns. For the indexed database API this would mean that sync and async interfaces could essentially look the same except sync would return completed values and async would return promises. I realize that defining a promise interface would have implications beyond the indexed database API, as the goal of promises is to provide a consistent interface for asynchronous interaction across components, but perhaps this would be a good time for the W3C to define such an API. It seems like the indexed database API would be a perfect interface to leverage promises. If you are interested in proposal, there is one from CommonJS here [1] (the get() and call() wouldn't apply here). With this interface, a promise.then(callback, errorHandler) function is the only function a promise would need to provide. [1] http://wiki.commonjs.org/wiki/Promises Very interesting. The general concept seems promising and fairly flexible. You can easily code in a similar style to normal async/callback semantics, but it seems like you have a lot more flexibility. I do have a few questions though. Are there any good examples of these used in the wild that you can point me towards? I used my imagination for prototyping up some examples, but it'd be great to see some real examples + be able to see the exact semantics used in those implementations. Promises are heavily used in the E programming language, the Twisted project (python). In JavaScript land, Dojo's Deferred's are an example of a form of promises and a number of SSJS projects including Node and Narwhal. To see some examples, you can look at the Dojo's docs [1] (note that Dojo's spells it addCallback and addErrback instead of then, however we are looking to possibly move to the CommonJS promise for Dojo 2.0). Here is somewhat random example of module that uses Deferred's [2] [1] http://api.dojotoolkit.org/jsdoc/1.3/dojo.Deferred [2]
Re: [IndexedDB] Promises (WAS: Seeking pre-LCWD comments for Indexed Database API; deadline February 2)
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 3/1/2010 2:52 PM, Jeremy Orlow wrote: Thanks for the pointers. I'm actually pretty sold on the general idea of promises, and my intuition is that there won't be a very big resource penalty for using an API like this rather than callbacks or what's currently specced. At the same time, it seems as though there isn't much of a standard in terms of the precise semantics and some of the techniques (such as optionally taking callbacks and not returning a promise if they are supplied) seems like a decent answer for pure javascript APIs, but maybe not as good for IDL and a standard like this. Do you guys have any recommendations for the precise semantics we'd use, if we used promises in IndexedDB? To get started, let me list what I'd propose and maybe you can offer counter proposals or feedback on what would or wouldn't work? Each method on a Request interface (the async ones in the spec) whose counterpart returns something other than void would instead return a Promise. Asynchronous counterparts to void-returning synchronous functions can still return promises. The promise would just resolve to undefined, but it still fulfills the role of indicating when the operation is complete. The promises would only have a then method which would take in an onsuccess and onerror callback. Both are optional. The onsuccess function should take in a single parameter which matches the return value of the synchronous counterpart. The onerror function should take in an IDBDatabaseError. If the callbacks are null, undefined, or omitted, they're ignored. If they're anything else, we should probably either raise an exception immediately or ignore them. Yes. If there's an error, all onerror callbacks would be called with the IDBDatabaseError. Yes. Exceptions within callbacks would be ignored. With CommonJS promises, the promise returned by the then() call goes into an error state if a callback throws an exception. For example, someAsyncOperation.then(successHandler, function(){ throw new Error(test) }) .then(null, function(error){ console.log(error); }); Would log the thrown error, effectively giving you a way of catching the error. Are you suggesting this as a simplification so that IndexedDB impls doesn't have to worry about recursive creation of promises? If so, I suppose that seems like a reasonable simplification to me. Although if promises are something that could be potentially reused in other specs, it would be nice to have a quality solution, and I don't think this is a big implementation burden, I've implemented the recursive capabilities in dozen or two lines of JS code. But if burden is too onerous, I am fine with the simplification. In terms of speccing, I'm not sure if we can get away with speccing one promise interface or whether we'd need to create one for each type of promise. Certainly the intent of promises is that there is exists only one generic promise interface that can be reused everywhere, at least from the JS perspective, not sure if the extra type constraints in IDL demand multiple interfaces to model promise's effectively parameterized generic type form. - -- Kris Zyp SitePen (503) 806-1841 http://sitepen.com -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.9 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAkuN6kkACgkQ9VpNnHc4zAwsewCfcqu8L1ZTSU0NUoAL5pG/i+uO A98An1y2XK2ylsVxVwOxjrsWbn4Jd+y0 =7yq3 -END PGP SIGNATURE-
Re: [IndexedDB] Promises (WAS: Seeking pre-LCWD comments for Indexed Database API; deadline February 2)
Thanks for the pointers. I'm actually pretty sold on the general idea of promises, and my intuition is that there won't be a very big resource penalty for using an API like this rather than callbacks or what's currently specced. At the same time, it seems as though there isn't much of a standard in terms of the precise semantics and some of the techniques (such as optionally taking callbacks and not returning a promise if they are supplied) seems like a decent answer for pure javascript APIs, but maybe not as good for IDL and a standard like this. Do you guys have any recommendations for the precise semantics we'd use, if we used promises in IndexedDB? To get started, let me list what I'd propose and maybe you can offer counter proposals or feedback on what would or wouldn't work? Each method on a Request interface (the async ones in the spec) whose counterpart returns something other than void would instead return a Promise. The promises would only have a then method which would take in an onsuccess and onerror callback. Both are optional. The onsuccess function should take in a single parameter which matches the return value of the synchronous counterpart. The onerror function should take in an IDBDatabaseError. If the callbacks are null, undefined, or omitted, they're ignored. If they're anything else, we should probably either raise an exception immediately or ignore them. If there's an error, all onerror callbacks would be called with the IDBDatabaseError. Exceptions within callbacks would be ignored. In terms of speccing, I'm not sure if we can get away with speccing one promise interface or whether we'd need to create one for each type of promise. On Thu, Feb 18, 2010 at 4:20 PM, Kris Zyp k...@sitepen.com wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 2/18/2010 5:31 AM, Jeremy Orlow wrote: On Wed, Jan 27, 2010 at 9:46 PM, Kris Zyp k...@sitepen.com mailto:k...@sitepen.com k...@sitepen.com wrote: * Use promises for async interfaces - In server side JavaScript, most projects are moving towards using promises for asynchronous interfaces instead of trying to define the specific callback parameters for each interface. I believe the advantages of using promises over callbacks are pretty well understood in terms of decoupling async semantics from interface definitions, and improving encapsulation of concerns. For the indexed database API this would mean that sync and async interfaces could essentially look the same except sync would return completed values and async would return promises. I realize that defining a promise interface would have implications beyond the indexed database API, as the goal of promises is to provide a consistent interface for asynchronous interaction across components, but perhaps this would be a good time for the W3C to define such an API. It seems like the indexed database API would be a perfect interface to leverage promises. If you are interested in proposal, there is one from CommonJS here [1] (the get() and call() wouldn't apply here). With this interface, a promise.then(callback, errorHandler) function is the only function a promise would need to provide. [1] http://wiki.commonjs.org/wiki/Promises Very interesting. The general concept seems promising and fairly flexible. You can easily code in a similar style to normal async/callback semantics, but it seems like you have a lot more flexibility. I do have a few questions though. Are there any good examples of these used in the wild that you can point me towards? I used my imagination for prototyping up some examples, but it'd be great to see some real examples + be able to see the exact semantics used in those implementations. Promises are heavily used in the E programming language, the Twisted project (python). In JavaScript land, Dojo's Deferred's are an example of a form of promises and a number of SSJS projects including Node and Narwhal. To see some examples, you can look at the Dojo's docs [1] (note that Dojo's spells it addCallback and addErrback instead of then, however we are looking to possibly move to the CommonJS promise for Dojo 2.0). Here is somewhat random example of module that uses Deferred's [2] [1] http://api.dojotoolkit.org/jsdoc/1.3/dojo.Deferred [2] http://download.dojotoolkit.org/release-1.4.1/dojo-release-1.4.1/dojox/rpc/JsonRest.js I see that you can supply an error handling callback to .then(), but does that only apply to the one operation? I could easily imagine emulating try/catch type semantics and have errors continue down the line of .then's until someone handles it. It might even make sense to allow the error handlers to re-raise (i.e. allow to bubble) errors so that later routines would get them as well. Yes, that's exactly right, errors can be
[IndexedDB] Promises (WAS: Seeking pre-LCWD comments for Indexed Database API; deadline February 2)
On Wed, Jan 27, 2010 at 9:46 PM, Kris Zyp k...@sitepen.com wrote: * Use promises for async interfaces - In server side JavaScript, most projects are moving towards using promises for asynchronous interfaces instead of trying to define the specific callback parameters for each interface. I believe the advantages of using promises over callbacks are pretty well understood in terms of decoupling async semantics from interface definitions, and improving encapsulation of concerns. For the indexed database API this would mean that sync and async interfaces could essentially look the same except sync would return completed values and async would return promises. I realize that defining a promise interface would have implications beyond the indexed database API, as the goal of promises is to provide a consistent interface for asynchronous interaction across components, but perhaps this would be a good time for the W3C to define such an API. It seems like the indexed database API would be a perfect interface to leverage promises. If you are interested in proposal, there is one from CommonJS here [1] (the get() and call() wouldn't apply here). With this interface, a promise.then(callback, errorHandler) function is the only function a promise would need to provide. [1] http://wiki.commonjs.org/wiki/Promises Very interesting. The general concept seems promising and fairly flexible. You can easily code in a similar style to normal async/callback semantics, but it seems like you have a lot more flexibility. I do have a few questions though. Are there any good examples of these used in the wild that you can point me towards? I used my imagination for prototyping up some examples, but it'd be great to see some real examples + be able to see the exact semantics used in those implementations. I see that you can supply an error handling callback to .then(), but does that only apply to the one operation? I could easily imagine emulating try/catch type semantics and have errors continue down the line of .then's until someone handles it. It might even make sense to allow the error handlers to re-raise (i.e. allow to bubble) errors so that later routines would get them as well. Maybe you'd even want it to bubble by default? What have other implementations done with this stuff? What is the most robust and least cumbersome for typical applications? (And, in te complete absence of real experience, are there any expert opinions on what might work?) Overall this seems fairly promising and not that hard to implement. Do others see pitfalls that I'm missing? J
Re: [IndexedDB] Promises (WAS: Seeking pre-LCWD comments for Indexed Database API; deadline February 2)
On Feb 18, 2010, at 4: 31AM, Jeremy Orlow wrote Very interesting. The general concept seems promising and fairly flexible. You can easily code in a similar style to normal async/callback semantics, but it seems like you have a lot more flexibility. I do have a few questions though. Are there any good examples of these used in the wild that you can point me towards? I used my imagination for prototyping up some examples, but it'd be great to see some real examples + be able to see the exact semantics used in those implementations. The node.js community has some experience with promises. Here was a recent discussion they had on promises and alternatives (although I think it was primarily syntax driven): http://groups.google.com/group/nodejs/browse_thread/thread/78ad3478317ee19c/625b1d0f013206fa If you're unfamiliar with node.js [1], it strives to always be asynchronous and non-blocking. There are a number of database wrapper modules, nearly all of which should give examples of using promises: http://wiki.github.com/ry/node/modules#database - Joe [1]: http://nodejs.org/
Re: [IndexedDB] Promises (WAS: Seeking pre-LCWD comments for Indexed Database API; deadline February 2)
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 2/18/2010 5:31 AM, Jeremy Orlow wrote: On Wed, Jan 27, 2010 at 9:46 PM, Kris Zyp k...@sitepen.com mailto:k...@sitepen.com wrote: * Use promises for async interfaces - In server side JavaScript, most projects are moving towards using promises for asynchronous interfaces instead of trying to define the specific callback parameters for each interface. I believe the advantages of using promises over callbacks are pretty well understood in terms of decoupling async semantics from interface definitions, and improving encapsulation of concerns. For the indexed database API this would mean that sync and async interfaces could essentially look the same except sync would return completed values and async would return promises. I realize that defining a promise interface would have implications beyond the indexed database API, as the goal of promises is to provide a consistent interface for asynchronous interaction across components, but perhaps this would be a good time for the W3C to define such an API. It seems like the indexed database API would be a perfect interface to leverage promises. If you are interested in proposal, there is one from CommonJS here [1] (the get() and call() wouldn't apply here). With this interface, a promise.then(callback, errorHandler) function is the only function a promise would need to provide. [1] http://wiki.commonjs.org/wiki/Promises Very interesting. The general concept seems promising and fairly flexible. You can easily code in a similar style to normal async/callback semantics, but it seems like you have a lot more flexibility. I do have a few questions though. Are there any good examples of these used in the wild that you can point me towards? I used my imagination for prototyping up some examples, but it'd be great to see some real examples + be able to see the exact semantics used in those implementations. Promises are heavily used in the E programming language, the Twisted project (python). In JavaScript land, Dojo's Deferred's are an example of a form of promises and a number of SSJS projects including Node and Narwhal. To see some examples, you can look at the Dojo's docs [1] (note that Dojo's spells it addCallback and addErrback instead of then, however we are looking to possibly move to the CommonJS promise for Dojo 2.0). Here is somewhat random example of module that uses Deferred's [2] [1] http://api.dojotoolkit.org/jsdoc/1.3/dojo.Deferred [2] http://download.dojotoolkit.org/release-1.4.1/dojo-release-1.4.1/dojox/rpc/JsonRest.js I see that you can supply an error handling callback to .then(), but does that only apply to the one operation? I could easily imagine emulating try/catch type semantics and have errors continue down the line of .then's until someone handles it. It might even make sense to allow the error handlers to re-raise (i.e. allow to bubble) errors so that later routines would get them as well. Yes, that's exactly right, errors can be raised/thrown and propagate (when an error handling callback is not provided) to the next promise, and be caught (with an error handler) just as you have expected from the analogous propagation of errors across stack frames in JS. Maybe you'd even want it to bubble by default? What have other implementations done with this stuff? What is the most robust and least cumbersome for typical applications? (And, in te complete absence of real experience, are there any expert opinions on what might work?) I think it is pretty clear you want propagation, just like with normal sync errors, it is very handy to have a catch/error handler low down in the stack to generically handle various errors. Overall this seems fairly promising and not that hard to implement. Do others see pitfalls that I'm missing? There are certainly numerous design decisions that can be made with promises. * If an error occurs and an error handler is not provided in the current event turn (note that an event handler can be provided at any point in the future), should the error be logged somewhere? * If an callback handler is added to an already fulfilled promise, should the callback be executed immediately or in the next event turn? Most JS impls execute immediately, but E suggests otherwise. * One pitfall that a number of prior implementations have made is in having callback's return value mutate the current promise instead of returning the new one, the CommonJS spec makes it clear that then() should return a new promise that receives the return values from the callback. - -- Kris Zyp SitePen (503) 806-1841 http://sitepen.com -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.9 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAkt9aNIACgkQ9VpNnHc4zAxMBgCfUG0/CVTgV15MBe8uQRDc6RPW