Re: yield and Promises
On 20/10/2011, at 23:37, Brendan Eich wrote:
On Oct 20, 2011, at 12:59 PM, Jorge wrote:
the assert_invariants() at the next line might run in another turn of the
event loop (when f() resumes), just as the callback does.
No. Nothing in JS today, since it lacks coroutines or call/cc, can suspend
under f and cause the continuation to be captured and then called in a later
event loop turn.
That's why I put the comment //might suspend execution !
*IF* it had coroutines or call/cc, then
the assert_invariants() at the next line might run in another turn of the
event loop (when f() resumes), just as the callback does.
and then, as far as I can see, the risks wrt invariants would be exactly the
same in the two cases:
//#1
assert_invariants();
function callBack () {
assert_invariants(); // perhaps yes, perhaps no. There's no guarantee.
};
setTimeout(callBack, 1e3);
return;
//#2
assert_invariants();
f(); //might suspend execution
assert_invariants(); // perhaps yes, perhaps no. There's no guarantee either.
return;
And my point is that the invariants not invariant anymore argument against
call/cc (that the node.js guys keep harping on again and again) does not hold
for this kind of async code written in cps because this kind of async code
written in cps does not guarantee it either.
On the other hand, *IF* we could suspend f(), instead of:
asyncFunction(request, cb);
function cb (e, response) {
if (e) //whatever
//our code continues here
}
we could simply write the above like this:
try {
response = asyncFunction(request); //might suspend execution
}
catch (e) {
//whatever
}
//our code continues here
And this has several (valuable, imo) advantages:
- We aren't trashing the call stack on every async call: we can finally debug
properly!
- We can (finally!) catch the exceptions where and when it matters.
- We can loop and control flow in the usual ways (at last!).
- It's the habitual style of coding that everybody knows already.
--
Jorge.
___
es-discuss mailing list
es-discuss@mozilla.org
https://mail.mozilla.org/listinfo/es-discuss
Re: yield and Promises
On 21/10/2011, at 11:07, Jorge wrote: And this has several (valuable, imo) advantages: - We aren't trashing the call stack on every async call: we can finally debug properly! - We can (finally!) catch the exceptions where and when it matters. - We can loop and control flow in the usual ways (at last!). - It's the habitual style of coding that everybody knows already. One more: - We won't have to keep pumping data upwards in the contexts in the closure (from the callback), and/or nesting them (both the contexts and the callbacks). -- Jorge ___ es-discuss mailing list es-discuss@mozilla.org https://mail.mozilla.org/listinfo/es-discuss
Re: yield and Promises
You can disagree with anything if you're allowed to change the terms of the
discussion. :)
Brendan said JS is run-to-completion, which means that if you call a function
and control returns to you, no intervening threads of control have executed in
the meantime. But then you changed his example to this:
//#1
assert_invariants();
function callBack () {
assert_invariants(); // perhaps yes, perhaps no. There's no guarantee.
};
setTimeout(callBack, 1e3);
return;
Now matter how you line up the whitespace, the semantics of a function does not
guarantee that the function will be called right now. When a programmer
explicitly puts something in a function (the function callBack here), they are
saying here is some code that can be run at any arbitrary time. They are
expressing that *explicitly*. Whereas in a semantics with
fibers/coroutines/call/cc:
//#2
assert_invariants();
f(); //might suspend execution
assert_invariants(); // perhaps yes, perhaps no. There's no guarantee either.
return;
the mere *calling* of any function is *implicitly* giving permission to suspend
the *entire continuation* (of the current event queue turn) and continue it at
any point later on, after any other threads of control may have executed.
If you want to claim these two things are equivalent, I feel pretty confident
predicting this conversation will quickly descend into the Turing tarpit...
Dave
___
es-discuss mailing list
es-discuss@mozilla.org
https://mail.mozilla.org/listinfo/es-discuss
Re: yield and Promises
On Oct 21, 2011, at 9:34 AM, John J Barton wrote: Can anyone summarize how these proposals relate to Kris Kowal / Kris Zyp / Mark Miller Q library: https://github.com/kriskowal/q Did you see https://github.com/kriskowal/q/tree/master/examples/async-generators yet? In my experience, reasoning about the code was much easier with Q than without Q. (Not something I found in trying generators). In order to say something that isn't subjective yet content-free other than negative, what was hard to reason about, and why? Can you give three examples? /be I found the documentation hard to follow since it assumes background I don't have and the examples were not interesting to me, but once I tried it I was pleasantly surprised. It does have ergonomic issues, an undefined and resolved promise work equally well, but I think this may be inexperience on my part. jjb ___ es-discuss mailing list es-discuss@mozilla.org https://mail.mozilla.org/listinfo/es-discuss ___ es-discuss mailing list es-discuss@mozilla.org https://mail.mozilla.org/listinfo/es-discuss
Re: yield and Promises
Hi Kris,
Your proposal has a lot of similarities to
http://wiki.ecmascript.org/doku.php?id=strawman:deferred_functions
which was proposed this past spring.
I'm not sure I follow what's top-down vs bottom-up about the two different
approaches. Let me suggest some terminology that has emerged in the proposal
process: I'll use generators to mean any single-frame, one-shot continuation
feature that's independent of the host event queue, and deferred functions to
mean any single-frame, one-shot continuation feature that is tied to the host
event queue by means of being automatically scheduled.
Generators directly solve a problem that is much less significant in normal
JS coding. While it is exciting that generators coupled with libraries give
us a much better tool for asynchronous use cases (the above can be coded with
libraryFunction(function*(){...}), my concern is that the majority use case
is the one that requires libraries rather than the minority case, and does
not promote interoperability.
It's true that generators require libraries in order to use them for writing
asynchronous code in direct style. And I agree with you and Alex and Arv that
there is a cost to not standardizing on those libraries. There are different
frameworks with similar but incompatible idioms for Deferred objects, Promises,
and the like, and they could be standardized.
A couple years later, I believe the landscape has dramatically changed, and
we indeed do have significant convergence on a promise API with the
thenable interface. From Dojo, to jQuery, to several server side libraries,
and apparently even Windows 8's JS APIs (from what I understand) all share an
intersection of APIs that include a then() method as a method to define a
promise and register a callback for when a promise is fulfilled (or fails).
This is an unusual level of convergence for a JS community that is so
diverse. I believe this gives evidence of well substantiated and tested
interface that can be used for top-controlled single-frame continuations that
can easily be specified, understood and used by developers.
But there's more to it than just the interface. You fix a particular scheduling
semantics when you put deferred functions into the language. I'm still learning
about the difference between the Deferred pattern and the Promises pattern, but
the former seems much more stateful than the latter: you enqueue another
listener onto an internal mutable queue. I'm not sure how much state can be
avoided with listeners (at the end of the day, callbacks have to be invoked in
some particular order), but that concerned me when I saw the deferred functions
proposal. I can't prove to you that that scheduling policy isn't the right one,
but I'm not ready to say it is.
So I'm not sure all scheduling policies are created equal. And with generators,
at least people have the freedom to try out different ones. I'm currently
trying one with task.js, and I hope others will try to come up with their own.
(There's also the added benefit that by writing the scheduler in JS, you can
instrument and build cool tools like record-and-reply debugging.)
Dave
___
es-discuss mailing list
es-discuss@mozilla.org
https://mail.mozilla.org/listinfo/es-discuss
Re: yield and Promises
On Fri, Oct 21, 2011 at 9:41 AM, Brendan Eich bren...@mozilla.com wrote: On Oct 21, 2011, at 9:34 AM, John J Barton wrote: Can anyone summarize how these proposals relate to Kris Kowal / Kris Zyp / Mark Miller Q library: https://github.com/kriskowal/q Did you see https://github.com/kriskowal/q/tree/master/examples/async-generators yet? Thanks, I think that page clarifies my issue with generators: they solve a problem I don't have. See below. In my experience, reasoning about the code was much easier with Q than without Q. (Not something I found in trying generators). In order to say something that isn't subjective yet content-free other than negative, what was hard to reason about, and why? Can you give three examples? I only mentioned generators since Zyp's proposal uses yield. Now I'm regretting it because I really wanted to highlight Q. My comment is entirely subjective and intended to be positive about Q. The examples on the async-generators page cites above are clearer than the ones on the MDC generators page because they focus on next(). The strong case for generators is support for generic, encapsulated iteration. Examples illustrating this power would go a long way to build the case for them IMO. Examples of quirky iteration do not. Now back to Zyp's key point: using async functionality for iteration, powerful or not, does not address the key use-case for async. In particular, Q simplifies joining parallel async operations (XHR, postMessages, 'load', 'progress' events). Of course it may well be that generators provide an elegant solution to this important problem, but I've not seen such examples. jjb ___ es-discuss mailing list es-discuss@mozilla.org https://mail.mozilla.org/listinfo/es-discuss
Re: yield and Promises
On Fri, Oct 21, 2011 at 10:20 AM, John J Barton johnjbar...@johnjbarton.com wrote: On Fri, Oct 21, 2011 at 9:41 AM, Brendan Eich bren...@mozilla.com wrote: On Oct 21, 2011, at 9:34 AM, John J Barton wrote: Can anyone summarize how these proposals relate to Kris Kowal / Kris Zyp / Mark Miller Q library: https://github.com/kriskowal/q In the credit where due dept., the original Q library is Tyler Close's ref_send library. Other related and prior work is linked to at http://wiki.ecmascript.org/doku.php?id=strawman:concurrency#see Did you see https://github.com/kriskowal/q/tree/master/examples/async-generators yet? Thanks, I think that page clarifies my issue with generators: they solve a problem I don't have. In that case, you might be equally uninterested ;) in http://wiki.ecmascript.org/doku.php?id=strawman:async_functions#reference_implementation which shows how to do the same thing with generators as proposed for ES-next. See below. In my experience, reasoning about the code was much easier with Q than without Q. (Not something I found in trying generators). In order to say something that isn't subjective yet content-free other than negative, what was hard to reason about, and why? Can you give three examples? I only mentioned generators since Zyp's proposal uses yield. Now I'm regretting it because I really wanted to highlight Q. My comment is entirely subjective and intended to be positive about Q. Thanks! I am positive about Q as well. And yes, I like Kris Kowal's implementation. The examples on the async-generators page cites above are clearer than the ones on the MDC generators page because they focus on next(). The strong case for generators is support for generic, encapsulated iteration. Examples illustrating this power would go a long way to build the case for them IMO. Examples of quirky iteration do not. Now back to Zyp's key point: using async functionality for iteration, powerful or not, does not address the key use-case for async. In particular, Q simplifies joining parallel async operations (XHR, postMessages, 'load', 'progress' events). Of course it may well be that generators provide an elegant solution to this important problem, but I've not seen such examples. Have you seen http://wiki.ecmascript.org/doku.php?id=strawman:concurrency#q.race and http://wiki.ecmascript.org/doku.php?id=strawman:concurrency#q.all ? If these don't address the joining you have in mind, could you post some examples? Thanks. jjb ___ es-discuss mailing list es-discuss@mozilla.org https://mail.mozilla.org/listinfo/es-discuss -- Cheers, --MarkM ___ es-discuss mailing list es-discuss@mozilla.org https://mail.mozilla.org/listinfo/es-discuss
Re: yield and Promises
On Oct 21, 2011, at 10:20 AM, John J Barton wrote: My comment is entirely subjective and intended to be positive about Q. We like Q too. :-) The examples on the async-generators page cites above are clearer than the ones on the MDC generators page because they focus on next(). The strong case for generators is support for generic, encapsulated iteration. Examples illustrating this power would go a long way to build the case for them IMO. Examples of quirky iteration do not. I agree, and thanks for the MDC comments -- I'll get some editing help deployed. Now back to Zyp's key point: using async functionality for iteration, powerful or not, does not address the key use-case for async. Not by itself. But to avoid rehashing, I'll stop here. In particular, Q simplifies joining parallel async operations (XHR, postMessages, 'load', 'progress' events). Of course it may well be that generators provide an elegant solution to this important problem, but I've not seen such examples. Not having to nest a callback and risk closure leaks (which Erik Corry nicely diagrammed in his JSConf.eu talk) is important. Please attend to this recurrent problem, about which I've been crystal clear. /be ___ es-discuss mailing list es-discuss@mozilla.org https://mail.mozilla.org/listinfo/es-discuss
Re: yield and Promises
On 21/10/2011, at 17:40, Eric Jacobs wrote:
Jorge,
Would it still be satisfying to you if instead of writing the call expression
like this:
try {
response = asyncFunction(request); //might suspend execution
}
catch (e) {
//whatever
}
//our code continues here
we needed to write it with an explicit annotation, like this:
response = yield asyncFunction(request); //might suspend execution
or perhaps this:
yield { response = asyncFunction(request); } //might suspend execution
or some other creative way of statically encoding the might suspend
execution condition into the syntax?
Yes, of course, it would be fine. Why ?
--
Jorge.
___
es-discuss mailing list
es-discuss@mozilla.org
https://mail.mozilla.org/listinfo/es-discuss
Re: yield and Promises
On 21/10/2011, at 21:23, Dean Landolt wrote:
On Fri, Oct 21, 2011 at 3:20 PM, Jorge jo...@jorgechamorro.com wrote:
On 21/10/2011, at 17:40, Eric Jacobs wrote:
Jorge,
Would it still be satisfying to you if instead of writing the call
expression like this:
try {
response = asyncFunction(request); //might suspend execution
}
catch (e) {
//whatever
}
//our code continues here
we needed to write it with an explicit annotation, like this:
response = yield asyncFunction(request); //might suspend execution
or perhaps this:
yield { response = asyncFunction(request); } //might suspend execution
or some other creative way of statically encoding the might suspend
execution condition into the syntax?
Yes, of course, it would be fine. Why ?
Because this is the fundamental difference between shallow and deep
continuations.
Yes, if we can write this:
try {
response = yield asyncFunction(request); //might suspend execution
}
catch (e) {
//whatever
}
and asyncFunction can suspend/resume then it's alright.
Why ?
--
Jorge.
___
es-discuss mailing list
es-discuss@mozilla.org
https://mail.mozilla.org/listinfo/es-discuss
Re: yield and Promises
Jorge wrote: or some other creative way of statically encoding the might suspend execution condition into the syntax? Yes, of course, it would be fine. Why ? Because this is the crux of the run-to-completion debate that we're immersed in. Right now, JS has run-to-completion, where completion is defined as the end of a method, or a yield statement. If the yield statement continues to be required for code paths which may transfer control to another context, then that won't change, and all the run-to-completion objections to coroutines disappear. IOW, there is no difference WRT run-to-completion between generator-yield and coroutine-yield. The catch is, of course, that all code which either can yield, or can call other functions which yield, must have the yield keyword there, to mark that run-to-completion invariants will end at that point. This seems like a very reasonable compromise to me. -Eric ___ es-discuss mailing list es-discuss@mozilla.org https://mail.mozilla.org/listinfo/es-discuss
Re: yield and Promises
On Oct 21, 2011, at 12:49 PM, Eric Jacobs wrote: The catch is, of course, that all code which either can yield, or can call other functions which yield, must have the yield keyword there, to mark that run-to-completion invariants will end at that point. This seems like a very reasonable compromise to me. If you have to yield at every point in a call chain that might reach a coroutine that captures a deep continuation, then you've really got a chain of shallow continuations. /be ___ es-discuss mailing list es-discuss@mozilla.org https://mail.mozilla.org/listinfo/es-discuss
Re: yield and Promises
On Fri, Oct 21, 2011 at 10:46 AM, Mark S. Miller erig...@google.com wrote: On Fri, Oct 21, 2011 at 10:20 AM, John J Barton johnjbar...@johnjbarton.com wrote: In particular, Q simplifies joining parallel async operations (XHR, postMessages, 'load', 'progress' events). Of course it may well be that generators provide an elegant solution to this important problem, but I've not seen such examples. Have you seen http://wiki.ecmascript.org/doku.php?id=strawman:concurrency#q.race and http://wiki.ecmascript.org/doku.php?id=strawman:concurrency#q.all ? If these don't address the joining you have in mind, could you post some examples? Thanks. Unfortunately I was not able to follow the comments on that page. (These strawman pages are hard to follow because they describe new things using new terminology). This code seems to do what I intended: https://github.com/johnjbarton/Purple/blob/master/chrome/extension/pea.js#L114 The structure is: start A, start B, when AB (start C, start D, when CD (we win))); The code marches right but for me the key is being able to predict the relative order of the calls. Of course this particular example is not good for comparing different alternatives. I guess the biggest win for Q comes in unconventional cases where async is used for remote communications and such examples are currently complex. jjb ___ es-discuss mailing list es-discuss@mozilla.org https://mail.mozilla.org/listinfo/es-discuss
Re: yield and Promises
Brendan Eich wrote: On Oct 21, 2011, at 12:49 PM, Eric Jacobs wrote: The catch is, of course, that all code which either can yield, or can call other functions which yield, must have the yield keyword there, to mark that run-to-completion invariants will end at that point. This seems like a very reasonable compromise to me. If you have to yield at every point in a call chain that might reach a coroutine that captures a deep continuation, then you've really got a chain of shallow continuations. /be As a language end-user, I'm not sure that I would (or should) be concerned about the distinction between a deep continuation and a chain of shallow continuations. After all, the stack is just a chain of frames, and the mechanism by which those frames are chained together is not of semantic importance to the language. That said, coroutinish features like having stacktraces that show multiple levels of blocking operations, and try/catch blocks that can catch over several blocking operations would be really nice to have. If those features can be built at the library level using shallow continuations or generator trampolines or what have you, I say great. -Eric ___ es-discuss mailing list es-discuss@mozilla.org https://mail.mozilla.org/listinfo/es-discuss
Re: yield and Promises
On Oct 21, 2011, at 2:03 PM, Eric Jacobs wrote: As a language end-user, I'm not sure that I would (or should) be concerned about the distinction between a deep continuation and a chain of shallow continuations. After all, the stack is just a chain of frames, and the mechanism by which those frames are chained together is not of semantic importance to the language. My point was simply that it ain't coroutines or deep continuations if you are required to say yield at each frame. That said, coroutinish features like having stacktraces that show multiple levels of blocking operations, and try/catch blocks that can catch over several blocking operations would be really nice to have. If those features can be built at the library level using shallow continuations or generator trampolines or what have you, I say great. That's the plan. Deep continuations are out for the two reasons (implementor diversity vs. interop, run-to-completion auditability) I gave. /be ___ es-discuss mailing list es-discuss@mozilla.org https://mail.mozilla.org/listinfo/es-discuss
Re: yield and Promises
On 10/21/2011 11:01 AM, David Herman wrote:
[snip]
But there's more to it than just the interface. You fix a particular
scheduling semantics when you put deferred functions into the
language. I'm still learning about the difference between the Deferred
pattern and the Promises pattern, but the former seems much more
stateful than the latter: you enqueue another listener onto an
internal mutable queue.
At least in the Dojo community (and I think Kowal does with Q as well),
we define a Deferred producer-sider constructor for creating promises,
with an API that can resolve() or reject() the generated promise. The
promise is then the consumer-side interface that allows consumer to
register a callback for the fulfillment or rejection of the promise
(with a then() method or a variety of other convenience functions). The
mutable state pattern was used in earlier versions of Dojo, but later we
switched to an API that keeps promises immutable except for legacy
methods, as we have reached consensus that mutable promises are bad.
Thus the terminology difference between mutable state and immutable
state is simply wrong vs right for us ;).
There are indeed different scheduling semantics to consider. With Dojo
(and I think jQuery as well), we have considered enqueuing callbacks
onto the event queue to unviable because historically the only mechanism
within the browser has been setTimeout (there is no setImmediate or
process.nextTick available) which has a rather large minimum delay that
can easily up add to noticeable and unacceptable introduction of delays
with a chain of a series of promises. Consequently our implementations
do not enqueue any callbacks for future turns, all callbacks are
executed in the same turn as the resolution of the promise, and due to
latency concerns we haven't really felt the freedom to explore other
scheduling semantics. This scheduling semantic has worked fine for us,
but I don't mind an alternate one. It looks like kriskowal/q does
enqueue, using a postMessage hack to enable faster enqueuing on newer
browsers.
On 10/21/2011 10:34 AM, John J Barton wrote:
Can anyone summarize how these proposals relate to Kris Kowal / Kris
Zyp / Mark Miller Q library:
https://github.com/kriskowal/q
The proposal was designed such that it should work smoothly with Kowal's
Q originating promises as well (acting like Q.when). For example, using
the opening example of delay function from the kriskowal/q readme, one
could write:
function(){
return afterOneSecond(yield delay(1000));
}
and it would be effectively the same as (with the possible exception of
scheduling policies):
function(){
return Q.when(delay(1000), afterOneSecond);
}
In my experience, reasoning about the code was much easier with Q than
without Q. (Not something I found in trying generators). I found the
documentation hard to follow since it assumes background I don't have
and the examples were not interesting to me, but once I tried it I was
pleasantly surprised. It does have ergonomic issues, an undefined and
resolved promise work equally well, but I think this may be
inexperience on my part.
Yes, kriskowal/q is an excellent library.
Thanks,
Kris
___
es-discuss mailing list
es-discuss@mozilla.org
https://mail.mozilla.org/listinfo/es-discuss
Re: yield and Promises
On 19/10/2011, at 23:34, Brendan Eich wrote:
The other objection is that (ignoring some evil native APIs such as sync XHR)
JS has run-to-completion execution model now. You can model
assert_invariants();
f();
assert_invariants_not_affected_by_f_etc();
where etc means functions called from f. No data races, no preemption
points even if voluntary -- the immediately preempted function may have
volunteered, but in programming in the large, the sum of its ancestors in all
call graphs may well *not* have volunteered to lose their invariants.
This second objection is not an implementor issue, rather a
security/integrity/pedagogy concern. It's a big one too.
Is run-to-completion so important, really ?
Because, if there's a callback involved, the invariants are not invariant
anymore, and that's the sole argument node.js guys keep harping on again and
again (wrongly imo) against any way of suspending/resuming f().
For example:
assert_invariants();
function callBack () {
assert_invariants(); // perhaps yes, perhaps no. There's no guarantee.
};
setTimeout(callBack, 1e3);
return;
So, as far as I can see, when dealing with asynchronous code, the risks in that
code are equivalent to the risks in this code:
assert_invariants();
f(); //might suspend execution
assert_invariants(); // perhaps yes, perhaps no. There's no guarantee either.
return;
But, in the first case you can't try/catch where it matters (which is
annoying), and you can't write your code linearly as if it were synchronous,
which is a (bit of a) pain.
So I must be missing something. What's it ?
--
Jorge.
___
es-discuss mailing list
es-discuss@mozilla.org
https://mail.mozilla.org/listinfo/es-discuss
Re: yield and Promises
On Thu, Oct 20, 2011 at 9:44 AM, Jorge jo...@jorgechamorro.com wrote:
On 19/10/2011, at 23:34, Brendan Eich wrote:
The other objection is that (ignoring some evil native APIs such as sync
XHR) JS has run-to-completion execution model now. You can model
assert_invariants();
f();
assert_invariants_not_affected_by_f_etc();
where etc means functions called from f. No data races, no preemption
points even if voluntary -- the immediately preempted function may have
volunteered, but in programming in the large, the sum of its ancestors in
all call graphs may well *not* have volunteered to lose their invariants.
This second objection is not an implementor issue, rather a
security/integrity/pedagogy concern. It's a big one too.
Is run-to-completion so important, really ?
Because, if there's a callback involved, the invariants are not invariant
anymore, and that's the sole argument node.js guys keep harping on again and
again (wrongly imo) against any way of suspending/resuming f().
For example:
assert_invariants();
function callBack () {
assert_invariants(); // perhaps yes, perhaps no. There's no guarantee.
};
setTimeout(callBack, 1e3);
return;
So, as far as I can see, when dealing with asynchronous code, the risks in
that code are equivalent to the risks in this code:
assert_invariants();
f(); //might suspend execution
assert_invariants(); // perhaps yes, perhaps no. There's no guarantee
either.
return;
But, in the first case you can't try/catch where it matters (which is
annoying), and you can't write your code linearly as if it were synchronous,
which is a (bit of a) pain.
So I must be missing something. What's it ?
Yes, I think what you're missing is the semantics intended by
run-to-completion. You claim the two cases above are equivalent but that's
not true at all. There's a *huge *difference between explicit preemption and
implicit preemption -- the latter is a hazard, plain and simple. Let's not
mix the two up and jeopardize getting the former into the language.
___
es-discuss mailing list
es-discuss@mozilla.org
https://mail.mozilla.org/listinfo/es-discuss
Re: yield and Promises
On Oct 20, 2011, at 6:44 AM, Jorge wrote:
On 19/10/2011, at 23:34, Brendan Eich wrote:
The other objection is that (ignoring some evil native APIs such as sync
XHR) JS has run-to-completion execution model now. You can model
assert_invariants();
f();
assert_invariants_not_affected_by_f_etc();
where etc means functions called from f. No data races, no preemption
points even if voluntary -- the immediately preempted function may have
volunteered, but in programming in the large, the sum of its ancestors in
all call graphs may well *not* have volunteered to lose their invariants.
This second objection is not an implementor issue, rather a
security/integrity/pedagogy concern. It's a big one too.
Is run-to-completion so important, really ?
Yes.
Birdie: You looking for an answer or an argument?
Margo Channing: An answer.
Birdie: No.
Margo Channing: Why not?
Birdie: Now you want an argument.
Because, if there's a callback involved, the invariants are not invariant
anymore,
What do you mean by if there's a callback involved?
What I sketched showed a function f being called. There is no preemption point
under a function call. If I had written g(function callback() {...})) then the
... would perhaps have run in a separate event loop turn. So what? That's not
issue.
and that's the sole argument node.js guys keep harping on again and again
(wrongly imo) against any way of suspending/resuming f().
You are changing the example to something not at issue. Callbacks run in
separate turns (by convention, better if defined as always, as for
setTimeout(0)).
For example:
assert_invariants();
function callBack () {
assert_invariants(); // perhaps yes, perhaps no. There's no guarantee.
};
setTimeout(callBack, 1e3);
return;
Here again, as with 'yield', the programmer explicitly opted out of
run-to-completion. The reader can see the 'function callBack' head and braced
body. This signals that the code is deferred and won't be executed until
invocation.
So, as far as I can see, when dealing with asynchronous code, the risks in
that code are equivalent to the risks in this code:
assert_invariants();
f(); //might suspend execution
assert_invariants(); // perhaps yes, perhaps no. There's no guarantee either.
return;
See above. You're now making every single call expression in an entire JS
codebase potentially a preemption point. That's bad for reasoning about
invariants, therefore bad for correctness, including security.
But, in the first case you can't try/catch where it matters (which is
annoying), and you can't write your code linearly as if it were synchronous,
which is a (bit of a) pain.
So I must be missing something. What's it ?
You changed the example to defer evaluation with a callback.passed down to
another function and then asserted the changed example was no different from a
direct call with no callback. That's different, because the function wrapping
explicitly defers evaluation of the function body.
/be
___
es-discuss mailing list
es-discuss@mozilla.org
https://mail.mozilla.org/listinfo/es-discuss
Re: yield and Promises
On 20/10/2011, at 18:38, Brendan Eich wrote:
On Oct 20, 2011, at 6:44 AM, Jorge wrote:
On 19/10/2011, at 23:34, Brendan Eich wrote:
The other objection is that (ignoring some evil native APIs such as sync
XHR) JS has run-to-completion execution model now. You can model
assert_invariants();
f();
assert_invariants_not_affected_by_f_etc();
where etc means functions called from f. No data races, no preemption
points even if voluntary -- the immediately preempted function may have
volunteered, but in programming in the large, the sum of its ancestors in
all call graphs may well *not* have volunteered to lose their invariants.
This second objection is not an implementor issue, rather a
security/integrity/pedagogy concern. It's a big one too.
Is run-to-completion so important, really ?
Yes.
Birdie: You looking for an answer or an argument?
Margo Channing: An answer.
Birdie: No.
Margo Channing: Why not?
Birdie: Now you want an argument.
Because, if there's a callback involved, the invariants are not invariant
anymore,
What do you mean by if there's a callback involved?
What I sketched showed a function f being called. There is no preemption
point under a function call. If I had written g(function callback() {...}))
then the ... would perhaps have run in a separate event loop turn. So what?
That's not issue.
and that's the sole argument node.js guys keep harping on again and again
(wrongly imo) against any way of suspending/resuming f().
You are changing the example to something not at issue. Callbacks run in
separate turns (by convention, better if defined as always, as for
setTimeout(0)).
For example:
assert_invariants();
function callBack () {
assert_invariants(); // perhaps yes, perhaps no. There's no guarantee.
};
setTimeout(callBack, 1e3);
return;
Here again, as with 'yield', the programmer explicitly opted out of
run-to-completion. The reader can see the 'function callBack' head and braced
body. This signals that the code is deferred and won't be executed until
invocation.
So, as far as I can see, when dealing with asynchronous code, the risks in
that code are equivalent to the risks in this code:
assert_invariants();
f(); //might suspend execution
assert_invariants(); // perhaps yes, perhaps no. There's no guarantee either.
return;
See above. You're now making every single call expression in an entire JS
codebase potentially a preemption point. That's bad for reasoning about
invariants, therefore bad for correctness, including security.
But, in the first case you can't try/catch where it matters (which is
annoying), and you can't write your code linearly as if it were synchronous,
which is a (bit of a) pain.
So I must be missing something. What's it ?
You changed the example to defer evaluation with a callback.passed down to
another function and then asserted the changed example was no different from
a direct call with no callback. That's different, because the function
wrapping explicitly defers evaluation of the function body.
I don't see how it's different: next to f() it says //might suspend execution:
the assert_invariants() at the next line might run in another turn of the event
loop (when f() resumes), just as the callback does.
?
--
Jorge.___
es-discuss mailing list
es-discuss@mozilla.org
https://mail.mozilla.org/listinfo/es-discuss
Re: yield and Promises
On Oct 20, 2011, at 12:59 PM, Jorge wrote: assert_invariants(); f(); //might suspend execution assert_invariants(); // perhaps yes, perhaps no. There's no guarantee either. return; (Please trim cited text -- I know gmail hides it, which is a bug, but most mail user agents show it, and think of the bandwidth!) I don't see how it's different: next to f() it says //might suspend execution: You wrote that comment, not me. I wrote assert_invariants(); f(); assert_invariants_not_affected_by_f_etc(); and this is how JS works today. This is run-to-completion. It's important not to break it. the assert_invariants() at the next line might run in another turn of the event loop (when f() resumes), just as the callback does. No. Nothing in JS today, since it lacks coroutines or call/cc, can suspend under f and cause the continuation to be captured and then called in a later event loop turn. That's the point. /be___ es-discuss mailing list es-discuss@mozilla.org https://mail.mozilla.org/listinfo/es-discuss
yield and Promises
The topic of single-frame continuations has been discussed here before,
with the current state of ES.next moving towards generators that are
based on and similar to Mozilla's JS1.7 implementation. Generators are a
powerful addition to the language and welcome (at least by me). However,
I believe that this still leaves a distinct gap in functionality that
forces the majority use case for single-frame continuations to be
handled by libraries. A language feature that OOTB must rely on
libraries to fulfill the most wanted use cases seems like than ideal.
I believe one could separate these single-frame continuations (or
coroutines, looking at it from the perspective of the behavior of the
function) into two categories. There are bottom-up controlled
continuations, where the caller of the coroutine function controls when
the function will resume execution. I think is equivalent to a
generator. Generator functions return an object with an interface for
resuming the execution (and providing values for the continuation) of
the coroutine function.
There are are also top-down controlled continuations. Here coroutine
functions can suspend execution when given an object (typically from one
of the functions it calls) that provides the interface to resume
execution. Resuming execution therefore is controlled by values returned
from callees instead of from the caller. It is worth noting that
bottom-up controllers can be turned into a top-down controller and vice
versa with the use of libraries (one can go either way).
I believe that the overwhelming need that is continually and constantly
expressed and felt in the JS community in terms of handling asynchronous
activity is fundamentally a cry for top-down controlled single-frame
continuations (obviously not always stated in such terms, but that is
the effective need/solution). In terms of an actual code example,
essentially what is desired is to be able to write functions like:
element.onclick = function(){
// suspend execution after doSomethingAsync() to wait for result
var result = some operator doSomethingAsync();
// resume and do something else
alert(result);
};
Generators directly solve a problem that is much less significant in
normal JS coding. While it is exciting that generators coupled with
libraries give us a much better tool for asynchronous use cases (the
above can be coded with libraryFunction(function*(){...}), my concern is
that the majority use case is the one that requires libraries rather
than the minority case, and does not promote interoperability.
Now why should we consider something now when previous alternatives to
generators have failed to gain traction? Previous proposals have avoided
a specifying a direct interface on top-down objects to leave the door
open for different possible interfaces for resuming executions, or
different possible promise styles. We have wisely deferred to
libraries when different possible approaches have yet to be explored
within the JS community. A couple years ago there were numerous
approaches being explored. However to truly follow through with this
strategy we should then proceed with language design when convergence
does in fact take place. A couple years later, I believe the landscape
has dramatically changed, and we indeed do have significant convergence
on a promise API with the thenable interface. From Dojo, to jQuery, to
several server side libraries, and apparently even Windows 8's JS APIs
(from what I understand) all share an intersection of APIs that include
a then() method as a method to define a promise and register a callback
for when a promise is fulfilled (or fails). This is an unusual level of
convergence for a JS community that is so diverse. I believe this gives
evidence of well substantiated and tested interface that can be used for
top-controlled single-frame continuations that can easily be specified,
understood and used by developers.
My proposal is to allow the use of the yield keyword in standard
functions (not just generator function*'s) with the following semantics:
The yield operator is prefix operator that takes a single operand
(variant of AssignmentExpression, just as within generator function*s).
When a yield operator is encountered in the execution of a standard
function (not a generator), the operand value is examined. If the value
is an object with a then property that is a function (the object is
AKA promise), the execution will suspend, preserving the context for
when the execution is resumed. The operand's then function will be
called with a resume function as the first argument, and a fail
function as the second argument. If and when the resume function is
called, execution of the suspended function will resume. The first
argument of the call to the resume function will be provided as the
result of the evaluation yield operator within the resumed execution. If
the fail function is called,
Re: yield and Promises
This is a really great idea, Kris! A few comments inline...
On Wed, Oct 19, 2011 at 1:11 PM, Kris Zyp k...@sitepen.com wrote:
The topic of single-frame continuations has been discussed here before,
with the current state of ES.next moving towards generators that are based
on and similar to Mozilla's JS1.7 implementation. Generators are a powerful
addition to the language and welcome (at least by me). However, I believe
that this still leaves a distinct gap in functionality that forces the
majority use case for single-frame continuations to be handled by libraries.
A language feature that OOTB must rely on libraries to fulfill the most
wanted use cases seems like than ideal.
I believe one could separate these single-frame continuations (or
coroutines, looking at it from the perspective of the behavior of the
function) into two categories. There are bottom-up controlled continuations,
where the caller of the coroutine function controls when the function will
resume execution. I think is equivalent to a generator. Generator functions
return an object with an interface for resuming the execution (and providing
values for the continuation) of the coroutine function.
There are are also top-down controlled continuations. Here coroutine
functions can suspend execution when given an object (typically from one of
the functions it calls) that provides the interface to resume execution.
Resuming execution therefore is controlled by values returned from callees
instead of from the caller. It is worth noting that bottom-up controllers
can be turned into a top-down controller and vice versa with the use of
libraries (one can go either way).
I believe that the overwhelming need that is continually and constantly
expressed and felt in the JS community in terms of handling asynchronous
activity is fundamentally a cry for top-down controlled single-frame
continuations (obviously not always stated in such terms, but that is the
effective need/solution). In terms of an actual code example, essentially
what is desired is to be able to write functions like:
element.onclick = function(){
// suspend execution after doSomethingAsync() to wait for result
var result = some operator doSomethingAsync();
// resume and do something else
alert(result);
};
Generators directly solve a problem that is much less significant in normal
JS coding. While it is exciting that generators coupled with libraries give
us a much better tool for asynchronous use cases (the above can be coded
with libraryFunction(function*(){...}), my concern is that the majority use
case is the one that requires libraries rather than the minority case, and
does not promote interoperability.
Now why should we consider something now when previous alternatives to
generators have failed to gain traction? Previous proposals have avoided a
specifying a direct interface on top-down objects to leave the door open for
different possible interfaces for resuming executions, or different possible
promise styles. We have wisely deferred to libraries when different
possible approaches have yet to be explored within the JS community. A
couple years ago there were numerous approaches being explored. However to
truly follow through with this strategy we should then proceed with language
design when convergence does in fact take place. A couple years later, I
believe the landscape has dramatically changed, and we indeed do have
significant convergence on a promise API with the thenable interface. From
Dojo, to jQuery, to several server side libraries, and apparently even
Windows 8's JS APIs (from what I understand) all share an intersection of
APIs that include a then() method as a method to define a promise and
register a callback for when a promise is fulfilled (or fails). This is an
unusual level of convergence for a JS community that is so diverse. I
believe this gives evidence of well substantiated and tested interface that
can be used for top-controlled single-frame continuations that can easily be
specified, understood and used by developers.
My proposal is to allow the use of the yield keyword in standard
functions (not just generator function*'s) with the following semantics: The
yield operator is prefix operator that takes a single operand (variant of
AssignmentExpression, just as within generator function*s). When a yield
operator is encountered in the execution of a standard function (not a
generator), the operand value is examined. If the value is an object with a
then property that is a function (the object is AKA promise), the
execution will suspend, preserving the context for when the execution is
resumed. The operand's then function will be called with a resume
function as the first argument, and a fail function as the second
argument. If and when the resume function is called, execution of the
suspended function will resume. The first argument of the call to the
resume
Re: yield and Promises
Kris Zyp wrote:
I believe that the overwhelming need that is continually and
constantly expressed and felt in the JS community in terms of handling
asynchronous activity is fundamentally a cry for top-down controlled
single-frame continuations (obviously not always stated in such terms,
but that is the effective need/solution). In terms of an actual code
example, essentially what is desired is to be able to write functions
like:
element.onclick = function(){
// suspend execution after doSomethingAsync() to wait for result
var result = some operator doSomethingAsync();
// resume and do something else
alert(result);
};
I really like the direction that this is going, but I'm curious: Why not
look into having full coroutines support? Coroutines support the above
pattern well, plus they can be used to implement generators using the
same mechanisms (although slightly differently than JS1.7/Python
generators.) To allow code to suspend execution as you have shown above
while avoiding reentrancy, you'll need some kind of fork or spawn
primitive as well, and coroutines provide a nice paradigm for that.
-Eric
Kris Zyp wrote:
I believe that the
overwhelming need that is continually and constantly expressed
and felt in the JS community in terms of handling asynchronous
activity is fundamentally a cry for top-down controlled
single-frame continuations (obviously not always stated in such
terms, but that is the effective need/solution). In terms of an
actual code example, essentially what is desired is to be able
to write functions like:
element._onclick_ = function(){
// suspend execution after doSomethingAsync() to wait for
result
var result = some operator doSomethingAsync();
// resume and do something else
alert(result);
};
I really like the direction that this is going, but I'm curious: Why
not look into having full coroutines support? Coroutines support the
above pattern well, plus they can be used to implement generators
using the same mechanisms (although slightly differently than
JS1.7/Python generators.) To allow code to suspend execution as you
have shown above while avoiding reentrancy, you'll need some kind of
"fork" or "spawn" primitive as well, and coroutines provide a nice
paradigm for that.
-Eric
___
es-discuss mailing list
es-discuss@mozilla.org
https://mail.mozilla.org/listinfo/es-discuss
Re: yield and Promises
On 10/19/2011 12:29 PM, Dean Landolt wrote:
This is a really great idea, Kris! A few comments inline...
[snip]
If the value is not an object with a then property that is a
function, the operand value is the immediate result of the
evaluation of the yield expression and execution is not suspended.
Here is a simple example of usage:
use strict
function delay(ms){
// a function that will yield for the given milliseconds
yield {
then: function(resume){
setTimeout(resume, ms);
}
}
}
IIUC you're proposing language-level support for promises, right?
There's no getting around it -- you're spec'ing an interface for the
unary yield operator to interact with. So why not go all out and have
the language stratify `then` for you with private names?
That's fine with me.
[snip]
Obviously one could choose a different keyword to use. I'd imagine
await is one alternative. The drawback of yield is that it
does behave differently than a yield in a generator. However,
generators behave quite differently anyway, and top-controlled
yield shares a very important commonality. Using one keyword
means there is only a single operator that can result in a
suspension of execution within a function body, making easier to
spot such occurrences and look for possible points where certain
variants should not be anticipated. Of course it is also nice to
avoid proliferation of keywords and introducing new previously
unreserved keywords.
Any thoughts on how this should interplay with generators? One
side-effect of overloading yield is that it becomes impossible to wait
for a promise inside a generator -- is this a feature or a bug?
I think it is a feature, as I don't believe they both forms can be used
very coherently together in the same function. Consider a separate
await operator inside a generator. If you execute this operator with
an unresolved promise, the function is supposed to return (a promise),
but in a generator when a return is encountered it throws a
StopIteration. It hardly seems useful to have an (await somePromise)
immediately halting the generator. If you want to use promises within a
generator, I believe the correct usage would be to propagate the promise
out to the generator controller and then yield from there:
function* slowGenerator(){
while(true){
yield delay(50);
}
}
let seq = slowGenerator();
yield seq.next();
yield seq.next();
There are also have been suggestions about potentially have
language support for queuing different operations on to promises
(gets, puts, and posts). I don't think proposal precludes such
possibilities (it only precludes the possibility of opaque
promises that have no interface, but the majority of devs I have
talked to are pretty opposed to that idea, so that doesn't seem
like likely possibility).
I assume that if a function that yields, when called with a yield
prefix, will return a promise -- is this correct?
Yes.
What if there exists a yield in the function but the function returns
without hitting a yield in the codepath? No promise then?
No promise will be returned.
I believe it is critical that we may maintain a principle of locality
such that:
(function(){
if(false){
valid statement
}
else return true;
})();
will always return true, regardless of the operators placed within the
if statement's body.
Thanks,
Kris
___
es-discuss mailing list
es-discuss@mozilla.org
https://mail.mozilla.org/listinfo/es-discuss
Re: yield and Promises
On Oct 19, 2011, at 12:55 PM, Eric Jacobs wrote:
Kris Zyp wrote:
I believe that the overwhelming need that is continually and
constantly expressed and felt in the JS community in terms of handling
asynchronous activity is fundamentally a cry for top-down controlled
single-frame continuations (obviously not always stated in such terms,
but that is the effective need/solution). In terms of an actual code
example, essentially what is desired is to be able to write functions
like:
element.onclick = function(){
// suspend execution after doSomethingAsync() to wait for result
var result = some operator doSomethingAsync();
// resume and do something else
alert(result);
};
I really like the direction that this is going, but I'm curious: Why not
look into having full coroutines support?
Asked and answered many times, e.g.
https://mail.mozilla.org/pipermail/es-discuss/2006-June/003497.html
on the implementation problems with requiring suspending across native frames.
The other objection is that (ignoring some evil native APIs such as sync XHR)
JS has run-to-completion execution model now. You can model
assert_invariants();
f();
assert_invariants_not_affected_by_f_etc();
where etc means functions called from f. No data races, no preemption points
even if voluntary -- the immediately preempted function may have volunteered,
but in programming in the large, the sum of its ancestors in all call graphs
may well *not* have volunteered to lose their invariants.
This second objection is not an implementor issue, rather a
security/integrity/pedagogy concern. It's a big one too.
/be
Coroutines support the above
pattern well, plus they can be used to implement generators using the
same mechanisms (although slightly differently than JS1.7/Python
generators.) To allow code to suspend execution as you have shown above
while avoiding reentrancy, you'll need some kind of fork or spawn
primitive as well, and coroutines provide a nice paradigm for that.
-Eric
..html___
es-discuss mailing list
es-discuss@mozilla.org
https://mail.mozilla.org/listinfo/es-discuss
___
es-discuss mailing list
es-discuss@mozilla.org
https://mail.mozilla.org/listinfo/es-discuss
Re: yield and Promises
On Oct 19, 2011, at 10:11 AM, Kris Zyp wrote: The topic of single-frame continuations has been discussed here before, with the current state of ES.next moving towards generators that are based on and similar to Mozilla's JS1.7 implementation. Generators are a powerful addition to the language and welcome (at least by me). However, I believe that this still leaves a distinct gap in functionality that forces the majority use case for single-frame continuations to be handled by libraries. A language feature that OOTB must rely on libraries to fulfill the most wanted use cases seems like than ideal. Yeah, batteries included languages such as Python win. We aspire to that. But apart from your (or anyone else's) proposal, JS is not Python in particular ways that make it hard to rush OOTB built-ins to do promises or deferred functions on top. First, Python has a protocol for breaking compatibility, and since traditionally CPython was provisioned in single systems or server machine rooms by sysadmins (built from source, even), sysadmins could update when ready. This results in some versionitis pain for sure, but at least within an administrative domain of authority, you could suit yourself. On the web, there's no such firewalling at DNS source of authority boundaries. You want your web content loaded by as many browsers as possible. So we don't get many bites at the compatibility break apple. Browser vendors do not gain market share by being first to break compatibility, they lose. The game theory is perverse and merciless. Second, TC39 sucks at library design. There, I said it. The right place to solve complex, higher-level, better-batteries-included-with-solar-charger problems is github.com. TC39 will make mistakes, and rushing will enshrine them in standards we cannot back away from easily or at all (first point). I believe one could separate these single-frame continuations (or coroutines, looking at it from the perspective of the behavior of the function) into two categories. Suggest avoiding coroutines, as Simula or Lua coroutines that capture deep continuations are not the same as generators. See my previous post. Don't get me wrong: carry on, here and (better, because of user testing) on github. But do not expect rushed standardization or TC39 picks the winner up front. /be___ es-discuss mailing list es-discuss@mozilla.org https://mail.mozilla.org/listinfo/es-discuss
Re: yield and Promises
On Oct 19, 2011, at 2:34 PM, Brendan Eich wrote: The other objection is that (ignoring some evil native APIs such as sync XHR) JS has run-to-completion execution model now. You can model assert_invariants(); f(); assert_invariants_not_affected_by_f_etc(); Contrast with generators, where you opt-in frame-by-frame: assert_invariants(); yield f(); yay_i_lost_my_invariants_on_purpose(); This requires trampolining, a scheduler. Delegated yield (yield*, yield from in PEP 380) helps. No free lunch. But you don't lose invariants due to some callee-of-a-callee suspending, as you would with coroutines. /be___ es-discuss mailing list es-discuss@mozilla.org https://mail.mozilla.org/listinfo/es-discuss
