Nice work! Now I've got more motivation to get LuvMonkey into a more usable state.
On Sat, May 19, 2012 at 4:06 PM, Bruno Jouhier <[email protected]> wrote: > I'm not sure I get it but I'll try to answer. > > What I'm describing in the post is how logic can be expressed with > generators rather than callbacks. I'm assuming that the low level calls are > callback-style. So, there is no reference to any specific I/O library > and/or to back pressure. > > The back pressure problem is a problem that I'm handling in streamline's > streams module. And I'm handling it with a simple pair of async calls: > stream.read(cb) and stream.write(cb, buffer) that are small wrappers around > node streams. > > The read call pauses and resumes the underlying stream based on some > configurable high/low mark buffering limits (you can set them to 0 but then > the stream will pause every time it needs to buffer a chunk). > > The write call deals with the drain event under the hood. If the lower > level call write call returns true, the callback is called immediately > (streamline trampolines so there is no risk of stack overflow in callback > mode). If it returns false, the callback is triggered by the drain event. > > Pump loops can be written as: > > while (data = input.read(_)) > output.write(_, data); > > In callback mode, streamline transforms this loop into something like: > > (function loop() { > input.read(function(err, data) { > if (err) return cb(err); > if (data) > output.write(function(err) { > if (err) return cb(err); > loop(): > }, data); > else cb(); > }); > })(); > > In generators mode it transforms it into: > > while (data = yield input.read(_)) > yield output.write(_, data) > > The code looks very different but it execute just like the callback code > above. The input.read and output.write calls will use "invoke" and > callbacks to interact with the underlying node streams APIs. The run loop > that I've given in my post will exit at spot (c) every time a callback is > pending and the callback will reactivate it by calling resume; so, even > though the code does not look async and callback driven it is actually > completely async and callback driven. > > With this pump loop, backpressure happens naturally when the two calls are > combined together. If the output is slower than the input when pumping from > an input stream to an output stream, the write call will start to wait for > drain events. This will naturally stop the pump loop. The input stream will > continue to receive data events but it will just buffer because read won't > be called by the pump loop any more. When buffering goes over the high > mark, the input stream will be paused. Then, at some point, the output > stream will receive a drain event. It will call its callback, which will > resume the pump loop. Read will be called and will get data that has been > buffered. The input stream will be resumed when buffering goes below the > low mark, etc., etc. If the drain event comes before the input reaches the > high water mark, the loop will be resumed and the input stream won't be > paused, which is what we want. > > So, even though the pump loop is written as a simple while (data = > input.read(_)) output.write(_, data), it does handle the back pressure. > > Bruno > > > On Saturday, May 19, 2012 9:40:03 PM UTC+2, Mikeal Rogers wrote: >> >> How do you handle back pressure? >> >> On May 19, 2012, at May 19, 20129:51 AM, Bruno Jouhier wrote: >> >> Yes, I fixed it. Thanks. >> >> On Saturday, May 19, 2012 3:15:33 PM UTC+2, Matthew Hazlett wrote: >>> >>> On 5/19/2012 6:20 AM, Bruno Jouhier wrote: >>> > http://bjouhier.wordpress.com/**2012/05/18/asynchronous-** >>> javascript-with-generators-an-**experiment/<http://bjouhier.wordpress.com/2012/05/18/asynchronous-javascript-with-generators-an-experiment/> >>> >>> shouldn't that be print(num) not print(n) >>> >>> >> -- >> Job Board: http://jobs.nodejs.org/ >> Posting guidelines: https://github.com/joyent/**node/wiki/Mailing-List-** >> Posting-Guidelines<https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines> >> You received this message because you are subscribed to the Google >> Groups "nodejs" group. >> To post to this group, send email to [email protected] >> To unsubscribe from this group, send email to >> nodejs+unsubscribe@**googlegroups.com<[email protected]> >> For more options, visit this group at >> http://groups.google.com/**group/nodejs?hl=en?hl=en<http://groups.google.com/group/nodejs?hl=en?hl=en> >> >> >> > On Saturday, May 19, 2012 9:40:03 PM UTC+2, Mikeal Rogers wrote: >> >> How do you handle back pressure? >> >> On May 19, 2012, at May 19, 20129:51 AM, Bruno Jouhier wrote: >> >> Yes, I fixed it. Thanks. >> >> On Saturday, May 19, 2012 3:15:33 PM UTC+2, Matthew Hazlett wrote: >>> >>> On 5/19/2012 6:20 AM, Bruno Jouhier wrote: >>> > http://bjouhier.wordpress.com/**2012/05/18/asynchronous-** >>> javascript-with-generators-an-**experiment/<http://bjouhier.wordpress.com/2012/05/18/asynchronous-javascript-with-generators-an-experiment/> >>> >>> shouldn't that be print(num) not print(n) >>> >>> >> -- >> Job Board: http://jobs.nodejs.org/ >> Posting guidelines: https://github.com/joyent/**node/wiki/Mailing-List-** >> Posting-Guidelines<https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines> >> You received this message because you are subscribed to the Google >> Groups "nodejs" group. >> To post to this group, send email to [email protected] >> To unsubscribe from this group, send email to >> nodejs+unsubscribe@**googlegroups.com<[email protected]> >> For more options, visit this group at >> http://groups.google.com/**group/nodejs?hl=en?hl=en<http://groups.google.com/group/nodejs?hl=en?hl=en> >> >> >> > On Saturday, May 19, 2012 9:40:03 PM UTC+2, Mikeal Rogers wrote: >> >> How do you handle back pressure? >> >> On May 19, 2012, at May 19, 20129:51 AM, Bruno Jouhier wrote: >> >> Yes, I fixed it. Thanks. >> >> On Saturday, May 19, 2012 3:15:33 PM UTC+2, Matthew Hazlett wrote: >>> >>> On 5/19/2012 6:20 AM, Bruno Jouhier wrote: >>> > http://bjouhier.wordpress.com/**2012/05/18/asynchronous-** >>> javascript-with-generators-an-**experiment/<http://bjouhier.wordpress.com/2012/05/18/asynchronous-javascript-with-generators-an-experiment/> >>> >>> shouldn't that be print(num) not print(n) >>> >>> >> -- >> Job Board: http://jobs.nodejs.org/ >> Posting guidelines: https://github.com/joyent/**node/wiki/Mailing-List-** >> Posting-Guidelines<https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines> >> You received this message because you are subscribed to the Google >> Groups "nodejs" group. >> To post to this group, send email to [email protected] >> To unsubscribe from this group, send email to >> nodejs+unsubscribe@**googlegroups.com<[email protected]> >> For more options, visit this group at >> http://groups.google.com/**group/nodejs?hl=en?hl=en<http://groups.google.com/group/nodejs?hl=en?hl=en> >> >> >> > On Saturday, May 19, 2012 9:40:03 PM UTC+2, Mikeal Rogers wrote: >> >> How do you handle back pressure? >> >> On May 19, 2012, at May 19, 20129:51 AM, Bruno Jouhier wrote: >> >> Yes, I fixed it. Thanks. >> >> On Saturday, May 19, 2012 3:15:33 PM UTC+2, Matthew Hazlett wrote: >>> >>> On 5/19/2012 6:20 AM, Bruno Jouhier wrote: >>> > http://bjouhier.wordpress.com/**2012/05/18/asynchronous-** >>> javascript-with-generators-an-**experiment/<http://bjouhier.wordpress.com/2012/05/18/asynchronous-javascript-with-generators-an-experiment/> >>> >>> shouldn't that be print(num) not print(n) >>> >>> >> -- >> Job Board: http://jobs.nodejs.org/ >> Posting guidelines: https://github.com/joyent/**node/wiki/Mailing-List-** >> Posting-Guidelines<https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines> >> You received this message because you are subscribed to the Google >> Groups "nodejs" group. >> To post to this group, send email to [email protected] >> To unsubscribe from this group, send email to >> nodejs+unsubscribe@**googlegroups.com<[email protected]> >> For more options, visit this group at >> http://groups.google.com/**group/nodejs?hl=en?hl=en<http://groups.google.com/group/nodejs?hl=en?hl=en> >> >> >> -- > Job Board: http://jobs.nodejs.org/ > Posting guidelines: > https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines > You received this message because you are subscribed to the Google > Groups "nodejs" group. > To post to this group, send email to [email protected] > To unsubscribe from this group, send email to > [email protected] > For more options, visit this group at > http://groups.google.com/group/nodejs?hl=en?hl=en > -- Job Board: http://jobs.nodejs.org/ Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines You received this message because you are subscribed to the Google Groups "nodejs" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/nodejs?hl=en?hl=en
