You can always call `stream.read`, at any time. This is how data is *pulled*off the stream (instead of it being pushed to you, whether you're ready or not). Because of this you won't lose any data. With new streams there's no real notion of a paused state -- it's always paused. Once you grok that it may not seem so counter-intuitive.
The `readable` event is like a corollary to `drain` -- there to tell you that it's worth bothering with a call to read. You don't *have* to listen for it -- a (needlessly inefficient) stream reader could just as easily poll stream.read for new data periodically. On Mon, Mar 25, 2013 at 4:42 PM, Michael Jackson <[email protected]>wrote: > readable is emitted after you've actually started reading. >> > > That's not what it says in the > docs<http://nodejs.org/api/stream.html#stream_event_readable> > . > > ### > Event: 'readable' > When there is data ready to be consumed, this event will fire. > When this event emits, call the read() method to consume the data. > ### > > Calling stream.read *before* you get the "readable" event is totally > counterintuitive. > > -- > Michael Jackson > @mjackson > > In your example, you dont ever `response.read()`, so no readable event >> is ever emitted. >> >> As you said, streams start in paused state and ready to be read. >> >> On 03/25/13 22:28, Michael Jackson wrote: >> > Is it correct to assume that a Readable won't emit the "readable" event >> > until you're registered for it? >> > >> > Reading through the streams2 docs, I was under the impression that all >> > streams start out paused and don't start emitting data until you add >> > either a "data" (for old streams) or a "readable" listener. For new >> > streams, this should mean that they don't emit "readable" until at least >> > one listener is registered. Otherwise we still need to do some buffering >> > in order to capture all the data. >> > >> > For example, this code misses the readable event on node 0.10: >> > >> > var http = require('http'); >> > >> > http.get('http://www.google.com', function (response) { >> > console.log('got response with status ' + response.statusCode); >> > >> > setTimeout(function () { >> > response.on('readable', function () { >> > console.log('readable'); >> > }); >> > >> > response.on('end', function () { >> > console.log('end'); >> > }); >> > }, 5); >> > }); >> > >> > Here's my shell session: >> > >> > $ node -v >> > v0.10.0 >> > $ node http-test.js >> > got response with status 200 >> > $ >> > >> > Is this the correct behavior? >> > >> > -- >> > Michael Jackson >> > @mjackson >> > >> > >> > On Thu, Mar 21, 2013 at 4:27 PM, Isaac Schlueter <[email protected] >> > <mailto:[email protected]>> wrote: >> > >> > re old-mode >> > >> > Yes, that's fine. If you just want to get all the data asap, use >> > on('data', handler). It'll work great, and it's still very fast. >> > pause()/resume(), the whole bit. (The difference is that it won't >> > emit data until you're listening, and pause() will *actually* >> pause.) >> > >> > >> > Re read(cb) >> > >> > It's problematic for reasons that I've discussed all of the places >> > where it's been brought up. That horse is dead, let's stop beating >> > it. (There were a few other proposals as well, btw. Reducibles and >> > some other monadic approaches come to mind.) >> > >> > >> > Re pipe() vs looping around read() vs custom Writable vs on('data') >> > >> > Whatever works for your case is fine. It's flexible on purpose, and >> > allows more types of consumption than streams1, and creating custom >> > writables is easier than it was in streams1. >> > >> > If you find something that the API can't do for you, or find >> yourself >> > doing a lot of backflips or overriding a lot of methods to get your >> > stuff working, then let's chat about it in a github issue. You >> might >> > be missing something, or you might have found a genuine shortcoming >> in >> > the API. >> > >> > >> > >> > On Thu, Mar 21, 2013 at 2:01 PM, Sigurgeir Jonsson >> > <[email protected] <mailto:[email protected]>> >> > wrote: >> > > Thanks for all the answers. I almost forgot to look back at this >> > thread as >> > > the custom writeStreams have exceeded the high expectation I had >> > already for >> > > Streams 2. >> > > For me, the reference manual was a little confusing, as there are >> > complete >> > > examples on using the read method, no mention of "reading" >> through a >> > > writeStream endpoint. >> > > >> > > Marco, I agree that that read has more detailed control of minimum >> > incoming >> > > content. However I wonder if it would be more efficient to >> default >> > > pipe.chunkSize to a "lowWatermark" of the receiver (if defined). >> > This >> > > lowWatermark could be adjusted dynamically and the callback in the >> > writable >> > > should keep sequence of events under control? >> > > >> > > Anyway, thanks Node team, I'm very impressed! >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > On Wednesday, March 20, 2013 4:45:32 AM UTC-4, Marco Rogers wrote: >> > >> >> > >> @Nathan's response is right. Creating a writable stream is >> > preferable in >> > >> most cases. But I wanted to add a little context to that. If >> > you're dealing >> > >> with a base readable stream, it's just pushing chunks of data at >> > you off the >> > >> wire. Your first task is to collect those chunks into meaningful >> > data. So >> > >> IMO the reason creating a writable stream is preferable is >> because it >> > >> prompts you not just read off the stream, but to create semantics >> > around >> > >> what the new stream is supposed to be. The api reflects this >> > opinion and >> > >> that's why creating writable streams feels like the more natural >> > way, and >> > >> the ugliness of dealing with read() is wrapped up in the pipe() >> > method. It >> > >> was kind of designed that way. >> > >> >> > >> But the read() api was also designed for a use case. It's meant >> > to handle >> > >> low/high water marks effectively, as well as enable more >> > optimized special >> > >> parsing by reading off specific lengths of chunks. These were >> > things that >> > >> people kept needing, but the old api didn't support well. If you >> were >> > >> writing a library for a special parser, you might write a custom >> > Writable >> > >> stream and inside it you would be using the read(n) api to >> > control *how* you >> > >> read data off the socket. I hope that makes sense. >> > >> >> > >> :Marco >> > >> >> > >> On Monday, March 18, 2013 11:06:48 AM UTC-7, Sigurgeir Jonsson >> wrote: >> > >>> >> > >>> The new streams have excellent support for high/low watermarks >> and >> > >>> auto-pausing/resuming, but the documentation confuses me a >> little... >> > >>> particularly the read method. >> > >>> >> > >>> When I read the new docs for the first time I was under the >> > impression >> > >>> that the optimal way to become a user of a stream is to write >> > loops around >> > >>> the read functio. However in practice I find myself simply >> > writing custom >> > >>> writeStreams and use the callback to control upstream pressure >> > (in addition >> > >>> to source Watermarks if needed). Here is an example where I >> > move the >> > >>> output to a queue that executes a custom function in parallel >> (i.e. >> > >>> uploading to a database) >> https://gist.github.com/ZJONSSON/5189249 >> > >>> >> > >>> Are there any benefits to using the read method directly on a >> > stream vs. >> > >>> piping to a custom Writable stream? >> > > >> > > -- >> > > -- >> > > Job Board: http://jobs.nodejs.org/ >> > > Posting guidelines: >> > > >> https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines >> > > You received this message because you are subscribed to the Google >> > > Groups "nodejs" group. >> > > To post to this group, send email to [email protected] >> > <mailto:[email protected]> >> > > To unsubscribe from this group, send email to >> > > [email protected] >> > <mailto:nodejs%[email protected]> >> > > For more options, visit this group at >> > > http://groups.google.com/group/nodejs?hl=en?hl=en >> > > >> > > --- >> > > You received this message because you are subscribed to the Google >> > Groups >> > > "nodejs" group. >> > > To unsubscribe from this group and stop receiving emails from it, >> > send an >> > > email to [email protected] >> > <mailto:nodejs%[email protected]>. >> > > For more options, visit https://groups.google.com/groups/opt_out. >> > > >> > > >> > >> > -- >> > -- >> > Job Board: http://jobs.nodejs.org/ >> > Posting guidelines: >> > https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines >> > You received this message because you are subscribed to the Google >> > Groups "nodejs" group. >> > To post to this group, send email to [email protected] >> > <mailto:[email protected]> >> > To unsubscribe from this group, send email to >> > [email protected] >> > <mailto:nodejs%[email protected]> >> > For more options, visit this group at >> > http://groups.google.com/group/nodejs?hl=en?hl=en >> > >> > --- >> > You received this message because you are subscribed to the Google >> > Groups "nodejs" group. >> > To unsubscribe from this group and stop receiving emails from it, >> > send an email to [email protected] >> > <mailto:nodejs%[email protected]>. >> > For more options, visit https://groups.google.com/groups/opt_out. >> > >> > >> > >> > -- >> > -- >> > Job Board: http://jobs.nodejs.org/ >> > Posting guidelines: >> > https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines >> > You received this message because you are subscribed to the Google >> > Groups "nodejs" group. >> > To post to this group, send email to [email protected] >> > To unsubscribe from this group, send email to >> > [email protected] >> > For more options, visit this group at >> > http://groups.google.com/group/nodejs?hl=en?hl=en >> > >> > --- >> > You received this message because you are subscribed to the Google >> > Groups "nodejs" group. >> > To unsubscribe from this group and stop receiving emails from it, send >> > an email to [email protected]. >> > For more options, visit https://groups.google.com/groups/opt_out. >> > >> > >> >> > -- > -- > Job Board: http://jobs.nodejs.org/ > Posting guidelines: > https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines > You received this message because you are subscribed to the Google > Groups "nodejs" group. > To post to this group, send email to [email protected] > To unsubscribe from this group, send email to > [email protected] > For more options, visit this group at > http://groups.google.com/group/nodejs?hl=en?hl=en > > --- > You received this message because you are subscribed to the Google Groups > "nodejs" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > For more options, visit https://groups.google.com/groups/opt_out. > > > -- -- Job Board: http://jobs.nodejs.org/ Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines You received this message because you are subscribed to the Google Groups "nodejs" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/nodejs?hl=en?hl=en --- You received this message because you are subscribed to the Google Groups "nodejs" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/groups/opt_out.
