new streams don't emit data events without a listener
On Mon, Mar 25, 2013 at 4:28 PM, Michael Jackson <[email protected]>wrote: > Is it correct to assume that a Readable won't emit the "readable" event > until you're registered for it? > > Reading through the streams2 docs, I was under the impression that all > streams start out paused and don't start emitting data until you add either > a "data" (for old streams) or a "readable" listener. For new streams, this > should mean that they don't emit "readable" until at least one listener is > registered. Otherwise we still need to do some buffering in order to > capture all the data. > > For example, this code misses the readable event on node 0.10: > > var http = require('http'); > > http.get('http://www.google.com', function (response) { > console.log('got response with status ' + response.statusCode); > > setTimeout(function () { > response.on('readable', function () { > console.log('readable'); > }); > > response.on('end', function () { > console.log('end'); > }); > }, 5); > }); > > Here's my shell session: > > $ node -v > v0.10.0 > $ node http-test.js > got response with status 200 > $ > > Is this the correct behavior? > > -- > Michael Jackson > @mjackson > > > On Thu, Mar 21, 2013 at 4:27 PM, Isaac Schlueter <[email protected]> wrote: > >> re old-mode >> >> Yes, that's fine. If you just want to get all the data asap, use >> on('data', handler). It'll work great, and it's still very fast. >> pause()/resume(), the whole bit. (The difference is that it won't >> emit data until you're listening, and pause() will *actually* pause.) >> >> >> Re read(cb) >> >> It's problematic for reasons that I've discussed all of the places >> where it's been brought up. That horse is dead, let's stop beating >> it. (There were a few other proposals as well, btw. Reducibles and >> some other monadic approaches come to mind.) >> >> >> Re pipe() vs looping around read() vs custom Writable vs on('data') >> >> Whatever works for your case is fine. It's flexible on purpose, and >> allows more types of consumption than streams1, and creating custom >> writables is easier than it was in streams1. >> >> If you find something that the API can't do for you, or find yourself >> doing a lot of backflips or overriding a lot of methods to get your >> stuff working, then let's chat about it in a github issue. You might >> be missing something, or you might have found a genuine shortcoming in >> the API. >> >> >> >> On Thu, Mar 21, 2013 at 2:01 PM, Sigurgeir Jonsson >> <[email protected]> wrote: >> > Thanks for all the answers. I almost forgot to look back at this thread >> as >> > the custom writeStreams have exceeded the high expectation I had >> already for >> > Streams 2. >> > For me, the reference manual was a little confusing, as there are >> complete >> > examples on using the read method, no mention of "reading" through a >> > writeStream endpoint. >> > >> > Marco, I agree that that read has more detailed control of minimum >> incoming >> > content. However I wonder if it would be more efficient to default >> > pipe.chunkSize to a "lowWatermark" of the receiver (if defined). This >> > lowWatermark could be adjusted dynamically and the callback in the >> writable >> > should keep sequence of events under control? >> > >> > Anyway, thanks Node team, I'm very impressed! >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > On Wednesday, March 20, 2013 4:45:32 AM UTC-4, Marco Rogers wrote: >> >> >> >> @Nathan's response is right. Creating a writable stream is preferable >> in >> >> most cases. But I wanted to add a little context to that. If you're >> dealing >> >> with a base readable stream, it's just pushing chunks of data at you >> off the >> >> wire. Your first task is to collect those chunks into meaningful data. >> So >> >> IMO the reason creating a writable stream is preferable is because it >> >> prompts you not just read off the stream, but to create semantics >> around >> >> what the new stream is supposed to be. The api reflects this opinion >> and >> >> that's why creating writable streams feels like the more natural way, >> and >> >> the ugliness of dealing with read() is wrapped up in the pipe() >> method. It >> >> was kind of designed that way. >> >> >> >> But the read() api was also designed for a use case. It's meant to >> handle >> >> low/high water marks effectively, as well as enable more optimized >> special >> >> parsing by reading off specific lengths of chunks. These were things >> that >> >> people kept needing, but the old api didn't support well. If you were >> >> writing a library for a special parser, you might write a custom >> Writable >> >> stream and inside it you would be using the read(n) api to control >> *how* you >> >> read data off the socket. I hope that makes sense. >> >> >> >> :Marco >> >> >> >> On Monday, March 18, 2013 11:06:48 AM UTC-7, Sigurgeir Jonsson wrote: >> >>> >> >>> The new streams have excellent support for high/low watermarks and >> >>> auto-pausing/resuming, but the documentation confuses me a little... >> >>> particularly the read method. >> >>> >> >>> When I read the new docs for the first time I was under the impression >> >>> that the optimal way to become a user of a stream is to write loops >> around >> >>> the read functio. However in practice I find myself simply writing >> custom >> >>> writeStreams and use the callback to control upstream pressure (in >> addition >> >>> to source Watermarks if needed). Here is an example where I move the >> >>> output to a queue that executes a custom function in parallel (i.e. >> >>> uploading to a database) https://gist.github.com/ZJONSSON/5189249 >> >>> >> >>> Are there any benefits to using the read method directly on a stream >> vs. >> >>> piping to a custom Writable stream? >> > >> > -- >> > -- >> > Job Board: http://jobs.nodejs.org/ >> > Posting guidelines: >> > https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines >> > You received this message because you are subscribed to the Google >> > Groups "nodejs" group. >> > To post to this group, send email to [email protected] >> > To unsubscribe from this group, send email to >> > [email protected] >> > For more options, visit this group at >> > http://groups.google.com/group/nodejs?hl=en?hl=en >> > >> > --- >> > You received this message because you are subscribed to the Google >> Groups >> > "nodejs" group. >> > To unsubscribe from this group and stop receiving emails from it, send >> an >> > email to [email protected]. >> > For more options, visit https://groups.google.com/groups/opt_out. >> > >> > >> >> -- >> -- >> Job Board: http://jobs.nodejs.org/ >> Posting guidelines: >> https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines >> You received this message because you are subscribed to the Google >> Groups "nodejs" group. >> To post to this group, send email to [email protected] >> To unsubscribe from this group, send email to >> [email protected] >> For more options, visit this group at >> http://groups.google.com/group/nodejs?hl=en?hl=en >> >> --- >> You received this message because you are subscribed to the Google Groups >> "nodejs" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> For more options, visit https://groups.google.com/groups/opt_out. >> >> >> > -- > -- > Job Board: http://jobs.nodejs.org/ > Posting guidelines: > https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines > You received this message because you are subscribed to the Google > Groups "nodejs" group. > To post to this group, send email to [email protected] > To unsubscribe from this group, send email to > [email protected] > For more options, visit this group at > http://groups.google.com/group/nodejs?hl=en?hl=en > > --- > You received this message because you are subscribed to the Google Groups > "nodejs" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > For more options, visit https://groups.google.com/groups/opt_out. > > > -- -- Job Board: http://jobs.nodejs.org/ Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines You received this message because you are subscribed to the Google Groups "nodejs" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/nodejs?hl=en?hl=en --- You received this message because you are subscribed to the Google Groups "nodejs" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/groups/opt_out.
