Is it correct to assume that a Readable won't emit the "readable" event
until you're registered for it?
Reading through the streams2 docs, I was under the impression that all
streams start out paused and don't start emitting data until you add either
a "data" (for old streams) or a "readable" listener. For new streams, this
should mean that they don't emit "readable" until at least one listener is
registered. Otherwise we still need to do some buffering in order to
capture all the data.
For example, this code misses the readable event on node 0.10:
var http = require('http');
http.get('http://www.google.com', function (response) {
console.log('got response with status ' + response.statusCode);
setTimeout(function () {
response.on('readable', function () {
console.log('readable');
});
response.on('end', function () {
console.log('end');
});
}, 5);
});
Here's my shell session:
$ node -v
v0.10.0
$ node http-test.js
got response with status 200
$
Is this the correct behavior?
--
Michael Jackson
@mjackson
On Thu, Mar 21, 2013 at 4:27 PM, Isaac Schlueter <[email protected]> wrote:
> re old-mode
>
> Yes, that's fine. If you just want to get all the data asap, use
> on('data', handler). It'll work great, and it's still very fast.
> pause()/resume(), the whole bit. (The difference is that it won't
> emit data until you're listening, and pause() will *actually* pause.)
>
>
> Re read(cb)
>
> It's problematic for reasons that I've discussed all of the places
> where it's been brought up. That horse is dead, let's stop beating
> it. (There were a few other proposals as well, btw. Reducibles and
> some other monadic approaches come to mind.)
>
>
> Re pipe() vs looping around read() vs custom Writable vs on('data')
>
> Whatever works for your case is fine. It's flexible on purpose, and
> allows more types of consumption than streams1, and creating custom
> writables is easier than it was in streams1.
>
> If you find something that the API can't do for you, or find yourself
> doing a lot of backflips or overriding a lot of methods to get your
> stuff working, then let's chat about it in a github issue. You might
> be missing something, or you might have found a genuine shortcoming in
> the API.
>
>
>
> On Thu, Mar 21, 2013 at 2:01 PM, Sigurgeir Jonsson
> <[email protected]> wrote:
> > Thanks for all the answers. I almost forgot to look back at this thread
> as
> > the custom writeStreams have exceeded the high expectation I had already
> for
> > Streams 2.
> > For me, the reference manual was a little confusing, as there are
> complete
> > examples on using the read method, no mention of "reading" through a
> > writeStream endpoint.
> >
> > Marco, I agree that that read has more detailed control of minimum
> incoming
> > content. However I wonder if it would be more efficient to default
> > pipe.chunkSize to a "lowWatermark" of the receiver (if defined). This
> > lowWatermark could be adjusted dynamically and the callback in the
> writable
> > should keep sequence of events under control?
> >
> > Anyway, thanks Node team, I'm very impressed!
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > On Wednesday, March 20, 2013 4:45:32 AM UTC-4, Marco Rogers wrote:
> >>
> >> @Nathan's response is right. Creating a writable stream is preferable in
> >> most cases. But I wanted to add a little context to that. If you're
> dealing
> >> with a base readable stream, it's just pushing chunks of data at you
> off the
> >> wire. Your first task is to collect those chunks into meaningful data.
> So
> >> IMO the reason creating a writable stream is preferable is because it
> >> prompts you not just read off the stream, but to create semantics around
> >> what the new stream is supposed to be. The api reflects this opinion and
> >> that's why creating writable streams feels like the more natural way,
> and
> >> the ugliness of dealing with read() is wrapped up in the pipe() method.
> It
> >> was kind of designed that way.
> >>
> >> But the read() api was also designed for a use case. It's meant to
> handle
> >> low/high water marks effectively, as well as enable more optimized
> special
> >> parsing by reading off specific lengths of chunks. These were things
> that
> >> people kept needing, but the old api didn't support well. If you were
> >> writing a library for a special parser, you might write a custom
> Writable
> >> stream and inside it you would be using the read(n) api to control
> *how* you
> >> read data off the socket. I hope that makes sense.
> >>
> >> :Marco
> >>
> >> On Monday, March 18, 2013 11:06:48 AM UTC-7, Sigurgeir Jonsson wrote:
> >>>
> >>> The new streams have excellent support for high/low watermarks and
> >>> auto-pausing/resuming, but the documentation confuses me a little...
> >>> particularly the read method.
> >>>
> >>> When I read the new docs for the first time I was under the impression
> >>> that the optimal way to become a user of a stream is to write loops
> around
> >>> the read functio. However in practice I find myself simply writing
> custom
> >>> writeStreams and use the callback to control upstream pressure (in
> addition
> >>> to source Watermarks if needed). Here is an example where I move the
> >>> output to a queue that executes a custom function in parallel (i.e.
> >>> uploading to a database) https://gist.github.com/ZJONSSON/5189249
> >>>
> >>> Are there any benefits to using the read method directly on a stream
> vs.
> >>> piping to a custom Writable stream?
> >
> > --
> > --
> > Job Board: http://jobs.nodejs.org/
> > Posting guidelines:
> > https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
> > You received this message because you are subscribed to the Google
> > Groups "nodejs" group.
> > To post to this group, send email to [email protected]
> > To unsubscribe from this group, send email to
> > [email protected]
> > For more options, visit this group at
> > http://groups.google.com/group/nodejs?hl=en?hl=en
> >
> > ---
> > You received this message because you are subscribed to the Google Groups
> > "nodejs" group.
> > To unsubscribe from this group and stop receiving emails from it, send an
> > email to [email protected].
> > For more options, visit https://groups.google.com/groups/opt_out.
> >
> >
>
> --
> --
> Job Board: http://jobs.nodejs.org/
> Posting guidelines:
> https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
> You received this message because you are subscribed to the Google
> Groups "nodejs" group.
> To post to this group, send email to [email protected]
> To unsubscribe from this group, send email to
> [email protected]
> For more options, visit this group at
> http://groups.google.com/group/nodejs?hl=en?hl=en
>
> ---
> You received this message because you are subscribed to the Google Groups
> "nodejs" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to [email protected].
> For more options, visit https://groups.google.com/groups/opt_out.
>
>
>
--
--
Job Board: http://jobs.nodejs.org/
Posting guidelines:
https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
You received this message because you are subscribed to the Google
Groups "nodejs" group.
To post to this group, send email to [email protected]
To unsubscribe from this group, send email to
[email protected]
For more options, visit this group at
http://groups.google.com/group/nodejs?hl=en?hl=en
---
You received this message because you are subscribed to the Google Groups
"nodejs" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
For more options, visit https://groups.google.com/groups/opt_out.
