The buffer size of a pipe between your process and ls should fill up eventually and ls's calls to write to stdout should eventually block not allowing ls to continue until you've consumed the buffer.
At least that's my theory - maybe you aren't applying backpressure right? On Wed, Oct 8, 2014 at 5:52 PM, Aseem Kishore <[email protected]> wrote: > Thanks Forrest for the confirmation! I'll subscribe. > > And thanks all for the streaming `ls [-f]` suggestion. I should have > mentioned I had tried that, but it doesn't really solve this problem. > > The reason is because unless you are processing each file instantly, the > backpressure is going to build up, as `ls` keeps outputting names but your > app isn't processing them -- so the memory usage is going to build up > inside Node either way. Pausing the stream only pauses the output stream, > causing Node to buffer up the data; it can't magically pause `ls` execution. > > At least, that's my understanding. Happy to be corrected. > > Aseem > > -- > Job board: http://jobs.nodejs.org/ > New group rules: > https://gist.github.com/othiym23/9886289#file-moderation-policy-md > Old group rules: > https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines > --- > You received this message because you are subscribed to the Google Groups > "nodejs" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To post to this group, send email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/nodejs/2b131db2-d83c-411a-9963-7452044dafdc%40googlegroups.com > <https://groups.google.com/d/msgid/nodejs/2b131db2-d83c-411a-9963-7452044dafdc%40googlegroups.com?utm_medium=email&utm_source=footer> > . > > For more options, visit https://groups.google.com/d/optout. > -- Job board: http://jobs.nodejs.org/ New group rules: https://gist.github.com/othiym23/9886289#file-moderation-policy-md Old group rules: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines --- You received this message because you are subscribed to the Google Groups "nodejs" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/nodejs/CAPJ5V2YHp8eBW5rd77DjX%2BYxwc%2Bqtff1s0E4sNRwz6cyHBkafw%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
