My implementation and Mikeal's implementation were very early drafts of bufferedstream. Mjijackson's looks pretty robust and I usually point people at that (haven't used it personally). It even handles the case of end() before you've read the buffered data out. Usage should be pretty straight forward.
bufferedStream.pause(); // make sure to pause before piping so it doesn't propogate to the read steam req.pipe(bufferedStream); // do some async stuff and resume in the callback The thing to be aware of here is is that if you pass the bufferedStream around instead of the actual req, it wil not behave the same. This is a known issue with the current piping paradigm. :Marco On Thursday, June 28, 2012 8:28:10 AM UTC-7, Domenic Denicola wrote: > > The scenario I'm trying to solve is: > * Get an incoming HTTP request stream > * Look at the headers and use them to do some authorization logic, > asynchronously > * When that logic returns successfully, pipe the request stream to > another stream > * When that logic returns a failure, discard the incoming request stream > and don't leak memory. > > My initial thought was to do `req.pause()` before the async authorization, > then `req.resume()` in the callback. But this breaks because `req.pause` is > useless [1]. It is also a bad idea because, from what I understand, it > sends some TCP-level pause signal, creating unnecessary roundtrips. > > My second try was to use Connect's `util.pause` [2] on the stream, and > resume in the callback. Note that `util.pause` really is just a way to > buffer data/end events. This breaks sporadically, because sometimes the > request data is small and all of it gets emitted by the time I finish > authorization and call resume. The result is then a non-readable stream > emitting data and end events, which doesn't work great with pipe or really > anything. See [3] for an example of how this bit me. > > I'm now thinking about trying something like Mikeal's > morestreams.BufferedStream [4], mjijackson's BufferedStream [5], or Marco's > BufferedStream gist [6]. Has anyone used these? Comparative opinions? Only > mjijackson's is documented. What would the use look like? Mikeal gives an > example in the original thread [7]; is that still solid? Is it easy to > ensure you don't leak memory? > > Is there a better solution? This has to be a pretty common problem, right? > One particular pain point when I've tried to use such buffered stream > abstractions before is that you lose all the useful properties of the > request object, so I guess I have to carry around two objects: request (for > headers etc.) and requestDataStream. Has anyone made that work better? I > was looking at streamlinejs's wrappers [8], which seemed potentially > helpful, but streamlinejs is kind of weird. > > > [1]: https://groups.google.com/d/topic/nodejs/yv6Dl-O-wYk/discussion > [2]: > https://github.com/senchalabs/connect/blob/e6ca949dcd6083f6d10ca7896e9050157b4bc979/lib/utils.js#L231-276 > [3]: https://github.com/aheckmann/gm/issues/65 > [4]: https://github.com/mikeal/morestreams > [5]: https://github.com/mjijackson/bufferedstream/ > [6]: https://gist.github.com/850957 > [7]: https://groups.google.com/d/msg/nodejs/yv6Dl-O-wYk/h_yuaQQLXtsJ > [8]: > https://github.com/Sage/streamlinejs/blob/master/lib/streams/server/streams.md > -- Job Board: http://jobs.nodejs.org/ Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines You received this message because you are subscribed to the Google Groups "nodejs" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/nodejs?hl=en?hl=en
