Hey , check out substack's stream article and look for the section about backpressure . I think it will do exactly what you are looking for here.
https://github.com/substack/stream-handbook - Kelsey On Dec 12, 2012 5:36 AM, "Paul Connolley" <[email protected]> wrote: > Hi there > > Long time lurker, first time poster. I've been working on a module for the > last couple of weeks as a bit of a training exercise. I´ve been digging in > to node for the last 6 months and I´m trying to make all my modules > streaming. The latest exercise is a reverse image proxy for mapping tiles > (Openlayers) with caching that actually would have some application in > frontend JS that I´ve written. > > I want to store ancillary data externally and then attach it to the stream > when piping (akin to how Request and Filed attach relevant header info) and > I obviously want to do this asynchronously. My initial thought was to > override the pipe method on the stream to load the metadata from file and > then call the real pipe once the ancillary data has been retrieved. My > concern is whether I would run the risk of losing the streamed image data > while waiting for the metadata to load. > > Would it be advisable to keep a buffer of incoming stream data while > waiting for the metadata and then emit from the buffer or would it be as > simple as pausing and resuming the incoming pipe? At the moment, I´m > storing JSON in flat files but I´ve just been refactoring so that I could > plug it in to redis or memcached or whatnot. > > As soon as I get to my desk, I will push my current code up to github in > case it´s necessary but any advice would definitely be welcomed. > Alternatively, if I´m taking the wrong route in solving this problem, any > criticism and redirection would be well received. > > Thanks, > connrs. > > -- > Job Board: http://jobs.nodejs.org/ > Posting guidelines: > https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines > You received this message because you are subscribed to the Google > Groups "nodejs" group. > To post to this group, send email to [email protected] > To unsubscribe from this group, send email to > [email protected] > For more options, visit this group at > http://groups.google.com/group/nodejs?hl=en?hl=en > -- Job Board: http://jobs.nodejs.org/ Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines You received this message because you are subscribed to the Google Groups "nodejs" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/nodejs?hl=en?hl=en
