On Tue, Jul 2, 2013 at 7:53 AM, Rusty Conover <[email protected]> wrote: > Hi, > > I have a stream that has many listeners waiting for it to drain before more > data will be written. When it fires a drain event, the first listener who > processes the drain event will generally cause the stream to fill up again > (calls .write() until it returns false), but the other listeners to the > drain event continue to be called even if the stream is no longer "drained" > because they are simply in the callback list. > > If I have 10 listeners that want to write to 1 stream, the first listener > gets the drain event and writes enough data to fill the stream's high > watermark buffer, then listeners 2-10 also write their data causing node to > buffer up more data on the WritableStream's buffer list. Basically for > every one drain event, I have nine other listeners buffering data that they > shouldn't (the thundering herd of other listeners that should be sleeping). > > I propose adding some new functionality to the event interface that allows > events being emitted to allow "conditional emission". That is a listener > should be able to "swallow" an event and not propagate it to the other > listeners of that same event. In that way one drain event will only be > propagated to the first listener and if that listener fills the stream up, > the drain event will not be propagated to the other listeners causing less > data to be buffered and memory to be used. If the listener does not fill up > the stream the event should be allowed to continue propagating to the other > listeners. > > Being that the event module is frozen, what is the best path for > implementation and adoption if this is reasonable? > > Rusty
Before we go into the merits of such functionality, what is a plausible multiple writers/single reader scenario? The only one I can come up is logging, where you probably don't care about the exact order of writes. By the way, this 'thundering writers' issue seems like it's straightforward to solve by sticking in an in-between stream that wakes up writers selectively. -- -- Job Board: http://jobs.nodejs.org/ Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines You received this message because you are subscribed to the Google Groups "nodejs" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/nodejs?hl=en?hl=en --- You received this message because you are subscribed to the Google Groups "nodejs" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/groups/opt_out.
