On Dec 19, 2013, at 10:25 AM, Matthieu Monrocq <[email protected]>
wrote:
> On Thu, Dec 19, 2013 at 7:23 PM, Kevin Ballard <[email protected]> wrote:
> Here’s an example from where I use an infinite queue.
>
> I have an IRC bot, written in Go. The incoming network traffic of this bot is
> handled in one goroutine, which parses each line into its components, and
> enqueues the result on a channel. The channel is very deliberately made
> infinite (via a separate goroutine that stores the infinite buffer in a local
> slice). The reason it’s infinite is because the bot needs to be resilient
> against the case where either the consumer unexpectedly blocks, or the
> network traffic spikes. The general assumption is that, under normal
> conditions, the consumer will always be able to keep up with the producer (as
> the producer is based on network traffic and not e.g. a tight CPU loop
> generating messages as fast as possible). Backpressure makes no sense here,
> as you cannot put backpressure on the network short of letting the socket
> buffer fill up, and letting the socket buffer fill up with cause the IRC
> network to disconnect you. So the overriding goal here is to prevent network
> disconnects, while assuming that the consumer will be able to catch up if it
> ever gets behind.
>
> This particular use case very explicitly wants a dynamically-sized infinite
> channel. I suppose an absurdly large channel would be acceptable, because if
> the consumer ever gets e.g. 100,000 lines behind then it’s in trouble
> already, but I’d rather not have the memory overhead of a
> statically-allocated gigantic channel buffer.
>
> I feel the need to point out that the producer could locally queue the
> messages before sending over the channel if it were bounded.
No it can’t. Most of the time, the producer is blocked waiting to read from the
socket. If it’s locally queued the messages, and the channel empties out, the
producer will still be blocked on the socket and won’t be able to send any of
the queued messages.
-Kevin
_______________________________________________
Rust-dev mailing list
[email protected]
https://mail.mozilla.org/listinfo/rust-dev
