On Tue, Dec 24, 2013 at 9:48 PM, Patrick Walton <pcwal...@mozilla.com> wrote: > On 12/24/13 5:45 PM, Daniel Micay wrote: >> >> Bounded channels present the choice between blocking (with an optional >> timeout) when the channel is full or making a non-blocking call with >> custom handling of the error case. > > > I prefer the latter. > > >> Unbounded channels don't provide a >> way to write robust systems without layering on application-level >> message passing and book-keeping. It doesn't absolve you of handling >> deadlocks by considering the bounds because you're just going to get a >> far worse out-of-memory condition instead. > > > I don't think OOM is far worse than a deadlock. On the server side both > amount to "get a page and log into the machine and restart the affected > process". Which is of course very bad, but I don't think one is that much > worse than the other.
It's worse than a deadlock because out-of-memory causes significant issues for the rest of the system. It will cause a massive amount of swapping, which without an SSD can cause a catastrophic increase in latency where the system will just churn for ages. On a system without overcommit, many processes are going to hit failures in `malloc` and may have to flush buffers and terminate. In a system with overcommit, the misbehaving Rust process will hopefully be killed but it's not a sure thing since another process may be using a lot more memory for a valid reason. I think the ability to deadlock is overstated... as it's not going to happen without two-way communication. If you have clear consumers and producers then it's not an issue you can hit. If you don't, then you can still have a well-defined communication protocol handling the issue. > Anyway, I'm OK with bounded channels to avoid OOM, but I'm pretty unsure > about the following three things: > > 1. Blocking on the sender side when the queue is full. This causes > deadlocks. You can create the equivalent using two channels if you really > want this behavior. When an unbounded queue is full, a process is killed. It's not really a point in favour of unbounded queues in my opinion. > 2. Allocating a flat array with the size of the bound up front for channels. > In a producer/consumer scenario, you want the bound to be high to maximize > parallelism, without paying a constant memory usage penalty. It makes sense to allocate it up-front for some lock-free queues but not a classical one built on a mutex and two condition variables. I doubt that a lock-free algorithm with have any significant wins against a Linux mutex or Windows critical section, and on Haswell lock elision erases the theoretical benefits too. > 3. Having a bound of 1 by default. The default should allow for parallelism. A bound of 1 by default seems pretty stupid. I've never understood why Go does this... it belongs in a separate type. > Haskell may be a nice thing to look at here: the unbounded channel is the > primitive, and bounded channels are built on top of unbounded ones using an > `MVar` to rate limit. Such a thing could be easily built in Rust using a > channel paired with an `AtomicUint`. The performance does matter though and this seems like it would slow things down. If it's built on top of the lock-free channels it will be doing needless atomic operations. _______________________________________________ Rust-dev mailing list Rust-dev@mozilla.org https://mail.mozilla.org/listinfo/rust-dev
