On 10/28/2013 10:02 PM, Simon Ruggier wrote:
Greetings fellow Rustians!
First of all, thanks for working on such a great language. I really
like the clean syntax, increased safety, separation of data from
function definitions, and freedom from having to declare duplicate
method prototypes in header files.
I've been working on an alternate way to communicate between tasks in
Rust, following the same approach as the LMAX Disruptor.[1] I'm hoping
to eventually offer a superset of the functionality in the pipes API,
and replace them as the default communication mechanism between tasks.
Just as with concurrency in general, my main motivation in
implementing this is to improve performance. For more information
about the disruptor approach, there's a lot of information linked from
their home page, in a variety of formats.
This is really exciting work. Thanks for pursuing it. I've been
interested in exploring something like Disruptor in Rust. The current
channel types in Rust are indeed slow, and fixing them is the topic of
https://github.com/mozilla/rust/issues/8568.
This is my first major contribution of new functionality to an
open-source project, so I didn't want to discuss it in advance until I
had a working system to demonstrate. I currently have a very basic
proof of concept that achieves almost two orders of magnitude better
performance than the pipes API. On my hardware[2], I currently see
throughput of about 27 million items per second when synchronizing
with a double-checked wait condition protocol between sender and
receivers, 80+ million items with no blocking (i.e. busy waiting), and
anywhere from 240,000 to 600,000 when using pipes. The LMAX Disruptor
library gets up to 110 million items per second on the same hardware
(using busy waiting and yielding), so there's definitely still room
for significant improvement.
Those are awesome results!
I've put the code up on GitHub (I'm using rustc from master).[3]
Currently, single and multi-stage pipelines of receivers are
supported, while many features are missing, like multiple concurrent
senders, multiple concurrent receivers, or mutation of the items as
they pass through the pipeline. However, given what I have so far, now
is probably the right time to start soliciting feedback and advice.
I'm looking for review, suggestions/constructive criticism, and
guidance about contributing this to the Rust codebase.
I'm not deeply familiar with Disruptor, but I believe that it uses
bounded queues. My general feeling thus far is that, as the general
'go-to' channel type, people should not be using bounded queues that
block the sender when full because of the potential for unexpected
deadlocks. I could be convinced otherwise though if it's just not
possible to have reasonably fast unbounded channels. Note that I don't
think it's critical for the general-purpose channel to be as fast as
possible - it's more important to be convenient.
Please keep working on this. I'm excited to see your results.
Thanks,
Simon
[1] http://lmax-exchange.github.io/disruptor/
[2] A 2.66GHz Intel P8800 CPU running in a Thinkpad T500 on Linux x86_64
[3] https://github.com/sruggier/rust-disruptor
_______________________________________________
Rust-dev mailing list
[email protected]
https://mail.mozilla.org/listinfo/rust-dev
_______________________________________________
Rust-dev mailing list
[email protected]
https://mail.mozilla.org/listinfo/rust-dev
