On 04/25/2013 10:42 PM, james wrote:
On 24/04/2013 23:38, Brian Anderson wrote:
What I am focused on now is the design of a synchronous API for
sending and receiving streams of bytes, along with an implementation
built on an internal *asynchronous* I/O interface attached to the
scheduler event loop. This includes reading and writing files, TCP,
UDP, Unix sockets, stdio, pipes, adapters for string parsing,
compression, memory buffers, etc.
This always concerns me - the design will tend towards having state on
the stack etc and 'pull' processing rather than focussing on a clean
design for a chain of state machines that handle 'push' of data coming
from AIO completion events.
Now I know that with say Erlang this is less concerning because
processes are very lightweight there and you have process-specific GC,
and I guess Rust has an ambition to be like that - but is it ready to
handle this sort of IO framework in that way?
I share your concern. It's likely that Rust tasks will never scale as
well as Erlang and I suspect that we will ultimately want an async API
that does not impose the expense of 1 task per I/O stream. I hope though
that the current approach will scale 'pretty well'.
Specifically, will your IO framework be practical with 50,000
connections in reasonable memory (say a constrained 32 bit process)?
I do think 50,000 I/O-performing tasks in a 32-bit process is within our
reach. Last year I demonstrated 500,000 simultaneous Rust tasks (that
did nothing but wait) in a 32-bit process, but there have been severe
regressions since. On Linux I think we can get the initial allocation
overhead for a single task into the ~2k range for sure.
If not, I wonder if there should be more effort on the AIO layer
first, and then consider a synchronous shim on top.
Also - can I ask that any AIO layer be integrated with a
semaphore-type system natively, then at least one can write some
subprocess and use shared memory and integrate it into the main loop.
I don't understand this question.
-Brian
_______________________________________________
Rust-dev mailing list
Rust-dev@mozilla.org
https://mail.mozilla.org/listinfo/rust-dev
