Hi Jim,
You have a lot of good observations here, of which I may be qualified to
respond to a few.
On 10/05/2011 08:53 AM, Jim Peters wrote:
As a newcomer to Rust, it seems to me that it would be a
simplification to express an iterator using a port, channel and task,
to re-use existing parts of the compiler instead of making an iterator
a special case. So the iterator code (notionally) runs as a task and
passes back values over the channel, and the foreach loop pulls in
those values one by one from the port. (The task creation could be
hidden behind the 'iter' calling syntax.)
Expressing iteration with tasks is very attractive.
However, this would force the compiler to do quite a lot of
optimisation if this was going to work efficiently (e.g. as
efficiently as a conventional for loop).
It would definitely require a lot of compiler magic, which is somewhat
contrary to Rust's goal of making everything have a well-defined cost-model.
I have thought for a while about tasklet style implementations
(Erlang/Go/whatever), and the big problem I still don't have a clear
answer to is about scheduling and optimising all those tasks.
Hopefully Rust has some kind of a rough plan for dealing with this?
Honestly, I don't think there's a solid plan for this yet. The short
term goal for tasks has been to get something useful working.
I think if tasks and channels/etc can't be optimised down like this
they're not going to be fast enough for widespread use in algorithms,
and programmers will have to ask themselves at each point whether they
are a good choice or not.
Making task communication fast is a big concern. Programmers should
definitely be comfortable writing rust programs in a task-oriented way
by default, but I think it is probably the case that programmers will
have to consider which abstractions will provide the appropriate
performance for their use case. In general, Rust prefers to provide
multiple abstractions with clear performance costs, than one abstraction
with a clever implementation.
For example, let's say I have a loop from 1 to 100,000, and there is a
small chunk of work I need to do for each value. The obvious approach
is to spawn a task for each of these 100,000 values in order to make
use of all my cores. How is Rust going to deal with this?
- Allocate 100,000 stack frames for all the tasks in a global pool,
and work through them in parallel on N cores? (Memory explosion)
I think this is the goal. Rust will be getting stack growth soon and
stacks should start very small.
Reading the Wiki, it suggests that the task/port/channel-related stuff
was being moved out to a library, which means the compiler can't
optimise it -- or was that just the old design?
All task stuff is in the library now, partly because it allows more
freedom to experiment. It does take away the compiler's ability to do
any special optimizations with tasks.
There's definitely a lot of opportunity to improve and influence the
design of the task system still.
-Brian
_______________________________________________
Rust-dev mailing list
[email protected]
https://mail.mozilla.org/listinfo/rust-dev
