On Oct 6, 2011, at 2:37 PM, Brian Anderson wrote: > Hi Jim, > > You have a lot of good observations here, of which I may be qualified to > respond to a few. > > On 10/05/2011 08:53 AM, Jim Peters wrote: >> As a newcomer to Rust, it seems to me that it would be a >> simplification to express an iterator using a port, channel and task, >> to re-use existing parts of the compiler instead of making an iterator >> a special case. So the iterator code (notionally) runs as a task and >> passes back values over the channel, and the foreach loop pulls in >> those values one by one from the port. (The task creation could be >> hidden behind the 'iter' calling syntax.) > > Expressing iteration with tasks is very attractive.
It's definitely nice from the standpoint of having fewer languages forms for users to learn and implementers to debug. It reminds me of how task join and status notifications got a lot simpler when we implemented them in terms of messages. However, iterators need to be really fast, and in most cases the compiler should be able to optimize them into a really tight loop. I really doubt Rust's task system will ever be able to be this fast. >> For example, let's say I have a loop from 1 to 100,000, and there is a >> small chunk of work I need to do for each value. The obvious approach >> is to spawn a task for each of these 100,000 values in order to make >> use of all my cores. How is Rust going to deal with this? >> >> - Allocate 100,000 stack frames for all the tasks in a global pool, >> and work through them in parallel on N cores? (Memory explosion) > > I think this is the goal. Rust will be getting stack growth soon and stacks > should start very small. Even without stack growth, we have a few programs that manage to spawn ten thousands of threads. This requires a small hack to shrink the default stack size, but it does work. In general, Rust programmers shouldn't have to worry about keeping track of how many tasks they are creating. >> Reading the Wiki, it suggests that the task/port/channel-related stuff >> was being moved out to a library, which means the compiler can't >> optimise it -- or was that just the old design? > > All task stuff is in the library now, partly because it allows more freedom > to experiment. It does take away the compiler's ability to do any special > optimizations with tasks. The key benefit of having tasks and communication is that we can innovate in the task system without having to modify the compiler. Tasks and communication are still an integral part of the language, even if their interface is only through the standard library. For example, communication safety is enforced by the kind system. The runtime is also very much aware of tasks, and many dynamic scheduling optimizations can happen here. For example, one scheduling trick I think would be worth trying is making it so when you send a message to a port and the task that owns the port is blocked on that port, the sending task could yield directly to the receiving task. I think this might improve performance for tasks that primarily receive small messages, do a minimal amount of processing, and then go back to waiting. There's a spectrum of things various languages call tasks. Rust's are more like lightweight threads. As such, they should be really cheap to create and you can comfortable create many thousands of them, but they will probably always cost more than a function call, for example. Other languages have lighter task-like things that could be used for things like iterators, but they tend to require extensive code transformations to implement efficiently. Rust's design philosophy seems to eschew these high level code transformations in favor of making it easier to predict what output the compiler will generate for various programs. -Eric _______________________________________________ Rust-dev mailing list [email protected] https://mail.mozilla.org/listinfo/rust-dev
