dsimcha wrote:
I've created an alpha release of parallelFuture, a high-level parallelization
library for D2. Right now, it has a task pool, futures, parallel foreach, and
parallel map.
Here's the (IMHO) coolest example:
auto pool = new ThreadPool();
// Assuming we have a function isPrime(), print all
// prime numbers from 0 to uint.max, testing for primeness
// in parallel.
auto myRange = iota(0, uint.max);
foreach(num; pool.parallel(myRange)) {
if(isPrime(num)) {
synchronized writeln(num);
}
}
The interface is there and it seems to work, although it has not been
extensively stress tested yet. Some of the implementation details could
admittedly use some cleaning up, and I would appreciate help from some
threading gurus on improving my queue (right now it's a naive synchronized
singly-linked list) and getting condition mutexes to work properly. (Right
now, I'm using atomic polling followed by sleeping for 1 millisecond in a lot
of places. It's a kludge, but it seems to work reasonably well in practice.)
The code is at:
http://dsource.org/projects/scrapple/browser/trunk/parallelFuture/parallelFuture.d
The docs are at:
http://cis.jhu.edu/~dsimcha/parallelFuture.html
Nice. About the tasks:
In .Net a worker thread is created for each cpu core. Each worker thread
has its own local queue of tasks which it initially retrieves from the
global queue but if the tasks creates new tasks it adds to its local
queue directly for less contention. When it finishes completing a task
it takes the next from the back of its local queue to take advantage of
cache (like a stack). The tasks at the front of the queue can be stolen
from another worker thread if its local queue and the global queue are
both empty to maximize cpu usage.
Also the task's wait for complete is only considered complete if all of
the tasks it created too are complete (kinda like recursion).
What is the implementation or plans like for this.
