On Wednesday, 19 February 2014 at 09:21:48 UTC, Tarman wrote:
Hi,
We're doing some "super computing" "big data" style stuff with
D. We have a system where we're comparing associative arrays
with billions of entries.
However in this system we need to fairly consider possible
solutions for many "units" at a time within a single thread.
So we'd like to... say, first iterate over the first 10 million
for each "unit" then iterate over the next 10 million for the
next unit so that each unit gets a fair share of CPU time.
However in this case we can't:
int count = 0;
foreach (item; associativeArray) {
if (++count == 10_000_000) {
// here we wish to somehow save the iteration state
}
}
Then on the next iteration:
foreach (resume from previous iteration point) {
}
We've tried copying the keys into a non-associative array and
sure this works, but it is far far far less optimal than an
equivalent C++ solution we wrote where we use an
std::unordered_set and can simply store the iterator.
Probably not an answer to your question, but if possible, you can
use a fiber and yield when you should give control to another
fiber.
http://dlang.org/phobos/core_thread.html#.Fiber
