Re: spawn X different workers & wait for results from all of them

2015-09-08 Thread Justin Whear via Digitalmars-d-learn 
On Sat, 05 Sep 2015 12:21:33 +0200, Robert M. Münch wrote:

> My "pieceOfWork" is not the same. So I don't have the case: Do 4 time
> this 1thing. Instead, do 1 time these 4 things.

Ah, so you want to receive one each of various types?  Something like 
this might work (untested):

// These could be inferred using std.traits.ReturnType
alias ResultTypes = AliasSeq!(int, float, Baz);

bool received(T) = false;
bool receivedAll()
{
foreach (T; ResultTypes)
if (!received!T) return false;
return true;
}

while (!receivedAll)
{
receive(
(int x)   { received!int = true; /* other work... */ },
(float x) { received!float = true; /* other work... */ },
(Baz x)   { received!Baz = true; /* other work... */ }
);
}

Re: spawn X different workers & wait for results from all of them

2015-09-05 Thread Robert M. Münch via Digitalmars-d-learn 

On 2015-09-04 17:32:48 +, Justin Whear said:


How would receive know?


Well, it could be pretty simple. At the moment:

receive(
int ...,
long ...,
myStruct ...
)

Will wait for one out of the three. So it's an OR.

reveive_all(
int ...,
long ...,
myStruct ...
)

would finish if every type was returned once (maybe at least once). 
With this I would have an AND combination.



If you're using std.concurrency, the receiving
function needs to encode doneness, e.g.

const numJobs = 4;
foreach (_; 0 .. numJobs)
receive(...);


That's what you can do if the type is the same. Or just put four 
receives sequentially in the code. The not so nice side effect is, that 
you define an order for processing but the tasks might finish in an 
other order. So, possible work is blocked.



Or you could use std.parallelism:

foreach (pieceOfWork; parallel(listOfWork))
doIt(pieceOfWork);


My "pieceOfWork" is not the same. So I don't have the case: Do 4 time 
this 1thing. Instead, do 1 time these 4 things.


--
Robert M. Münch
http://www.saphirion.com
smarter | better | faster

Re: spawn X different workers & wait for results from all of them

2015-09-05 Thread thedeemon via Digitalmars-d-learn 
On Thursday, 3 September 2015 at 16:50:21 UTC, Robert M. Münch 
wrote:

Hi, I'm not sure how to best implement the following:

1. I have 4 different tasks to do.
2. All can run in parallel
3. Every task will return some result that I need

Now how to best do it?


I think the Task and taskPool from std.parallelism are a good fit.
Something like:

auto task1 = task!fun1(params1);
auto task2 = task!fun2(params2);
auto task3 = task!fun3(params3);
auto task4 = task!fun4(params4);

taskPool.put(task1);
taskPool.put(task2);
taskPool.put(task3);
taskPool.put(task4);

auto res1 = task1.workForce();
auto res2 = task2.workForce();
auto res3 = task3.workForce();
auto res4 = task4.workForce();

//here we have all the results, they were calculated in parallel

Re: spawn X different workers & wait for results from all of them

2015-09-05 Thread Robert M. Münch via Digitalmars-d-learn 

On 2015-09-05 15:44:02 +, thedeemon said:


I think the Task and taskPool from std.parallelism are a good fit.
Something like:

auto task1 = task!fun1(params1);
auto task2 = task!fun2(params2);
auto task3 = task!fun3(params3);
auto task4 = task!fun4(params4);

taskPool.put(task1);
taskPool.put(task2);
taskPool.put(task3);
taskPool.put(task4);

auto res1 = task1.workForce();
auto res2 = task2.workForce();
auto res3 = task3.workForce();
auto res4 = task4.workForce();

//here we have all the results, they were calculated in parallel


Thanks, that looks pretty good. Going to x-check how this could be used.

--
Robert M. Münch
http://www.saphirion.com
smarter | better | faster

Re: spawn X different workers & wait for results from all of them

2015-09-04 Thread Justin Whear via Digitalmars-d-learn 
On Thu, 03 Sep 2015 18:50:21 +0200, Robert M. Münch wrote:

> Hi, I'm not sure how to best implement the following:
> 
> 1. I have 4 different tasks to do.
> 2. All can run in parallel 3. Every task will return some result that I
> need
> 
> Now how to best do it? When using receive() it fires on the first hit
> and the function continues. It's like a receive(OR), one hit and it's
> done. So, I would need something like a receive(ALL) that continues only
> of all results (messages) have been received.
> 
> Is something like this available or do I have to build it myself?

How would receive know?  If you're using std.concurrency, the receiving 
function needs to encode doneness, e.g.

const numJobs = 4;
foreach (_; 0 .. numJobs)
receive(...);

Or you could use std.parallelism:

foreach (pieceOfWork; parallel(listOfWork))
doIt(pieceOfWork);