Re: how to print progress of a long running parallel() loop?

[Ali Çehreli via Digitalmars-d-learn]

On 12/7/20 4:48 AM, Dukc wrote:

On Monday, 7 December 2020 at 08:16:50 UTC, mw wrote:

r = Parallel(n_jobs=2, verbose=10)(delayed(sleep)(.2) for _ in range(10))

to print out the progress.

How to do this in D's parallel loop?

thanks.


Allocate a `shared int` before the foreach loop. In the loop when, let's 
say `!(i & 0xFFF)`,  atomically increment the shared variable and print 
it's number.


Disclaimer: no experience about using `shared`, but this is what I'd try.


Yes, that would be the way but I don't know the effect of using shared 
or not. But because the progress needs to be somehow reported, an atomic 
variable may not be sufficient (unless a separate thread is reading it, 
etc.)


This issue came up during one of my recent presentations at this point:

  https://www.youtube.com/watch?v=dRORNQIB2wA&feature=youtu.be&t=1527

I used a synchronized block in that code because it did not affect 
performance in my use case.


Ali

Re: how to print progress of a long running parallel() loop?

[Dukc via Digitalmars-d-learn]

On Monday, 7 December 2020 at 08:16:50 UTC, mw wrote:
r = Parallel(n_jobs=2, verbose=10)(delayed(sleep)(.2) for _ in 
range(10))


to print out the progress.

How to do this in D's parallel loop?

thanks.


Allocate a `shared int` before the foreach loop. In the loop 
when, let's say `!(i & 0xFFF)`,  atomically increment the shared 
variable and print it's number.


Disclaimer: no experience about using `shared`, but this is what 
I'd try.

how to print progress of a long running parallel() loop?

[mw via Digitalmars-d-learn]

https://dlang.org/phobos/std_parallelism.html#.parallel

auto logs = new double[1_000_000];

foreach (i, ref elem; parallel(logs))
{
elem = log(i + 1.0);
}


In Python, using joblib, I can use `verbose` level like this:

r = Parallel(n_jobs=2, verbose=10)(delayed(sleep)(.2) for _ in 
range(10))


to print out the progress.

How to do this in D's parallel loop?

thanks.