Perhaps SharedArrays are what you need here?
http://docs.julialang.org/en/release-0.3/stdlib/parallel/?highlight=sharedarray#Base.SharedArray
Reading from a shared array in workers is fine, but when different workers try
to update the same part of that array you will get racy behaviour and most
likely not the correct result.
Can you somehow re-formulate your problem along these lines, using a map and
reduce approach using a pure function?
@everywhere function myfunc_pure(startindex)
result = zeros(Int,10)
for i in startindex + (0:19) # 20 iterations
result[mod(i,length(result))+1] += 1
end
result
end
reduce(+,pmap(myfunc_pure, 1:5)) # 5 blocks of 20 iterations
Like this you don't have a shared mutable state and thus no risk for mess-ups.
Am 13.03.2015 um 00:56 schrieb Pieter Barendrecht <[email protected]>:
> I'm wondering how to save data/results in a parallel for-loop. Let's assume
> there is a single Int64 array, initialised using zeros() before starting the
> for-loop. In the for-loop (typically ~100,000 iterations, that's the reason
> I'm interested in parallel processing) the entries of this Int64 array should
> be increased (based on the results of an algorithm that's invoked in the
> for-loop).
>
> Everything works fine when using just a single proc, but I'm not sure how to
> modify the code such that, when using e.g. addprocs(4), the data/results
> stored in the Int64 array can be processed once the for-loop ends. The
> algorithm (a separate function) is available to all procs (using the
> require() function). Just using the Int64 array in the for-loop (using
> @parallel for k=1:100000) does not work as each proc receives its own copy,
> so after the for-loop it contains just zeros (as illustrated in a set of
> slides on the Julia language). I guess it involves @spawn and fetch() and/or
> pmap(). Any suggestions or examples would be much appreciated :).
