>
>
> It looks like that SO answer is moving data into the global scope of each
> worker. It is probably worth experimenting with but I'd be worried about
> performance implications of non-const global variables. It's probably the
> case that this is still a win for my use case though. Thanks for the link.
>
Why not declare input vector as const?
I have a similar requirement for a simulation. I found it convenient
to wrap everything that is required on workers into a module.
module SphericalHarmonicTransforms
export spherical_harmonic_transforms
const input = Vector{Float64}(10^7)
...
function spherical_harmonic_transforms(idx)
coefficients = Vector{Complex128}(10^6)
... # reference input vector here
return coefficients
end
end
Then to propagate to all workers, just write using
SphericalHarmonicTransforms
function sim()
idx = 1
limit = 10000
nextidx() = (myidx = idx; idx += 1; myidx)
@sync for worker in workers()
@async while true
myidx = nextidx()
myidx ≤ limit || break
coefficients = remotecall_fetch(worker,
spherical_harmonic_transforms, myidx)
write_results_to_disk(coefficients)
end
end
end
addprocs()
using SphericalHarmonicTransforms
sim()
