Hi,
I'm having some trouble figuring out exactly how I'm supposed to use
SharedArrays - I might just be misunderstanding them or else something
odd is happening with them.
I'm trying to do some parallel computing which looks a bit like this
test case:
function createdata(shared)
const n = 1000
if shared
A = SharedArray(Uint, (n, n))
else
A = Array(Uint, (n, n))
end
for i = 1:n, j = 1:n
A[i, j] = rand(Uint)
end
return n, A
end
function mainfunction(r; shared = false)
n, A = createdata(shared)
i = 1
nextidx() = (idx = i; i += 1; idx)
@sync begin
for p in workers()
@async begin
while true
idx = nextidx()
if idx > r
break
end
found, s = remotecall_fetch(p, parfunction, n, A)
end
end
end
end
end
function parfunction(n::Int, A::Array{Uint, 2})
# possibly do some other computation here independent of shared
arrays
s = sum(A)
return false, s
end
function parfunction(n::Int, A::SharedArray{Uint, 2})
s = sum(A)
return false, s
end
If I then start julia with e.g. two worker processes, so julia -p 2, the
following happens:
julia> require("testpar.jl")
julia> @time mainfunction(1000, shared = false)
elapsed time: 15.717117365 seconds (8448701068 bytes allocated)
julia> @time mainfunction(1000, shared = true)
elapsed time: 6.068758627 seconds (56713996 bytes allocated)
julia> rmprocs([2, 3])
:ok
julia> @time mainfunction(1000, shared = false)
elapsed time: 0.717638344 seconds (40357664 bytes allocated)
julia> @time mainfunction(1000, shared = true)
elapsed time: 0.702174085 seconds (32680628 bytes allocated)
So, with a normal array it's slow as expected, and it is faster with the
shared array, but what seems to happen is that with the normal array cpu
usage is 100 % on two cores but with the shared array cpu usage spikes
for a fraction of a second and then for the remaining nearly 6 seconds
it's at around 10 %. Can anyone reproduce this? Am I just doing
something wrong with shared arrays.
Slightly related note: is there now a way to create a random shared
array? https://github.com/JuliaLang/julia/pull/4939 and the latest docs
don't mention this.
