Use a SharedArray.
--Tim
On Friday, May 01, 2015 05:57:54 PM Constantin Berzan wrote:
> I have a parallel application where I want to modify some objects from
> worker processes, and get the modified values back into the main process.
> Is there a clean way to do this in Julia?
>
> Here is a contrived example illustrating the problem (run with "julia -p 2
> myfile.jl"):
>
> @everywhere function mutate_arr!(arr, x)
>
> > println(arr)
> > push!(arr, x)
> > println(arr)
> > nothing
> >
> > end
> >
> >
> > arr = [1, 2, 3]
> > println(arr)
> > remotecall_fetch(2, mutate_arr!, arr, 4)
> > println(arr)
>
> This fails with the following error:
>
> exception on 2: ERROR: cannot resize array with shared data
>
> > in push! at ./array.jl:459
> > in mutate_arr! at .../myfile.jl:3
> > in anonymous at multi.jl:855
> > in run_work_thunk at multi.jl:621
> > in anonymous at task.jl:855
>
> Is it the case that remotecall and @spawn do not allow modifying the
> function arguments? If so, this should be documented.
>
> My second attempt uses a RemoteRef to pass the array explicitly:
>
> @everywhere function mutate_arr!(arr, x)
>
> > println(arr)
> > push!(arr, x)
> > println(arr)
> > nothing
> >
> > end
>
> @everywhere function wrapper_mutate_arr!(ref, x)
>
> > local_arr = take!(ref)
> > mutate_arr!(local_arr, x)
> > put!(ref, local_arr)
> > nothing
> >
> > end
>
> arr = [1, 2, 3]
>
> > println(arr)
>
> arr_ref = RemoteRef(2)
>
> > put!(arr_ref, arr)
> > remotecall_fetch(2, wrapper_mutate_arr!, arr_ref, 4)
> > arr = take!(arr_ref)
> > println(arr)
>
> This fails with the same error ("cannot resize array with shared data").
> What is shared here, and why?
>
> If I use replace local_arr = take!(ref) with local_arr = copy(take!(ref)),
> it works. But I think this creates an additional copy (one for the
> serialization, and one for the copy() call), which I would like to avoid.
>
> In summary: If I have a function like mutate_arr! that I want to offload to
> a worker process, what is the right way to do it? Obviously the functional
> way of thinking is to avoid mutation entirely, but if I already have a
> function like that, what should I do? Just use an additional copy() as
> above?
>
> (In case it matters, I'm using Julia 0.3.4.)
>
> Thanks,
> Constantin
