I'm trying to understand the most Julian way to perform a particular
parallel programming task. Suppose I need function foo from module.jl to be
available everywhere. Let's call the following code map_foo.jl:
@everywhere include("module.jl")
@everywhere using MyModule
pmap(foo,1:100)
That works fine, except when module.jl itself has other dependencies on
other modules:
module MyModule
using DataStructures
export foo
function foo(i)
return Queue(i)
end
end # module
In this case, it works to call
julia map_foo.jl
but when I call
julia -p 2 map_foo.jl
I get the following error
Warning: requiring "DataStructures" did not define a corresponding module.
Warning: requiring "DataStructures" did not define a corresponding module.
exception on exception on 2: 3: ERROR: ERROR: Queue not definedQueue not
defined
in
in foo at /Users/madeleineudell/Dropbox/pestilli_icme_life
(1)/src/julia/questions/module.jl:7
in anonymous at multi.jl:834
in run_work_thunk at multi.jl:575
in anonymous at task.jl:834
foo at /Users/madeleineudell/Dropbox/pestilli_icme_life
(1)/src/julia/questions/module.jl:7
in anonymous at multi.jl:834
in run_work_thunk at multi.jl:575
in anonymous at task.jl:834
Does anyone know how I can successfully chain dependencies like this when
using parallelism? Calling @everywhere on the import call in module.jl also
doesn't fix the problem, strangely enough.
Of course, if I could put all my code into shared memory, I'd be much
happier. I just saw an update <https://github.com/JuliaLang/julia/pull/4939>
adding
shared memory arrays, but I don't know if there's a way to get shared
memory code!
