Hello -
I have the following problem: I would like to find a good way to import a
collection of user-defined functions across several workers. Some of my
functions are defined on DataFrames, but "using DataFrames" is not getting
me anywhere on the other workers.
I think the problem I am running into may result from some combination of
the rules of scope and the command "using".
Alternatively, given what I want to do, maybe running this with "julia -p
N" is not the best way to make use of N workers in the way I want to.
I open several workers: "julia -p 3"
"using DataFrames" - this brings DataFrames into the 'main' worker's scope
(I think), but not into the scope of subordinate workers.
"using ProjectModule" - I am trying to load a module across all workers
which contains several functions I have written (maybe this is not the best
way to accomplish this task?)
This error is returned:
LoadError: LoadError: UndefVarError: DataFrame not defined
ProjectModule looks something like
module ProjectModule
include("function1.jl")
export function1
end
where function1 is defined as
function1(input::DataFrame)
#do something to input
end
I have tried a few things:
- Running "@everywhere using DataFrames" from within the main worker (this
has worked once or twice - that is, I can then use function1 on a different
worker - but it isn't consistent)
- Opening the workers at the outset using julia -p N -L ProjectModule.jl...
(repeated N times) I get: "LoadError: UndefVarError: DataFrames not
defined"
- I also put "using DataFrames" into the ProjectModule.jl file. The
program definitely hated that. (Specifically: I was warned that I was
"overwriting DataFrames".)
Is there a better way to load both the DataFrames package and the functions
I have written across a couple of workers?
Thanks!
ABB
