That does work. Thank you very much.
Is it possible that a difference is made by using addprocs() after I have
the REPL running vs. starting the program as "julia -p N" for some N? I
ask because I am (nearly) sure my module was defined the way you suggest
earlier today and it was giving me the error.
On Wednesday, June 1, 2016 at 8:02:26 PM UTC-5, Greg Plowman wrote:
>
> I find that putting everything required on workers into a module is the
> way to go.
> Then just use using Module (after adding workers)
>
> This works for me (v0.4.5):
>
> ProjectModule.jl:
> module ProjectModule
> using DataFrames
> include("function1.jl")
> export function1
> end
>
>
> function1.jl:
> function function1(input::DataFrame)
> # do something to input
> println("function1 here")
> end
>
> Now test:
> addprocs()
> using DataFrames
> using ProjectModule
> df = DataFrame(A = 1:4, B = ["M", "F", "F", "M"])
> pmap(function1, fill(df,10))
>
>
>
> On Thursday, June 2, 2016 at 8:24:52 AM UTC+10, ABB wrote:
>
>> Hello -
>>
>> I have the following problem: I would like to find a good way to import a
>> collection of user-defined functions across several workers. Some of my
>> functions are defined on DataFrames, but "using DataFrames" is not getting
>> me anywhere on the other workers.
>>
>> I think the problem I am running into may result from some combination of
>> the rules of scope and the command "using".
>>
>> Alternatively, given what I want to do, maybe running this with "julia -p
>> N" is not the best way to make use of N workers in the way I want to.
>>
>> I open several workers: "julia -p 3"
>>
>> "using DataFrames" - this brings DataFrames into the 'main' worker's
>> scope (I think), but not into the scope of subordinate workers.
>>
>> "using ProjectModule" - I am trying to load a module across all workers
>> which contains several functions I have written (maybe this is not the best
>> way to accomplish this task?)
>>
>> This error is returned:
>>
>> LoadError: LoadError: UndefVarError: DataFrame not defined
>>
>> ProjectModule looks something like
>>
>> module ProjectModule
>> include("function1.jl")
>>
>> export function1
>> end
>>
>> where function1 is defined as
>>
>> function1(input::DataFrame)
>> #do something to input
>> end
>>
>> I have tried a few things:
>>
>> - Running "@everywhere using DataFrames" from within the main worker
>> (this has worked once or twice - that is, I can then use function1 on a
>> different worker - but it isn't consistent)
>>
>> - Opening the workers at the outset using julia -p N -L
>> ProjectModule.jl... (repeated N times) I get: "LoadError: UndefVarError:
>> DataFrames not defined"
>>
>> - I also put "using DataFrames" into the ProjectModule.jl file. The
>> program definitely hated that. (Specifically: I was warned that I was
>> "overwriting DataFrames".)
>>
>> Is there a better way to load both the DataFrames package and the
>> functions I have written across a couple of workers?
>>
>> Thanks!
>>
>> ABB
>>
>
