[julia-users] Scheduling external programs in slurm with Julia and ClusterManagers

[William Patterson]

Hi, I am new to Julia and this is my first post in this group.

I am writing a script in Julia that schedules external programs with Slurm 
on a Rocks Cluster. When it came to actually scheduling the jobs I ran into 
difficulties getting external programs to run inside a function.

The code below is a snippet from my program that is causing issues 
independently:
using ClusterManagers

@everywhere function test_run_job()
    return readall(`echo test`)
end

function schedule_jobs()
    job_num = 10

    addprocs(SlurmManager(job_num))

    output = []
    pids = []
    for i in workers()
        out, pid = fetch(@spawnat i (test_run_job(), getpid()))
        push!(output, out)
        push!(pids, pid)
    end
    println(output)

    for i in workers()
        rmprocs(i)
    end
end

When I run the code above as-is i get the error below:
ERROR: LoadError: On worker 2: 10
function test_run_job not defined on process 2
 in error at ./error.jl:21
 in anonymous at serialize.jl:526
 in anonymous at multi.jl:1364
 in anonymous at multi.jl:910
 in run_work_thunk at multi.jl:651
 in run_work_thunk at multi.jl:660
 in anonymous at task.jl:58
 in remotecall_fetch at multi.jl:737
 in call_on_owner at multi.jl:783
 in fetch at multi.jl:801
 in schedule_jobs at the/path/to/my/script/myscript.jl:15
 in include at ./boot.jl:261
 in include_from_node1 at ./loading.jl:304
 in process_options at ./client.jl:280
 in _start at ./client.jl:378

Can someone please explain what I could be doing wrong?
Thank you!