Hi julians,
Parallel computing is not my focus, but I would like to run a julia script
in parallel in my PC in order to save time. The PC has an 8-core processor
so I would like to run at least 4 replicates of my script (test1.jl) at the
same time, each one with one different parameter.
So far I was working on a cluster and I was using qsub in order to submit
my jobs. I didn't need to configure anything. I had a quota of 20 jobs, so
if I launched 200 jobs 20 of them would run and the others would be in a
queue waiting for their moment to run.
Now I would like to run my scripts in my machine and I would like to do
something similar. For me it is fine if I wait for the current jobs to
finish running in order to launch another bunch of jobs. As well as it is
fine if I launch a new job for each experiment that finishes. The easiest
option the best :)
My script does not receive any file as input, but it writes the outputs to
unique files (the file names are unique for each job).
Do you recommend a good way to do it in Julia? Let's say my script is
called "test1.jl". So far I was trying the following code to call my jobs:
function main()
parameters = collect(0.001:0.001:0.6);
@parallel for(p in parameters)
command1 = `/home/cdesantana/Downloads/julia -p 4
/home/cdesantana/Experiments/test1.jl $p`;
run(command1);
end
end
main();
I really don't understand what is happening here, but I am sure it is not
working 4 jobs of my scripts... :(
cdesantana@c-de-santana:~/Data/Dendritics/poster2016$ ps aux|grep julia
cdesant+ 1964 3.7 1.3 9214696 107356 pts/10 Sl 15:20 0:01
/home/cdesantana/Downloads/julia/julia script.jl
cdesant+ 1994 92.2 2.7 9489844 214388 pts/10 Rl 15:20 0:31
/home/cdesantana/Downloads/julia/julia -p4
/home/cdesantana/Experiments/test1.jl 1 0.001
cdesant+ 2013 8.7 1.5 9189424 120340 ? Ssl 15:20 0:02
/home/cdesantana/Downloads/julia/usr/bin/julia -Cnative
-J/home/cdesantana/Downloads/julia/usr/lib/julia/sys.so --bind-to
192.168.89.174 --worker
cdesant+ 2016 9.2 1.6 9288520 131600 ? Ssl 15:20 0:03
/home/cdesantana/Downloads/julia/usr/bin/julia -Cnative
-J/home/cdesantana/Downloads/julia/usr/lib/julia/sys.so --bind-to
192.168.89.174 --worker
cdesant+ 2018 6.7 1.6 9353972 131772 ? Ssl 15:20 0:02
/home/cdesantana/Downloads/julia/usr/bin/julia -Cnative
-J/home/cdesantana/Downloads/julia/usr/lib/julia/sys.so --bind-to
192.168.89.174 --worker
cdesant+ 2023 7.7 1.6 9419496 131604 ? Ssl 15:20 0:02
/home/cdesantana/Downloads/julia/usr/bin/julia -Cnative
-J/home/cdesantana/Downloads/julia/usr/lib/julia/sys.so --bind-to
192.168.89.174 --worker
cdesant+ 2159 0.0 0.0 11716 892 pts/10 S+ 15:21 0:00 grep
--color=auto julia
I was expecting that it would be running my script 4 times, each one with
one different value of $p (0.001, 0.002, 0.003, 0.004).
Any suggestion? My machine has ubuntu installed so I am fine if I need to
combine Linux programs with Julia.
Many thanks in advance for any help!
Best,
Charles
--
Um axé! :)
--
Charles Novaes de Santana, PhD
https://github.com/cndesantana
