On Tue, Aug 26, 2014 at 2:15 PM, Douglas A. Augusto <daaugu...@gmail.com> wrote:
> Is there a way to inform GNU Parallel that a certain job consumes more than
> one slot so that the job is properly scheduled?
There is no such way per job, but there is per invocation of GNU
Parallel using the % in --jobs:
parallel --S server1,server2 --jobs 50% my_2_core_command ::: *
This will run one instance of my_2_core_command for every 2 cores in
server1 and server2.
I have never in practice seen or even heard of a situation where the
number of cores used depended on the arguments, so I don't think it
happens very often for GNU Parallel use cases.
It does, of course, happen often for cluster queue systems, but GNU
Parallel is not intended to become a full replacement for cluster
queue systems like PBS, Sun Grid Engine and Torque.
If many users start to request the option and can argue why a full
cluster queue systems is not the right way to go for them, I might be
persuaded to consider it.
> More specifically, I am looking for an option "--slots" like:
>
> parallel -j 3 --slots {3} "echo {1} sleeping processes for {2} seconds;
> mpiexec -n {1} sleep {2}" ::: 2 3 ::: 5 10 ::: 2 3
I assume you want to add an '--xapply' to that - otherwise it does not
make any sense to me.
If your goal is to not overload a server, look at --load: The idea
here being that --load 100% will only start another job if there is at
least 1 core sitting idle.
parallel -j 100% --load 100% --xapply "echo {1} sleeping processes
for {2} seconds; mpiexec -n {1} sleep {2}" ::: 2 3 ::: 5 10
(Crappy example because 'sleep' will make the core be sitting idle).
/Ole
