hey Reuti, I wrote a python api using networkx and a database layer called clue so i've removed a few of the flags.
qsub -r y -l mem_free=1.9G,s_vmem=2G,hbatch=1 -pe smp 1 -N sshot3__out__rs_mantra1__ifd_gen -q *@@ifd_gen -P rnd -o /production/tmp/gridjobs/lars/fluffy/grid_feed1.rvb9CO -e /production/tmp/gridjobs/lars/fluffy/grid_feed1.rvb9CO -p 0 -v GRIDJOB,ATTEMPTS,STEPFRAME,BLOCKONERROR,GRIDSHOT,SLOTS -t 1-50:5 /tmp/gridTask_sshot3__out__rs_mantra1__ifd_gen.30851.0.sh qsub -r y -l mem_free=1.9G,s_vmem=2G -pe smp 4 -N sshot3__out__rs_mantra1__rs_mantra_tile1__mantra_render_split_1_seed_0 -q *@@atoms -P rnd -o /production/tmp/gridjobs/lars/fluffy/grid_feed1.rvb9CO -e /production/tmp/gridjobs/lars/fluffy/grid_feed1.rvb9CO -p 0 -hold_jid_ad 47673 -v SPLITTOTAL,SEEDTOTAL,GRIDJOB,ATTEMPTS,STEPFRAME,BLOCKONERROR,SEED,SPLIT,SEEDNUM,GRIDSHOT,SLOTS -t 1-50:1 /tmp/gridTask_sshot3__out__rs_mantra1__rs_mantra_tile1__mantra_render_split_1_seed_0.30771.0.sh qsub -r y -l mem_free=1.9G,s_vmem=2G -pe smp 4 -N sshot3__out__rs_mantra1__rs_mantra_tile1__mantra_render_split_3_seed_0 -q *@@atoms -P rnd -o /production/tmp/gridjobs/lars/fluffy/grid_feed1.rvb9CO -e /production/tmp/gridjobs/lars/fluffy/grid_feed1.rvb9CO -p 0 -hold_jid_ad 47673 -v SPLITTOTAL,SEEDTOTAL,GRIDJOB,ATTEMPTS,STEPFRAME,BLOCKONERROR,SEED,SPLIT,SEEDNUM,GRIDSHOT,SLOTS -t 1-50:1 /tmp/gridTask_sshot3__out__rs_mantra1__rs_mantra_tile1__mantra_render_split_3_seed_0.30753.0.sh qsub -r y -l mem_free=1.9G,s_vmem=2G -pe smp 4 -N sshot3__out__rs_mantra1__rs_mantra_tile1__mantra_render_split_2_seed_0 -q *@@atoms -P rnd -o /production/tmp/gridjobs/lars/fluffy/grid_feed1.rvb9CO -e /production/tmp/gridjobs/lars/fluffy/grid_feed1.rvb9CO -p 0 -hold_jid_ad 47673 -v SPLITTOTAL,SEEDTOTAL,GRIDJOB,ATTEMPTS,STEPFRAME,BLOCKONERROR,SEED,SPLIT,SEEDNUM,GRIDSHOT,SLOTS -t 1-50:1 /tmp/gridTask_sshot3__out__rs_mantra1__rs_mantra_tile1__mantra_render_split_2_seed_0.30657.0.sh qsub -r y -l mem_free=1.9G,s_vmem=2G -pe smp 4 -N sshot3__out__rs_mantra1__rs_mantra_tile1__mantra_render_split_0_seed_0 -q *@@atoms -P rnd -o /production/tmp/gridjobs/lars/fluffy/grid_feed1.rvb9CO -e /production/tmp/gridjobs/lars/fluffy/grid_feed1.rvb9CO -p 0 -hold_jid_ad 47673 -v SPLITTOTAL,SEEDTOTAL,GRIDJOB,ATTEMPTS,STEPFRAME,BLOCKONERROR,SEED,SPLIT,SEEDNUM,GRIDSHOT,SLOTS -t 1-50:1 /tmp/gridTask_sshot3__out__rs_mantra1__rs_mantra_tile1__mantra_render_split_0_seed_0.30680.0.sh qsub -r y -l mem_free=1.9G,s_vmem=2G -pe smp 1 -N sshot3__out__rs_mantra1__rs_mantra_tile1__joinexr -q *@@allhosts -P rnd -o /production/tmp/gridjobs/lars/fluffy/grid_feed1.rvb9CO -e /production/tmp/gridjobs/lars/fluffy/grid_feed1.rvb9CO -p 0 -hold_jid_ad 47677,47674,47675,47676 -v GRIDJOB,ATTEMPTS,STEPFRAME,BLOCKONERROR,GRIDSHOT,SLOTS -t 1-50:1 /tmp/gridTask_sshot3__out__rs_mantra1__rs_mantra_tile1__joinexr.30998.0.sh now the command runs fine. but the behaviour of the queue seems to be to want to finish a PE job before moving on to a next one where as what i'd want is to do the splits in a round robin fashion. Lars On 13 December 2011 15:48, Reuti <[email protected]> wrote: > Hi, > > Am 13.12.2011 um 16:25 schrieb Lars van der bijl: > >> we have been running our sge for a while now but we implemented a new >> technique and I'm having trouble figuring out how to make the grid >> help with it. >> >> I have the following task / dependency structure. >> >> task1 >> >> task2_seed_0 = dependent on output of task1 >> task2_seed_1 = dependent on output of task1 >> task2_seed_2 = dependent on output of task1 >> task2_seed_3 = dependent on output of task1 > > how did you set up the job dependency and submission in detail? > > -- Reuti > > >> joiner = dependent on output of seed tasks. >> >> all of these are Parallel environments with a range from 1-100 and a batch >> of 1 >> >> now what I would like to happen is that the Grid finishes doing the >> first batch of task 1. >> then all 4 of the task2 will get there first batch started. >> >> what's currently happening is that sge tries to schedule and finish >> all of task2_seed_0 before moving on to the task2_seed_1. >> >> anyone have a idea on how to get this behaviour? >> >> Lars >> _______________________________________________ >> users mailing list >> [email protected] >> https://gridengine.org/mailman/listinfo/users > _______________________________________________ users mailing list [email protected] https://gridengine.org/mailman/listinfo/users
