Am 31.12.2011 um 19:08 schrieb Dave Love: > Reuti <[email protected]> writes: > >> If you have this often, another way to have more flexibility could be >> a parallel file system, so that any node can access the scratch >> data. Then you could split the jobs without any special setup. > > [We've had reasonable success with ~100 nodes just with NFS over 10Gbe > to a Sun x4500. It works better than an old Panasas system we had.] > >> For now: the step A of the job could use a `qalter` to add a "-l >> h=$HOSTNAME" to step B of this flow of steps. (All exechosts need to be >> submission hosts this way though). Besides a jobnumber, `qalter` can also >> you a job name which could make it easier to target the following step when >> you name the jobs properly. > > It occurs to me that this might be a use for something like per-user > complexes <https://arc.liv.ac.uk/trac/SGE/ticket/1387>.
How could this help to route the job to a certain host? > I'd be > interested in suggestions for implementation of something like that or > information on what LSF etc. provide. Yes, a similar feature is available in GNU batch* and I mentioned it some time ago on the SGE dev list too. The complexes (i.e. variables [per user, group, global]) are persistent between the jobs, and jobs can be started of if the value of a variable reaches a certain value: $ gbch-r -c 'trigger<5' -f S -s trigger+=1 ./process.sh $ gbch-r -c 'trigger=5' -f N -s trigger=0 ./final.sh Each start of process.sh will increase the variable "trigger" by one (and starts only if it's smaller than 5). The final.sh will start when "trigger" has reached 5, and set it to zero again when it exits. It's also possible to decrease the variable at the end of the job, so to have a counter representing the number of active kinds of this job. I don't know what LSF is offering though. -- Reuti *) http://www.gnu.org/software/gnubatch/ _______________________________________________ users mailing list [email protected] https://gridengine.org/mailman/listinfo/users
