I've been looking at the issue where parallel jobs can be killed because
they have exceeded h_vmem due to a large number of qrsh processes started
by the master task.
https://arc.liv.ac.uk/trac/SGE/ticket/694
It turns out that you can opt-out of qrsh contributing to job resource
limits, at least on Linux, by making qrsh remove any secondary groups in
the configured gid_range (man ge_conf) for your cluster.
I have successfully tested this by writing a small wrapper program to
qrsh. The downside is that it relies on a SUID privileged call to the libc
function "setgroups".
Has anyone already done this (by modifying qrsh directly or using a
wrapper) on a cluster? How have you got on?
Other than the security aspects surrounding a SUID binary, I can see that
the big issue is the potential of running out of memory on the compute
node if there are a very large number of compute nodes in the job.
I could be persuaded to write a patch to make this a configurable option,
but would prefer first to have some discussion on what people on this list
think the correct behaviour would be...
Thanks,
Mark
--
-----------------------------------------------------------------
Mark Dixon Email : [email protected]
HPC/Grid Systems Support Tel (int): 35429
Information Systems Services Tel (ext): +44(0)113 343 5429
University of Leeds, LS2 9JT, UK
-----------------------------------------------------------------
_______________________________________________
users mailing list
[email protected]
https://gridengine.org/mailman/listinfo/users
