Am 06.07.2011 um 23:00 schrieb Peskin, Eric: >> >>> What is the most reliable way to track core-hours, such that occupying 100 >>> cores for a day costs 100 times as much as occupying just 1 core for a day? >> >> Occupying or using? > > At the (very) fine grain (as in a context-switching time slice), these two > concepts are the same thing because a core can only run one process at a > time. But presumably that argument only applies to CPU (or UTIME or STIME) > and not to WALLCLOCK time. > > Even at the coarse grain, we have not seen SGE allocate more jobs to a node > than there are cores on that node. Is this where SGE's notion of "slots" > comes in? Is a "slot" the same as a core?
I would phrase it: it should be seen this way at least for serial jobs. Sometimes you have more slots defined per core as you have more than one queue instance per node and need additional measure to limit it again to match the core count. As long as you do this, you can also multiply wallclock time with the number of requested slots. The computed values might be different, if a parallel job is not running in parallel all the time, althought the user requested these additional slots. > At the moment, the cluster is swamped with jobs that have brought the load > on each node to the point that SGE considers all the queue instances to be in > an "alarm" state. What definition of "alarm" do you have? > Any further jobs are held in the "waiting" state (qw), and qlogin attempts > are simply rejected. You could define an additonal queue without alarm for `qlogin` in case you want to check the state of a job on a node. Small drawback: the exclusive feature can't be used then, as there is no way to keep one queue instance out of the exclusive constraint. > So certainly in a case like this, I would say that our resources are > occupied. The "alarm" feature by using one of the load values should be used with caution nowadays. IMO it was originally introduce to allow to oversubscribe a big SMP machine (former Sun Fire with Solaris as OS), with let's say 128 cores with maybe 192 slots as parallel jobs may not always run in parallel all the time. If the load got to high, the alarm would prohibit additonal jobs to start and in average you got a good usage of all cores without wasting some for "half-parallel" jobs. The problem with Linux is, that for some time now uninterruptible kernel tasks count as running. Hence a load somewhat higher than the number of installed cores can often be seen, although the "real" working processes match the number of installed cores (or maybe even lower in your case). So the "alarm" feature is of limited use. -- Reuti > ------------------------------------------------------------ > This email message, including any attachments, is for the sole use of the > intended recipient(s) and may contain information that is proprietary, > confidential, and exempt from disclosure under applicable law. Any > unauthorized review, use, disclosure, or distribution is prohibited. If you > have received this email in error please notify the sender by return email > and delete the original message. Please note, the recipient should check this > email and any attachments for the presence of viruses. The organization > accepts no liability for any damage caused by any virus transmitted by this > email. > ================================= > > _______________________________________________ users mailing list [email protected] https://gridengine.org/mailman/listinfo/users
