Hi Prentice,
The main point here is setting the attribute as CONSUMABLE and definning
in each execution host the attribute as the maximum memory you want to
be used in that node, generally RAM, but you might also consider virtual
memory.
This way, every time a user requests some amount of memory, that amount
is substracted from the "currently available" memory of the host. Then,
when one job requests more memory than what is there left, the job is
scheduled to another node.
As for the 3 options, h_vmem is a hard limit related with ulimit. If a
job exceeds that amount, the job is killed. mem_free is, as you said,
just a hint to account how many have been requested, but if a job
exceeds that, nothing happens. s_vmem, theoretically is a soft limit and
has some uses with jobs that catch the signals properly, but I have
never used it.
h_vmem is a better option if your main goal is avoid swapping, but it
leads, by design, to a somewhat "worse" usage of memory because users
will always request more memory than what they really need (if a job
uses generally 200Mb, but has one peak of 2Gb, you'll need to plan your
request for the 2Gb). On the other hand, mem_free is just a hint,
nothing enforces the memory usage. That means that users can request
100Mb for jobs using 40Gb and your nodes will swap as if there is no
tomorrow.
And as for the default values, it depends on which attribute you use. If
you use h_vmem, the default value will be the limit. If you set it to 0
or something too low, everything (not explicitly requesting memory) will
fail. If you set it too high, you will have jobs occupying lots of
memory that they aren't really using. If you use mem_free, as it is only
a hint, not much happens. Set it to totalmem/num_cores.
Personally, here we use h_vmem with a high default value (6Gb), but we
have trained all our users to use memory properly, and we have also
created some scripts for controlling the good and bad memory requesting.
Txema
El 25/05/12 20:34, Prentice Bisbal escribió:
Okay, this going to be a stupid question coming from someone who's been
on this list for years, but here goes...
I've just upgraded the RAM on a few cluster nodes to 32 GB (instead of
16 GB). A few users could benefit from this, so I'd like to be able to
specify h_vmem or mem_free or s_vmem for jobs so that jobs requiring RAM
16 GB go to this node. Digging through the google results, It looks
like I need to do the following:
1. Make whatever complex I choose to use requestable
2. Set a default value.
3. Profit.
It's step 2 that worries me. Must I set a default? My assumption is that
if I do, that default will be used for ALL jobs. Is that correct? I'm
worried because something tells me that no matter what default I use,
it's going to break something, somewhere for someone. Is it possible to
set default = NONE?
Second question, is which resource is the best to use for this? h_vmem,
s_vmem, or mem_free. I know mem_free is more of a "hint", and doesn't
guarantee anything, and h_vmem will kill a job submitter guesses wrong.
I've never seen anyone even mention s_vmem for this. Is there a reason
for that?
_______________________________________________
users mailing list
[email protected]
https://gridengine.org/mailman/listinfo/users
