I know that a couple of other folks have already weighed in on this topic, but since I've done a fair a amount of work on the RM in the last year, I thought that I would throw in my two cents.
To use Chris's words, when using the "fresh-out-of-the-box" version of the RM, both of the concepts of Capacity and Load are entirely arbitrary. They have no relation to any kind of resources available on your node machines. Therefore, if you give each job a load of 1 (regardless of the node resources required to run the job) and if you give a node a capacity of 10, the RM will try to always have 10 jobs running on that node. It does nothing to track resource usage on the node, so use of such a paradigm as the one that I just described could be wildly inefficient. Because these numbers are arbitrary, I recommend carefully investigating the availability of resources on your nodes and setting load and capacity levels using that information. For example, if you find that your jobs tend to be I/O bound when you have more than 3 running simultaneously on the same node, then you could set your job load to 1 and the node capacity to 3. If you wanted more granularity, you could easily set the load to 33 and the capacity to 100. Since these numbers are entirely arbitrary, you have the freedom to make such changes. Obviously, not all jobs will be the same, so you may want to assign different loads to different jobs and assign different capacities to nodes based upon the resources that each makes available. Gabe =) ________________________________________ From: Wong, Cynthia L (388J) [[email protected]] Sent: Friday, April 13, 2012 12:47 PM To: [email protected] Subject: Capacity vs Load in Resource Manager What is "capacity" in nodes.xml? I also see "Load" as a piece of metadata for "Job". What does that mean? What is the different between them? How are they used by Resource Manager? Does Resource Manager monitor "Cores", CPU, disk space, and memory of a node? Thanks, Cynthia -- Cynthia L. Wong Data Management Systems and Technologies Jet Propulsion Laboratory 4800 Oak Grove Drive, M/S 171-264, Pasadena, CA 91109-8099 Phone: 818/393-2572, Email: [email protected]
