-----Original Message-----
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] Behalf Of Andrew Morton
Sent: Friday, October 01, 2004 7:41 PM
To: Shailabh Nagar; [EMAIL PROTECTED]
Cc: [EMAIL PROTECTED]; [EMAIL PROTECTED]; [EMAIL PROTECTED];
[EMAIL PROTECTED]; [EMAIL PROTECTED]; [EMAIL PROTECTED];
[EMAIL PROTECTED]; [EMAIL PROTECTED]; [EMAIL PROTECTED];
[EMAIL PROTECTED]; [EMAIL PROTECTED]; [EMAIL PROTECTED];
[EMAIL PROTECTED]; [EMAIL PROTECTED]
Subject: [ckrm-tech] Re: [Lse-tech] [PATCH] cpusets - big numa cpu and
memory placement
Paul, I'm having second thoughts regarding a cpusets merge. Having gone
back and re-read the cpusets-vs-CKRM thread from mid-August, I am quite
unconvinced that we should proceed with two orthogonal resource
management/partitioning schemes.
And CKRM is much more general than the cpu/memsets code, and hence it
should be possible to realize your end-users requirements using an
appropriately modified CKRM, and a suitable controller.
I'd view the difficulty of implementing this as a test of the wisdom of
CKRM's design, actually.
The clearest statement of the end-user cpu and memory partitioning
requirement is this, from Paul:
Cpusets - Static Isolation:
The essential purpose of cpusets is to support isolating large,
long-running, multinode compute bound HPC (high performance
computing) applications or relatively independent service jobs,
on dedicated sets of processor and memory nodes.
The (unobtainable) ideal of cpusets is to provide perfect
isolation, for such jobs as:
1) Massive compute jobs that might run hours or days, on dozens
or hundreds of processors, consuming gigabytes or terabytes
of main memory. These jobs are often highly parallel, and
carefully sized and placed to obtain maximum performance
on NUMA hardware, where memory placement and bandwidth is
critical.
2) Independent services for which dedicated compute resources
have been purchased or allocated, in units of one or more
CPUs and Memory Nodes, such as a web server and a DBMS
sharing a large system, but staying out of each others way.
The essential new construct of cpusets is the set of dedicated
compute resources - some processors and memory. These sets have
names, permissions, an exclusion property, and can be subdivided
into subsets.
The cpuset file system models a hierarchy of 'virtual computers',
which hierarchy will be deeper on larger systems.
The average lifespan of a cpuset used for (1) above is probably
between hours and days, based on the job lifespan, though a couple
of system cpusets will remain in place as long as the system is
running. The cpusets in (2) above might have a longer lifespan;
you'd have to ask Simon Derr of Bull about that.
Now, even that is not a very good end-user requirement because it does
prejudge the way in which the requirement's solution should be
implemented.
Users don't require that their NUMA machines "model a hierarchy of
'virtual computers'". Users require that their NUMA machines implement
some particular behaviour for their work mix. What is that behaviour?
For example, I am unable to determine from the above whether the users
would be 90% satisfied with some close-enough ruleset which was
implemented
with even the existing CKRM cpu and memory governors.
So anyway, I want to reopen this discussion, and throw a huge spanner in
your works, sorry.
I would ask the CKRM team to tell us whether there has been any
progress in
this area, whether they feel that they have a good understanding
of the end
user requirement, and to sketch out a design with which CKRM could satisfy
that requirement.
Thanks.
-------------------------------------------------------
This SF.net email is sponsored by: IT Product Guide on ITManagersJournal
Use IT products in your business? Tell us what you think of them. Give us
Your Opinions, Get Free ThinkGeek Gift Certificates! Click to
find out more
http://productguide.itmanagersjournal.com/guidepromo.tmpl
_______________________________________________
ckrm-tech mailing list
https://lists.sourceforge.net/lists/listinfo/ckrm-tech
