Bogdan Costescu wrote:
On Sun, 28 Sep 2008, Jon Forrest wrote:
There are two philosophies on where a compute node's OS and basic
utilities should be located:
You forget a NFS-root setup, this doesn't require memory for the RAM
disk on which you later mount NFS dirs.
You're right. I should have mentioned it. It does fall into the
non-local install classification, although it has fewer of
the problems.
I prefer to look at the nodes as disposable, instead of "let's keep the
node up as long as possible", so I usually don't modify a running
system. Instead I modify the node "image" and reboot the nodes after the
current jobs finish - this is easy to do when using a queueing system
and is easy to hide from users when the typical jobs are longer than the
reboot time.
There are several ways to modify a running system, some more
dangerous than others. Probably the most dangerous is modifying
shared libraries and executables. Probably the least dangerous
is adding new files of any type. Most of the modifications I've
had to make involve editing text files, which hasn't caused any problems.
The trouble with rebooting nodes is that this takes human energy.
It's easier to keep nodes up as long possible, although this is not
a significant issue provided that the reboots are done as innocuously as
you describe.
However, on a modern multicore compute node this might just be a few
percent of the total RAM on the node.
This also depends on how much of the distribution you keep as part of
the node "image" and how you place the application software.
True. Some people, like the BusyBox people, have put a lot
of energy into coming up with "tiny versions of many common
UNIX utilities" in order to save memory in embedded systems.
Approach #2 requires much less time when a node is installed,
and a little less time when a node is booted.
I don't agree with you here as you probably have in mind a
kickstart-based install for approach #1 running upon each node boot.
True.
I use for a long time a different approach - the node "image" is copied
via rsync at boot time; the long waiting time for installing the RPMs
and running whatever configuration scripts happens only once when the
node "image" is prepared, the nodes only copy it - and it's a full copy
only the first time they boot with a new disk, afterwards it's the rsync
magic which makes it finish within seconds while making it look like a
new disk :-)
This is a good idea. Can you write more about this?
Cordially,
--
Jon Forrest
Research Computing Support
College of Chemistry
173 Tan Hall
University of California Berkeley
Berkeley, CA
94720-1460
510-643-1032
[EMAIL PROTECTED]
_______________________________________________
Beowulf mailing list, Beowulf@beowulf.org
To change your subscription (digest mode or unsubscribe) visit
http://www.beowulf.org/mailman/listinfo/beowulf
