Re: Compute Farm, Part II (nobody mentioned nfsroot!)
I'm suprised no-one has mentioned nfsroot to-date. Based on its docs, it could do this quite well. But tryin to understand the docs is hell, and getting it to work even worse, especially when I got no response to my probs f/the maintainer. -- Terrence Brannon * [EMAIL PROTECTED] * http://lnc.usc.edu/~brannon USC, HNB, 3614 Watt Way, Los Angeles, CA 90089-2520 * (213) 740-3397 Great Milk Debate... http://www.milk.com vs. http://www.notmilk.com -- TO UNSUBSCRIBE FROM THIS MAILING LIST: e-mail the word unsubscribe to [EMAIL PROTECTED] . Trouble? e-mail to [EMAIL PROTECTED] .
Re: Compute Farm
On Tue, 10 Feb 1998, Tim Sailer wrote: The powers-that-be around here have almost decided to scrap Slowaris x86 for the 200 machine PPro compouter farm, and go with Linux... I need to convince them to use Debian and not RH. They want to be able to configure 1 machine and mirror the setup to each machine in the farm, without having to go to each of the remaining 199 and configure it by hand. Someone told them that RH made this easy, and no other dist could do it! Other people have given some excellent advice already. I may be doing something like this in the near future, in a much smaller scale (20+2 node farm aiming at 4+ Gflop/s). Reading the documentation, bootp seems like a good option for network configuration detailts, but, as Craig Sanders pointed out, not a very dependable one. In a more fixed situation, it seems better to statically configure the machines using smart scripts... I don't know if this is what you are up to, but you may want to check out Beowulf's home page (shows up in Yahoo! rather easily). It's RH based, but I don't see anything there that's Red Hat specific. There are a couple of kernel patches and and bunch of rpm's. I recall there's PVM and MPP. Drake Diedrich has already packaged PVM and dqs. If I ever get the time, I'd like to try to make a Debian version of the requiered packages. Marcelo PS: What I'm still pondering is exactly how RH does this easy. Last time I poked at RH, it didn't had anything to automate this kind of task. PPS: Since neither RH nor Debian actually have anything built-in for this task, Debian is still a better choice: several times ppl in the developer's list have expressed interest in further developing Deity into this direction. I know this doesn't buy you anything right now, but it may be worth mentioning. -- TO UNSUBSCRIBE FROM THIS MAILING LIST: e-mail the word unsubscribe to [EMAIL PROTECTED] . Trouble? e-mail to [EMAIL PROTECTED] .
Re: Compute Farm
Hi all,
Except cfengine there is another hopefully better because debian specific
solution I discovered recently installing hamm. I mean
# dpkg --root=/another_machine_root_dir deb_file
Think about the great possibilities it gives! Let's assume
# mkdir /var/lib/dpkg/DebianFarm
# for i in $FarmMachineNames ; do {
mkdir /var/lib/dpkg/DebianFarm/$i;
mount -t nfs $i:/ /var/lib/dpkg/$i
}; done
Then we can at any time do:
# for i in $FarmMachineNames ; do
dpkg --install --root=/var/lib/dpkg/DebianFarm/$i deb_package;
done
Of course it doesn't eliminate cfengine because such scripts may serve
only for instalation purposes. But installing the full system takes at
least an hour counting from the first dselect start. In this particular
case it allows to save 200 hours.
BTW, it appears to be neccessary to have a possibility of using dselect
to choose packages and resolve all conflicts/dependencies and then export
package names IN PROPER ORDER (this is probably one of hamm's bugs) to a
file which could be an argument for the script proposed above. Is it
possible?
-- Jerzy Kakol
--
TO UNSUBSCRIBE FROM THIS MAILING LIST: e-mail the word unsubscribe to
[EMAIL PROTECTED] .
Trouble? e-mail to [EMAIL PROTECTED] .
Re: Compute Farm
Marcelo E. Magallón wrote: PPS: Since neither RH nor Debian actually have anything built-in for this task, Debian is still a better choice: several times ppl in the developer's list have expressed interest in further developing Deity into this direction. I know this doesn't buy you anything right now, but it may be worth mentioning. Plans have existed from the beginning to allow deity to work at administering several machine concurently. However, for complexity reasons, that part of the design was left on hold until a later revision of Deity. We are working towards a 1.0 version of deity. I wouldn't expect multi-machine capability until 3.0 or so. Behan -- Behan Webster mailto:[EMAIL PROTECTED] +1-613-224-7547 http://www.verisim.com/ -- TO UNSUBSCRIBE FROM THIS MAILING LIST: e-mail the word unsubscribe to [EMAIL PROTECTED] . Trouble? e-mail to [EMAIL PROTECTED] .
Re: Compute Farm, Part II
you wouldn'tr want to share /etc tho...because then that shares everything...which isn't always good (wouldn't want them all to have the same IP adress) tho you could mount a shared version of etc and have the shared configs be sym links to the shared mounted version but.. something along the idea of netbsd's mount_union could do this. mount the shared /etc behind the local file system, so that files not present locally are pulled off the server. rick -- These opinions will not be those of ISU until it pays my retainer. -- TO UNSUBSCRIBE FROM THIS MAILING LIST: e-mail the word unsubscribe to [EMAIL PROTECTED] . Trouble? e-mail to [EMAIL PROTECTED] .
Re: Compute Farm
On Wed, 11 Feb 1998, Gergely Madarasz wrote: On Wed, 11 Feb 1998, Craig Sanders wrote: Tim, from my experiences with (much smaller scale but still essentially the same) mass-duplication of debian, all you need to do is duplicate the hard disk and change half a dozen (or less) files under /etc: /etc/hostname /etc/init.d/network /etc/hosts no need for changing these files, bootp can handle them (there is an example rc.bootp which can be put in /etc/init.d/network) that is true. however, i would be inclined to do it using dd to dupe the disks and a sh/ed/sed/perl script to modify the config files purely because bootp introduces a single point of failure to the network - if the bootp server dies then the other machines can't boot until it's fixed. The script method is also a lot simpler and more flexible - you can auto-configure anything you like with a suitable smart script. bootp only allows you to change the network details. either way will work, and it is possible to have redundancy in the bootp setup (just build another bootp or dhcp server and configure the two to work together without conflicting). craig -- craig sanders -- TO UNSUBSCRIBE FROM THIS MAILING LIST: e-mail the word unsubscribe to [EMAIL PROTECTED] . Trouble? e-mail to [EMAIL PROTECTED] .
Re: Compute Farm, Part II
If they are all pretty similar configs, then it sounds like a very easy solution is to use rdist or rsync. On Wed, 11 Feb 1998, Tim Sailer wrote: I guess I didn't explain real well the first time, although I enjoyed the thread.. On this compute farm, they want to make changes to 1 machine, as in adding a package, changing a config file, etc and having the resultant changes reflected on the other 199 machines, without having to go to each machine and tweak it. Installing the machines will be a short, but intense process, but they are looking for long range admin solutions. Thanks, Tim -- (work) [EMAIL PROTECTED] / (home) [EMAIL PROTECTED] - http://www.buoy.com/~tps Nihil illegitemi carborvndvm. --anon ** Disclaimer: My views/comments/beliefs, as strange as they are, are my own.** -- TO UNSUBSCRIBE FROM THIS MAILING LIST: e-mail the word unsubscribe to [EMAIL PROTECTED] . Trouble? e-mail to [EMAIL PROTECTED] . -- TO UNSUBSCRIBE FROM THIS MAILING LIST: e-mail the word unsubscribe to [EMAIL PROTECTED] . Trouble? e-mail to [EMAIL PROTECTED] .
Re: Compute Farm, Part II
On Wed, 11 Feb 1998, Tim Sailer wrote: I guess I didn't explain real well the first time, although I enjoyed the thread.. On this compute farm, they want to make changes to 1 machine, as in adding a package, changing a config file, etc and having the resultant changes reflected on the other 199 machines, without having to go to each machine and tweak it. Installing the machines will be a short, but intense process, but they are looking for long range admin solutions. isn't this sort of thing precisely what cfengine is for? cfengine is available as a debian package. rdist/rsync/ssh can also be useful for remote admin too... BTW, what makes them think that RH can do this any easier than debian? looks like cfengine is orphaned at the momentit has been updated to libc6 already, though. /debian/dists/unstable/main/binary-i386$ dpkg -I admin/cfengine_1.3.19-2.deb new debian package, version 2.0. size 229684 bytes: control archive= 1078 bytes. 79 bytes, 3 lines * conffiles 818 bytes,21 lines control 665 bytes,21 lines * postinst #!/bin/sh 90 bytes, 7 lines * prerm#!/bin/sh Package: cfengine Version: 1.3.19-2 Section: admin Priority: optional Architecture: i386 Depends: libc6 Installed-Size: 428 Maintainer: (orphaned) Description: A tool for configuring and maintaining network machines The main purpose of cfengine is to allow the system administrator to create a single central file which will define how every host on a network should be configured. . cfengine is also useful as an interpreter for a general scripting language for ordinary users. It is handy for tidying up junk files and for maintaining `watchdog' scripts to manage access rights and permissions on files when collaborating with other users. . It takes a while to set up cfengine for a network (especially an already existing network), but once that is done you will wonder how you ever lived without it! craig -- Craig Sanders Systems Administrator VICNET- Victoria's Network http://www.vicnet.net.au Ph: +61 3 9669 9684Fax: +61 3 9669 9805 ** nothing is true, everything is permitted ** -- TO UNSUBSCRIBE FROM THIS MAILING LIST: e-mail the word unsubscribe to [EMAIL PROTECTED] . Trouble? e-mail to [EMAIL PROTECTED] .
Re: Compute Farm, Part II
Tim, IIRC Brian White has advocated cfengine for this task. HTH Lindsay =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-= Lindsay Allen [EMAIL PROTECTED] Perth, Western Australia voice +61 8 9316 248632.0125S 115.8445Evk6lj Debian Unix =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-= On Wed, 11 Feb 1998, Tim Sailer wrote: I guess I didn't explain real well the first time, although I enjoyed the thread.. On this compute farm, they want to make changes to 1 machine, as in adding a package, changing a config file, etc and having the resultant changes reflected on the other 199 machines, without having to go to each machine and tweak it. Installing the machines will be a short, but intense process, but they are looking for long range admin solutions. Thanks, Tim -- (work) [EMAIL PROTECTED] / (home) [EMAIL PROTECTED] - http://www.buoy.com/~tps Nihil illegitemi carborvndvm. --anon ** Disclaimer: My views/comments/beliefs, as strange as they are, are my own.** -- TO UNSUBSCRIBE FROM THIS MAILING LIST: e-mail the word unsubscribe to [EMAIL PROTECTED] . Trouble? e-mail to [EMAIL PROTECTED] . -- TO UNSUBSCRIBE FROM THIS MAILING LIST: e-mail the word unsubscribe to [EMAIL PROTECTED] . Trouble? e-mail to [EMAIL PROTECTED] .
Re: Compute Farm, Part II
Craig Sanders wrote: On Wed, 11 Feb 1998, Tim Sailer wrote: I guess I didn't explain real well the first time, although I enjoyed the thread.. On this compute farm, they want to make changes to 1 machine, as in adding a package, changing a config file, etc and having the resultant changes reflected on the other 199 machines, without having to go to each machine and tweak it. Installing the machines will be a short, but intense process, but they are looking for long range admin solutions. isn't this sort of thing precisely what cfengine is for? cfengine is available as a debian package. Doh! I knew there was something out there to do this. That's why I asked the list. Looks like my brain is full.. when something new goes in, something old leaks out.. :) rdist/rsync/ssh can also be useful for remote admin too... BTW, what makes them think that RH can do this any easier than debian? I have no idea. The guy who is spouting that nonsense is considered a 'loose canon', but he's very vocal, and attracted a lot of attention. [clip] Description: A tool for configuring and maintaining network machines The main purpose of cfengine is to allow the system administrator to create a single central file which will define how every host on a network should be configured. This is exactly what they need I think. Thanks to everyone for the pointers! Tim -- (work) [EMAIL PROTECTED] / (home) [EMAIL PROTECTED] - http://www.buoy.com/~tps They tell me my job is easy... anyone can do it. Why doesn't anyone else want it? -- me ** Disclaimer: My views/comments/beliefs, as strange as they are, are my own.** -- TO UNSUBSCRIBE FROM THIS MAILING LIST: e-mail the word unsubscribe to [EMAIL PROTECTED] . Trouble? e-mail to [EMAIL PROTECTED] .
Re: Compute Farm
But I would suggest reading the article anyway. If RedHat makes it easy
to install on many machines, the group in the article did NOT use such a
solution. Instead, they spent time writing some scripts and setting up
a generic hdd that they sent to their hardware vendor. The scripts were
designed to customize each machine as it arrived. I have never really
seen a solution to this problem (although I find it hard to imagine that
RedHat has solved it - it seems tough to me).
Paul
On 10-Feb-98 Hunter H Marshall wrote:
Behan Webster wrote:
Hunter H Marshall wrote:
Tim Sailer wrote:
They want to be able
to configure 1 machine and mirror the setup to each machine
See if the recent aricle in Linux Journal might be
of help. The article concerned the use of 160
Alpha Linux boxes for graphics rendering. I belive
it was the Jan '98 issue.
http://www.ssc.com/lj/
Since Tim wants reason to use Debian instead of Redhat, and the article
you suggest talks of using Redhat, that probably won't be very helpful.
I do not have the article in hand. I was afraid of
something like that!
:-{
hunter red-faced marshall
--
TO UNSUBSCRIBE FROM THIS MAILING LIST: e-mail the word unsubscribe to
[EMAIL PROTECTED] .
Trouble? e-mail to [EMAIL PROTECTED] .
--
Paul Rightley DX-3 Hydrodynamics, MS P940
Los Alamos National LaboratoryLos Alamos, NM 87545
Phone: (505)667-0460 Fax: (505)665-3359
--
--
TO UNSUBSCRIBE FROM THIS MAILING LIST: e-mail the word unsubscribe to
[EMAIL PROTECTED] .
Trouble? e-mail to [EMAIL PROTECTED] .
Re: Compute Farm, Part II
On Wed, 11 Feb 1998, Tim Sailer wrote: Craig Sanders wrote: isn't this sort of thing precisely what cfengine is for? cfengine is available as a debian package. Doh! I knew there was something out there to do this. That's why I asked the list. Looks like my brain is full.. when something new goes in, something old leaks out.. :) rdist/rsync/ssh can also be useful for remote admin too... BTW, what makes them think that RH can do this any easier than debian? I am not sure if this solves the problem of package installation, unless you actually force a mirror of all files, not just the configuration files in /etc. Carlo --- Carlo U. Segre Associate Professor of Physics Illinois Institute of Technology Chicago, IL 60616 Voice:(312) 567-3498 FAX: (312) 567-3494 http://www.iit.edu/~segre [EMAIL PROTECTED] -- TO UNSUBSCRIBE FROM THIS MAILING LIST: e-mail the word unsubscribe to [EMAIL PROTECTED] . Trouble? e-mail to [EMAIL PROTECTED] .
Re: Compute Farm, Part II
Lindsay Allen wrote: Tim, IIRC Brian White has advocated cfengine for this task. If I may, he advocated it because we used it here at Verisim to manage our workstations and servers. Unfortunately we found although it did work, it was rather clumsy, and didn't scale well in our environment. I have written a script (called cfile) which has all the functionality of what we had cfengine doing for us before. It needs a bit of a work (since we've been using it for 4 months we've noticed some places it could be improved). All in all though, it seems to do the trick of managing a small network of near identical machines (hardware wise). It can also specially configure certain machines to provide services (dns, web, ftp, etc). I am confident that it would scale much larger. Tim, if you are interested in seeing my developement code let me know. It's implementation will be taking a big change, but it's basic working will be the same. (i.e. the current version of cfile will give you an idea of it's capabilities and features). I plan on making a debian package of it once I am done my overhaul and make it even more generic (I want to use this same tool on my home network, here at Verisim, and on several other friend's networks). Let me know. Behan -- Behan Webster mailto:[EMAIL PROTECTED] +1-613-224-7547 http://www.verisim.com/ -- TO UNSUBSCRIBE FROM THIS MAILING LIST: e-mail the word unsubscribe to [EMAIL PROTECTED] . Trouble? e-mail to [EMAIL PROTECTED] .
Re: Compute Farm, Part II
Behan Webster wrote: Lindsay Allen wrote: Tim, IIRC Brian White has advocated cfengine for this task. If I may, he advocated it because we used it here at Verisim to manage our workstations and servers. Unfortunately we found although it did work, it was rather clumsy, and didn't scale well in our environment. I have written a script (called cfile) which has all the functionality of what we had cfengine doing for us before. It needs a bit of a work (since we've been using it for 4 months we've noticed some places it could be improved). All in all though, it seems to do the trick of managing a small network of near identical machines (hardware wise). It can also specially configure certain machines to provide services (dns, web, ftp, etc). I am confident that it would scale much larger. Tim, if you are interested in seeing my developement code let me know. It's implementation will be taking a big change, but it's basic working will be the same. (i.e. the current version of cfile will give you an idea of it's capabilities and features). I plan on making a debian package of it once I am done my overhaul and make it even more generic (I want to use this same tool on my home network, here at Verisim, and on several other friend's networks). I'll pass the info along, Behan. Thanks. At this point, I'm not officially involved, although this may end up being a new job here at the Lab for me. For some reason, '[EMAIL PROTECTED]' gets routed to my mailbox.. :) Tim -- (work) [EMAIL PROTECTED] / (home) [EMAIL PROTECTED] - http://www.buoy.com/~tps They tell me my job is easy... anyone can do it. Why doesn't anyone else want it? -- me ** Disclaimer: My views/comments/beliefs, as strange as they are, are my own.** -- TO UNSUBSCRIBE FROM THIS MAILING LIST: e-mail the word unsubscribe to [EMAIL PROTECTED] . Trouble? e-mail to [EMAIL PROTECTED] .
RE: Compute Farm, Part II
That shouldn't be hard Have you though tof using 1 or 2 computers as fileservers? After a quick read of teh FHS (filesystem standard) the setup is made to allow the shareing of large parts of the filesystem between computers. Then anytime you add a program to the system (usually in /usr/bin etc) it is added to all of them as it is you will probably want to share the home directories for users anyway /usr itself can not only be shared...but can be mounted READ ONLY to prevent its destructiuon (you would want it writeable maybe on one or two machines so that you can administer it) of course the main problem is if the fileserver computer fails... but there should be solutions for that too... I havn;'t done much myself with it... it would probably mean using NFS or some other system that allows shareing between computers A great setup that I remember from when I was upgrading the computers at a large department store (big rollout) was that they had 2 computers acting as the main servers... 1 just mirrored the main one..and if the main one went down..the Alternate took over ...and the rest of the store didn't even know the difference I don't know how hard that would be to setupbut it should be possible from what I know... RedHat's system is mainly to set 1 machine up and have an easy way to clone it (I believ ethey call it kickstart...you do the setup once...then just pop it in and walk away...it dows the install automatically) as for the Change 1 config file that could be done too... you wouldn'tr want to share /etc tho...because then that shares everything...which isn't always good (wouldn't want them all to have the same IP adress) tho you could mount a shared version of etc and have the shared configs be sym links to the shared mounted version but...im just pulling these ideas off the top of my head... bottom line is it can be done...its just a matter of how you want to do it and what your needs are exactly I mean...you don't really want every system running servers (or do you?) I dunno...just my $.02 On Wed, 11 Feb 1998, Tim Sailer wrote: I guess I didn't explain real well the first time, although I enjoyed the thread.. On this compute farm, they want to make changes to 1 machine, as in adding a package, changing a config file, etc and having the resultant changes reflected on the other 199 machines, without having to go to each machine and tweak it. Installing the machines will be a short, but intense process, but they are looking for long range admin solutions. Thanks, Tim -- TO UNSUBSCRIBE FROM THIS MAILING LIST: e-mail the word unsubscribe to [EMAIL PROTECTED] . Trouble? e-mail to [EMAIL PROTECTED] .
Re: Compute Farm
Tim Sailer wrote: They want to be able to configure 1 machine and mirror the setup to each machine See if the recent aricle in Linux Journal might be of help. The article concerned the use of 160 Alpha Linux boxes for graphics rendering. I belive it was the Jan '98 issue. http://www.ssc.com/lj/ hunter -- TO UNSUBSCRIBE FROM THIS MAILING LIST: e-mail the word unsubscribe to [EMAIL PROTECTED] . Trouble? e-mail to [EMAIL PROTECTED] .
Re: Compute Farm
Hunter H Marshall wrote: Tim Sailer wrote: They want to be able to configure 1 machine and mirror the setup to each machine See if the recent aricle in Linux Journal might be of help. The article concerned the use of 160 Alpha Linux boxes for graphics rendering. I belive it was the Jan '98 issue. http://www.ssc.com/lj/ Since Tim wants reason to use Debian instead of Redhat, and the article you suggest talks of using Redhat, that probably won't be very helpful. Behan -- Behan Webster mailto:[EMAIL PROTECTED] +1-613-224-7547 http://www.verisim.com/ -- TO UNSUBSCRIBE FROM THIS MAILING LIST: e-mail the word unsubscribe to [EMAIL PROTECTED] . Trouble? e-mail to [EMAIL PROTECTED] .
Re: Compute Farm
Behan Webster wrote:
Hunter H Marshall wrote:
Tim Sailer wrote:
They want to be able
to configure 1 machine and mirror the setup to each machine
See if the recent aricle in Linux Journal might be
of help. The article concerned the use of 160
Alpha Linux boxes for graphics rendering. I belive
it was the Jan '98 issue.
http://www.ssc.com/lj/
Since Tim wants reason to use Debian instead of Redhat, and the article
you suggest talks of using Redhat, that probably won't be very helpful.
I do not have the article in hand. I was afraid of
something like that!
:-{
hunter red-faced marshall
--
TO UNSUBSCRIBE FROM THIS MAILING LIST: e-mail the word unsubscribe to
[EMAIL PROTECTED] .
Trouble? e-mail to [EMAIL PROTECTED] .
RE: Compute Farm
Tim Sailer [EMAIL PROTECTED] wrote: They want to be able to configure 1 machine and mirror the setup to each machine in the farm, without having to go to each of the remaining 199 and configure it by hand. Someone told them that RH made this easy, and no other dist could do it! Well...there are many many ways to do itand it depends on what the setup is How homogeneous are the machines? for the most partif the hardware on em isn't terribly strange... it would be littl emore than an issue of mirroring 1 hard drive image over to all of the machines With a network setup that could be quite easy Hell I just recently was moving from one PC to another and I did a Drag and Dropjust took out the hard drive and pu tit in the new machine and boom...up came linux once the BIOS was happy (486/DX66 from a 486SX/25 system...differnt amount of RAM, diff vid card...) There is apcakage in debian that I have not played with but saw...which allows you to setup 1 computer andinstall the package,... then it makes a boot disk...goto any machine..pop in the boot disk... no install..instant workstation! and it is setup to do everything the server can do (by default) then come questions...is there going to be a single network filesystem? for the most part (as per the FHS) you should be able to share /usr beween all machines (assuming the same rchitecture of course) same for /home etc Suposedly redhat kickstart is suposed to be able to automate this... but with a tiny bit more work it should be very easily doable with debian hmmm just a slightly evil thought anyone tried cat /dev/hda1 /dev/hdb1 assuming hda1 and hdb1 are similar partition sizes and types -Steve -- TO UNSUBSCRIBE FROM THIS MAILING LIST: e-mail the word unsubscribe to [EMAIL PROTECTED] . Trouble? e-mail to [EMAIL PROTECTED] .
Re: Compute Farm
Stephen Carpenter wrote: Tim Sailer [EMAIL PROTECTED] wrote: They want to be able to configure 1 machine and mirror the setup to each machine in the farm, without having to go to each of the remaining 199 and configure it by hand. Someone told them that RH made this easy, and no other dist could do it! Well...there are many many ways to do itand it depends on what the setup is How homogeneous are the machines? They are going to be almost identical. for the most partif the hardware on em isn't terribly strange... it would be littl emore than an issue of mirroring 1 hard drive image over to all of the machines With a network setup that could be quite easy Hell I just recently was moving from one PC to another and I did a Drag and Dropjust took out the hard drive and pu tit in the new machine and boom...up came linux once the BIOS was happy (486/DX66 from a 486SX/25 system...differnt amount of RAM, diff vid card...) There is apcakage in debian that I have not played with but saw...which allows you to setup 1 computer andinstall the package,... then it makes a boot disk...goto any machine..pop in the boot disk... no install..instant workstation! and it is setup to do everything the server can do (by default) then come questions...is there going to be a single network filesystem? for the most part (as per the FHS) you should be able to share /usr beween all machines (assuming the same rchitecture of course) same for /home etc Suposedly redhat kickstart is suposed to be able to automate this... but with a tiny bit more work it should be very easily doable with debian All that would work if you wanted to take the machines offline to do that, but these machines are going to be handling 30MB/min of streaming data for 6 months at a clip. There's no way that would work. :( hmmm just a slightly evil thought anyone tried cat /dev/hda1 /dev/hdb1 assuming hda1 and hdb1 are similar partition sizes and types Tried it... doesnt work. Tim -- (work) [EMAIL PROTECTED] / (home) [EMAIL PROTECTED] - http://www.buoy.com/~tps Nihil illegitemi carborvndvm. --anon ** Disclaimer: My views/comments/beliefs, as strange as they are, are my own.** -- TO UNSUBSCRIBE FROM THIS MAILING LIST: e-mail the word unsubscribe to [EMAIL PROTECTED] . Trouble? e-mail to [EMAIL PROTECTED] .
Re: Compute Farm
Hunter H Marshall [EMAIL PROTECTED] writes: Tim Sailer wrote: They want to be able to configure 1 machine and mirror the setup to each machine See if the recent aricle in Linux Journal might be of help. The article concerned the use of 160 Alpha Linux boxes for graphics rendering. I belive it was the Jan '98 issue. http://www.ssc.com/lj/ Look at http://www.ssc.com/lj/issue46/2494.html They provided a master disk to the manufacturer who copied it to all of the other machines. A script changed all the needed values (IP, Hostname, etc.). With debian you can use, dpkg --get-selections, dpkg --set-selections and installation via ftp. A problem is, that you can`t leave the installation process unattended, because you have to answer some questions e.g. about precedence of mime applications or setup for x11. AFIAK the new installation program will address this issue (questions asked first, then installation, automatic setup for farms --- it`s expected to come with Debian 2.1). At the moment, copying the disks seems to be the best solution. HTH, Jens --- [EMAIL PROTECTED] Key from: http://www.weh.rwth-aachen.de/~jens/public.asc Key ID: 2048/E451C639 Jens Ritter Key fingerprint: 5F 3D 43 1E 24 1E CC 48 1E 05 93 3A A7 10 73 37 -- TO UNSUBSCRIBE FROM THIS MAILING LIST: e-mail the word unsubscribe to [EMAIL PROTECTED] . Trouble? e-mail to [EMAIL PROTECTED] .
Re: Compute Farm
All that would work if you wanted to take the machines offline to do that, but these machines are going to be handling 30MB/min of streaming data for 6 months at a clip. There's no way that would work. :( hmmm just a slightly evil thought anyone tried cat /dev/hda1 /dev/hdb1 assuming hda1 and hdb1 are similar partition sizes and types Tried it... doesnt work. you can't use cat, but if the drives are identical you can use dd (We use that to clone identical copies of drives here). As for package maintenance, once you have one machine set up the way you want, you can get the package list using dpkg --get-selections, save that to a file. Then reverse it (ie: dpkg --set-selections filename) on the next machine after updating the package list. Then you you don't need to use dselect or do much work on the other machines and maintenance becomes pretty painless. Come these things into an install/update script and then you can pretty much automate everything except for answering all the questions that come up during an install (in my case, I just keep tabs of these config files on the master machine and rdist them across afterwards). This is how I keep our 50 or so linux machines in sync... If you are using hamm the dpkg-mountable method is pretty nice since it only loads packages that are needed as opposed to recursing down all the directories. -- TO UNSUBSCRIBE FROM THIS MAILING LIST: e-mail the word unsubscribe to [EMAIL PROTECTED] . Trouble? e-mail to [EMAIL PROTECTED] .
Re: Compute Farm
On Tue, 10 Feb 1998, Tim Sailer wrote: Stephen Carpenter wrote: hmmm just a slightly evil thought anyone tried cat /dev/hda1 /dev/hdb1 assuming hda1 and hdb1 are similar partition sizes and types Tried it... doesnt work. The correct answer is dd if=/dev/hda1 of=hdb1 bs=1024. This doesn't care what is one the drive, even win 95 (known by experience), and it will act like a perfect mirror. The proper thing to do is to nfs export everything on one system, make a boot disk that can mount the nfs drive, partition/mount the new drive, and edit any specific info (hostname and ip) all by a simple boot script. It's way above my head, but when set up right, would be almost as easy to do 100 as it is to do 5. I think this is what the titanic team did (in the lj article). I don't think they used redhat's easy method. Brandon - Brandon Mitchell [EMAIL PROTECTED] We all know linux is great... it PGP: finger -l [EMAIL PROTECTED] does infinite loops in 5 seconds Phone: (757) 221-4847 --Linus Torvalds -- TO UNSUBSCRIBE FROM THIS MAILING LIST: e-mail the word unsubscribe to [EMAIL PROTECTED] . Trouble? e-mail to [EMAIL PROTECTED] .
Re: Compute Farm
On Tue, 10 Feb 1998, Tim Sailer wrote: Stephen Carpenter wrote: Tim Sailer [EMAIL PROTECTED] wrote: They want to be able to configure 1 machine and mirror the setup to each . . . hmmm just a slightly evil thought anyone tried cat /dev/hda1 /dev/hdb1 assuming hda1 and hdb1 are similar partition sizes and types Tried it... doesnt work. A more evil one, assuming /mnt is setup to mount /dev/hdb .. (umount /mnt;cd /;tar cvzf - ./)|(mount /mnt; cd /mnt;tar xvzf -) ?? J. -- TO UNSUBSCRIBE FROM THIS MAILING LIST: e-mail the word unsubscribe to [EMAIL PROTECTED] . Trouble? e-mail to [EMAIL PROTECTED] .
Re: Compute Farm
On Tue, 10 Feb 1998 [EMAIL PROTECTED] wrote: On Tue, 10 Feb 1998, Tim Sailer wrote: Stephen Carpenter wrote: hmmm just a slightly evil thought anyone tried cat /dev/hda1 /dev/hdb1 assuming hda1 and hdb1 are similar partition sizes and types Tried it... doesnt work. The correct answer is dd if=/dev/hda1 of=hdb1 bs=1024. This doesn't care what is one the drive, even win 95 (known by experience), and it will act like a perfect mirror. The proper thing to do is to nfs export everything on one system, make a boot disk that can mount the nfs drive, partition/mount the new drive, and edit any specific info (hostname and ip) all by a simple boot script. It's way above my head, but when set up right, would be almost as easy to do 100 as it is to do 5. I think this is what the titanic team did (in the lj article). I don't think they used redhat's easy method. I used this (well, similar) method to install 58 machines in the computer lab. They're all same, with 2.1G hard drives. I installed both linux and windoze95 (well, it wasnt me, but a collegue ;)) on one machine (about 50%-50% for linux and w95). I made a boot disk, which sets up the network with bootp, booted the installed machine off it, nfs mounted the server, and then gzipped the whole disk (/dev/hda) with dd if=/dev/hda | gzip -c /mnt/diskimage.gz. Then I booted another machine with the bootdisk, mounted the server and did a gunzip -c /mnt/diskimage | dd of=/dev/hda. I run 4-10 of these simultaneously (had to start them at the same time, so they dont slow down because of reading different parts of diskimage.gz). I didn't need to edit any bootup scripts, everything is setup thru bootp. It all went perfectly, no need for partitioning either, /dev/hda contains that info too. Windows95 had some problems though, it cannot get its name from the bootp server, and it had to install another network card driver because its pnp serial number changed. With linux this was no problem, I used a pnpdump/sed script on /etc/isapnp.conf before running isapnp. So this installation went perfectly, the only problem is that it is slow. 58*480Megs (this was the diskimage.gz size) thru a 10Mb network ... well, it took around 10 hours. Since then I thought of a better, faster way to do this, I've already written the programs for it, just cant test it yet. So the idea is: run a program on the already installed machine, which opens /dev/hda and then udp broadcasts it to the network. Each client catches the broadcasted packets, which has some information about block number, block size and a crc, checks it, writes it to the disk, and marks the block received. When the server finishes, it sends a block with length 0, then the clients can send querys to the server for the missed blocks. I'll try these programs in a few weeks... I expect it to finish the installation of the whole lab in less than an hour. Greg -- Madarasz Gergely [EMAIL PROTECTED] [EMAIL PROTECTED] It's practically impossible to look at a penguin and feel angry. Egy pingvinre gyakorlatilag lehetetlen haragosan nezni. HuLUG: http://www.cab.u-szeged.hu/local/linux/ -- TO UNSUBSCRIBE FROM THIS MAILING LIST: e-mail the word unsubscribe to [EMAIL PROTECTED] . Trouble? e-mail to [EMAIL PROTECTED] .
Re: Compute Farm
On Tue, 10 Feb 1998, Tim Sailer wrote: The powers-that-be around here have almost decided to scrap Slowaris x86 for the 200 machine PPro compouter farm, and go with Linux... I need to convince them to use Debian and not RH. They want to be able to configure 1 machine and mirror the setup to each machine in the farm, without having to go to each of the remaining 199 and configure it by hand. Someone told them that RH made this easy, and no other dist could do it! Someone help me out here! This will be running the RHIC project in 1999 here.. I'd love to see Debian get that kind of exposure! Tim, from my experiences with (much smaller scale but still essentially the same) mass-duplication of debian, all you need to do is duplicate the hard disk and change half a dozen (or less) files under /etc: /etc/hostname /etc/init.d/network /etc/hosts and maybe some others, depending on your particular needs and your particular hardware setup. e.g. if the hardware is not identical (in particular, same network card) and identically configured then you will also need to change /etc/modules. you probably don't need to change /etc/mailname or the mail configuration - just make them all masquerade as the domain name if they need to send. BTW, it will be much easier to do if all the machines have identical hard disks. then you can just install debian on one disk and use dd to duplicate the disk, and then run a script to 'individualise' the hostname and ip address details. FWIW, i auto-build debian based dialin servers (to be installed at schools for staff and students) by installing the latest stable release, and then run a fairly simple script which asks for the host domain name, and ip address and then uses ed to modify a handful of config filesthe ones listed above plus stuff in /etc/ppp it really is easy to do. if you need any help at all i am more than willing to lend a hand... I find consistency in design to be one of the nicest features of debian: I can build a standard machine which can be made to perform any function (workstation, file server, ppp dialin server, mail server, squid proxy, etc) just by installing the relevant packages. it gives me an extraordinarily high level of consistency across the machines i have to look after...they only differences are just the network config, and the config files for one or two packages. best of all, if a machine happens to die and i have kept a backup copy of it's unique config files i can rebuild it very quickly. squid is a good example - twice i've had the hard drives on a squid box fail under the heavy load. rebuilding the box was very easy (it takes me about half a day to build a debian box including unpacking all the hardware from the packing cases). While i was building the replacement box, I used linux's IP aliasing to make another machine on the network pretend to be the proxy server (but with only a tiny 100mb cache spool) so that customers weren't inconvenienced by the downtime. As far as I am concerned, the cheaper price of commodity hardware is far more valuable than any X-hour support contract offered by a commercial *nix vendor. In the X hours i have to wait for their engineer to come out, i can completely rebuild the machine from spare parts that i have lying around. Spare parts that I can afford because they are a third (or less) of the price of the high end gear. It is undeniable that, for example, Sun harware is more reliable than PC clone hardwarebut it isn't reliable enough to be worth 3 to 5 times the price. we've got some $30,000 (Australian $) sun boxes at work which were bought a few months before i started thereif i had been around at the time, we almost certainly would have bought $5,000 to $10,000 PC boxes plus a few thousand dollars worth of spare parts instead. power supplies, hard disks, memory, motherboards, cpus...enough to build a whole machine or two. including the cost of the support contract, this would have saved us over $20,000 per machine. we have 3 of these machines. over $60,000 could have been saved. OTOH, it's nice to have some ultra sparcs to play withi may even be able to rebuild one as a debian sparc box by re-arranging server loads more efficiently :-) craig -- craig sanders -- TO UNSUBSCRIBE FROM THIS MAILING LIST: e-mail the word unsubscribe to [EMAIL PROTECTED] . Trouble? e-mail to [EMAIL PROTECTED] .
Re: Compute Farm
On Wed, 11 Feb 1998, Craig Sanders wrote: Tim, from my experiences with (much smaller scale but still essentially the same) mass-duplication of debian, all you need to do is duplicate the hard disk and change half a dozen (or less) files under /etc: /etc/hostname /etc/init.d/network /etc/hosts no need for changing these files, bootp can handle them (there is an example rc.bootp which can be put in /etc/init.d/network) Greg -- Madarasz Gergely [EMAIL PROTECTED] [EMAIL PROTECTED] It's practically impossible to look at a penguin and feel angry. Egy pingvinre gyakorlatilag lehetetlen haragosan nezni. HuLUG: http://www.cab.u-szeged.hu/local/linux/ -- TO UNSUBSCRIBE FROM THIS MAILING LIST: e-mail the word unsubscribe to [EMAIL PROTECTED] . Trouble? e-mail to [EMAIL PROTECTED] .
