thanks Chris. lemme take advantage of your message to clarify some mailing list issues. if you have questions/feedback that concerns SystemConfigurator or SystemInstaller (or aren't really sure), please use the sisuite-users list. not all SIS developers are on systemimager-discuss. systemimager-only questions are also welcome on that list - though i think si-discuss has a larger user community at the moment.
personally, i'd like to see all the users of this list migrate over there (and maybe just make si-discuss an alias) - having multiple user lists is confusing, and ends up segregating our users. (leaving the TOFU for the sisuite-discuss list) On Wed, Mar 19, 2003 at 12:54:44PM -0500, Chris Dagdigian wrote: > > I am a long-time user of SystemImager 2.X who was forced into the > world of SI 3.0 when I started seeing bioinformatics cluster nodes > with gigabit ethernet cards and no cdrom drives. This config > effectively negates all the SI 2.x workarounds I had come up with :) > > The following really ugly workarounds are discussed: > > o methods of working around bootloader autoinstall issues > o turning on DMA for IDE disks to speed rsync during autoinstall > o combining Redhat kickstart w/ systemimager autoinstall > o A fix for GENTOO Linux autoinstalls > > > Apology: > This is going to be one of those messages from a complaining end-user > that lists problems and hacks but does not give the developers any > actual useful information. I was pressed for time over the past two > weeks and just had to make SI work by any means necessary. Once I can > replicate the problems in the home lab I hope to be able to submit > actual fixes. I should apologize for the length of this message as well > > Thanks: > Michael Athanas @ BioTeam found the hdparm performance > tweak. Andy Fant and Dann Fraizer had an IRC chat about devfs and bind > issues that ended up solving the Gentoo Linux autoinstall issues. > > > The core issue: > > Systemimager 3.0 is fantastic. Systemconfigurator however is less so. In > my recent experience it will reliably fail to successfully install a > bootloader on 4 out of 5 golden-client machine types during the > autoinstallation process. I have never seen it successfully autoinstall > a grub bootloader ever. > > The only time I can get SI to work flawlessly with systemconfigurator > doing its thing is when I install Redhat + lilo bootloader onto a Sun > LX50 rackmount. The LX50 is a dual Pentium III with LVD SCSI drives > requiring the aix7xxx module. Every other machine I've tried (all with > single simple IDE disks located at /dev/hda) will fail in the > bootloader/systemconfigurator process. > > Lessons learned: > > I've used Systemimager 3.x on three clusters in the past 2 weeks and > have painfully learned a few workarounds, performance tweaks and some > 'base conditions' that need to be true in my golden client OS configuration. > > The base OS conditions for getting SI autoinstallation to work in the > settings I've been in: > > (1) /boot needs to be right at the beginning of the disk, ideally in > /dev/hda1 or /dev/hda2 > > (2) LILO bootloader must be used if possible. Prepare for pain if you > are required to use GRUB (* may not be true) > > (3) Clear any references to partition labels from /etc/fstab and > /etc/lilo.conf if you are a Redhat user. This is the number one > cause of kernel panics after a node boots a freshly autoinstalled > disk. The /etc/fstab file LABLE= entries needs to be replaced with an > actual reference to the real disk and partition. In Redhat 8.0 I > noticed that redhat now changes /etc/lilo.conf to do things like > "append=root=label=/" instead of the tried-and true > "root=/dev/hda3". This cleanup of /etc/fstab and /etc/lilo.conf > needs to be done before you run 'prepareclient' and grab the > updated system image via 'getimage'. > > (4) Get used to the fact that you are going to have to hand-edit your > <clientImage>.master autoinstall scripts. 100% of the time now I > have found that I have needed to comment out the call to > systemconfigurator and replace it with shell commands that > manually chroot to /a/ and install a bootloader > > > After keeping those four things in mind I can now install a > golden-client node with a reasonable hope that I'll actually be able > to do a successful autoinstall from boot media or PXE. > > Ok moving on... > > I figured that it would be useful to document the workarounds and > performance tweaks that were found over the last two weeks when > working on multiple AMD-based cluster nodes running Redhat and Gentoo > linux. > > > (1) Massive rsync/imaging speed tweak for users with golden-clients > that have DMA-enabled IDE disk drives > > Discovery: The SI autoinstall media does not turn on DMA on your > golden-client's disk drive. This makes the rsync and imaging > process very very slow even over GigabitEthernet networks. > > By enabling DMA on our client disk drive we were able to improve > the speed of the systemimager autoinstall process by 5x - 10x. It > was a huge and very real performance gain. > > Redhat quick solution: Hack your autoinstall script to bring over > the 'hdparm' program via rsync before you start the giant rsync > process. Once you bring hdparm over to /bin you can insert the > following command into your autoinstall script: > > /bin/hdparm -d 1 /dev/hda > echo "Disk writes optimized!" > > Gentoo Linux solution: The hdparm program from Gentoo would not > run in the SI autoinstall environment due to glibc > differences. You can either compile it statically or just grab the > hdparm binary from a redhat install and stick it on the rsync > server somewhere (which is what we did) > > > > (2) The Cheezy Redhat Kickstart workaround for the time-impaired > > It's embarrassing to admit but my first workaround for not being > able to have SI correctly install a bootloader was to set up the > cluster such that I could use a Redhat Kickstart boot floppy to > perform a NFS-based autoinstall. The Redhat installer has no > problem creating the bootloader and you can craft the kickstart > config in a way that forces the MBR to be zeroed out and all the > partitions to be wiped clean. > > In the ks.cfg file there is a section at the end called "%post" > where one can run additional commands. I used this section to > script an install of the SI RPMs followed by a couple of > "updateclient -no-bootloader" commands. > > It was trivial to develop and looked something like this: > > %post > mkdir /mnt/tmp > mount -t nfs 10.0.0.1:/si-stuff/ /mnt/tmp > rpm -iv /mnt/tmp/*.rpm > updateclient -server 10.0.0.1 -image myImage -no-bootloader > unmount /mnt/tmp > > Although cheezy this method of using Redhat NFS-install plus a > kickstart boot floppy allowed me to accomplish the goal of being > able to boot a cluster node off a floppy disk and walk away while > the system self-installs and updates itself. > > I don't use this method any more now that I can generally install > the bootloader by hand in my .master scripts but I'd recommend > this approach to anyone who already has redhat media NFS mounted > and is frustrated by SI bootloader install issues. I can provide > the ks.cfg files for RH7.3 and RH8.0 if anyone needs to see examples. > > > (3) Current approach: Disable systemconfigurator and roll your own > bootloader install > > The way that I get SI autoinstalls to work now is to bypass > systemconfigurator now and just install my own bootloader in the > chrooted environment > > Note: I thought this approach only worked with Lilo bootloaders as > I was having trouble running /sbin/grub-install in this context > but Richard Cleary has just reported success with grub as well. > > Problem: The autoinstall script that is downloaded to the client > and run contains a call to systemconfigurator which is supposed to > handle the bootloader install and (apparently) some network stuff > as well. My experience has been that systemconfigurator just does > not work reliably and almost always will fail. > > Solution: Disable systemconfigurator and install bootloader by > hand. > > Example: > > This is what I commented out of my client-image.master script: > > ### BEGIN systemconfigurator ### > # Configure the client's hardware, network interface, and boot loader. > #chroot /a/ systemconfigurator --configsi --stdin <<EOL || shellout > # > #[NETWORK] > #HOSTNAME = $HOSTNAME > #DOMAINNAME = $DOMAINNAME > # > #[INTERFACE0] > #DEVICE = eth0 > #TYPE = dhcp > #EOL > #### END systemconfigurator ### > > Note: systemconfigurator is obviously trying to hostname and > network stuff as well. Since my cluster nodes always get IP and > hostname from a DHCP server I don't care about having > systemconfigurator set this for me during an autoinstall > process. Your mileage may vary. > > This is what I replaced it with: > > ## > echo "Bioteam: Installing lilo onto client system disk" > chroot /a/ /sbin/lilo -v || shellout > ## > > Richard Cleary has reported that this approach works with > Grub as follows: > > ## > chroot /a/ /sbin/grub-install || shellout > ## > > > > (4) GENTOO Linux specific workaround > > There is a bug database entry for SI getimage being unable to > handle devfs filesystems. > > Not sure if this was why /dev on my gentoo client install was > empty but it provided a good hint as to why the bootloader install > failed because "/dev/hda" was not found! > > The workaround was suggested by Dann via an IRC conversation with > Andy Fant at Tufts. The suggestion was to use "mount -o bind" to > create a workable /dev filesystem. > > I needed to edit the master autoinstall script as follows: > > Comment out this section: > > ### BEGIN mount proc in image for tools like System Configurator ### > #echo "mkdir -p /a/proc || shellout" > #mkdir -p /a/proc || shellout > #echo "mount proc /a/proc -t proc -o defaults || shellout" > #mount proc /a/proc -t proc -o defaults || shellout > #### END mount proc in image for tools like System Configurator ### > > Replace it with this: > > echo "Doing Gentoo specific bind stuff..." > mkdir -p /a/proc || shellout > mkdir -p /a/dev || shellou > mount -o bind /dev /a/dev || shellout > mount -o bind /proc /a/proc || shellout > echo "Done doing bind stuff..." > > > > > > > -- > Chris Dagdigian, <[EMAIL PROTECTED]> > BioTeam Inc. - Independent Bio-IT & Informatics consulting > Office: 617-666-6454, Mobile: 617-877-5498, Fax: 425-699-0193 > PGP KeyID: 83D4310E Yahoo IM: craffi Web: http://bioteam.net > > > > ------------------------------------------------------- > This SF.net email is sponsored by: Does your code think in ink? > You could win a Tablet PC. Get a free Tablet PC hat just for playing. > What are you waiting for? > http://ads.sourceforge.net/cgi-bin/redirect.pl?micr5043en > _______________________________________________ > SystemImager-discuss mailing list > [EMAIL PROTECTED] > https://lists.sourceforge.net/lists/listinfo/systemimager-discuss > -- [EMAIL PROTECTED] ------------------------------------------------------- This SF.net email is sponsored by: Does your code think in ink? You could win a Tablet PC. Get a free Tablet PC hat just for playing. What are you waiting for? http://ads.sourceforge.net/cgi-bin/redirect.pl?micr5043en _______________________________________________ Sisuite-users mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/sisuite-users
