Hi all, this patch is not like the usual bug fix or minor improvements. I'm not considering it a major architectural change, but surely it does change something in the SystemImager management.
For this I'd like to receive some feedbacks (also a simple "I like it" or "I don't like it, because <...>" is enough). See the patch below. If there's no objection I'll merge it in the next unstable release 3.9.4. -Andrea PS if you want to test this stuff, the pre-release debs and RPMs (of 3.9.3.svn4159 or greater) will be available soon from here: http://download.systemimager.org/~arighi/systemimager/ -------- Original Message -------- Subject: [systemimager-commits] r4159 - in trunk: . doc/man etc initrd_source/skel/etc/init.d lib/SystemImager sbin var/lib/systemimager/scripts/post-install var/lib/systemimager/scripts/pre-install Date: Mon, 13 Aug 2007 05:22:16 -0500 From: [EMAIL PROTECTED] Reply-To: [email protected] To: [EMAIL PROTECTED] Author: arighi Date: 2007-08-13 05:22:15 -0500 (Mon, 13 Aug 2007) New Revision: 4159 Added: trunk/sbin/si_clusterconfig Modified: trunk/Makefile trunk/doc/man/Makefile trunk/etc/cluster.xml trunk/initrd_source/skel/etc/init.d/functions trunk/lib/SystemImager/Server.pm trunk/sbin/si_addclients trunk/systemimager.spec trunk/var/lib/systemimager/scripts/post-install/README trunk/var/lib/systemimager/scripts/pre-install/README Log: - implemented host group concept for autoinstall script and pre/post install scripts This is a complete restyling of how autoinstall and pre/post install scripts are chosen by the imaging clients. The new command si_clusterconfig has been introduced to edit the configuration of the groups of your clients (/etc/systemimager/cluster.xml). The command also provides to create a cache of the XML configuration in a plain text file (/var/lib/systemimager/scripts/cluster.txt), easily parsable from the minimal busybox environment of the imaging clients. Now the autoinstall script is chosen using the following algorithm: - if SCRIPTNAME is explicitly specified by installation parameter, use that script - otherwise chose one of the following script (first hit wins = first file present in $SCRIPTS_DIR wins): - HOSTNAME.sh | HOSTNAME.master - GROUPNAME.sh | GROUPNAME.master - BASE_HOSTNAME.sh | BASE_HOSTNAME.master - IMAGENAME.sh | IMAGENAME.master - pre/post install scripts are executed in the following order: - [0-9][0-9]all.* - [0-9][0-9]IMAGENAME.* - [0-9][0-9]BASE_HOSTNAME.* - [0-9][0-9]GROUPNAME.* - [0-9][0-9]HOSTNAME.* This approach allows to define a hierarchical set of scripts that are executed on the imaging client from the most global to the most local. In this way a more local script (i.e. for a single host) can have override the effects of one or more global scripts (i.e. group -> all and host -> group). This approach preserves in part the backward compatibilty: everything defined in prevsious versions is still executed in the same target of clients, but the order of execution could change. HINT: double check your scripts and take a look at the READMEs. Modified: trunk/Makefile =================================================================== --- trunk/Makefile 2007-08-13 07:40:26 UTC (rev 4158) +++ trunk/Makefile 2007-08-13 10:22:15 UTC (rev 4159) @@ -167,7 +167,7 @@ KBOOT_CONF_DEST = $(ETC)/systemimager/kboot.cfg BINARIES := si_mkautoinstallcd si_mkautoinstalldiskette si_mkautoinstalldisk si_psh si_pcp si_pushoverrides -SBINARIES := si_addclients si_cpimage si_getimage si_mkdhcpserver si_mkdhcpstatic si_mkautoinstallscript si_mkbootserver si_mvimage si_pushupdate si_pushinstall si_rmimage si_mkrsyncd_conf si_mkclientnetboot si_netbootmond si_mkbootpackage si_monitor si_monitortk si_installbtimage +SBINARIES := si_addclients si_cpimage si_getimage si_mkdhcpserver si_mkdhcpstatic si_mkautoinstallscript si_mkbootserver si_mvimage si_pushupdate si_pushinstall si_rmimage si_mkrsyncd_conf si_mkclientnetboot si_netbootmond si_mkbootpackage si_monitor si_monitortk si_installbtimage si_clusterconfig CLIENT_SBINARIES := si_updateclient si_prepareclient COMMON_BINARIES = si_lsimage Modified: trunk/doc/man/Makefile =================================================================== --- trunk/doc/man/Makefile 2007-08-13 07:40:26 UTC (rev 4158) +++ trunk/doc/man/Makefile 2007-08-13 10:22:15 UTC (rev 4159) @@ -42,7 +42,8 @@ si_pushinstall.8 \ si_pushoverrides.8 \ si_mkautoinstalldisk.8 \ - si_netbootmond.8 + si_netbootmond.8 \ + si_clusterconfig.8 CLIENT_MAN = si_prepareclient.8 si_updateclient.8 @@ -88,6 +89,9 @@ si_netbootmond.8: $(TOPDIR)/sbin/si_netbootmond pod2man --release=$(SYSTEMIMAGER_VERSION) -s 8 -c "SystemImager" $< $@ +si_clusterconfig.8: $(TOPDIR)/sbin/si_clusterconfig + pod2man --release=$(SYSTEMIMAGER_VERSION) -s 8 -c "SystemImager" $< $@ + %.5 %.8: %.sgml %.content.sgml docbook2man $< mv $(shell echo $@ | tr '[:lower:]' '[:upper:]') $@ Modified: trunk/etc/cluster.xml =================================================================== --- trunk/etc/cluster.xml 2007-08-13 07:40:26 UTC (rev 4158) +++ trunk/etc/cluster.xml 2007-08-13 10:22:15 UTC (rev 4159) @@ -1,5 +1,12 @@ <?xml version='1.0' standalone='yes'?> <!-- + +********************************* WARNING ************************************** + +This file has been generated by si_clusterconfig(8), do not edit manually! + +********************************* WARNING ************************************** + This is the main configuration file to describe the topology of your clients and your image server informations. @@ -11,6 +18,8 @@ to the clients) - si_pushoverrides (SystemImager program to transfer and keep synchronized files to the clients via overrides) + + See comments below for more details. --> <xml> <!-- The image server hostname. --> Modified: trunk/initrd_source/skel/etc/init.d/functions =================================================================== --- trunk/initrd_source/skel/etc/init.d/functions 2007-08-13 07:40:26 UTC (rev 4158) +++ trunk/initrd_source/skel/etc/init.d/functions 2007-08-13 10:22:15 UTC (rev 4159) @@ -171,6 +171,8 @@ echo "BITTORRENT_POLLING_TIME=$BITTORRENT_POLLING_TIME" >> /tmp/variables.txt echo "BITTORRENT_SEED_WAIT=$BITTORRENT_SEED_WAIT" >> /tmp/variables.txt echo "BITTORRENT_UPLOAD_MIN=$BITTORRENT_UPLOAD_MIN" >> /tmp/variables.txt + + echo "GROUPNAMES=\"$GROUPNAMES\"" >> /tmp/variables.txt fi } # @@ -1248,6 +1250,13 @@ # ################################################################################ # +get_group_name() { + [ -f ${SCRIPTS_DIR}/cluster.txt ] && \ + GROUPNAMES=`grep ${HOSTNAME} ${SCRIPTS_DIR}/cluster.txt | cut -d: -f2 | tr "\n" ' '` +} +# +################################################################################ +# choose_autoinstall_script() { logmsg @@ -1260,6 +1269,10 @@ # get_base_hostname + # Get group name (defined in /etc/systemimager/cluster.xml on the image + # server). -AR- + get_group_name + # # If SCRIPTNAME is specified as a kernel append, or via local.cfg, then use that script. # @@ -1273,15 +1286,21 @@ # # If SCRIPTNAME was not specified, choose one, in order of preference. First hit wins. # Order of preference is: - # IMAGENAME oscar_image.master (if this is specified, then the user _really_ wants it) - # HOSTNAME.sh oscar_node187.sh (then most specific -- the node) - # BASE_HOSTNAME.master oscar_node.sh (to least specific -- the class) + # HOSTNAME (i.e. node001.sh) + # GROUPNAMES (i.e. Login.sh) - see /etc/systemimager/cluster.xml on the image server + # BASE_HOSTNAME (i.e. node.sh) + # IMAGENAME (i.e. ubuntu7_04.sh) # - if [ ! -z $IMAGENAME ]; then - SCRIPTNAMES="${SCRIPTS_DIR}/${IMAGENAME}.master ${SCRIPTS_DIR}/${HOSTNAME}.sh ${SCRIPTS_DIR}/${BASE_HOSTNAME}.master" - else - SCRIPTNAMES="${SCRIPTS_DIR}/${HOSTNAME}.sh ${SCRIPTS_DIR}/${BASE_HOSTNAME}.master" - fi + [ ! -z $HOSTNAME ] && \ + SCRIPTNAMES="${SCRIPTNAMES} ${SCRIPTS_DIR}/${HOSTNAME}.sh ${SCRIPTS_DIR}/${HOSTNAME}.master" + for GROUPNAME in $GROUPNAMES; do + SCRIPTNAMES="${SCRIPTNAMES} ${SCRIPTS_DIR}/${GROUPNAME}.sh ${SCRIPTS_DIR}/${GROUPNAME}.master" + done + unset GROUPNAME + [ ! -z $BASE_HOSTNAME ] && \ + SCRIPTNAMES="${SCRIPTNAMES} ${SCRIPTS_DIR}/${BASE_HOSTNAME}.sh ${SCRIPTS_DIR}/${BASE_HOSTNAME}.master" + [ ! -z $IMAGENAME ] && \ + SCRIPTNAMES="${SCRIPTNAMES} ${SCRIPTS_DIR}/${IMAGENAME}.sh ${SCRIPTS_DIR}/${IMAGENAME}.master" fi # @@ -1346,14 +1365,22 @@ get_base_hostname + # Get group name (defined in /etc/systemimager/cluster.xml on the image + # server). -AR- + get_group_name + if [ -e "${SCRIPTS_DIR}/pre-install/" ]; then cd ${SCRIPTS_DIR}/pre-install/ + PRE_INSTALL_SCRIPTS="$PRE_INSTALL_SCRIPTS `ls | grep "^[0-9][0-9]all\..*"`" PRE_INSTALL_SCRIPTS="$PRE_INSTALL_SCRIPTS `ls | grep "^[0-9][0-9]${IMAGENAME}\..*"`" PRE_INSTALL_SCRIPTS="$PRE_INSTALL_SCRIPTS `ls | grep "^[0-9][0-9]${BASE_HOSTNAME}\..*"`" + for GROUPNAME in ${GROUPNAMES}; do + PRE_INSTALL_SCRIPTS="$PRE_INSTALL_SCRIPTS `ls | grep "^[0-9][0-9]${GROUPNAME}\..*"`" + done + unset GROUPNAME PRE_INSTALL_SCRIPTS="$PRE_INSTALL_SCRIPTS `ls | grep "^[0-9][0-9]${HOSTNAME}\..*"`" - PRE_INSTALL_SCRIPTS="$PRE_INSTALL_SCRIPTS `ls | grep "^[0-9][0-9]all\..*"`" # Now, to get rid of those pesky newlines. -BEF- PRE_INSTALL_SCRIPTS=`echo $PRE_INSTALL_SCRIPTS | tr '\n' ' '` @@ -1386,6 +1413,10 @@ get_base_hostname + # Get group name (defined in /etc/systemimager/cluster.xml on the image + # server). -AR- + get_group_name + if [ -e "${SCRIPTS_DIR}/post-install/" ]; then # make a copy of variables.txt available to post-install scripts -BEF- @@ -1393,16 +1424,14 @@ cd ${SCRIPTS_DIR}/post-install/ + POST_INSTALL_SCRIPTS="$POST_INSTALL_SCRIPTS `ls | grep "^[0-9][0-9]all\..*"`" POST_INSTALL_SCRIPTS="$POST_INSTALL_SCRIPTS `ls | grep "^[0-9][0-9]${IMAGENAME}\..*"`" POST_INSTALL_SCRIPTS="$POST_INSTALL_SCRIPTS `ls | grep "^[0-9][0-9]${BASE_HOSTNAME}\..*"`" + for GROUPNAME in ${GROUPNAMES}; do + POST_INSTALL_SCRIPTS="$POST_INSTALL_SCRIPTS `ls | grep "^[0-9][0-9]${GROUPNAME}\..*"`" + done POST_INSTALL_SCRIPTS="$POST_INSTALL_SCRIPTS `ls | grep "^[0-9][0-9]${HOSTNAME}\..*"`" - POST_INSTALL_SCRIPTS="$POST_INSTALL_SCRIPTS `ls | grep "^[0-9][0-9]all\..*"`" - for OVERRIDE in $OVERRIDES - do - POST_INSTALL_SCRIPTS="$POST_INSTALL_SCRIPTS `ls | grep "^[0-9][0-9]${OVERRIDE}\..*"`" - done - # Now, to get rid of those pesky newlines. -BEF- POST_INSTALL_SCRIPTS=`echo $POST_INSTALL_SCRIPTS | tr '\n' ' '` Modified: trunk/lib/SystemImager/Server.pm =================================================================== --- trunk/lib/SystemImager/Server.pm 2007-08-13 07:40:26 UTC (rev 4158) +++ trunk/lib/SystemImager/Server.pm 2007-08-13 10:22:15 UTC (rev 4159) @@ -1833,7 +1833,7 @@ } else { $overrides = ''; } - print $MASTER_SCRIPT q([ -z $OVERRIDES ] && ) . qq(OVERRIDES="$script_name $overrides"\n); + print $MASTER_SCRIPT q([ -z $OVERRIDES ] && ) . qq(OVERRIDES="$script_name \$GROUPNAMES \$HOSTNAME $overrides"\n); last SWITCH; } Modified: trunk/sbin/si_addclients =================================================================== --- trunk/sbin/si_addclients 2007-08-13 07:40:26 UTC (rev 4158) +++ trunk/sbin/si_addclients 2007-08-13 10:22:15 UTC (rev 4159) @@ -452,7 +452,9 @@ if ($hosts) { create_links($hosts, $script, $autoinstall_script_dir); } elsif ($groups) { - create_links_group($groups, $script, $autoinstall_script_dir); + foreach my $g (split(/,| |\n/, $groups)) { + create_links($g, $script, $autoinstall_script_dir); + } } else { die "ERROR: something unexpected... groups or hosts were not defined!\n"; } @@ -497,17 +499,6 @@ } -# Usage: create_links_group($grouplist, $script, $autoinstall_script_dir); -sub create_links_group { - - my ($grouplist, $script, $autoinstall_script_dir) = @_; - - foreach my $node (SystemImager::HostRange::expand_groups($grouplist)) { - my $cmd = "cd $autoinstall_script_dir && ln -sf $script.master $node.sh"; - !system($cmd) or die "Can't $cmd!"; - } -} - # Usage: $ret = add_hosts_entries($ip_range, $domain_name, @all_hosts); sub add_hosts_entries { my ($ip_range, $domain_name, @all_hosts) = @_; Added: trunk/sbin/si_clusterconfig =================================================================== --- trunk/sbin/si_clusterconfig (rev 0) +++ trunk/sbin/si_clusterconfig 2007-08-13 10:22:15 UTC (rev 4159) @@ -0,0 +1,222 @@ +#!/usr/bin/perl -w +# +# "SystemImager" +# +# Copyright (C) 2007 Andrea Righi <[EMAIL PROTECTED]> + +use lib "USR_PREFIX//lib/systemimager/perl"; +use strict; +use Getopt::Long; +use File::Copy; +use XML::Simple; +use SystemImager::Config; +use SystemImager::HostRange; +use vars qw($config); + +my $VERSION = "SYSTEMIMAGER_VERSION_STRING"; + +my $program_name = "si_clusterconfig"; + +my $version_info = << "EOF"; +$program_name (part of SystemImager) v$VERSION + +Copyright (C) 2007 Andrea Righi <[EMAIL PROTECTED]> + +This is free software; see the source for copying conditions. There is NO +warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. +EOF + +my $help_info = $version_info . <<"EOF"; + +Usage: $program_name [OPTION]... + +Options: (options can be presented in any order and can be abbreviated) + --help, -h Display this output. + + --version, -v Display version and copyright information. + +EOF + +Getopt::Long::Configure("posix_default"); +Getopt::Long::Configure("no_gnu_compat"); +Getopt::Long::Configure("bundling"); + +GetOptions( + "help|h" => \my $help, + "version|v" => \my $version, +) or die("$help_info"); + +select(STDERR); +$| = 1; +select(STDOUT); +$| = 1; + +### BEGIN evaluate commad line options ### + +if ($help) { + print "$help_info"; + exit(0); +} + +if ($version) { + print "$version_info"; + exit(0); +} + +### END evaluate commad line options ### + +# This program must be run as root. +unless ($< == 0) { + die("FATAL: $program_name requires root privileges.\n"); +} + +# Get cluster configuration file. +my $database = $SystemImager::HostRange::database; + +# Get a valid system editor. +my $editor = ''; +if (defined($ENV{'EDITOR'})) { + $editor = $ENV{'EDITOR'}; +} else { + chomp($editor = `(type -p vim || type -p vi || type -p nano || type -p nano-tiny) 2>/dev/null`); +} + +while (1) { + # Edit the configuration file. + system "$editor $database"; + + # Check syntax after editing. + eval { XMLin($database, ForceArray => 1) }; + if ($@) { + print STDERR "ERROR: syntax error in $database!\n"; + print STDERR "[EMAIL PROTECTED]"; + print "Edit again? (y | n) "; + chomp(my $ans = <STDIN>); + if (lc($ans) eq 'y') { + next; + } else { + print STDERR "WARNING: $database still contains errors! Please fix it.\n"; + exit(1); + } + } + last; +} + +my $script_dir = $config->autoinstall_script_dir(); +my $output_file; + +# Cache XML info into a plain text file. This is needed to easily parse this +# file in the busybox environment when the clients are imaging. +$output_file = $script_dir . '/cluster.txt'; +print "Caching XML configuration to $output_file... "; +if (!XML_info_to_plain_file($output_file)) { + print "[ OK ]\n"; +} else { + die("\nERROR: couldn't cache XML info to $output_file!\n"); +} + +# Synchronize /etc/hosts to the hosts file read by the imaging clients. +# XXX: not really dependent on cluster.xml, but it's better to synchronize +# /etc/hosts with the hosts known by SystemImager sometimes... -AR- +$output_file = $script_dir . '/hosts'; +print "Synchronizing /etc/hosts to $output_file... "; +if (!sync_etc_hosts($output_file)) { + print "[ OK ]\n"; +} else { + die("\nERROR: couldn't synchronize /etc/hosts to $output_file!\n"); +} + +# Well done. +exit(0); + +# Usage: XML_info_to_plain_file($output_file) +# Description: +# Convert the cluster configuration defined from the XML file +# /etc/systemimager/cluster.xml into a plain txt file given as argument. +sub XML_info_to_plain_file +{ + my $dest = shift; + + my $xml = XMLin($database, ForceArray => 1); + + open(OUT, ">$dest") or return -1; + + # Resolve the list of groups in nodenames. + foreach my $group (@{$xml->{'group'}}) { + foreach my $node (@{$group->{'node'}}) { + print OUT $node . ':' . $group->{'name'}[0] . "\n"; + } + } + close(OUT); + + return 0; +} + +# Usage: sync_etc_hosts($destination_file) +# Description: +# Copy /etc/hosts to $destination_file if they differ. +sub sync_etc_hosts +{ + my $dest = shift; + + system "diff /etc/hosts $dest >/dev/null 2>&1"; + if ($?) { + copy('/etc/hosts', $dest) or return -1; + } + return 0; +} + +__END__ + +=head1 NAME + +si_clusterconfig - Configure SystemImager cluster topology + +=head1 SYNOPSIS + +si_clusterconfig [OPTION]... + +=head1 DESCRIPTION + +B<si_clusterconfig> is a wrapper to edit the configuration of the cluster +topology, defined in the XML file B</etc/systemimager/cluster.xml>. + +After the editing this command provides to automatically update all the required +informations, that depends on the cluster configuration (groups, hosts, etc). + +=head1 OPTIONS + +=over 8 + +=item B<--help | -h> + +Display a short help. + +=item B<--version | -V> + +Display version and copyright information. + +=head1 SEE ALSO + +systemimager(8), si_pcp(8), si_psh(8), /etc/systemimager/cluster.xml + +=head1 AUTHOR + +Andrea Righi <[EMAIL PROTECTED]>. + +=head1 COPYRIGHT AND LICENSE + +Copyright 2003 by Andrea Righi <[EMAIL PROTECTED]>. + +This program is free software; you can redistribute it and/or modify +it under the terms of the GNU General Public License as published by +the Free Software Foundation; either version 2 of the License, or +(at your option) any later version. + +This program is distributed in the hope that it will be useful, +but WITHOUT ANY WARRANTY; without even the implied warranty of +MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +GNU General Public License for more details. + +=cut + Property changes on: trunk/sbin/si_clusterconfig ___________________________________________________________________ Name: svn:executable + * Modified: trunk/systemimager.spec =================================================================== --- trunk/systemimager.spec 2007-08-13 07:40:26 UTC (rev 4158) +++ trunk/systemimager.spec 2007-08-13 10:22:15 UTC (rev 4159) @@ -875,6 +875,7 @@ %prefix/sbin/si_rmimage %prefix/sbin/si_monitor %prefix/sbin/si_monitortk +%prefix/sbin/si_clusterconfig %prefix/bin/si_mk* %prefix/bin/si_psh %prefix/bin/si_pcp Modified: trunk/var/lib/systemimager/scripts/post-install/README =================================================================== --- trunk/var/lib/systemimager/scripts/post-install/README 2007-08-13 07:40:26 UTC (rev 4158) +++ trunk/var/lib/systemimager/scripts/post-install/README 2007-08-13 10:22:15 UTC (rev 4159) @@ -57,15 +57,13 @@ 99all.harmless_example_script Classes include: - - $IMAGENAME (Ie: my_compute_image) - - $BASE_HOSTNAME (Ie: compute) - - $HOSTNAME (Ie: compute07) - - all - - $OVERRIDE + - all (all clients) + - $IMAGENAME (i.e. ubuntu7_04) + - $GROUPNAMES (i.e. Login) + - $BASE_HOSTNAME (i.e. node) + - $HOSTNAME (i.e. node07) Scripts are ordered and executed first by class, then by number within - a class. If you have multiple $OVERRIDE directories, they will be - executed in the order specified in your auto-install script. + a class. - # /* vi: set et ts=2 tw=72: */ Modified: trunk/var/lib/systemimager/scripts/pre-install/README =================================================================== --- trunk/var/lib/systemimager/scripts/pre-install/README 2007-08-13 07:40:26 UTC (rev 4158) +++ trunk/var/lib/systemimager/scripts/pre-install/README 2007-08-13 10:22:15 UTC (rev 4159) @@ -32,14 +32,13 @@ 99all.harmless_example_script Classes include: - - $IMAGENAME (Ie: my_compute_image) - - $BASE_HOSTNAME (Ie: compute) - - $HOSTNAME (Ie: compute07) - - all + - all (all clients) + - $IMAGENAME (i.e. ubuntu7_04) + - $GROUPNAMES (i.e. Login) + - $BASE_HOSTNAME (i.e. node) + - $HOSTNAME (i.e. node07) Scripts are ordered and executed first by class, then by number within - a class. If you have multiple $OVERRIDE directories, they will be - executed in the order specified in your auto-install script. + a class. - # /* vi: set et ts=2 tw=72: */ ------------------------------------------------------------------------- This SF.net email is sponsored by: Splunk Inc. Still grepping through log files to find problems? Stop. Now Search log events and configuration files using AJAX and a browser. Download your FREE copy of Splunk now >> http://get.splunk.com/ _______________________________________________ sisuite-devel mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/sisuite-devel
