Hi Ibrahim, Thanks for the update. I'll add this in on your behalf.
We'll most likely push the Ubuntu specific stuff into the Linux/Ubuntu.pm OS module. Since there will be subtle differences in the paths and commands. Aaron On Sun, Oct 9, 2011 at 3:41 PM, Jeries, Ibrahim (Student) <ibrahim.jeri...@student.anglia.ac.uk> wrote: > I've hit exactly the same issues with BackTrack (Ubuntu based) and CentOS 6.0 > when I tried to capture them as base images. > So my work around was to modify the generate_rc_local module in Linux.pm > starting at Line 2164 with an inner anonymous array to: > 1) Check the OS type and based on that info to: > a) Delete the 70-persistant-net.rules file > b) The load the correct script based on architecture > > 3) On CentOS 6.0 only, the rc.local location had to also be modified due to > change in its architecture. > 4) I've just noticed now that the CentOS code assembly path requires > modification to match the new rc.local location. > > Please note, due to project time constraints I had to stop the implantation > and testing and therefore I acknowledge that the script is somewhat limited, > however I do hope the idea/script can be accepted to show my gratitude to the > VCL for such a brilliant innovation. > > Credits to Andy for the Ubuntu script available at > (http://www.mail-archive.com/vcl-dev@incubator.apache.org/msg01069.html), > many thanks Andy Kurth. > > Here's what the final script looks like, it's been tested and it works > despite showing some warning during the imagining process. > > sub generate_rc_local { > my $self = shift; > if (ref($self) !~ /linux/i) { > notify($ERRORS{'CRITICAL'}, 0, "subroutine was called as a > function, it must be called as a class method"); > return 0; > } > > my $request_id = $self->data->get_request_id(); > my $management_node_keys = $self->data->get_management_node_keys(); > my $computer_short_name = $self->data->get_computer_short_name(); > my $computer_node_name = $self->data->get_computer_node_name(); > > # Inner sub to check if the OS is Ubuntu variant, if it is, run the > first rc.local script to ensure compatibility > # Modified by IJ to overcome the VCL limitation. > > my $OS_value = sub { > # return `ssh -i /etc/vcl/vcl.key > $computer_node_name grep -io "ubuntu" /proc/version | wc -l 2>&1`; > return `ssh -i $management_node_keys $computer_node_name grep > -io "ubuntu" /proc/version | wc -l 2>&1`; > }; > > my $count = &$OS_value(); > if ($count >= "1") > { > # Delete the Ubuntu persistent network rule generated on boot. > # This is to prevent locking NIC eth0 + eth1 to the non VCL MAC > addresses > system(my $del_net_rules = `ssh -i $management_node_keys > $computer_node_name /bin/rm -f /etc/udev/rules.d/70-persistent-net.rules`); > > my @array2print; > > push(@array2print, '#!/bin/bash' . "\n"); > push(@array2print, 'function set_config {' . "\n"); > push(@array2print, ' if [ $# -ne 3 ]' . "\n"); > push(@array2print, ' then' . "\n"); > push(@array2print, ' echo "usage: set_config > [config_file] [keyword] [value]"' . "\n"); > push(@array2print, ' exit 1' . "\n"); > push(@array2print, ' fi' . "\n"); > push(@array2print, '' . "\n"); > push(@array2print, ' config_file=$1' . "\n"); > push(@array2print, ' keyword=$2' . "\n"); > push(@array2print, ' value=$3' . "\n"); > push(@array2print, '' . "\n"); > push(@array2print, ' if [ $value == \'#\' ]' . "\n"); > push(@array2print, ' then' . "\n"); > push(@array2print, ' #echo "Commenting $keyword lines > in $config_file"' . "\n"); > push(@array2print, ' sed -i -r -e "s/^[ #]*($keyword > .*)/#\1/" $config_file' . "\n"); > push(@array2print, ' else' . "\n"); > push(@array2print, ' if [ `grep -i -r -c "^[ > #]*$keyword " $config_file` == \'0\' ]' . "\n"); > push(@array2print, ' then' . "\n"); > push(@array2print, ' #echo "Adding $keyword > value to $config_file"' . "\n"); > push(@array2print, ' echo "$keyword $value" > >> $config_file' . "\n"); > push(@array2print, ' else' . "\n"); > push(@array2print, ' escaped_value=$(echo > $value | sed -e \'s/\//\\\\\//g\')' . "\n"); > push(@array2print, ' #echo Setting $keyword > to $value in $config_file' . "\n"); > push(@array2print, ' sed -i -r -e "s/^[ #]*($keyword).*/\1 > $escaped_value/" $config_file' . "\n"); > push(@array2print, ' fi' . "\n"); > push(@array2print, ' fi' . "\n"); > push(@array2print, '' . "\n"); > push(@array2print, ' #grep -i -r "^[ #]*$keyword" > $config_file' . "\n"); > push(@array2print, '' . "\n"); > push(@array2print, 'return 1;' . "\n"); > push(@array2print, '}' . "\n"); > push(@array2print, '' . "\n"); > push(@array2print, 'clear' . "\n"); > push(@array2print, 'cp /etc/ssh/sshd_config > /etc/ssh/sshd_config.ORIG' . "\n"); > push(@array2print, '' . "\n"); > push(@array2print, 'set_config \'/etc/ssh/sshd_config\' > \'StrictModes\' \'no\'' . "\n"); > push(@array2print, 'set_config \'/etc/ssh/sshd_config\' > \'X11Forwarding\' \'yes\'' . "\n"); > push(@array2print, 'set_config \'/etc/ssh/sshd_config\' > \'KeyRegenerationInterval\' \'0\'' . "\n"); > push(@array2print, 'set_config \'/etc/ssh/sshd_config\' > \'MaxStartups\' \'#\'' . "\n"); > push(@array2print, '' . "\n"); > push(@array2print, 'cp /etc/ssh/sshd_config > /etc/ssh/external_sshd_config' . "\n"); > push(@array2print, '' . "\n"); > push(@array2print, 'set_config \'/etc/ssh/external_sshd_config\' > \'PidFile\' \'/var/run/ext_sshd.pid\'' . "\n"); > push(@array2print, '' . "\n"); > push(@array2print, 'sed -i -r -e "s/^[ #]*AllowUsers.*//g" > /etc/ssh/sshd_config' . "\n"); > push(@array2print, 'sed -i -r -e "s/^[ #]*AllowUsers.*//g" > /etc/ssh/external_sshd_config' . "\n"); > push(@array2print, '' . "\n"); > push(@array2print, 'sed -i -r -e "s/^[ #]*ListenAddress.*//g" > /etc/ssh/sshd_config' . "\n"); > push(@array2print, 'sed -i -r -e "s/^[ #]*ListenAddress.*//g" > /etc/ssh/external_sshd_config' . "\n"); > push(@array2print, '' . "\n"); > push(@array2print, 'IP0=$(ifconfig eth0 | grep \'inet addr\' | awk > \'{print $2}\' | awk -F: \'{print $2}\')' . "\n"); > push(@array2print, 'IP1=$(ifconfig eth1 | grep \'inet addr\' | awk > \'{print $2}\' | awk -F: \'{print $2}\')' . "\n"); > push(@array2print, 'echo "IP eth0: $IP0"' . "\n"); > push(@array2print, 'echo "IP eth1: $IP1"' . "\n"); > push(@array2print, '' . "\n"); > push(@array2print, 'echo "AllowUsers root" >> /etc/ssh/sshd_config' > . "\n"); > push(@array2print, '' . "\n"); > push(@array2print, 'echo "ListenAddress $IP0" >> > /etc/ssh/sshd_config' . "\n"); > push(@array2print, 'echo "ListenAddress $IP1" >> > /etc/ssh/external_sshd_config' . "\n"); > push(@array2print, '' . "\n"); > push(@array2print, 'cp /etc/init.d/ssh /etc/init.d/ext_ssh' . "\n"); > push(@array2print, 'sed -i -r -e "s/(ext_)?sshd\.pid/ext_sshd.pid/g" > /etc/init.d/ext_ssh' . "\n"); > push(@array2print, 'sed -i -r -e "s/\"sshd\"/\"ext_sshd\"/g" > /etc/init.d/ext_ssh' . "\n"); > push(@array2print, 'sed -i -r -e > "s/(.*init-functions)/\1\n\nSSHD_OPTS=\"-f > \/etc\/ssh\/external_sshd_config\"/" /etc/init.d/ext_ssh' . "\n"); > push(@array2print, '' . "\n"); > push(@array2print, 'echo' . "\n"); > push(@array2print, 'echo Stopping sshd services...' . "\n"); > push(@array2print, 'service ssh stop' . "\n"); > push(@array2print, 'sleep 2' . "\n"); > push(@array2print, 'service ext_ssh stop' . "\n"); > push(@array2print, 'sleep 2' . "\n"); > push(@array2print, '' . "\n"); > push(@array2print, 'rm -f /var/run/*sshd*pid' . "\n"); > push(@array2print, '' . "\n"); > push(@array2print, 'echo' . "\n"); > push(@array2print, 'echo Starting sshd services...' . "\n"); > push(@array2print, 'service ssh start' . "\n"); > push(@array2print, 'sleep 2' . "\n"); > push(@array2print, 'service ext_ssh start' . "\n"); > push(@array2print, '' . "\n"); > push(@array2print, 'echo' . "\n"); > push(@array2print, 'echo sshd processes:' . "\n"); > push(@array2print, 'pgrep -fl "sbin.sshd"' . "\n"); > push(@array2print, 'echo' . "\n"); > push(@array2print, 'echo sshd.pid: `cat /var/run/sshd.pid`' . "\n"); > push(@array2print, 'echo ext_sshd.pid: `cat /var/run/ext_sshd.pid`' > . "\n"); > push(@array2print, 'sleep 5' . "\n"); > push(@array2print, 'dhclient eth1' . "\n"); > > # write to tmpfile > my $tmpfile = "/tmp/$request_id.rc.local"; > if (open(TMP, ">$tmpfile")) > { > print TMP @array2print; > close(TMP); > } > else { > #print "could not write $tmpfile $!\n"; > notify($ERRORS{'OK'}, 0, "could not write $tmpfile $!"); > return 0; > } > } > # If the machine isn't Ubuntu then load the default CentOS script. > else { > > system(my $del_net_rules = `ssh -i $management_node_keys > $computer_node_name /bin/rm -f /etc/udev/rules.d/70-persistent-net.rules`); > > my @array2print; > > push(@array2print, '# This script will be executed after all the other > init scripts.' . "\n\n"); > push(@array2print, '# WARNING --- VCL IMAGE CREATORS --- WARNING' . > "\n\n"); > push(@array2print, '# This file will get overwritten during image > capture. Any customizations' . "\n"); > push(@array2print, '# should be put into /etc/init.d/vcl_post_reserve > or /etc/init.d/vcl_post_load' . "\n"); > push(@array2print, '# Note these files do not exist by default.' . > "\n\n"); > push(@array2print, 'touch /var/lock/subsys/local' . "\n\n"); > push(@array2print, 'cp /etc/ssh/sshd_config /etc/ssh/sshd_config.ORIG' > . "\n"); > push(@array2print, 'cp /etc/ssh/sshd_config /etc/ssh/sshd_config.ORIG' > . "\n"); > push(@array2print, 'cp /etc/init.d/sshd /etc/init.d/sshd.ORIG' . > "\n\n"); > push(@array2print, 'cp /etc/ssh/sshd_config > /etc/ssh/external_sshd_config' . "\n"); > push(@array2print, 'cp /etc/init.d/sshd /etc/init.d/ext_sshd' . > "\n\n"); > push(@array2print, '# Replace sshd with ext_sshd.pid in > /etc/init.d/ext_sshd' . "\n"); > push(@array2print, 'sed -i -r -e "s/(ext_)?sshd\.pid/ext_sshd.pid/g" > /etc/init.d/ext_sshd' ."\n\n"); > push(@array2print, '# Find ext_sshd.pid and add "OPTIONS="-f > /etc/ssh/external_sshd_config" on new line after it' . "\n"); > push(@array2print, 'sed -i -r -e "s/(ext_sshd.pid)/\1\n\nOPTIONS=\"-f > \/etc\/ssh\/external_sshd_config\"/" /etc/init.d/ext_sshd' . "\n\n"); > push(@array2print, 'IP0=$(ifconfig eth0 | grep inet | awk \'{print > $2}\' | awk -F: \'{print $2}\')' . "\n"); > push(@array2print, 'IP1=$(ifconfig eth1 | grep inet | awk \'{print > $2}\' | awk -F: \'{print $2}\')' . "\n\n"); > push(@array2print, 'sed -i -e \'/.*AllowUsers .*$/d\' > /etc/ssh/sshd_config' . "\n"); > push(@array2print, 'sed -i -e \'/.*ListenAddress .*/d\' > /etc/ssh/sshd_config' . "\n"); > push(@array2print, 'sed -i -e \'/.*ListenAddress .*/d\' > /etc/ssh/external_sshd_config' . "\n\n"); > push(@array2print, 'echo "AllowUsers root" >> /etc/ssh/sshd_config' . > "\n"); > push(@array2print, 'echo "ListenAddress $IP0" >> /etc/ssh/sshd_config' > . "\n"); > push(@array2print, 'echo "ListenAddress $IP1" >> > /etc/ssh/external_sshd_config' . "\n\n"); > push(@array2print, '/etc/rc.d/init.d/ext_sshd stop' . "\n"); > push(@array2print, '/etc/rc.d/init.d/sshd stop' . "\n"); > push(@array2print, 'sleep 2' . "\n\n"); > push(@array2print, '/etc/rc.d/init.d/sshd start' . "\n"); > push(@array2print, '/etc/rc.d/init.d/ext_sshd start' . "\n"); > > #write to tmpfile > my $tmpfile = "/tmp/$request_id.rc.local"; > if (open(TMP, ">$tmpfile")) > { > print TMP @array2print; > close(TMP); > } > else { > #print "could not write $tmpfile $!\n"; > notify($ERRORS{'OK'}, 0, "could not write $tmpfile $!"); > return 0; > } > } > my $tmpfile = "/tmp/$request_id.rc.local"; > #copy to node > if (run_scp_command($tmpfile, > "$computer_node_name:/etc/rc.d/rc.local", $management_node_keys)) { > } > else{ > return 0; > } > > # Assemble the command > my $command = "chmod +rx /etc/rc.local"; > > # Execute the command > my ($exit_status, $output) = run_ssh_command($computer_node_name, > $management_node_keys, $command, '', '', 1); > if (defined($exit_status) && $exit_status == 0) { > notify($ERRORS{'OK'}, 0, "executed $command, exit status: > $exit_status"); > } > elsif (defined($exit_status)) { > notify($ERRORS{'WARNING'}, 0, "setting rx on /etc/rc.local > returned a non-zero exit status: $exit_status"); > return; > } > else { > notify($ERRORS{'WARNING'}, 0, "failed to run SSH command to > execute script_path"); > return 0; > } > > unlink($tmpfile); > > return 1; > } > > > > -----Original Message----- > From: Aaron Peeler [mailto:fapee...@ncsu.edu] > Sent: 07 October 2011 12:57 > To: vcl-dev@incubator.apache.org > Subject: Re: After capturing a Fedora 15 image, ext_sshd stop kills all SSH > access > > Hi Pete, > > Thanks for the info. I'm in the process of adding Fedora15 now, so I'll > shortly hit this issue. > > I agree we'll need to adapt the pre_capture steps of Linux to account for > such changes. Either as you mention below, touch|no touch options or to > correctly account for the the issues with kilproc $SSHD. > > Aaron > > > On Thu, Oct 6, 2011 at 8:33 PM, Peter Dimitrios <petedag...@gmail.com> wrote: >> I've got a base Fedora 15 image which works fine within VCL after >> monkeying with the ext_sshd scripts. Now, when I go to create a new >> image based on it, the new image is captured correctly, but the >> problem with "ext_sshd stop" during the reservation process >> resurfaces. >> >> I've manually edited the /etc/init.d/ext_sshd and /etc/init.d/sshd >> scripts to correctly use the $PID_FILE (see >> http://www.mail-archive.com/vcl-user@incubator.apache.org/msg00156.htm >> l), >> so the SSH servers can be independently started/stopped correctly. >> But, it seems that generate_ext_sshd_init in Linux.pm doesn't tweak >> the ext_sshd script in the same manner, so after the capture I need to >> tweak it again. >> >> I wonder if eventually we might want to enhance the capture script to >> respect some sort of flag in the image to tell the capture script to >> leave some things alone. >> >> One approach might be to source a file from the image itself (such as >> /root/vcl-custom-capture.pm) that can allow an image creator to >> customize or even override some of the operations performed during >> image capture. In fact, that might be a more general-purpose way of >> providing an "escape hatch" for one-off image customizations. As I >> look at Fedora 15 and its continuing migration towards using systemctl >> rather than sysv init, many of the techniques that Linux.pm and other >> VCL modules perform directly on files will become more fragile over >> time; so either more work may need to be poured into the internal VCL >> scripts. Some sort of "escape hatch" like sourcing a file from the >> image (if it exists) could be quite helpful. >> >> FWIW, the sshd stop script that kills the sshd process (rather than >> using the PID_FILE) was documented in 2005 ( >> http://www.redhat.com/archives/rhl-list/2005-November/msg04593.html ), >> which was solved there by additionally creating an ext_sshd copy or >> symlink for the sshd executable. Also, as another aside, the >> discussion from 2010 at >> http://www.linuxquestions.org/questions/linux-software-2/multiple-ssh- >> daemons-fc11-787230/ reveals that some issues with SELinux that may >> need to be addressed in some environments. >> >> I guess few people bother to create another SSH daemon, and we all >> work around it, so it hasn't been a priority for anyone to fix the >> killproc $SSHD issue in the base distro. >> > > > > -- > Aaron Peeler > Program Manager > Virtual Computing Lab > NC State University > > All electronic mail messages in connection with State business which are sent > to or received by this account are subject to the NC Public Records Law and > may be disclosed to third parties. > > > -- Aaron Peeler Program Manager Virtual Computing Lab NC State University All electronic mail messages in connection with State business which are sent to or received by this account are subject to the NC Public Records Law and may be disclosed to third parties.
