I've hit exactly the same issues with BackTrack (Ubuntu based) and CentOS 6.0
when I tried to capture them as base images.
So my work around was to modify the generate_rc_local module in Linux.pm
starting at Line 2164 with an inner anonymous array to:
1) Check the OS type and based on that info to:
a) Delete the 70-persistant-net.rules file
b) The load the correct script based on architecture
3) On CentOS 6.0 only, the rc.local location had to also be modified due to
change in its architecture.
4) I've just noticed now that the CentOS code assembly path requires
modification to match the new rc.local location.
Please note, due to project time constraints I had to stop the implantation and
testing and therefore I acknowledge that the script is somewhat limited,
however I do hope the idea/script can be accepted to show my gratitude to the
VCL for such a brilliant innovation.
Credits to Andy for the Ubuntu script available at
(http://www.mail-archive.com/vcl-dev@incubator.apache.org/msg01069.html), many
thanks Andy Kurth.
Here's what the final script looks like, it's been tested and it works despite
showing some warning during the imagining process.
sub generate_rc_local {
my $self = shift;
if (ref($self) !~ /linux/i) {
notify($ERRORS{'CRITICAL'}, 0, "subroutine was called as a
function, it must be called as a class method");
return 0;
}
my $request_id = $self->data->get_request_id();
my $management_node_keys = $self->data->get_management_node_keys();
my $computer_short_name = $self->data->get_computer_short_name();
my $computer_node_name = $self->data->get_computer_node_name();
# Inner sub to check if the OS is Ubuntu variant, if it is, run the
first rc.local script to ensure compatibility
# Modified by IJ to overcome the VCL limitation.
my $OS_value = sub {
# return `ssh -i /etc/vcl/vcl.key $computer_node_name
grep -io "ubuntu" /proc/version | wc -l 2>&1`;
return `ssh -i $management_node_keys $computer_node_name grep
-io "ubuntu" /proc/version | wc -l 2>&1`;
};
my $count = &$OS_value();
if ($count >= "1")
{
# Delete the Ubuntu persistent network rule generated on boot.
# This is to prevent locking NIC eth0 + eth1 to the non VCL MAC
addresses
system(my $del_net_rules = `ssh -i $management_node_keys
$computer_node_name /bin/rm -f /etc/udev/rules.d/70-persistent-net.rules`);
my @array2print;
push(@array2print, '#!/bin/bash' . "\n");
push(@array2print, 'function set_config {' . "\n");
push(@array2print, ' if [ $# -ne 3 ]' . "\n");
push(@array2print, ' then' . "\n");
push(@array2print, ' echo "usage: set_config
[config_file] [keyword] [value]"' . "\n");
push(@array2print, ' exit 1' . "\n");
push(@array2print, ' fi' . "\n");
push(@array2print, '' . "\n");
push(@array2print, ' config_file=$1' . "\n");
push(@array2print, ' keyword=$2' . "\n");
push(@array2print, ' value=$3' . "\n");
push(@array2print, '' . "\n");
push(@array2print, ' if [ $value == \'#\' ]' . "\n");
push(@array2print, ' then' . "\n");
push(@array2print, ' #echo "Commenting $keyword lines
in $config_file"' . "\n");
push(@array2print, ' sed -i -r -e "s/^[ #]*($keyword
.*)/#\1/" $config_file' . "\n");
push(@array2print, ' else' . "\n");
push(@array2print, ' if [ `grep -i -r -c "^[
#]*$keyword " $config_file` == \'0\' ]' . "\n");
push(@array2print, ' then' . "\n");
push(@array2print, ' #echo "Adding $keyword
value to $config_file"' . "\n");
push(@array2print, ' echo "$keyword $value" >>
$config_file' . "\n");
push(@array2print, ' else' . "\n");
push(@array2print, ' escaped_value=$(echo
$value | sed -e \'s/\//\\\\\//g\')' . "\n");
push(@array2print, ' #echo Setting $keyword to
$value in $config_file' . "\n");
push(@array2print, ' sed -i -r -e "s/^[ #]*($keyword).*/\1
$escaped_value/" $config_file' . "\n");
push(@array2print, ' fi' . "\n");
push(@array2print, ' fi' . "\n");
push(@array2print, '' . "\n");
push(@array2print, ' #grep -i -r "^[ #]*$keyword"
$config_file' . "\n");
push(@array2print, '' . "\n");
push(@array2print, 'return 1;' . "\n");
push(@array2print, '}' . "\n");
push(@array2print, '' . "\n");
push(@array2print, 'clear' . "\n");
push(@array2print, 'cp /etc/ssh/sshd_config
/etc/ssh/sshd_config.ORIG' . "\n");
push(@array2print, '' . "\n");
push(@array2print, 'set_config \'/etc/ssh/sshd_config\'
\'StrictModes\' \'no\'' . "\n");
push(@array2print, 'set_config \'/etc/ssh/sshd_config\'
\'X11Forwarding\' \'yes\'' . "\n");
push(@array2print, 'set_config \'/etc/ssh/sshd_config\'
\'KeyRegenerationInterval\' \'0\'' . "\n");
push(@array2print, 'set_config \'/etc/ssh/sshd_config\'
\'MaxStartups\' \'#\'' . "\n");
push(@array2print, '' . "\n");
push(@array2print, 'cp /etc/ssh/sshd_config
/etc/ssh/external_sshd_config' . "\n");
push(@array2print, '' . "\n");
push(@array2print, 'set_config \'/etc/ssh/external_sshd_config\'
\'PidFile\' \'/var/run/ext_sshd.pid\'' . "\n");
push(@array2print, '' . "\n");
push(@array2print, 'sed -i -r -e "s/^[ #]*AllowUsers.*//g"
/etc/ssh/sshd_config' . "\n");
push(@array2print, 'sed -i -r -e "s/^[ #]*AllowUsers.*//g"
/etc/ssh/external_sshd_config' . "\n");
push(@array2print, '' . "\n");
push(@array2print, 'sed -i -r -e "s/^[ #]*ListenAddress.*//g"
/etc/ssh/sshd_config' . "\n");
push(@array2print, 'sed -i -r -e "s/^[ #]*ListenAddress.*//g"
/etc/ssh/external_sshd_config' . "\n");
push(@array2print, '' . "\n");
push(@array2print, 'IP0=$(ifconfig eth0 | grep \'inet addr\' | awk
\'{print $2}\' | awk -F: \'{print $2}\')' . "\n");
push(@array2print, 'IP1=$(ifconfig eth1 | grep \'inet addr\' | awk
\'{print $2}\' | awk -F: \'{print $2}\')' . "\n");
push(@array2print, 'echo "IP eth0: $IP0"' . "\n");
push(@array2print, 'echo "IP eth1: $IP1"' . "\n");
push(@array2print, '' . "\n");
push(@array2print, 'echo "AllowUsers root" >> /etc/ssh/sshd_config' .
"\n");
push(@array2print, '' . "\n");
push(@array2print, 'echo "ListenAddress $IP0" >>
/etc/ssh/sshd_config' . "\n");
push(@array2print, 'echo "ListenAddress $IP1" >>
/etc/ssh/external_sshd_config' . "\n");
push(@array2print, '' . "\n");
push(@array2print, 'cp /etc/init.d/ssh /etc/init.d/ext_ssh' . "\n");
push(@array2print, 'sed -i -r -e "s/(ext_)?sshd\.pid/ext_sshd.pid/g"
/etc/init.d/ext_ssh' . "\n");
push(@array2print, 'sed -i -r -e "s/\"sshd\"/\"ext_sshd\"/g"
/etc/init.d/ext_ssh' . "\n");
push(@array2print, 'sed -i -r -e
"s/(.*init-functions)/\1\n\nSSHD_OPTS=\"-f \/etc\/ssh\/external_sshd_config\"/"
/etc/init.d/ext_ssh' . "\n");
push(@array2print, '' . "\n");
push(@array2print, 'echo' . "\n");
push(@array2print, 'echo Stopping sshd services...' . "\n");
push(@array2print, 'service ssh stop' . "\n");
push(@array2print, 'sleep 2' . "\n");
push(@array2print, 'service ext_ssh stop' . "\n");
push(@array2print, 'sleep 2' . "\n");
push(@array2print, '' . "\n");
push(@array2print, 'rm -f /var/run/*sshd*pid' . "\n");
push(@array2print, '' . "\n");
push(@array2print, 'echo' . "\n");
push(@array2print, 'echo Starting sshd services...' . "\n");
push(@array2print, 'service ssh start' . "\n");
push(@array2print, 'sleep 2' . "\n");
push(@array2print, 'service ext_ssh start' . "\n");
push(@array2print, '' . "\n");
push(@array2print, 'echo' . "\n");
push(@array2print, 'echo sshd processes:' . "\n");
push(@array2print, 'pgrep -fl "sbin.sshd"' . "\n");
push(@array2print, 'echo' . "\n");
push(@array2print, 'echo sshd.pid: `cat /var/run/sshd.pid`' . "\n");
push(@array2print, 'echo ext_sshd.pid: `cat /var/run/ext_sshd.pid`' .
"\n");
push(@array2print, 'sleep 5' . "\n");
push(@array2print, 'dhclient eth1' . "\n");
# write to tmpfile
my $tmpfile = "/tmp/$request_id.rc.local";
if (open(TMP, ">$tmpfile"))
{
print TMP @array2print;
close(TMP);
}
else {
#print "could not write $tmpfile $!\n";
notify($ERRORS{'OK'}, 0, "could not write $tmpfile $!");
return 0;
}
}
# If the machine isn't Ubuntu then load the default CentOS script.
else {
system(my $del_net_rules = `ssh -i $management_node_keys
$computer_node_name /bin/rm -f /etc/udev/rules.d/70-persistent-net.rules`);
my @array2print;
push(@array2print, '# This script will be executed after all the other
init scripts.' . "\n\n");
push(@array2print, '# WARNING --- VCL IMAGE CREATORS --- WARNING' .
"\n\n");
push(@array2print, '# This file will get overwritten during image
capture. Any customizations' . "\n");
push(@array2print, '# should be put into /etc/init.d/vcl_post_reserve
or /etc/init.d/vcl_post_load' . "\n");
push(@array2print, '# Note these files do not exist by default.' .
"\n\n");
push(@array2print, 'touch /var/lock/subsys/local' . "\n\n");
push(@array2print, 'cp /etc/ssh/sshd_config /etc/ssh/sshd_config.ORIG'
. "\n");
push(@array2print, 'cp /etc/ssh/sshd_config /etc/ssh/sshd_config.ORIG'
. "\n");
push(@array2print, 'cp /etc/init.d/sshd /etc/init.d/sshd.ORIG' .
"\n\n");
push(@array2print, 'cp /etc/ssh/sshd_config
/etc/ssh/external_sshd_config' . "\n");
push(@array2print, 'cp /etc/init.d/sshd /etc/init.d/ext_sshd' . "\n\n");
push(@array2print, '# Replace sshd with ext_sshd.pid in
/etc/init.d/ext_sshd' . "\n");
push(@array2print, 'sed -i -r -e "s/(ext_)?sshd\.pid/ext_sshd.pid/g"
/etc/init.d/ext_sshd' ."\n\n");
push(@array2print, '# Find ext_sshd.pid and add "OPTIONS="-f
/etc/ssh/external_sshd_config" on new line after it' . "\n");
push(@array2print, 'sed -i -r -e "s/(ext_sshd.pid)/\1\n\nOPTIONS=\"-f
\/etc\/ssh\/external_sshd_config\"/" /etc/init.d/ext_sshd' . "\n\n");
push(@array2print, 'IP0=$(ifconfig eth0 | grep inet | awk \'{print
$2}\' | awk -F: \'{print $2}\')' . "\n");
push(@array2print, 'IP1=$(ifconfig eth1 | grep inet | awk \'{print
$2}\' | awk -F: \'{print $2}\')' . "\n\n");
push(@array2print, 'sed -i -e \'/.*AllowUsers .*$/d\'
/etc/ssh/sshd_config' . "\n");
push(@array2print, 'sed -i -e \'/.*ListenAddress .*/d\'
/etc/ssh/sshd_config' . "\n");
push(@array2print, 'sed -i -e \'/.*ListenAddress .*/d\'
/etc/ssh/external_sshd_config' . "\n\n");
push(@array2print, 'echo "AllowUsers root" >> /etc/ssh/sshd_config' .
"\n");
push(@array2print, 'echo "ListenAddress $IP0" >> /etc/ssh/sshd_config'
. "\n");
push(@array2print, 'echo "ListenAddress $IP1" >>
/etc/ssh/external_sshd_config' . "\n\n");
push(@array2print, '/etc/rc.d/init.d/ext_sshd stop' . "\n");
push(@array2print, '/etc/rc.d/init.d/sshd stop' . "\n");
push(@array2print, 'sleep 2' . "\n\n");
push(@array2print, '/etc/rc.d/init.d/sshd start' . "\n");
push(@array2print, '/etc/rc.d/init.d/ext_sshd start' . "\n");
#write to tmpfile
my $tmpfile = "/tmp/$request_id.rc.local";
if (open(TMP, ">$tmpfile"))
{
print TMP @array2print;
close(TMP);
}
else {
#print "could not write $tmpfile $!\n";
notify($ERRORS{'OK'}, 0, "could not write $tmpfile $!");
return 0;
}
}
my $tmpfile = "/tmp/$request_id.rc.local";
#copy to node
if (run_scp_command($tmpfile, "$computer_node_name:/etc/rc.d/rc.local",
$management_node_keys)) {
}
else{
return 0;
}
# Assemble the command
my $command = "chmod +rx /etc/rc.local";
# Execute the command
my ($exit_status, $output) = run_ssh_command($computer_node_name,
$management_node_keys, $command, '', '', 1);
if (defined($exit_status) && $exit_status == 0) {
notify($ERRORS{'OK'}, 0, "executed $command, exit status:
$exit_status");
}
elsif (defined($exit_status)) {
notify($ERRORS{'WARNING'}, 0, "setting rx on /etc/rc.local
returned a non-zero exit status: $exit_status");
return;
}
else {
notify($ERRORS{'WARNING'}, 0, "failed to run SSH command to
execute script_path");
return 0;
}
unlink($tmpfile);
return 1;
}
-----Original Message-----
From: Aaron Peeler [mailto:fapee...@ncsu.edu]
Sent: 07 October 2011 12:57
To: vcl-dev@incubator.apache.org
Subject: Re: After capturing a Fedora 15 image, ext_sshd stop kills all SSH
access
Hi Pete,
Thanks for the info. I'm in the process of adding Fedora15 now, so I'll shortly
hit this issue.
I agree we'll need to adapt the pre_capture steps of Linux to account for such
changes. Either as you mention below, touch|no touch options or to correctly
account for the the issues with kilproc $SSHD.
Aaron
On Thu, Oct 6, 2011 at 8:33 PM, Peter Dimitrios <petedag...@gmail.com> wrote:
> I've got a base Fedora 15 image which works fine within VCL after
> monkeying with the ext_sshd scripts. Now, when I go to create a new
> image based on it, the new image is captured correctly, but the
> problem with "ext_sshd stop" during the reservation process
> resurfaces.
>
> I've manually edited the /etc/init.d/ext_sshd and /etc/init.d/sshd
> scripts to correctly use the $PID_FILE (see
> http://www.mail-archive.com/vcl-user@incubator.apache.org/msg00156.htm
> l),
> so the SSH servers can be independently started/stopped correctly.
> But, it seems that generate_ext_sshd_init in Linux.pm doesn't tweak
> the ext_sshd script in the same manner, so after the capture I need to
> tweak it again.
>
> I wonder if eventually we might want to enhance the capture script to
> respect some sort of flag in the image to tell the capture script to
> leave some things alone.
>
> One approach might be to source a file from the image itself (such as
> /root/vcl-custom-capture.pm) that can allow an image creator to
> customize or even override some of the operations performed during
> image capture. In fact, that might be a more general-purpose way of
> providing an "escape hatch" for one-off image customizations. As I
> look at Fedora 15 and its continuing migration towards using systemctl
> rather than sysv init, many of the techniques that Linux.pm and other
> VCL modules perform directly on files will become more fragile over
> time; so either more work may need to be poured into the internal VCL
> scripts. Some sort of "escape hatch" like sourcing a file from the
> image (if it exists) could be quite helpful.
>
> FWIW, the sshd stop script that kills the sshd process (rather than
> using the PID_FILE) was documented in 2005 (
> http://www.redhat.com/archives/rhl-list/2005-November/msg04593.html ),
> which was solved there by additionally creating an ext_sshd copy or
> symlink for the sshd executable. Also, as another aside, the
> discussion from 2010 at
> http://www.linuxquestions.org/questions/linux-software-2/multiple-ssh-
> daemons-fc11-787230/ reveals that some issues with SELinux that may
> need to be addressed in some environments.
>
> I guess few people bother to create another SSH daemon, and we all
> work around it, so it hasn't been a priority for anyone to fix the
> killproc $SSHD issue in the base distro.
>
--
Aaron Peeler
Program Manager
Virtual Computing Lab
NC State University
All electronic mail messages in connection with State business which are sent
to or received by this account are subject to the NC Public Records Law and may
be disclosed to third parties.
