I'm attempting to create a base Linux image and keep getting stuck in the same spot. The image is capturing properly, but when Linux is going through post load, something breaks ssh. I don't understand why "/etc/init.d/ext_sshd stop" would halt ssh on the private interface?!?
2010-07-14 09:21:30|28645|68:72|reload|esxthin.pm:load(607)|going to sleep 5 seconds, waiting for computer to start SSH. Try 1 2010-07-14 09:21:35|26430|vcld:main(165)|lastcheckin time updated for management node 2: 2010-07-14 09:21:35 2010-07-14 09:21:36|28645|68:72|reload|esxthin.pm:load(607)|going to sleep 5 seconds, waiting for computer to start SSH. Try 2 2010-07-14 09:21:40|26430|vcld:main(165)|lastcheckin time updated for management node 2: 2010-07-14 09:21:40 2010-07-14 09:21:41|28645|68:72|reload|esxthin.pm:load(607)|going to sleep 5 seconds, waiting for computer to start SSH. Try 3 2010-07-14 09:21:45|26430|vcld:main(165)|lastcheckin time updated for management node 2: 2010-07-14 09:21:45 2010-07-14 09:21:46|28645|68:72|reload|esxthin.pm:load(607)|going to sleep 5 seconds, waiting for computer to start SSH. Try 4 2010-07-14 09:21:50|26430|vcld:main(165)|lastcheckin time updated for management node 2: 2010-07-14 09:21:50 2010-07-14 09:21:51|28645|68:72|reload|esxthin.pm:load(607)|going to sleep 5 seconds, waiting for computer to start SSH. Try 5 2010-07-14 09:21:55|26430|vcld:main(165)|lastcheckin time updated for management node 2: 2010-07-14 09:21:55 2010-07-14 09:21:56|28645|68:72|reload|esxthin.pm:load(607)|going to sleep 5 seconds, waiting for computer to start SSH. Try 6 2010-07-14 09:22:00|26430|vcld:main(165)|lastcheckin time updated for management node 2: 2010-07-14 09:22:00 2010-07-14 09:22:02|28645|68:72|reload|esxthin.pm:load(607)|going to sleep 5 seconds, waiting for computer to start SSH. Try 7 2010-07-14 09:22:05|26430|vcld:main(165)|lastcheckin time updated for management node 2: 2010-07-14 09:22:05 2010-07-14 09:22:07|28645|68:72|reload|esxthin.pm:load(607)|going to sleep 5 seconds, waiting for computer to start SSH. Try 8 2010-07-14 09:22:10|26430|vcld:main(165)|lastcheckin time updated for management node 2: 2010-07-14 09:22:10 2010-07-14 09:22:12|28645|68:72|reload|esxthin.pm:load(607)|going to sleep 5 seconds, waiting for computer to start SSH. Try 9 2010-07-14 09:22:15|26430|vcld:main(165)|lastcheckin time updated for management node 2: 2010-07-14 09:22:15 2010-07-14 09:22:17|28645|68:72|reload|esxthin.pm:load(607)|going to sleep 5 seconds, waiting for computer to start SSH. Try 10 2010-07-14 09:22:20|26430|vcld:main(165)|lastcheckin time updated for management node 2: 2010-07-14 09:22:20 2010-07-14 09:22:23|28645|68:72|reload|esxthin.pm:load(607)|going to sleep 5 seconds, waiting for computer to start SSH. Try 11 2010-07-14 09:22:25|26430|vcld:main(165)|lastcheckin time updated for management node 2: 2010-07-14 09:22:25 2010-07-14 09:22:28|28645|68:72|reload|utils.pm:run_ssh_command(6180)|executing SSH command on vmwarerhel54-base33-v0: |28645|68:72|reload| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p 22 -x vmwarerhel54-base33-v0 'uname -s' 2>&1 2010-07-14 09:22:28|28645|68:72|reload|utils.pm:run_ssh_command(6262)|run_ssh_command output: |28645|68:72|reload| Linux 2010-07-14 09:22:28|28645|68:72|reload|utils.pm:run_ssh_command(6276)|SSH command executed on vmwarerhel54-base33-v0, returning (0, "Linux") 2010-07-14 09:22:28|28645|68:72|reload|esxthin.pm:load(595)|vmwarerhel54-base33-v0 now has active sshd running 2010-07-14 09:22:28|28645|68:72|reload|utils.pm:insertloadlog(4710)|inserted computer=516, dynamicDHCPaddress, collecting dynamic IP address for node 2010-07-14 09:22:28|28645|68:72|reload|esxthin.pm:load(618)|Attempting to query vmclient for its public IP... 2010-07-14 09:22:28|28645|68:72|reload|utils.pm:getdynamicaddress(1605)|PrivateIP address for vmwarerhel54-base33-v0 collected 192.168.131.254 2010-07-14 09:22:28|28645|68:72|reload|utils.pm:run_ssh_command(6180)|executing SSH command on vmwarerhel54-base33-v0: |28645|68:72|reload| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p 22 -x vmwarerhel54-base33-v0 '/sbin/ifconfig |grep inet' 2>&1 2010-07-14 09:22:29|28645|68:72|reload|utils.pm:run_ssh_command(6262)|run_ssh_command output: |28645|68:72|reload| inet addr:192.168.131.254 Bcast:192.168.131.255 Mask:255.255.254.0 |28645|68:72|reload| inet6 addr: fe80::250:56ff:fe00:3da/64 Scope:Link |28645|68:72|reload| inet addr:128.82.130.171 Bcast:128.82.131.255 Mask:255.255.254.0 |28645|68:72|reload| inet6 addr: fe80::250:56ff:fe00:3db/64 Scope:Link |28645|68:72|reload| inet addr:127.0.0.1 Mask:255.0.0.0 |28645|68:72|reload| inet6 addr: ::1/128 Scope:Host 2010-07-14 09:22:29|28645|68:72|reload|utils.pm:run_ssh_command(6276)|SSH command executed on vmwarerhel54-base33-v0, returning (0, "inet addr:192.168.131.254 Bcas...") 2010-07-14 09:22:29|28645|68:72|reload|utils.pm:getdynamicaddress(1652)|dynamic IP address for vmwarerhel54-base33-v0 collected: 128.82.130.171 2010-07-14 09:22:29|28645|68:72|reload|esxthin.pm:load(622)|Got dynamic address from vmclient, attempting to update database 2010-07-14 09:22:29|28645|68:72|reload|utils.pm:update_computer_address(7067)|computer 516 IP address 128.82.130.171 updated in database 2010-07-14 09:22:29|28645|68:72|reload|esxthin.pm:load(623)|succesfully updated IPaddress of node vmwarerhel54-base33-v0 2010-07-14 09:22:29|28645|68:72|reload|esxthin.pm:load(653)|post_load() has been implemented by the OS module, returning 1 2010-07-14 09:22:29|28645|68:72|reload|new.pm:reload_image(666)|rh5image-base33-v0 was successfully reloaded on vmwarerhel54-base33-v0 2010-07-14 09:22:29|28645|68:72|reload|utils.pm:insertloadlog(4710)|inserted computer=516, loadimagecomplete, rh5image-base33-v0 was successfully reloaded on vmwarerhel54-base33-v0 2010-07-14 09:22:29|28645|68:72|reload|utils.pm:update_currentimage(6897)|updating computer 516: image=33, imagerevision=69 2010-07-14 09:22:29|28645|68:72|reload|utils.pm:update_currentimage(6927)|updated currentimageid and imagerevision id for computer id 516 2010-07-14 09:22:29|28645|68:72|reload|new.pm:reload_image(677)|updated computer table for vmwarerhel54-base33-v0: currentimageid=33 2010-07-14 09:22:29|28645|68:72|reload|new.pm:reload_image(687)|VCL::Module::OS::Linux->post_load() subroutine exists 2010-07-14 09:22:29|28645|68:72|reload|new.pm:reload_image(690)|calling VCL::Module::OS::Linux->post_load() subroutine 2010-07-14 09:22:29|28645|68:72|reload|utils.pm:insertloadlog(4710)|inserted computer=516, info, calling VCL::Module::OS::Linux->post_load() subroutine 2010-07-14 09:22:29|28645|68:72|reload|Linux.pm:post_load(178)|initiating Linux post_load: rh5image-base33-v0 on vmwarerhel54-base33-v0 2010-07-14 09:22:29|28645|68:72|reload|Linux.pm:changepasswd(756)|password for vmwarerhel54-base33-v0 is rgFfamLveBkL2g9 2010-07-14 09:22:29|28645|68:72|reload|utils.pm:run_ssh_command(6180)|executing SSH command on vmwarerhel54-base33-v0: |28645|68:72|reload| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p 22 -x vmwarerhel54-base33-v0 'cat /etc/shadow |grep -v root >> /tmp/shadow.vmwarerhel54-base33-v0' 2>&1 2010-07-14 09:22:29|28645|68:72|reload|utils.pm:run_ssh_command(6262)|run_ssh_command output: |28645|68:72|reload| none 2010-07-14 09:22:29|28645|68:72|reload|utils.pm:run_ssh_command(6272)|SSH command executed on vmwarerhel54-base33-v0, command: |28645|68:72|reload| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p 22 -x vmwarerhel54-base33-v0 'cat /etc/shadow |grep -v root >> /tmp/shadow.vmwarerhel54-base33-v0' 2>&1 |28645|68:72|reload| returning (1, "none") 2010-07-14 09:22:29|28645|68:72|reload|Linux.pm:changepasswd(771)|collected /etc/shadow file from vmwarerhel54-base33-v0 2010-07-14 09:22:29|28645|68:72|reload|utils.pm:run_scp_command(6403)|attempt 1/3: executing SCP command: /usr/bin/scp -B -i /etc/vcl/vcl.key -P 22 -p -r /tmp/shadow.vmwarerhel54-base33-v0 vmwarerhel54-base33-v0:/etc/shadow 2>&1 2010-07-14 09:22:30|28645|68:72|reload|utils.pm:run_scp_command(6453)|scp successful: attempt 1/3, exit status: 0, output: |28645|68:72|reload| Warning: Permanently added 'vmwarerhel54-base33-v0,192.168.131.254' (RSA) to the list of known hosts. 2010-07-14 09:22:30|28645|68:72|reload|Linux.pm:changepasswd(773)|copied updated /etc/shadow file to vmwarerhel54-base33-v0 2010-07-14 09:22:30|28645|68:72|reload|utils.pm:run_ssh_command(6180)|executing SSH command on vmwarerhel54-base33-v0: |28645|68:72|reload| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p 22 -x vmwarerhel54-base33-v0 'chmod 600 /etc/shadow' 2>&1 2010-07-14 09:22:30|28645|68:72|reload|utils.pm:run_ssh_command(6262)|run_ssh_command output: |28645|68:72|reload| none 2010-07-14 09:22:30|28645|68:72|reload|utils.pm:run_ssh_command(6276)|SSH command executed on vmwarerhel54-base33-v0, returning (0, "none") 2010-07-14 09:22:30|28645|68:72|reload|Linux.pm:changepasswd(775)|updated permissions to 600 on /etc/shadow file on vmwarerhel54-base33-v0 2010-07-14 09:22:30|28645|68:72|reload|Linux.pm:post_load(181)|successfully changed root password on vmwarerhel54-base33-v0 2010-07-14 09:22:30|28645|68:72|reload|utils.pm:run_ssh_command(6180)|executing SSH command on vmwarerhel54-base33-v0: |28645|68:72|reload| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p 22 -x vmwarerhel54-base33-v0 '/etc/init.d/ext_sshd stop' 2>&1 2010-07-14 09:22:30|26430|vcld:main(165)|lastcheckin time updated for management node 2: 2010-07-14 09:22:30 2010-07-14 09:22:30|28645|68:72|reload|utils.pm:run_ssh_command(6262)|run_ssh_command output: |28645|68:72|reload| Stopping sshd: [ OK ] 2010-07-14 09:22:30|28645|68:72|reload|utils.pm:run_ssh_command(6276)|SSH command executed on vmwarerhel54-base33-v0, returning (0, "Stopping sshd: [ OK ]") 2010-07-14 09:22:30|28645|68:72|reload|utils.pm:run_ssh_command(6180)|executing SSH command on vmwarerhel54-base33-v0: |28645|68:72|reload| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p 22 -x vmwarerhel54-base33-v0 'sed -ie "/^AllowUsers .*/d" /etc/ssh/external_sshd_config' 2>&1 |28645|68:72|reload| ---- WARNING ---- |28645|68:72|reload| 2010-07-14 09:22:30|28645|68:72|reload|utils.pm:run_ssh_command(6252)|attempt 1/3: failed to execute SSH command on vmwarerhel54-base33-v0: sed -ie "/^AllowUsers .*/d" /etc/ssh/external_sshd_config, exit status: 255, SSH exits with the exit status of the remote command or with 255 if an error occurred, output: |28645|68:72|reload| ssh output (sed -ie "/...): ssh: connect to host vmwarerhel54-base33-v0 port 22: Connection refused |28645|68:72|reload| ( 0) utils.pm, notify (line: 737) |28645|68:72|reload| (-1) utils.pm, run_ssh_command (line: 6252) |28645|68:72|reload| (-2) Linux.pm, post_load (line: 199) |28645|68:72|reload| (-3) new.pm, reload_image (line: 692) |28645|68:72|reload| (-4) new.pm, process (line: 266) |28645|68:72|reload| (-5) vcld, make_new_child (line: 594) |28645|68:72|reload| (-6) vcld, main (line: 341) 2010-07-14 09:22:30|28645|68:72|reload|utils.pm:run_ssh_command(6172)|sleeping for 2 seconds before making next SSH attempt 2010-07-14 09:22:32|28645|68:72|reload|utils.pm:run_ssh_command(6184)|attempt 2/3: executing SSH command on vmwarerhel54-base33-v0: |28645|68:72|reload| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p 22 -x vmwarerhel54-base33-v0 'sed -ie "/^AllowUsers .*/d" /etc/ssh/external_sshd_config' 2>&1 |28645|68:72|reload| ---- WARNING ---- |28645|68:72|reload| 2010-07-14 09:22:32|28645|68:72|reload|utils.pm:run_ssh_command(6252)|attempt 2/3: failed to execute SSH command on vmwarerhel54-base33-v0: sed -ie "/^AllowUsers .*/d" /etc/ssh/external_sshd_config, exit status: 255, SSH exits with the exit status of the remote command or with 255 if an error occurred, output: |28645|68:72|reload| ssh output (sed -ie "/...): ssh: connect to host vmwarerhel54-base33-v0 port 22: Connection refused |28645|68:72|reload| ( 0) utils.pm, notify (line: 737) |28645|68:72|reload| (-1) utils.pm, run_ssh_command (line: 6252) |28645|68:72|reload| (-2) Linux.pm, post_load (line: 199) |28645|68:72|reload| (-3) new.pm, reload_image (line: 692) |28645|68:72|reload| (-4) new.pm, process (line: 266) |28645|68:72|reload| (-5) vcld, make_new_child (line: 594) |28645|68:72|reload| (-6) vcld, main (line: 341) 2010-07-14 09:22:32|28645|68:72|reload|utils.pm:run_ssh_command(6172)|sleeping for 15 seconds before making next SSH attempt 2010-07-14 09:22:35|26430|vcld:main(165)|lastcheckin time updated for management node 2: 2010-07-14 09:22:35 2010-07-14 09:22:40|26430|vcld:main(165)|lastcheckin time updated for management node 2: 2010-07-14 09:22:40 2010-07-14 09:22:45|26430|vcld:main(165)|lastcheckin time updated for management node 2: 2010-07-14 09:22:45 2010-07-14 09:22:47|28645|68:72|reload|utils.pm:run_ssh_command(6184)|attempt 3/3: executing SSH command on vmwarerhel54-base33-v0: |28645|68:72|reload| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p 22 -x vmwarerhel54-base33-v0 'sed -ie "/^AllowUsers .*/d" /etc/ssh/external_sshd_config' 2>&1 |28645|68:72|reload| ---- WARNING ---- |28645|68:72|reload| 2010-07-14 09:22:47|28645|68:72|reload|utils.pm:run_ssh_command(6252)|attempt 3/3: failed to execute SSH command on vmwarerhel54-base33-v0: sed -ie "/^AllowUsers .*/d" /etc/ssh/external_sshd_config, exit status: 255, SSH exits with the exit status of the remote command or with 255 if an error occurred, output: |28645|68:72|reload| ssh output (sed -ie "/...): ssh: connect to host vmwarerhel54-base33-v0 port 22: Connection refused |28645|68:72|reload| ( 0) utils.pm, notify (line: 737) |28645|68:72|reload| (-1) utils.pm, run_ssh_command (line: 6252) |28645|68:72|reload| (-2) Linux.pm, post_load (line: 199) |28645|68:72|reload| (-3) new.pm, reload_image (line: 692) |28645|68:72|reload| (-4) new.pm, process (line: 266) |28645|68:72|reload| (-5) vcld, make_new_child (line: 594) |28645|68:72|reload| (-6) vcld, main (line: 341) |28645|68:72|reload| ---- WARNING ---- |28645|68:72|reload| 2010-07-14 09:22:47|28645|68:72|reload|utils.pm:run_ssh_command(6285)|failed to run SSH command after 3 attempts, command: /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p 22 -x vmwarerhel54-base33-v0 'sed -ie "/^AllowUsers .*/d" /etc/ssh/external_sshd_config' 2>&1, exit status: 255, output: |28645|68:72|reload| ssh output (sed -ie "/...): ssh: connect to host vmwarerhel54-base33-v0 port 22: Connection refused |28645|68:72|reload| ( 0) utils.pm, notify (line: 737) |28645|68:72|reload| (-1) utils.pm, run_ssh_command (line: 6285) |28645|68:72|reload| (-2) Linux.pm, post_load (line: 199) |28645|68:72|reload| (-3) new.pm, reload_image (line: 692) |28645|68:72|reload| (-4) new.pm, process (line: 266) |28645|68:72|reload| (-5) vcld, make_new_child (line: 594) |28645|68:72|reload| (-6) vcld, main (line: 341) 2010-07-14 09:22:47|28645|68:72|reload|utils.pm:mail(1348)|SUCCESS -- Sending mail To: gha...@odu.edu, PROBLEM -- Linux.pm |28645|68:72|reload| ---- CRITICAL ---- |28645|68:72|reload| 2010-07-14 09:22:47|28645|68:72|reload|Linux.pm:post_load(204)|failed to clear AllowUsers from external_sshd_config |28645|68:72|reload| ( 0) utils.pm, notify (line: 737) |28645|68:72|reload| (-1) Linux.pm, post_load (line: 204) |28645|68:72|reload| (-2) new.pm, reload_image (line: 692) |28645|68:72|reload| (-3) new.pm, process (line: 266) |28645|68:72|reload| (-4) vcld, make_new_child (line: 594) |28645|68:72|reload| (-5) vcld, main (line: 341) 2010-07-14 09:22:47|28645|68:72|reload|Linux.pm:post_load(207)|calling clear_private_keys 2010-07-14 09:22:47|28645|68:72|reload|Linux.pm:clear_private_keys(224)|perparing to clear known identity keys 2010-07-14 09:22:47|28645|68:72|reload|utils.pm:run_ssh_command(6180)|executing SSH command on vmwarerhel54-base33-v0: |28645|68:72|reload| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p 22 -x vmwarerhel54-base33-v0 '/bin/rm -f /root/.ssh/id_rsa /root/.ssh/id_rsa.pub' 2>&1 |28645|68:72|reload| ---- WARNING ---- |28645|68:72|reload| 2010-07-14 09:22:47|28645|68:72|reload|utils.pm:run_ssh_command(6252)|attempt 1/3: failed to execute SSH command on vmwarerhel54-base33-v0: /bin/rm -f /root/.ssh/id_rsa /root/.ssh/id_rsa.pub, exit status: 255, SSH exits with the exit status of the remote command or with 255 if an error occurred, output: |28645|68:72|reload| ssh output (/bin/rm -f...): ssh: connect to host vmwarerhel54-base33-v0 port 22: Connection refused |28645|68:72|reload| ( 0) utils.pm, notify (line: 737) |28645|68:72|reload| (-1) utils.pm, run_ssh_command (line: 6252) |28645|68:72|reload| (-2) Linux.pm, clear_private_keys (line: 231) |28645|68:72|reload| (-3) Linux.pm, post_load (line: 209) |28645|68:72|reload| (-4) new.pm, reload_image (line: 692) |28645|68:72|reload| (-5) new.pm, process (line: 266) |28645|68:72|reload| (-6) vcld, make_new_child (line: 594) 2010-07-14 09:22:47|28645|68:72|reload|utils.pm:run_ssh_command(6172)|sleeping for 2 seconds before making next SSH attempt 2010-07-14 09:22:49|28645|68:72|reload|utils.pm:run_ssh_command(6184)|attempt 2/3: executing SSH command on vmwarerhel54-base33-v0: |28645|68:72|reload| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p 22 -x vmwarerhel54-base33-v0 '/bin/rm -f /root/.ssh/id_rsa /root/.ssh/id_rsa.pub' 2>&1 |28645|68:72|reload| ---- WARNING ---- |28645|68:72|reload| 2010-07-14 09:22:50|28645|68:72|reload|utils.pm:run_ssh_command(6252)|attempt 2/3: failed to execute SSH command on vmwarerhel54-base33-v0: /bin/rm -f /root/.ssh/id_rsa /root/.ssh/id_rsa.pub, exit status: 255, SSH exits with the exit status of the remote command or with 255 if an error occurred, output: |28645|68:72|reload| ssh output (/bin/rm -f...): ssh: connect to host vmwarerhel54-base33-v0 port 22: Connection refused |28645|68:72|reload| ( 0) utils.pm, notify (line: 737) |28645|68:72|reload| (-1) utils.pm, run_ssh_command (line: 6252) |28645|68:72|reload| (-2) Linux.pm, clear_private_keys (line: 231) |28645|68:72|reload| (-3) Linux.pm, post_load (line: 209) |28645|68:72|reload| (-4) new.pm, reload_image (line: 692) |28645|68:72|reload| (-5) new.pm, process (line: 266) |28645|68:72|reload| (-6) vcld, make_new_child (line: 594) 2010-07-14 09:22:50|28645|68:72|reload|utils.pm:run_ssh_command(6172)|sleeping for 15 seconds before making next SSH attempt 2010-07-14 09:22:50|26430|vcld:main(165)|lastcheckin time updated for management node 2: 2010-07-14 09:22:50 Gerhard Hartl Old Dominion University | ODU
