Summary: some issues fixed, but still can't reload/reserve an image. -- I've fixed my networking issues; I don't see anywhere in the docs about having the Private NICs in management node's /etc/hosts and the Publics in DNS but found it in the perl modules. -- Images (either windows 7 or 8.1, cygwin 64-bit or 32-bit) capture but are all unable to reload. This includes images taken from a working VCL (2.2 with ESXi 4.1) via "update/change image", shutting down the VM, exporting to an OVA file, deploying in the new one (VCL 2.3.3 with ESXi 5.5u1), running gen_node_key.sh against it, then trying to capture.
Thanks for any help anyone can give me. jeff Error email follows: From: [email protected] To: [email protected], Date: 08/07/2014 09:31 PM Subject: PROBLEM -- vcl|40:40|reload|State.pm|vm0401>node4-vcl|vmwarewin7-win8116-v0|vclreload reservation failed on vm0401: process failed after trying to load or make available ------------------------------------------------------------------------ time: 2014-08-07 21:31:08 caller: State.pm:reservation_failed(240) ( 0) State.pm, reservation_failed (line: 240) (-1) new.pm, process (line: 342) (-2) vcld, make_new_child (line: 571) (-3) vcld, main (line: 350) ------------------------------------------------------------------------ management node: vcl.marist.edu reservation PID: 5606 parent vcld PID: 17215 request ID: 40 reservation ID: 40 request state/laststate: reload/image request start time: 2014-08-07 21:30:57 request end time: 2014-08-07 22:00:57 for imaging: no log ID: none computer: vm0401 computer id: 2 computer type: virtualmachine computer eth0 MAC address: 00:50:56:04:01:01 computer eth1 MAC address: 00:50:56:04:01:02 computer private IP address: 172.20.101.41 computer public IP address: 10.10.4.41 computer in block allocation: no provisioning module: VCL::Module::Provisioning::VMware::VMware vm host: node4-vcl vm host ID: 1 vm host computer ID: 1 vm profile: VMware ESXi - local & network storage vm profile VM path: datastore1 vm profile repository path: <undefined> vm profile datastore path: nfs-datastore vm profile disk type: image: vmwarewin7-win8116-v0 image display name: win81 image ID: 16 image revision ID: 15 image size: 1450 MB use Sysprep: no root access: yes image owner ID: 1 image owner affiliation: Local image revision date created: 2014-08-07 21:19:07 image revision production: yes OS module: VCL::Module::OS::Windows::Version_6::7 user: vclreload user name: vcl reload user ID: 2 user affiliation: Local ------------------------------------------------------------------------ RECENT LOG ENTRIES FOR THIS PROCESS: |5606|40:40|reload| Inodes: Total: 2147483647 Free: 2147483647 2014-08-07 21:31:07|5606|40:40|reload|utils.pm:run_ssh_command(5034)|SSH command executed on node4-vcl, returning (0, "File: "/vmfs/volumes/datastore...") 2014-08-07 21:31:07|5606|40:40|reload|Linux.pm:get_available_space(1702)|space available on volume on node4-vcl containing '/vmfs/volumes/datastore1': 145,546,543,104 bytes - 142,135,296.0 KB - 138,804.0 MB - 135.55 GB 2014-08-07 21:31:07|5606|40:40|reload|utils.pm:run_ssh_command(4902)|executing SSH command on node4-vcl: |5606|40:40|reload| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x node4-vcl 'stat -f "/vmfs/volumes/nfs-datastore"' 2>&1 2014-08-07 21:31:07|5606|40:40|reload|utils.pm:run_ssh_command(5020)|run_ssh_command output: |5606|40:40|reload| File: "/vmfs/volumes/nfs-datastore" |5606|40:40|reload| ID: 1ac4ade4b8af7569 Namelen: 127 Type: nfs |5606|40:40|reload| Block size: 4096 |5606|40:40|reload| Blocks: Total: 999091486 Free: 632905864 Available: 582160930 |5606|40:40|reload| Inodes: Total: 507510784 Free: 507510639 2014-08-07 21:31:07|5606|40:40|reload|utils.pm:run_ssh_command(5034)|SSH command executed on node4-vcl, returning (0, "File: "/vmfs/volumes/nfs-datas...") 2014-08-07 21:31:07|5606|40:40|reload|Linux.pm:get_available_space(1702)|space available on volume on node4-vcl containing '/vmfs/volumes/nfs-datastore': 2,384,531,169,280 bytes - 2,328,643,720.0 KB - 2,274,066.1 MB - 2,220.77 GB - 2.17 TB 2014-08-07 21:31:07|5606|40:40|reload|VMware.pm:is_vmx_vmdk_volume_shared(2292)|checking if vmx and vmdk base directory paths appear to be on the same volume: |5606|40:40|reload| vmx base directory path: '/vmfs/volumes/datastore1' |5606|40:40|reload| vmdk base directory path: '/vmfs/volumes/nfs-datastore' |5606|40:40|reload| vmx volume total space: 146,565,758,976 bytes - 143,130,624.0 KB - 139,776.0 MB - 136.50 GB |5606|40:40|reload| vmdk volume total space: 4,092,278,726,656 bytes - 3,996,365,944.0 KB - 3,902,701.1 MB - 3,811.23 GB - 3.72 TB |5606|40:40|reload| vmx volume available space: 145,546,543,104 bytes - 142,135,296.0 KB - 138,804.0 MB - 135.55 GB |5606|40:40|reload| vmdk volume available space: 2,384,531,169,280 bytes - 2,328,643,720.0 KB - 2,274,066.1 MB - 2,220.77 GB - 2.17 TB 2014-08-07 21:31:07|5606|40:40|reload|VMware.pm:is_vmx_vmdk_volume_shared(2305)|vmx and vmdk base directory paths do not appear to be on the same volume based on the total and available space 2014-08-07 21:31:07|5606|40:40|reload|VMware.pm:get_vmdk_base_directory_path_shared(3566)|using VM profile datastore path as the vmdk base directory path: nfs-datastore 2014-08-07 21:31:07|5606|40:40|reload|utils.pm:run_ssh_command(4902)|executing SSH command on node4-vcl: |5606|40:40|reload| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x node4-vcl 'stat -f "/vmfs/volumes/datastore1"' 2>&1 2014-08-07 21:31:08|5606|40:40|reload|utils.pm:run_ssh_command(5020)|run_ssh_command output: |5606|40:40|reload| File: "/vmfs/volumes/datastore1" |5606|40:40|reload| ID: d5a1049e464bb5dc Namelen: 127 Type: vmfs |5606|40:40|reload| Block size: 1048576 |5606|40:40|reload| Blocks: Total: 139776 Free: 138804 Available: 138804 |5606|40:40|reload| Inodes: Total: 2147483647 Free: 2147483647 2014-08-07 21:31:08|5606|40:40|reload|utils.pm:run_ssh_command(5034)|SSH command executed on node4-vcl, returning (0, "File: "/vmfs/volumes/datastore...") 2014-08-07 21:31:08|5606|40:40|reload|Linux.pm:get_available_space(1702)|space available on volume on node4-vcl containing '/vmfs/volumes/datastore1': 145,546,543,104 bytes - 142,135,296.0 KB - 138,804.0 MB - 135.55 GB 2014-08-07 21:31:08|5606|40:40|reload|VMware.pm:get_vm_os_configuration(4895)|returning matching 'win7-x86_64' OS configuration: vmwarewin7, image architecture: x86_64 |5606|40:40|reload| : { |5606|40:40|reload| : "cpu_socket_limit" => 2, |5606|40:40|reload| : "ethernet-virtualDev" => "e1000", |5606|40:40|reload| : "guestOS" => "windows7-64", |5606|40:40|reload| : "memsize" => 2048, |5606|40:40|reload| : "scsi-virtualDev" => "lsiLogic" |5606|40:40|reload| : } 2014-08-07 21:31:08|5606|40:40|reload|VMware.pm:get_vm_ram(5055)|image minimum RAM value (0 MB) is too low for the windows7-64 guest OS, adjusting to 2048 MB 2014-08-07 21:31:08|5606|40:40|reload|VMware.pm:get_vm_additional_vmx_bytes_required(5721)|2147483648 additional bytes required for VM vmem file 2014-08-07 21:31:08|5606|40:40|reload|utils.pm:run_ssh_command(4902)|executing SSH command on node4-vcl: |5606|40:40|reload| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x node4-vcl 'stat /vmfs/volumes/datastore1/vm0401_16\-v0/vm0401_16\-v0\.vmx' 2>&1 2014-08-07 21:31:08|5606|40:40|reload|utils.pm:run_ssh_command(5020)|run_ssh_command output: |5606|40:40|reload| stat: can't stat '/vmfs/volumes/datastore1/vm0401_16-v0/vm0401_16-v0.vmx': No such file or directory 2014-08-07 21:31:08|5606|40:40|reload|utils.pm:run_ssh_command(5030)|SSH command executed on node4-vcl, command: |5606|40:40|reload| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x node4-vcl 'stat /vmfs/volumes/datastore1/vm0401_16\-v0/vm0401_16\-v0\.vmx' 2>&1 |5606|40:40|reload| returning (1, "stat: can't stat '/vmfs/volume...") 2014-08-07 21:31:08|5606|40:40|reload|VMware.pm:get_vmdk_base_directory_path_shared(3566)|using VM profile datastore path as the vmdk base directory path: nfs-datastore 2014-08-07 21:31:08|5606|40:40|reload|utils.pm:run_ssh_command(4902)|executing SSH command on node4-vcl: |5606|40:40|reload| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x node4-vcl 'stat /vmfs/volumes/nfs\-datastore/vmwarewin7\-win8116\-v0/vmwarewin7\-win8116\-v0\.vmdk' 2>&1 2014-08-07 21:31:08|5606|40:40|reload|utils.pm:run_ssh_command(5020)|run_ssh_command output: |5606|40:40|reload| File: /vmfs/volumes/nfs-datastore/vmwarewin7-win8116-v0/vmwarewin7-win8116-v0.vmdk |5606|40:40|reload| Size: 557 Blocks: 8 IO Block: 131072 regular file |5606|40:40|reload| Device: b8af75691ac4ade4h/13307984518184218084d Inode: 232554499 Links: 1 |5606|40:40|reload| Access: (0600/-rw-------) Uid: ( 0/ root) Gid: ( 0/ root) |5606|40:40|reload| Access: 2014-08-08 01:30:45.000000000 |5606|40:40|reload| Modify: 2014-08-08 01:30:45.000000000 |5606|40:40|reload| Change: 2014-08-08 01:30:45.000000000 2014-08-07 21:31:08|5606|40:40|reload|utils.pm:run_ssh_command(5034)|SSH command executed on node4-vcl, returning (0, "File: /vmfs/volumes/nfs-datast...") 2014-08-07 21:31:08|5606|40:40|reload|Linux.pm:file_exists(1449)|'/vmfs/volumes/nfs-datastore/vmwarewin7-win8116-v0/vmwarewin7-win8116-v0.vmdk' exists on node4-vcl, files: 1, directories: 0, links: 0 |5606|40:40|reload| ---- WARNING ---- |5606|40:40|reload| 2014-08-07 21:31:08|5606|40:40|reload|VMware.pm:check_vmhost_disk_space(2393)|failed to determine additional bytes required for the vmx and vmdk directories on VM host node4-vcl |5606|40:40|reload| ( 0) VMware.pm, check_vmhost_disk_space (line: 2393) |5606|40:40|reload| (-1) VMware.pm, load (line: 475) |5606|40:40|reload| (-2) new.pm, reload_image (line: 671) |5606|40:40|reload| (-3) new.pm, process (line: 291) |5606|40:40|reload| (-4) vcld, make_new_child (line: 571) |5606|40:40|reload| (-5) vcld, main (line: 350) |5606|40:40|reload| ---- WARNING ---- |5606|40:40|reload| 2014-08-07 21:31:08|5606|40:40|reload|VMware.pm:load(477)|failed to determine if enough disk space is available on VM host node4-vcl |5606|40:40|reload| ( 0) VMware.pm, load (line: 477) |5606|40:40|reload| (-1) new.pm, reload_image (line: 671) |5606|40:40|reload| (-2) new.pm, process (line: 291) |5606|40:40|reload| (-3) vcld, make_new_child (line: 571) |5606|40:40|reload| (-4) vcld, main (line: 350) |5606|40:40|reload| ---- WARNING ---- |5606|40:40|reload| 2014-08-07 21:31:08|5606|40:40|reload|new.pm:reload_image(676)|vmwarewin7-win8116-v0 failed to load on vm0401, returning |5606|40:40|reload| ( 0) new.pm, reload_image (line: 676) |5606|40:40|reload| (-1) new.pm, process (line: 291) |5606|40:40|reload| (-2) vcld, make_new_child (line: 571) |5606|40:40|reload| (-3) vcld, main (line: 350) 2014-08-07 21:31:08|5606|40:40|reload|utils.pm:insertloadlog(3665)|inserted computer=2, loadimagefailed, vmwarewin7-win8116-v0 failed to load on vm0401 |5606|40:40|reload| ---- WARNING ---- |5606|40:40|reload| 2014-08-07 21:31:08|5606|40:40|reload|new.pm:process(339)|failed to load vm0401 with vmwarewin7-win8116-v0 |5606|40:40|reload| ( 0) new.pm, process (line: 339) |5606|40:40|reload| (-1) vcld, make_new_child (line: 571) |5606|40:40|reload| (-2) vcld, main (line: 350) 2014-08-07 21:31:08|5606|40:40|reload|DataStructure.pm:get_computer_state_name(2436)|attempting to retrieve current state of computer vm0401 from the database 2014-08-07 21:31:08|5606|40:40|reload|DataStructure.pm:get_computer_state_name(2467)|retrieved current state of computer vm0401 from the database: reloading 2014-08-07 21:31:08|5606|40:40|reload|DataStructure.pm:_automethod(847)|data structure updated, hash path: $self->request_data->{reservation}{40}{computer}{state}{name}, data identifier: computer_state_name, data: |5606|40:40|reload| : "reloading" 2014-08-07 21:31:08|5606|40:40|reload|DataStructure.pm:get_computer_private_ip_address(1641)|attempting to retrieve private IP address for computer: vm0401 2014-08-07 21:31:08|5606|40:40|reload|OS.pm:get_file_contents(1897)|retrieved 77 lines from file on vcl: '/etc/hosts' 2014-08-07 21:31:08|5606|40:40|reload|DataStructure.pm:get_computer_private_ip_address(1697)|returning IP address from /etc/hosts file: 172.20.101.41 2014-08-07 21:31:08|5606|40:40|reload|utils.pm:is_inblockrequest(5760)|zero rows were returned from database select 2014-08-07 21:31:08|5606|40:40|reload|DataStructure.pm:get_image_affiliation_name(2121)|image owner id: 1 2014-08-07 21:31:08|5606|40:40|reload|DataStructure.pm:retrieve_user_data(1412)|attempting to retrieve and store data for user: user.id = '1' 2014-08-07 21:31:08|5606|40:40|reload|DataStructure.pm:retrieve_user_data(1475)|data has been retrieved for user: admin (id: 1)
