Thanks for all the help, We were able to fix the issue with the banner connection. I have also made some the changes you suggested and noticed an decrease of failed VM's due to SSH time out.
The system is doing a Firewall check each time when a VM spins up, Is there any way to disable this. I notice it takes longer to get a VM up when it does this full check. -Alex On Mon, Mar 4, 2013 at 6:29 PM, Dmitri Chebotarov <[email protected]> wrote: > Alex, > > You can try to increase ssh_command ConnectionTimeout. > It's in utils.pm file, in sub run_ssh_command, variable name 'my > $ssh_command=…'. > I have mine set to ConnectTimeout=10. > > Can you also check that DNS server used by VCL (VM host, mgm node) is > operational? > > > Thank you, > -- > Dmitri Chebotarov > Virtual Computing Lab Systems Engineer, TSD - Ent Servers & Messaging > 223 Aquia Building, Ffx, MSN: 1B5 > Phone: (703) 993-6175 > Fax: (703) 993-3404 > > > On Monday, March 4, 2013 at 19:40 , Young h Oh wrote: > >> Alex, >> >> Have you checked the process of "ssh" or "vcld" in your management node? >> ($ps -aux |grep ssh). >> >> If there is any open ssh process to the blades, it could cause the problem. >> >> Hope this helps. >> >> Thanks, >> -------------------------------------------------------------------- >> Young Hyun Oh >> IBM Software Group >> >> Alexander Patterson ---03/04/2013 05:16:41 PM---I'm having errors with >> Connection timed out during banner exchange errors currently >> >> From: Alexander Patterson <[email protected] >> (mailto:[email protected])> >> To: [email protected] (mailto:[email protected]), >> Cc: "[email protected] (mailto:[email protected])" >> <[email protected] (mailto:[email protected])> >> Date: 03/04/2013 05:16 PM >> Subject: Re: VCL Error Problem >> >> >> >> I'm having errors with Connection timed out during banner exchange >> errors currently >> >> I can SSH from my management Node to my Blades no problem >> I do have a user name and password for my VM host profile, but it >> shouldn't need one from the management node to the blades >> Correct I don't have SDK installed. >> >> I'm in the VCL IRC chat >> http://webchat.freenode.net/ >> #asfvcl >> >> Trying to trouble shoot this issue as well are down and have students >> and professors waiting for it to come back up. >> -Alex >> >> On Mon, Mar 4, 2013 at 11:30 AM, Dmitri Chebotarov <[email protected] >> (mailto:[email protected])> wrote: >> > Alex, >> > >> > I believe vcld uses different key located in /etc/vcl/vcl.key. >> > Try to run this command from management node as 'root' and see if you can >> > connect to the VM host: >> > >> > /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o >> > UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=10 >> > -l root -p 22 -x 'VM-host (vclnfs01?)' >> > >> > Also, do you have username/password set in the VM host profile? >> > This info is used when the management node has SDK installed - your system >> > doesn't have it. >> > Try to set username/password to blank if you have anything in those >> > fields, restart vcld and try again. >> > >> > Thanks. >> > >> > On Mar 3, 2013, at 12:17 , Alexander Patterson >> > <[email protected] >> > (mailto:[email protected])> wrote: >> > >> > > I changed it to 600 and I can ssh from my management node to my VM's >> > > >> > > I checked the VM's and the SSH is on. >> > > >> > > My whole system is down and I'm getting the same error on all my >> > > blades. Could anyone shed some light on what the issue is? >> > > >> > > VCL::inuse object could not be created and initialized >> > > ------------------------------------------------------------------------ >> > > time: 2013-03-03 09:17:11 >> > > caller: vcld:make_new_child(574) >> > > ( 0) vcld, make_new_child (line: 574) >> > > (-1) vcld, main (line: 350) >> > > ------------------------------------------------------------------------ >> > > management node: vclnfs01.calstate.edu >> > > reservation PID: 18366 >> > > 2013-03-03 >> > > 09:16:55|18366|107987:101619|inuse|vcld:make_new_child(558)|vcld >> > > environment variable set to 0 for this process >> > > 2013-03-03 >> > > 09:16:55|18366|107987:101619|inuse|DataStructure.pm:_initialize(643)|image >> > > ID argument was specified: noimage, DataStructure object will contain >> > > image information for the production imagerevision of this image >> > > 2013-03-03 >> > > 09:16:55|18366|107987:101619|inuse|DataStructure.pm:_initialize(674)|retrieved >> > > data for imagerevision ID: 4 >> > > 2013-03-03 >> > > 09:16:55|18366|107987:101619|inuse|DataStructure.pm:_initialize(690)|retrieved >> > > data for image ID: 4 >> > > 2013-03-03 >> > > 09:16:55|18366|107987:101619|inuse|Module.pm:create_mn_os_object(376)|VCL::Module::OS::Linux::ManagementNode >> > > module loaded >> > > 2013-03-03 >> > > 09:16:55|18366|107987:101619|inuse|Module.pm:new(196)|VCL::Module::OS::Linux::ManagementNode >> > > object created for image noimage, address: f8f34c0 >> > > 2013-03-03 >> > > 09:16:55|18366|107987:101619|inuse|DataStructure.pm:_automethod(847)|data >> > > structure updated, hash path: >> > > $self->request_data->{reservation}{0}{computer}{hostname}, data >> > > identifier: computer_hostname, data: >> > > |18366|107987:101619|inuse| : "vclnfs01.calstate.edu" >> > > 2013-03-03 >> > > 09:16:55|18366|107987:101619|inuse|DataStructure.pm:_automethod(847)|data >> > > structure updated, hash path: >> > > $self->request_data->{reservation}{0}{computer}{NODENAME}, data >> > > identifier: computer_node_name, data: >> > > |18366|107987:101619|inuse| : "vclnfs01" >> > > 2013-03-03 >> > > 09:16:55|18366|107987:101619|inuse|DataStructure.pm:_automethod(847)|data >> > > structure updated, hash path: >> > > $self->request_data->{reservation}{0}{computer}{SHORTNAME}, data >> > > identifier: computer_short_name, data: >> > > |18366|107987:101619|inuse| : "vclnfs01" >> > > 2013-03-03 >> > > 09:16:55|18366|107987:101619|inuse|DataStructure.pm:_automethod(847)|data >> > > structure updated, hash path: >> > > $self->request_data->{reservation}{0}{computer}{IPaddress}, data >> > > identifier: computer_ip_address, data: >> > > |18366|107987:101619|inuse| : "137.145.56.10" >> > > 2013-03-03 >> > > 09:16:55|18366|107987:101619|inuse|Module.pm:create_mn_os_object(381)|VCL::Module::OS::Linux::ManagementNode >> > > OS object created, address: f8f34c0 >> > > 2013-03-03 >> > > 09:16:55|18366|107987:101619|inuse|Module.pm:new(192)|VCL::inuse >> > > object created for state inuse, address: f24deb0 >> > > 2013-03-03 >> > > 09:16:55|18366|107987:101619|inuse|State.pm:initialize(88)|obtained >> > > a database handle for this state process, stored as $ENV{dbh} >> > > 2013-03-03 >> > > 09:16:55|18366|107987:101619|inuse|State.pm:check_image_os(771)|no >> > > corrections need to be made, not an imaging request, returning 1 >> > > 2013-03-03 >> > > 09:16:55|18366|107987:101619|inuse|DataStructure.pm:is_parent_reservation(1004)|returning >> > > true: parent reservation ID for this request: 101619 >> > > 2013-03-03 >> > > 09:16:55|18366|107987:101619|inuse|utils.pm:rename_vcld_process(6752)|reservation >> > > count: 1 >> > > 2013-03-03 >> > > 09:16:55|18366|107987:101619|inuse|utils.pm:rename_vcld_process(6771)|PARENTIMAGE: >> > > 1 >> > > 2013-03-03 >> > > 09:16:55|18366|107987:101619|inuse|utils.pm:rename_vcld_process(6772)|SUBIMAGE: >> > > 0 >> > > 2013-03-03 >> > > 09:16:55|18366|107987:101619|inuse|utils.pm:rename_vcld_process(6790)|renamed >> > > process to 'vcld 107987:101619 inuse vmguest-113>vclesxi21 >> > > vmwarewincw7vista-ITSTrain150-v6 sd8454' >> > > 2013-03-03 >> > > 09:16:55|18366|107987:101619|inuse|DataStructure.pm:is_parent_reservation(1004)|returning >> > > true: parent reservation ID for this request: 101619 >> > > 2013-03-03 >> > > 09:16:55|18366|107987:101619|inuse|DataStructure.pm:is_parent_reservation(1004)|returning >> > > true: parent reservation ID for this request: 101619 >> > > 2013-03-03 >> > > 09:16:55|18366|107987:101619|inuse|Module.pm:create_os_object(308)|VCL::Module::OS::Windows::Version_6::Vista >> > > module loaded >> > > 2013-03-03 >> > > 09:16:55|18366|107987:101619|inuse|Module.pm:new(196)|VCL::Module::OS::Windows::Version_6::Vista >> > > object created for image vmwarewincw7vista-ITSTrain150-v6, address: >> > > faa2760 >> > > 2013-03-03 >> > > 09:16:55|18366|107987:101619|inuse|Windows.pm:initialize(214)|beginning >> > > Windows module initialization >> > > 2013-03-03 >> > > 09:16:55|18366|107987:101619|inuse|Windows.pm:initialize(224)|Windows >> > > module initialization complete >> > > 2013-03-03 >> > > 09:16:55|18366|107987:101619|inuse|Module.pm:create_os_object(314)|VCL::Module::OS::Windows::Version_6::Vista >> > > OS object created, address: faa2760 >> > > 2013-03-03 >> > > 09:16:55|18366|107987:101619|inuse|State.pm:initialize(125)|computer >> > > is a VM, attempting to create VM host OS object >> > > 2013-03-03 >> > > 09:16:55|18366|107987:101619|inuse|DataStructure.pm:_initialize(619)|computer >> > > ID argument was specified, retrieving data for computer ID: 3 >> > > 2013-03-03 >> > > 09:16:55|18366|107987:101619|inuse|DataStructure.pm:_initialize(643)|image >> > > ID argument was specified: 4, DataStructure object will contain image >> > > information for the production imagerevision of this image >> > > 2013-03-03 >> > > 09:16:55|18366|107987:101619|inuse|DataStructure.pm:_initialize(674)|retrieved >> > > data for imagerevision ID: 4 >> > > 2013-03-03 >> > > 09:16:55|18366|107987:101619|inuse|DataStructure.pm:_initialize(690)|retrieved >> > > data for image ID: 4 >> > > 2013-03-03 >> > > 09:16:55|18366|107987:101619|inuse|Module.pm:create_vmhost_os_object(454)|attempting >> > > to load VM host OS module: VCL::Module::OS::Linux::UnixLab (image: 4) >> > > 2013-03-03 >> > > 09:16:55|18366|107987:101619|inuse|Module.pm:create_vmhost_os_object(460)|VM >> > > host OS module loaded: VCL::Module::OS::Linux::UnixLab >> > > 2013-03-03 >> > > 09:16:55|18366|107987:101619|inuse|Module.pm:new(196)|VCL::Module::OS::Linux::UnixLab >> > > object created for image noimage, address: e1de370 >> > > 2013-03-03 >> > > 09:16:55|18366|107987:101619|inuse|Module.pm:create_vmhost_os_object(465)|VCL::Module::OS::Linux::UnixLab >> > > OS object created, address: e1de370 >> > > 2013-03-03 >> > > 09:16:55|18366|107987:101619|inuse|Module.pm:create_provisioning_object(519)|VCL::Module::Provisioning::VMware::VMware >> > > module loaded >> > > 2013-03-03 >> > > 09:16:55|18366|107987:101619|inuse|Module.pm:new(200)|VCL::Module::Provisioning::VMware::VMware >> > > object created for computer vmguest-113, address: 1015aa70 >> > > 2013-03-03 >> > > 09:16:55|18366|107987:101619|inuse|VMware.pm:initialize(267)|initializing >> > > VCL::Module::Provisioning::VMware::VMware object >> > > 2013-03-03 >> > > 09:16:55|18366|107987:101619|inuse|DataStructure.pm:_initialize(619)|computer >> > > ID argument was specified, retrieving data for computer ID: 3 >> > > 2013-03-03 >> > > 09:16:55|18366|107987:101619|inuse|DataStructure.pm:_initialize(643)|image >> > > ID argument was specified: 4, DataStructure object will contain image >> > > information for the production imagerevision of this image >> > > 2013-03-03 >> > > 09:16:55|18366|107987:101619|inuse|DataStructure.pm:_initialize(674)|retrieved >> > > data for imagerevision ID: 4 >> > > 2013-03-03 >> > > 09:16:55|18366|107987:101619|inuse|DataStructure.pm:_initialize(690)|retrieved >> > > data for image ID: 4 >> > > 2013-03-03 >> > > 09:16:55|18366|107987:101619|inuse|VMware.pm:get_vmhost_datastructure(1397)|created >> > > DataStructure object for VM host: vclesxi21 >> > > 2013-03-03 >> > > 09:16:55|18366|107987:101619|inuse|VMware.pm:initialize(287)|VM >> > > profile assigned to vclesxi21: VMware ESXI Network Storage >> > > 2013-03-03 >> > > 09:16:55|18366|107987:101619|inuse|DataStructure.pm:_initialize(619)|computer >> > > ID argument was specified, retrieving data for computer ID: 3 >> > > 2013-03-03 >> > > 09:16:55|18366|107987:101619|inuse|DataStructure.pm:_initialize(643)|image >> > > ID argument was specified: 4, DataStructure object will contain image >> > > information for the production imagerevision of this image >> > > 2013-03-03 >> > > 09:16:55|18366|107987:101619|inuse|DataStructure.pm:_initialize(674)|retrieved >> > > data for imagerevision ID: 4 >> > > 2013-03-03 >> > > 09:16:55|18366|107987:101619|inuse|DataStructure.pm:_initialize(690)|retrieved >> > > data for image ID: 4 >> > > 2013-03-03 >> > > 09:16:55|18366|107987:101619|inuse|VMware.pm:get_vmhost_datastructure(1397)|created >> > > DataStructure object for VM host: vclesxi21 >> > > 2013-03-03 >> > > 09:16:55|18366|107987:101619|inuse|VMware.pm:get_vmhost_api_object(1500)|attempting >> > > to load VMware control module: >> > > VCL::Module::Provisioning::VMware::vSphere_SDK >> > > 2013-03-03 >> > > 09:16:55|18366|107987:101619|inuse|VMware.pm:get_vmhost_api_object(1506)|loaded >> > > VMware control module: VCL::Module::Provisioning::VMware::vSphere_SDK >> > > 2013-03-03 >> > > 09:16:55|18366|107987:101619|inuse|Module.pm:new(200)|VCL::Module::Provisioning::VMware::vSphere_SDK >> > > object created for computer vmguest-113, address: 10597100 >> > > 2013-03-03 >> > > 09:16:55|18366|107987:101619|inuse|vSphere_SDK.pm:initialize(98)|vSphere >> > > SDK for Perl does not appear to be installed on this managment node, >> > > unable to load VMware vSphere SDK Perl modules, error: >> > > |18366|107987:101619|inuse| Can't locate VMware/VIRuntime.pm in @INC >> > > (@INC contains: /usr/local/vcl/bin/../../../.. >> > > /usr/local/vcl/bin/../.. /usr/local/vcl/bin/../../.. >> > > /usr/local/vcl/bin/../../../../.. /usr/local/vcl/bin/.. >> > > /usr/local/vcl/bin/../lib >> > > /usr/lib64/perl5/site_perl/5.8.8/x86_64-linux-thread-multi >> > > /usr/lib/perl5/site_perl/5.8.8 /usr/lib/perl5/site_perl >> > > /usr/lib64/perl5/vendor_perl/5.8.8/x86_64-linux-thread-multi >> > > /usr/lib/perl5/vendor_perl/5.8.8 /usr/lib/perl5/vendor_perl >> > > /usr/lib64/perl5/5.8.8/x86_64-linux-thread-multi /usr/lib/perl5/5.8.8 >> > > .) at (eval 311) line 1. >> > > |18366|107987:101619|inuse| BEGIN failed--compilation aborted at (eval >> > > 311) line 1. >> > > 2013-03-03 >> > > 09:16:55|18366|107987:101619|inuse|VMware.pm:get_vmhost_api_object(1520)|API >> > > object could not be created: >> > > VCL::Module::Provisioning::VMware::vSphere_SDK >> > > 2013-03-03 >> > > 09:17:11|18366|107987:101619|inuse|OS.pm:is_ssh_responding(682)|vclesxi21 >> > > is NOT responding to SSH, SSH command failed, port 22: open, port 24: >> > > closed >> > > 2013-03-03 >> > > 09:17:11|18366|107987:101619|inuse|VMware.pm:initialize(307)|unable >> > > to control OS of VM host vclesxi21 using >> > > VCL::Module::OS::Linux::UnixLab OS object because VM host is not >> > > responding to SSH >> > > |18366|107987:101619|inuse| ---- WARNING ---- >> > > |18366|107987:101619|inuse| 2013-03-03 >> > > 09:17:11|18366|107987:101619|inuse|Module.pm:create_provisioning_object(530)|provisioning >> > > object could not be created, returning 0 >> > > |18366|107987:101619|inuse| ( 0) Module.pm, create_provisioning_object >> > > (line: 530) >> > > |18366|107987:101619|inuse| (-1) State.pm, initialize (line: 136) >> > > |18366|107987:101619|inuse| (-2) Module.pm, new (line: 207) >> > > |18366|107987:101619|inuse| (-3) vcld, make_new_child (line: 567) >> > > |18366|107987:101619|inuse| (-4) vcld, main (line: 350) >> > > |18366|107987:101619|inuse| ---- WARNING ---- >> > > |18366|107987:101619|inuse| 2013-03-03 >> > > 09:17:11|18366|107987:101619|inuse|State.pm:initialize(145)|failed to >> > > create provisioning object >> > > |18366|107987:101619|inuse| ( 0) State.pm, initialize (line: 145) >> > > |18366|107987:101619|inuse| (-1) Module.pm, new (line: 207) >> > > |18366|107987:101619|inuse| (-2) vcld, make_new_child (line: 567) >> > > |18366|107987:101619|inuse| (-3) vcld, main (line: 350) >> > > 2013-03-03 >> > > 09:17:11|18366|107987:101619|inuse|utils.pm:delete_computerloadlog_reservation(6396)|removing >> > > computerloadlog entries matching loadstate = begin >> > > 2013-03-03 >> > > 09:17:11|18366|107987:101619|inuse|utils.pm:delete_computerloadlog_reservation(6443)|deleted >> > > rows from computerloadlog for reservation id=101619 >> > > 2013-03-03 >> > > 09:17:11|18366|107987:101619|inuse|State.pm:DESTROY(929)|VCL::inuse >> > > process duration: 16 seconds >> > > 2013-03-03 >> > > 09:17:11|18366|107987:101619|inuse|utils.pm:get_management_node_info(5456)|retrieving >> > > current management node info for 'vclnfs01.calstate.edu' from >> > > database, cached data is stale: 60 seconds old >> > > 2013-03-03 >> > > 09:17:11|18366|107987:101619|inuse|utils.pm:getnewdbh(2474)|unable >> > > to use database handle stored in $ENV{dbh} >> > > 2013-03-03 >> > > 09:17:11|18366|107987:101619|inuse|utils.pm:getnewdbh(2531)|database >> > > handle stored in $ENV{dbh} >> > > 2013-03-03 >> > > 09:17:11|18366|107987:101619|inuse|utils.pm:get_management_node_info(5603)|retrieved >> > > management node info: 'vclnfs01.calstate.edu' (vclnfs01) >> > > 2013-03-03 >> > > 09:17:11|18366|107987:101619|inuse|DataStructure.pm:get_computer_private_ip_address(1641)|attempting >> > > to retrieve private IP address for computer: vmguest-113 >> > > 2013-03-03 >> > > 09:17:11|18366|107987:101619|inuse|OS.pm:get_file_contents(1897)|retrieved >> > > 531 lines from file on vclnfs01: '/etc/hosts' >> > > 2013-03-03 >> > > 09:17:11|18366|107987:101619|inuse|DataStructure.pm:get_computer_private_ip_address(1697)|returning >> > > IP address from /etc/hosts file: 10.10.10.192 >> > > 2013-03-03 >> > > 09:17:11|18366|107987:101619|inuse|utils.pm:is_inblockrequest(5760)|zero >> > > rows were returned from database select >> > > 2013-03-03 >> > > 09:17:11|18366|107987:101619|inuse|DataStructure.pm:get_image_affiliation_name(2121)|image >> > > owner id: 10 >> > > 2013-03-03 >> > > 09:17:11|18366|107987:101619|inuse|DataStructure.pm:retrieve_user_data(1412)|attempting >> > > to retrieve and store data for user: user.id = '10' >> > > 2013-03-03 >> > > 09:17:11|18366|107987:101619|inuse|DataStructure.pm:retrieve_user_data(1475)|data >> > > has been retrieved for user: wv4973 (id: 10) >> > > >> > > On Sat, Mar 2, 2013 at 4:17 PM, Alexander Patterson >> > > <[email protected] >> > > (mailto:[email protected])> wrote: >> > > > I'm also getting >> > > > >> > > > @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ >> > > > @ WARNING: UNPROTECTED PRIVATE KEY FILE! @ >> > > > @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ >> > > > Permissions 0744 for '/root/.ssh/id_rsa' are too open. >> > > > It is required that your private key files are NOT accessible by >> > > > others. >> > > > This private key will be ignored. >> > > > bad permissions: ignore key: /root/.ssh/id_rsa >> > > > >> > > > >> > > > It's asking for a password when I try to ssh from my management node >> > > > to my blades. I am logged in as the root admin >> > > > >> > > > On Sat, Mar 2, 2013 at 11:28 AM, Creech, David <[email protected] >> > > > (mailto:[email protected])> wrote: >> > > > > I ran into this issue a couple of weeks ago and with the assistance >> > > > > of Andy (NCSU) was able to determine the issue is related to one of >> > > > > our blades not being accessible via SSH (management node uses SSH to >> > > > > "check" on the reservation). >> > > > > >> > > > > Looking through the error log I see where it mentions vclesxi26 >> > > > > isn't accessible via SSH. >> > > > > >> > > > > > 2013-03-02 >> > > > > > 10:45:17|4983|108039:101671|inuse|OS.pm:is_ssh_responding(682)|vclesxi26 >> > > > > > is NOT responding to SSH, SSH command failed, port 22: open, port >> > > > > > 24: >> > > > > > closed >> > > > > >> > > > > >> > > > > Hope this helps... >> > > > > >> > > > > Thanks, >> > > > > >> > > > > David Creech >> > > > > ECU - ITCS Academic Technologies >> > > > > >> > > > > Sent from my iPhone >> > > > > >> > > > > On Mar 2, 2013, at 1:46 PM, "Alexander Patterson" >> > > > > <[email protected] >> > > > > (mailto:[email protected])> wrote: >> > > > > >> > > > > > VCL::inuse object could not be created and initialized >> > > > > > ------------------------------------------------------------------------ >> > > > > > time: 2013-03-02 10:45:17 >> > > > > > caller: vcld:make_new_child(574) >> > > > > > ( 0) vcld, make_new_child (line: 574) >> > > > > > (-1) vcld, main (line: 350) >> > > > > > ------------------------------------------------------------------------ >> > > > > > management node: vclnfs01.calstate.edu >> > > > > > reservation PID: 4983 >> > > > > > 2013-03-02 >> > > > > > 10:45:02|4983|108039:101671|inuse|vcld:make_new_child(558)|vcld >> > > > > > environment variable set to 0 for this process >> > > > > > 2013-03-02 >> > > > > > 10:45:02|4983|108039:101671|inuse|DataStructure.pm:_initialize(643)|image >> > > > > > ID argument was specified: noimage, DataStructure object will >> > > > > > contain >> > > > > > image information for the production imagerevision of this image >> > > > > > 2013-03-02 >> > > > > > 10:45:02|4983|108039:101671|inuse|DataStructure.pm:_initialize(674)|retrieved >> > > > > > data for imagerevision ID: 4 >> > > > > > 2013-03-02 >> > > > > > 10:45:02|4983|108039:101671|inuse|DataStructure.pm:_initialize(690)|retrieved >> > > > > > data for image ID: 4 >> > > > > > 2013-03-02 >> > > > > > 10:45:02|4983|108039:101671|inuse|Module.pm:create_mn_os_object(376)|VCL::Module::OS::Linux::ManagementNode >> > > > > > module loaded >> > > > > > 2013-03-02 >> > > > > > 10:45:02|4983|108039:101671|inuse|Module.pm:new(196)|VCL::Module::OS::Linux::ManagementNode >> > > > > > object created for image noimage, address: 18a0b840 >> > > > > > 2013-03-02 >> > > > > > 10:45:02|4983|108039:101671|inuse|DataStructure.pm:_automethod(847)|data >> > > > > > structure updated, hash path: >> > > > > > $self->request_data->{reservation}{0}{computer}{hostname}, data >> > > > > > identifier: computer_hostname, data: >> > > > > > |4983|108039:101671|inuse| : "vclnfs01.calstate.edu" >> > > > > > 2013-03-02 >> > > > > > 10:45:02|4983|108039:101671|inuse|DataStructure.pm:_automethod(847)|data >> > > > > > structure updated, hash path: >> > > > > > $self->request_data->{reservation}{0}{computer}{NODENAME}, data >> > > > > > identifier: computer_node_name, data: >> > > > > > |4983|108039:101671|inuse| : "vclnfs01" >> > > > > > 2013-03-02 >> > > > > > 10:45:02|4983|108039:101671|inuse|DataStructure.pm:_automethod(847)|data >> > > > > > structure updated, hash path: >> > > > > > $self->request_data->{reservation}{0}{computer}{SHORTNAME}, data >> > > > > > identifier: computer_short_name, data: >> > > > > > |4983|108039:101671|inuse| : "vclnfs01" >> > > > > > 2013-03-02 >> > > > > > 10:45:02|4983|108039:101671|inuse|DataStructure.pm:_automethod(847)|data >> > > > > > structure updated, hash path: >> > > > > > $self->request_data->{reservation}{0}{computer}{IPaddress}, data >> > > > > > identifier: computer_ip_address, data: >> > > > > > |4983|108039:101671|inuse| : "137.145.56.10" >> > > > > > 2013-03-02 >> > > > > > 10:45:02|4983|108039:101671|inuse|Module.pm:create_mn_os_object(381)|VCL::Module::OS::Linux::ManagementNode >> > > > > > OS object created, address: 18a0b840 >> > > > > > 2013-03-02 >> > > > > > 10:45:02|4983|108039:101671|inuse|Module.pm:new(192)|VCL::inuse >> > > > > > object created for state inuse, address: 183826f0 >> > > > > > 2013-03-02 >> > > > > > 10:45:02|4983|108039:101671|inuse|State.pm:initialize(88)|obtained >> > > > > > a database handle for this state process, stored as $ENV{dbh} >> > > > > > 2013-03-02 >> > > > > > 10:45:02|4983|108039:101671|inuse|State.pm:check_image_os(771)|no >> > > > > > corrections need to be made, not an imaging request, returning 1 >> > > > > > 2013-03-02 >> > > > > > 10:45:02|4983|108039:101671|inuse|DataStructure.pm:is_parent_reservation(1004)|returning >> > > > > > true: parent reservation ID for this request: 101671 >> > > > > > 2013-03-02 >> > > > > > 10:45:02|4983|108039:101671|inuse|utils.pm:rename_vcld_process(6752)|reservation >> > > > > > count: 1 >> > > > > > 2013-03-02 >> > > > > > 10:45:02|4983|108039:101671|inuse|utils.pm:rename_vcld_process(6771)|PARENTIMAGE: >> > > > > > 1 >> > > > > > 2013-03-02 >> > > > > > 10:45:02|4983|108039:101671|inuse|utils.pm:rename_vcld_process(6772)|SUBIMAGE: >> > > > > > 0 >> > > > > > 2013-03-02 >> > > > > > 10:45:02|4983|108039:101671|inuse|utils.pm:rename_vcld_process(6790)|renamed >> > > > > > process to 'vcld 108039:101671 inuse vmguest-196>vclesxi26 >> > > > > > vmwarewinxp-CSUEBSPSS18withOffice2007124-v9 iy8492' >> > > > > > 2013-03-02 >> > > > > > 10:45:02|4983|108039:101671|inuse|DataStructure.pm:is_parent_reservation(1004)|returning >> > > > > > true: parent reservation ID for this request: 101671 >> > > > > > 2013-03-02 >> > > > > > 10:45:02|4983|108039:101671|inuse|DataStructure.pm:is_parent_reservation(1004)|returning >> > > > > > true: parent reservation ID for this request: 101671 >> > > > > > 2013-03-02 >> > > > > > 10:45:02|4983|108039:101671|inuse|Module.pm:create_os_object(308)|VCL::Module::OS::Windows::Version_5::XP >> > > > > > module loaded >> > > > > > 2013-03-02 >> > > > > > 10:45:02|4983|108039:101671|inuse|Module.pm:new(196)|VCL::Module::OS::Windows::Version_5::XP >> > > > > > object created for image >> > > > > > vmwarewinxp-CSUEBSPSS18withOffice2007124-v9, >> > > > > > address: 18a253e0 >> > > > > > 2013-03-02 >> > > > > > 10:45:02|4983|108039:101671|inuse|Windows.pm:initialize(214)|beginning >> > > > > > Windows module initialization >> > > > > > 2013-03-02 >> > > > > > 10:45:02|4983|108039:101671|inuse|Windows.pm:initialize(224)|Windows >> > > > > > module initialization complete >> > > > > > 2013-03-02 >> > > > > > 10:45:02|4983|108039:101671|inuse|Module.pm:create_os_object(314)|VCL::Module::OS::Windows::Version_5::XP >> > > > > > OS object created, address: 18a253e0 >> > > > > > 2013-03-02 >> > > > > > 10:45:02|4983|108039:101671|inuse|State.pm:initialize(125)|computer >> > > > > > is a VM, attempting to create VM host OS object >> > > > > > 2013-03-02 >> > > > > > 10:45:02|4983|108039:101671|inuse|DataStructure.pm:_initialize(619)|computer >> > > > > > ID argument was specified, retrieving data for computer ID: 127 >> > > > > > 2013-03-02 >> > > > > > 10:45:02|4983|108039:101671|inuse|DataStructure.pm:_initialize(643)|image >> > > > > > ID argument was specified: 4, DataStructure object will contain >> > > > > > image >> > > > > > information for the production imagerevision of this image >> > > > > > 2013-03-02 >> > > > > > 10:45:02|4983|108039:101671|inuse|DataStructure.pm:_initialize(674)|retrieved >> > > > > > data for imagerevision ID: 4 >> > > > > > 2013-03-02 >> > > > > > 10:45:02|4983|108039:101671|inuse|DataStructure.pm:_initialize(690)|retrieved >> > > > > > data for image ID: 4 >> > > > > > 2013-03-02 >> > > > > > 10:45:02|4983|108039:101671|inuse|Module.pm:create_vmhost_os_object(454)|attempting >> > > > > > to load VM host OS module: VCL::Module::OS::Linux::UnixLab (image: >> > > > > > 4) >> > > > > > 2013-03-02 >> > > > > > 10:45:02|4983|108039:101671|inuse|Module.pm:create_vmhost_os_object(460)|VM >> > > > > > host OS module loaded: VCL::Module::OS::Linux::UnixLab >> > > > > > 2013-03-02 >> > > > > > 10:45:02|4983|108039:101671|inuse|Module.pm:new(196)|VCL::Module::OS::Linux::UnixLab >> > > > > > object created for image noimage, address: 18a38ab0 >> > > > > > 2013-03-02 >> > > > > > 10:45:02|4983|108039:101671|inuse|Module.pm:create_vmhost_os_object(465)|VCL::Module::OS::Linux::UnixLab >> > > > > > OS object created, address: 18a38ab0 >> > > > > > 2013-03-02 >> > > > > > 10:45:02|4983|108039:101671|inuse|Module.pm:create_provisioning_object(519)|VCL::Module::Provisioning::VMware::VMware >> > > > > > module loaded >> > > > > > 2013-03-02 >> > > > > > 10:45:02|4983|108039:101671|inuse|Module.pm:new(200)|VCL::Module::Provisioning::VMware::VMware >> > > > > > object created for computer vmguest-196, address: 19065120 >> > > > > > 2013-03-02 >> > > > > > 10:45:02|4983|108039:101671|inuse|VMware.pm:initialize(267)|initializing >> > > > > > VCL::Module::Provisioning::VMware::VMware object >> > > > > > 2013-03-02 >> > > > > > 10:45:02|4983|108039:101671|inuse|DataStructure.pm:_initialize(619)|computer >> > > > > > ID argument was specified, retrieving data for computer ID: 127 >> > > > > > 2013-03-02 >> > > > > > 10:45:02|4983|108039:101671|inuse|DataStructure.pm:_initialize(643)|image >> > > > > > ID argument was specified: 4, DataStructure object will contain >> > > > > > image >> > > > > > information for the production imagerevision of this image >> > > > > > 2013-03-02 >> > > > > > 10:45:02|4983|108039:101671|inuse|DataStructure.pm:_initialize(674)|retrieved >> > > > > > data for imagerevision ID: 4 >> > > > > > 2013-03-02 >> > > > > > 10:45:02|4983|108039:101671|inuse|DataStructure.pm:_initialize(690)|retrieved >> > > > > > data for image ID: 4 >> > > > > > 2013-03-02 >> > > > > > 10:45:02|4983|108039:101671|inuse|VMware.pm:get_vmhost_datastructure(1397)|created >> > > > > > DataStructure object for VM host: vclesxi26 >> > > > > > 2013-03-02 >> > > > > > 10:45:02|4983|108039:101671|inuse|VMware.pm:initialize(287)|VM >> > > > > > profile assigned to vclesxi26: VMware ESXI Network Storage >> > > > > > 2013-03-02 >> > > > > > 10:45:02|4983|108039:101671|inuse|DataStructure.pm:_initialize(619)|computer >> > > > > > ID argument was specified, retrieving data for computer ID: 127 >> > > > > > 2013-03-02 >> > > > > > 10:45:02|4983|108039:101671|inuse|DataStructure.pm:_initialize(643)|image >> > > > > > ID argument was specified: 4, DataStructure object will contain >> > > > > > image >> > > > > > information for the production imagerevision of this image >> > > > > > 2013-03-02 >> > > > > > 10:45:02|4983|108039:101671|inuse|DataStructure.pm:_initialize(674)|retrieved >> > > > > > data for imagerevision ID: 4 >> > > > > > 2013-03-02 >> > > > > > 10:45:02|4983|108039:101671|inuse|DataStructure.pm:_initialize(690)|retrieved >> > > > > > data for image ID: 4 >> > > > > > 2013-03-02 >> > > > > > 10:45:02|4983|108039:101671|inuse|VMware.pm:get_vmhost_datastructure(1397)|created >> > > > > > DataStructure object for VM host: vclesxi26 >> > > > > > 2013-03-02 >> > > > > > 10:45:02|4983|108039:101671|inuse|VMware.pm:get_vmhost_api_object(1500)|attempting >> > > > > > to load VMware control module: >> > > > > > VCL::Module::Provisioning::VMware::vSphere_SDK >> > > > > > 2013-03-02 >> > > > > > 10:45:02|4983|108039:101671|inuse|VMware.pm:get_vmhost_api_object(1506)|loaded >> > > > > > VMware control module: >> > > > > > VCL::Module::Provisioning::VMware::vSphere_SDK >> > > > > > 2013-03-02 >> > > > > > 10:45:02|4983|108039:101671|inuse|Module.pm:new(200)|VCL::Module::Provisioning::VMware::vSphere_SDK >> > > > > > object created for computer vmguest-196, address: 195e4eb0 >> > > > > > 2013-03-02 >> > > > > > 10:45:02|4983|108039:101671|inuse|vSphere_SDK.pm:initialize(98)|vSphere >> > > > > > SDK for Perl does not appear to be installed on this managment >> > > > > > node, >> > > > > > unable to load VMware vSphere SDK Perl modules, error: >> > > > > > |4983|108039:101671|inuse| Can't locate VMware/VIRuntime.pm in @INC >> > > > > > (@INC contains: /usr/local/vcl/bin/../../../.. >> > > > > > /usr/local/vcl/bin/../.. /usr/local/vcl/bin/../../.. >> > > > > > /usr/local/vcl/bin/../../../../.. /usr/local/vcl/bin/.. >> > > > > > /usr/local/vcl/bin/../lib >> > > > > > /usr/lib64/perl5/site_perl/5.8.8/x86_64-linux-thread-multi >> > > > > > /usr/lib/perl5/site_perl/5.8.8 /usr/lib/perl5/site_perl >> > > > > > /usr/lib64/perl5/vendor_perl/5.8.8/x86_64-linux-thread-multi >> > > > > > /usr/lib/perl5/vendor_perl/5.8.8 /usr/lib/perl5/vendor_perl >> > > > > > /usr/lib64/perl5/5.8.8/x86_64-linux-thread-multi >> > > > > > /usr/lib/perl5/5.8.8 >> > > > > > .) at (eval 320510) line 1. >> > > > > > |4983|108039:101671|inuse| BEGIN failed--compilation aborted at >> > > > > > (eval >> > > > > > 320510) line 1. >> > > > > > 2013-03-02 >> > > > > > 10:45:02|4983|108039:101671|inuse|VMware.pm:get_vmhost_api_object(1520)|API >> > > > > > object could not be created: >> > > > > > VCL::Module::Provisioning::VMware::vSphere_SDK >> > > > > > D >> > > > > > 2013-03-02 >> > > > > > 10:45:17|4983|108039:101671|inuse|VMware.pm:initialize(307)|unable >> > > > > > to control OS of VM host vclesxi26 using >> > > > > > VCL::Module::OS::Linux::UnixLab OS object because VM host is not >> > > > > > responding to SSH >> > > > > > |4983|108039:101671|inuse| ---- WARNING ---- >> > > > > > |4983|108039:101671|inuse| 2013-03-02 >> > > > > > 10:45:17|4983|108039:101671|inuse|Module.pm:create_provisioning_object(530)|provisioning >> > > > > > object could not be created, returning 0 >> > > > > > |4983|108039:101671|inuse| ( 0) Module.pm, >> > > > > > create_provisioning_object >> > > > > > (line: 530) >> > > > > > |4983|108039:101671|inuse| (-1) State.pm, initialize (line: 136) >> > > > > > |4983|108039:101671|inuse| (-2) Module.pm, new (line: 207) >> > > > > > |4983|108039:101671|inuse| (-3) vcld, make_new_child (line: 567) >> > > > > > |4983|108039:101671|inuse| (-4) vcld, main (line: 350) >> > > > > > |4983|108039:101671|inuse| ---- WARNING ---- >> > > > > > |4983|108039:101671|inuse| 2013-03-02 >> > > > > > 10:45:17|4983|108039:101671|inuse|State.pm:initialize(145)|failed >> > > > > > to >> > > > > > create provisioning object >> > > > > > |4983|108039:101671|inuse| ( 0) State.pm, initialize (line: 145) >> > > > > > |4983|108039:101671|inuse| (-1) Module.pm, new (line: 207) >> > > > > > |4983|108039:101671|inuse| (-2) vcld, make_new_child (line: 567) >> > > > > > |4983|108039:101671|inuse| (-3) vcld, main (line: 350) >> > > > > > 2013-03-02 >> > > > > > 10:45:17|4983|108039:101671|inuse|utils.pm:delete_computerloadlog_reservation(6396)|removing >> > > > > > computerloadlog entries matching loadstate = begin >> > > > > > 2013-03-02 >> > > > > > 10:45:17|4983|108039:101671|inuse|utils.pm:delete_computerloadlog_reservation(6443)|deleted >> > > > > > rows from computerloadlog for reservation id=101671 >> > > > > > 2013-03-02 >> > > > > > 10:45:17|4983|108039:101671|inuse|State.pm:DESTROY(929)|VCL::inuse >> > > > > > process duration: 15 seconds >> > > > > > 2013-03-02 >> > > > > > 10:45:17|4983|108039:101671|inuse|utils.pm:get_management_node_info(5456)|retrieving >> > > > > > current management node info for 'vclnfs01.calstate.edu' from >> > > > > > database, cached data is stale: 63 seconds old >> > > > > > 2013-03-02 >> > > > > > 10:45:17|4983|108039:101671|inuse|utils.pm:getnewdbh(2474)|unable >> > > > > > to use database handle stored in $ENV{dbh} >> > > > > > 2013-03-02 >> > > > > > 10:45:17|4983|108039:101671|inuse|utils.pm:getnewdbh(2531)|database >> > > > > > handle stored in $ENV{dbh} >> > > > > > 2013-03-02 >> > > > > > 10:45:17|4983|108039:101671|inuse|utils.pm:get_management_node_info(5603)|retrieved >> > > > > > management node info: 'vclnfs01.calstate.edu' (vclnfs01) >> > > > > > 2013-03-02 >> > > > > > 10:45:17|4983|108039:101671|inuse|DataStructure.pm:get_computer_private_ip_address(1641)|attempting >> > > > > > to retrieve private IP address for computer: vmguest-196 >> > > > > > 2013-03-02 >> > > > > > 10:45:17|4983|108039:101671|inuse|OS.pm:get_file_contents(1897)|retrieved >> > > > > > 531 lines from file on vclnfs01: '/etc/hosts' >> > > > > > 2013-03-02 >> > > > > > 10:45:17|4983|108039:101671|inuse|DataStructure.pm:get_computer_private_ip_address(1697)|returning >> > > > > > IP address from /etc/hosts file: 10.10.11.24 >> > > > > > 2013-03-02 >> > > > > > 10:45:17|4983|108039:101671|inuse|utils.pm:is_inblockrequest(5760)|zero >> > > > > > rows were returned from database select >> > > > > > 2013-03-02 >> > > > > > 10:45:17|4983|108039:101671|inuse|DataStructure.pm:get_image_affiliation_name(2121)|image >> > > > > > owner id: 10 >> > > > > > 2013-03-02 >> > > > > > 10:45:17|4983|108039:101671|inuse|DataStructure.pm:retrieve_user_data(1412)|attempting >> > > > > > to retrieve and store data for user: user.id = '10' >> > > > > > 2013-03-02 >> > > > > > 10:45:17|4983|108039:101671|inuse|DataStructure.pm:retrieve_user_data(1475)|data >> > > > > > has been retrieved for user: wv4973 (id: 10) >> > > > > > >> > > > > > -- >> > > > > > Thanks, >> > > > > > Alex Patterson >> > > > > > User Support Services >> > > > > > Operating System Analyst >> > > > > > California State University, East Bay >> > > > > >> > > > >> > > > >> > > > >> > > > >> > > > -- >> > > > Thanks, >> > > > Alex Patterson >> > > > User Support Services >> > > > Operating System Analyst >> > > > California State University, East Bay >> > > >> > > >> > > >> > > >> > > -- >> > > Thanks, >> > > Alex Patterson >> > > User Support Services >> > > Operating System Analyst >> > > California State University, East Bay >> > >> > >> > >> > >> > -- >> > Thank you, >> > >> > Dmitri Chebotarov >> > VCL Sys Eng, Engineering & Architectural Support, TSD - Ent Servers & >> > Messaging >> > 223 Aquia Building, Ffx, MSN: 1B5 >> > Phone: (703) 993-6175 | Fax: (703) 993-3404 >> >> >> >> >> -- >> Thanks, >> Alex Patterson >> User Support Services >> Operating System Analyst >> California State University, East Bay > > > -- Thanks, Alex Patterson User Support Services Operating System Analyst California State University, East Bay
