Jason, locking at this issue one can see;
- No SSVM is running (I know you know this)
- Two systemVM templates are in DOWNLOAD_ERROR state
(mysql> select * from cloud.template_host_ref;)
so there is no way to start the SSVM. The content of the secondary
storage appears intact, but installing the systemvm template using
cloud-install-sys-tmplt again makes sure there are no issues there.
Since this is a KVM, system, fixing the KVM system template first should
enable SSVM to start but this is not happening. The logs indicate that
the VMWARE template prevents this (the search finds this first and
decides to not continue).
Fixing the VMWARE template as well solves the problem and the manager
can spin up the SSVM and console proxy.
The logs indicate there have been 100% capacity on the secondary storage
so some issues there caused the SSVM to try to redownload the system vm
templates. I believe you said you destroyed the SSVM which means the
SSVM can not recover (for your config to allow the SSVM to recover you
need to set the SSVM proxy, but you should rather use URL that does not
require proxy in the first place).
Pasted in the script used to fixup the DB. NB; important to check the ID
mapping between vm_template and template_host_ref. Stop the manager when
using a script like this.
/Ove
#!/bin/bash
PWD=
WEBSRV=
if [ ! -d . ]; then
echo "fixing up KVM template"
# vm_template id=3 --> template_host_ref=3
mysql -u root --password=$PWD cloud -e "UPDATE template_host_ref
SET job_id=NULL WHERE id=3"
mysql -u root --password=$PWD cloud -e "UPDATE template_host_ref
SET download_pct=100 WHERE id=3"
mysql -u root --password=$PWD cloud -e "UPDATE template_host_ref
SET download_state='DOWNLOADED' WHERE id=3"
mysql -u root --password=$PWD cloud -e "UPDATE template_host_ref
SET error_str=NULL WHERE id=3"
mysql -u root --password=$PWD cloud -e "UPDATE template_host_ref
SET local_path=NULL WHERE id=3"
mysql -u root --password=$PWD cloud -e "UPDATE template_host_ref
SET install_path='template/tmpl/1/3/' WHERE id=3"
mysql -u root --password=$PWD cloud -e "UPDATE template_host_ref
SET url='http://$WEBSRV/iso/acton-systemvm-02062012.qcow2.bz2' WHERE id=3"
fi
if [ -d . ]; then
echo "fixing up vSphere template"
# vm_template id=8 --> template_host_ref=2
mysql -u root --password=$PWD cloud -e "UPDATE template_host_ref
SET job_id=NULL WHERE id=2"
mysql -u root --password=$PWD cloud -e "UPDATE template_host_ref
SET download_pct=100 WHERE id=2"
mysql -u root --password=$PWD cloud -e "UPDATE template_host_ref
SET download_state='DOWNLOADED' WHERE id=2"
mysql -u root --password=$PWD cloud -e "UPDATE template_host_ref
SET error_str=NULL WHERE id=2"
mysql -u root --password=$PWD cloud -e "UPDATE template_host_ref
SET local_path=NULL WHERE id=2"
mysql -u root --password=$PWD cloud -e "UPDATE template_host_ref
SET install_path='template/tmpl/1/8/' WHERE id=2"
mysql -u root --password=$PWD cloud -e "UPDATE template_host_ref
SET url='http://$WEBSRV/iso/burbank-systemvm-08012012.ova' WHERE id=2"
fi
On 06/04/2013 11:31 PM, Jason Pavao wrote:
Here is the exact command that i'm running-
[root@slc05hmi fmwcloud_secondary]#
/usr/lib64/cloud/common/scripts/storage/secondary/cloud-install-sys-tmplt -m
/mnt/fmwcloud_secondary -u
http://download.cloud.com/templates/acton/acton-systemvm-02062012.qcow2.bz2
-h kvm -F
--2013-06-04 15:22:41--
http://download.cloud.com/templates/acton/acton-systemvm-02062012.qcow2.bz2
Resolving www-proxy.us.oracle.com... 148.87.19.20
Connecting to www-proxy.us.oracle.com|148.87.19.20|:80... connected.
Proxy request sent, awaiting response... 200 OK
Length: 286036668 (273M) [binary/octet-stream]
Saving to:
â/usr/lib64/cloud/common/scripts/storage/secondary/a312260a-6386-40fa-a342-a8855e8d1b79.qcow2â
100%[===========================================================================================================================================>]
286,036,668 1.59M/s in 3m 58s
2013-06-04 15:26:39 (1.15 MB/s) -
â/usr/lib64/cloud/common/scripts/storage/secondary/a312260a-6386-40fa-a342-a8855e8d1b79.qcow2â
Uncompressing to
/usr/lib64/cloud/common/scripts/storage/secondary/a312260a-6386-40fa-a342-a8855e8d1b79.qcow2.tmp
(type bz2)...could take a long time
Moving to
/mnt/fmwcloud_secondary/template/tmpl/1/3///a312260a-6386-40fa-a342-a8855e8d1b79.qcow2...could
take a while
Successfully installed system VM template to
/mnt/fmwcloud_secondary/template/tmpl/1/3/
It indicates that it all successfully completes, however-
As you can see, no matter what I try I cannot bring the system vm
templates into a ready state.
Is their any additional flags or steps I need to make in order to get
these templates in a ready state?
Again- this environment has been trouble free, any help would be
appreciated!
Thanks.
-jason
On 6/4/2013 1:59 PM, Jason Pavao wrote:
More infos on this environment-
cs 4.0.2 running on oel6.3 with 2 kvm hvs.
This environment has been reliable until this sudden issue. Any help
suggestions would be appreciated
On 6/4/2013 11:59 AM, Jason Pavao wrote:
Hey Folks,
I am having a strange issue, my KVM system vm templates. My templates
indicate that the are "connection timed out". When I re-run my system
vm installation steps as outlines in the install doc-
/usr/lib64/cloud/common/scripts/storage/secondary/cloud-install-sys-tmplt
-m /mnt/fmwcloud_secondary -u
http://slcac475.us.oracle.com/iso/acton-systemvm-02062012.qcow2.bz2
-h kvm -f
Everything appears to complete, after a reboot of the management head
the kvm system vm templates continue to indicate "connection timed
out". and I see this is the mgmt log-
2013-06-04 12:58:21,771 DEBUG [cloud.server.StatsCollector]
(StatsCollector-1:null) There is no secondary storage VM for
secondary storage host
nfs://slcnas458.us.oracle.com/export/fmw_labmanager/fmwcloud
Any suggestions/help would be immensely appreciated.
--
Thanks.
-Jason
--
Ove Everlid
System Administrator / Architect / SDN & Linux hacker
Mobile: +46706662363
Office: +4618656913 (note EMEA Time Zone)
