Hi,

I had a successful update from 3.0.2->4.0.
Everything was perfect until I've enabled storage-clean-up in the global 
options.

At the next first time the storage clean-up thread run it marked all systemVM 
volumes/templates on the primary storages for removal.
It has literally went through all my system VMs and first deleted their volumes 
than their templates...
Then the system start to complain about file-not found messages, deploy 
failed...

I'm trying to recover from this but I cannot re-create the secondary storage VM 
due to:
2012-11-14 14:17:56,580 DEBUG [cloud.server.StatsCollector] 
(StatsCollector-1:null) There is no secondary storage VM for secondary storage 
host nfs://192.168.1.2/share/watford_cloud

But I end up with:
2012-11-14 14:19:55,241 DEBUG [utils.script.Script] (catalina-exec-6:null) 
Looking for createvolume.sh in 
/usr/lib64/cloud/common/scripts/storage/secondary/createvolume.sh
2012-11-14 14:19:55,241 INFO  [storage.template.DownloadManagerImpl] 
(catalina-exec-6:null) createvolume.sh found in 
/usr/lib64/cloud/common/scripts/storage/secondary/createvolume.sh
2012-11-14 14:19:55,243 DEBUG [storage.resource.NfsSecondaryStorageResource] 
(catalina-exec-6:null) Executing: /bin/bash -c ln -sf /mnt/SecStorage 
/var/www/html/copy
2012-11-14 14:19:55,304 DEBUG [storage.resource.NfsSecondaryStorageResource] 
(catalina-exec-6:null) Exit value is 1
2012-11-14 14:19:55,304 DEBUG [storage.resource.NfsSecondaryStorageResource] 
(catalina-exec-6:null) ln: creating symbolic link `/var/www/html/copy': No such 
file or directory
2012-11-14 14:19:55,304 WARN  [storage.resource.NfsSecondaryStorageResource] 
(catalina-exec-6:null) Error in linking  err=ln: creating symbolic link 
`/var/www/html/copy': No such file or directory
2012-11-14 14:19:55,305 INFO  [cloud.resource.ResourceManagerImpl] 
(catalina-exec-6:null) Unable to fully initialize the agent because no 
StartupCommands are returned
2012-11-14 14:19:55,305 INFO  [cloud.resource.ResourceManagerImpl] 
(catalina-exec-6:null) server resources successfully discovered by 
SecondaryStorage
2012-11-14 14:19:55,305 WARN  [cloud.api.ApiDispatcher] (catalina-exec-6:null) 
class com.cloud.api.ServerApiException : Failed to add secondary storage

This is issue is likely caused by the update procedure due to the systemVM were 
not using the latest template.
They were not re-deployed just re-started with the new systemVM template and 
possible some references were not updated in other DB tables so the clean-up 
script found them as not used and destroyed them all.
I'm lucky enough I always have test environment for everything.

Regards

Tamas Monos                                               DDI         
+44(0)2034687012
Chief Technical                                             Office    
+44(0)2034687000
Veber: The Hosting Specialists               Fax         +44(0)871 522 7057
http://www.veber.co.uk<http://www.veber.co.uk/>

Follow us on Twitter: 
www.twitter.com/veberhost<http://www.twitter.com/veberhost>
Follow us on Facebook: 
www.facebook.com/veberhost<http://www.facebook.com/veberhost>

Reply via email to