deepti dohare created CLOUDSTACK-356:
----------------------------------------
Summary: snapshot errors with multiple secondary storage in one
zone
Key: CLOUDSTACK-356
URL: https://issues.apache.org/jira/browse/CLOUDSTACK-356
Project: CloudStack
Issue Type: Bug
Components: Snapshot
Affects Versions: pre-4.0.0
Reporter: deepti dohare
Fix For: 4.1.0
BRIEF SYNOPSIS:
========================
Snapshots are failing inder some cases where the host has already mounted other
secondary storage and snapshot command has been issued to it
ENVIRONMENT
====================
CS 2.2.14 + XEN server
DETAILED DESCRIPTION
====================
1) Noticcing following error for some snapshots
2012-07-23 02:16:38,120 WARN [xen.resource.CitrixResourceBase]
(DirectAgent-281:null) Task failed! Task record: uuid:
f0642eda-6e40-8bca-1ffb-bbf52a0e307e
status: FAILURE
errorInfo: [XENAPI_PLUGIN_FAILURE, backupSnapshot, SROSError, Error reporting
error, unknown key File
/var/run/cloud_mount/1/snapshots/35/2521/a05404cd-e2c7-41cf-9972-aa019ab79494.vhd
does not exist.]
2) Cluster has investigated the issue and they found that mount points are not
identical for the xenservers to the secondary storages across a cluster (df) :
* host 1 : 10.16.36.64:/RAID10-DATA01/secsatxen002/snapshots 2147483648
240044032 1907439616 12% /var/run/cloud_mount/1/snapshots
* Host 2 : 10.16.36.64:/RAID10-DATA01/secsatxen001/snapshots 2147483648
569533440 1577950208 27% /var/run/cloud_mount/1/snapshots
Therefore, the contents are not identical in the 2 secondary storages (this is
right to my understanding, because multiple secondary storage permit to scale
horizontally) :
* Host 1 :
* -rw-r--r--+ 1 nobody nobody 307M 2012-07-23 16:11
07ba46c7-0908-4402-b20a-5ddd29a65d17.vhd
* -rw-r--r--+ 1 nobody nobody 301M 2012-07-18 02:18
1c378e6a-0eba-48d9-8545-ab9fa09b9ba0.vhd
* -rw-r--r--+ 1 nobody nobody 59M 2012-07-19 02:18
3b7e8c19-ba9c-433e-ac18-f52d4ad42be9.vhd
* -rw-r--r--+ 1 nobody nobody 97M 2012-07-20 02:19
4644362a-764d-4c66-a038-168127007a5f.vhd
* -rw-r--r--+ 1 nobody nobody 47M 2012-07-23 02:18
72e54c85-8245-4b84-b4b5-5a1ef2b44e52.vhd
* -rw-r--r--+ 1 nobody nobody 69M 2012-07-21 02:18
76301f19-fcc0-43ac-adab-cde2cf23bd87.vhd
* -rw-r--r--+ 1 nobody nobody 57M 2012-07-22 02:18
a05404cd-e2c7-41cf-9972-aa019ab79494.vhd
* Host 2 : empty
A simple workaround is to migrate the corresponding instance to the host with
the right mount. After further checking, this behavior is encountered on all
clusters, only rarely because only applying to instances that :
* Have a scheduled snapshot retention policy
* Have either migrated, been restarted on a different host than previous run
Though workaround is feasible but it is difficult to predict when the user
seems this probelm.
please check if there is any problem in cloudstack logic to ensure right mount
secondary mount point is available and create it if it doesnt exist
If this behaviour confirms to be a bug,please file one bug in jira.
REPRO STEPS
==================
You can try reproducing the scenario by havign two secondary storages and
continuously taking snapshots
EXPECTED BEHAVIOR
==================
Cloudstack should create correct mount of secondary storage if it doesnt exist.
It should never use other secondary storage mount point.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
