Joris van Lieshout created CLOUDSTACK-692:
---------------------------------------------
Summary: The StorageManager-Scavenger deletes snapshots that are
still in the process of being created at that time when the volume has older
snapshots that do need scavenging
Key: CLOUDSTACK-692
URL: https://issues.apache.org/jira/browse/CLOUDSTACK-692
Project: CloudStack
Issue Type: Bug
Security Level: Public (Anyone can view this level - this is the default.)
Components: Snapshot
Reporter: Joris van Lieshout
Priority: Minor
Hi there,
I think we ran into a bug due to a concurrence of circumstances regarding
snapshotting and the cleanup of snapshots.
The StorageManager-Scavenger instructs the StorageVM to delete a snapshot that
is still in the process of being created on a hypervisor at that time when the
volume has older snapshots that do need scavenging.
==== The SR gets mounted for the snapshot to be created on.
2012-12-16 08:02:53,831 DEBUG [xen.resource.CitrixResourceBase]
(DirectAgent-293:null) Host 192.168.###.42
OpaqueRef:fae7f8be-8cf1-7b84-3d30-7202e172b530: Created a SR; UUID is
1f7530d8-4615-c220-7f37-0
5862ddbfe3b device config is
{serverpath=/pool0/####-###-dc-1-sec1/snapshots/163/1161, server=192.168.###.14}
==== The SMlog on the xenserver show that at this time the snapshot is still
being created.
2012-12-16 08:37:08,768 DEBUG [agent.transport.Request]
(StorageManager-Scavenger-1:null) Seq 159-1958616345: Sending { Cmd , MgmtId:
345052433504, via: 159, Ver: v1, Flags: 100011, [{"CleanupSnapshot
BackupCommand":{"secondaryStoragePoolURL":"nfs://192.168.###.14/pool0/####-###-dc-1-sec1","dcId":2,"accountId":163,"volumeId":1161,"validBackupUUIDs":["b714a0ee-406e-4100-a75d-bc594391dca9","209bc1dd-f6
1a-486c-aecf-335590a907eb"],"wait":0}}] }
==== At this time we start seeing tapdisk errors on the XenServer indicating
that the vhd file is gone.
Dec 16 08:37:08 ####vm8 tapdisk[26553]: ERROR: errno -116 at vhd_complete:
/var/run/sr-mount/1f7530d8-4615-c220-7f37-05862ddbfe3b/073893a6-e9cb-4cf6-8070-c6cf771db5d7.vhd:
op: 2, lsec: 448131408, secs:
88, nbytes: 45056, blk: 109407, blk_offset: 330368935
Dec 16 08:37:08 ####vm8 tapdisk[26553]: ERROR: errno -116 at vhd_complete:
/var/run/sr-mount/1f7530d8-4615-c220-7f37-05862ddbfe3b/073893a6-e9cb-4cf6-8070-c6cf771db5d7.vhd:
op: 2, lsec: 448131496, secs: 40, nbytes: 20480, blk: 109407, blk_offset:
330368935
Dec 16 08:37:08 ####vm8 tapdisk[26553]: ERROR: errno -116 at vhd_complete:
/var/run/sr-mount/1f7530d8-4615-c220-7f37-05862ddbfe3b/073893a6-e9cb-4cf6-8070-c6cf771db5d7.vhd:
op: 4, lsec: 448131072, secs: 1, nbytes: 512, blk: 109407, blk_offset:
330368935
Dec 16 08:37:08 ####vm8 tapdisk[26553]: ERROR: errno -116 at
__tapdisk_vbd_complete_td_request: req tap-77.0: write 0x0058 secs @ 0x1ab5f150
- Stale NFS file handle
Dec 16 08:37:08 ####vm8 tapdisk[26553]: ERROR: errno -116 at
__tapdisk_vbd_complete_td_request: req tap-77.1: write 0x0028 secs @ 0x1ab5f1a8
- Stale NFS file handle
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
