Hi Gian, We faced the same situation as well. In our case, we just delete the db entries by running the following sql command < delete from snapshots where status="Allocated”; >
It doesn’t affect anything else. Best Regards Stavros > On 18 Mar 2016, at 15:56, Gian Paolo Buono <gianpaolo.bu...@gesca.it> wrote: > > > Hi all, > In my environment i have many snapshot in state "Allocated" but in table > snapshot_store_ref no record associated. > EX: > select id from snapshots where status = 'Allocated'; > 1020 > select * from snapshot_store_ref where snaphot_id = 1020; > Empty set (0.00 sec) > > Can i delete this entry or I must set Destroyed state ? > Thanks > On 03/18/2016 11:35 AM, Stavros Konstantaras wrote: > > It seems that CS 4.6 does not have enough logic to handle correctly the > snapshots. In my case, it was an expunged volume that used to create the > problem as snapshots of this volume still existed (both in DB and in > secondary storage). So I created a manual workaround to bypass this problem > by making our hands dirty in MySQL: > > - Stop CS service > > - In MySQL find the expunged volume in “volumes" table and keep the volume_id > > - In “snapshot_store_ref" table get the location of the snapshots by using > the mentioned volume_id. Then go to the secondary storage and delete all the > relevant files (since now we know the exact location) > > - Now it’s time to update the tables by running the following SQL commands: > <update snapshot_store_ref set state="Destroyed" where id=XXX;> <update > snapshots set status="Destroyed" where id=XXX;> where XXX is the volume id of > the second step. > > - Start CS service and check the snapshots output. The problem should be gone. > > Hope that helps some fellows. > > Regards > Stavros > > > ---------------------------- > Stavros Konstantaras > Science faculty Research IT support (FEIOG) > University of Amsterdam, Science Park 904, 1098 XH > > Fingerprint: E5E5 9B19 D1CD 88CD 4763 3465 A8DC 7C92 330F D59A > > > > On 16 Mar 2016, at 11:05, Stavros Konstantaras <s.konstanta...@uva.nl > <mailto:s.konstanta...@uva.nl>><mailto:s.konstanta...@uva.nl > <mailto:s.konstanta...@uva.nl>> wrote: > > Hi Adrian, > > Firstly, thanks for your extensive answer and the trick. My problem is > similar to the CLOUDSTACK-8845 issue. In our case we just try to view the > snapshots from cloudstack UI instead of listing them in cloud monkey. My > first thoughts are that probably some inconsistencies exist in the database > but unfortunately after running the following mysql commands I don’t get any > results. > > < select volume_id,name,status from cloud.snapshots where volume_id not in > (select id from cloud.volumes); > > < select * from cloud.snapshot_store_ref where volume_id not in (select id > from cloud.volumes); > > > Any other ideas to find out the inconsistencies and get rid of them? In > addition, I checked the log file of cloudstack-management server and I don’t > see the error you mentioned. I use CS 4.6 and XenServer 6.5 > > Thanks again. > > Regards > Stavros > > ---------------------------- > Stavros Konstantaras > Science faculty Research IT support (FEIOG) > University of Amsterdam, Science Park 904, 1098 XH > > Fingerprint: E5E5 9B19 D1CD 88CD 4763 3465 A8DC 7C92 330F D59A > > > > On 16 Mar 2016, at 07:43, Adrian Sender <asen...@testlabs.com.au > <mailto:asen...@testlabs.com.au>><mailto:asen...@testlabs.com.au > <mailto:asen...@testlabs.com.au>> wrote: > > Hi Stravos, > > A bit more info with ongoing snapshot issues with Xenserver.. your snapshots > may also be failing if your chain depth exceeds 30. > > Assuming your using LVMoHBA (hardware hba), easiest way to check this is as > follows: > > If the xenserver snapshot chain depth exceeds 30, you will not be able to take > any new snapshots. > > You may see a similar error: (note exact error will depend on CCP / CS > version.. > > 2015-11-18 10:57:42,133 WARN [c.c.h.x.r.XenServerStorageProcessor] > (DirectAgent-243:ctx-341ac146) (logid:65b51cbd) create snapshot operation > Failed for snapshotId: 36709, reason: SR_BACKEND_FAILURE_109The snapshot chain > is too long > SR_BACKEND_FAILURE_109The snapshot chain is too long > > > Example > ======= > [root@qh2-nsp04 ~]# vhd-util query -vsfd -n > /dev/VG_XenStorage-3965cb14-960e-d9d5-4ab9-608425f3ee20/VHD-7bf051ee-7762-4902-84be-4962e70a4e3e > -p > 10240 > 1145303552 > /dev/mapper/VG_XenStorage--3965cb14--960e--d9d5--4ab9--608425f3ee20-VHD--811a34ff--731a--4c36--92be--286831ab7393 > hidden: 0 > chain depth: 30 > > The best way to get the LVM path is as follows: > > mysql> select * from volumes where name='ROOT-3230' \G > > We are looking for the path, in this case > "7bf051ee-7762-4902-84be-4962e70a4e3e" > > Login to Xenserver console and do "lvscan | grep > 7bf051ee-7762-4902-84be-4962e70a4e3e" This will give us VG and LV. > > Then you can execute vhd-util query -vsfd -n > /dev/VG_XenStorage-3965cb14-960e-d9d5-4ab9-608425f3ee20/VHD-e00cd7a7-ae20-4b5d-a0e3-1c215d432ed4 > > Delete the broken snapshots from the primary storage in XenCenter. > > Regards, > Adrian Sender > > Testlabs Australia > Research Platform Services > M: +61. 487-440-732 > E: asen...@testlabs.com.au > <mailto:asen...@testlabs.com.au><mailto:asen...@testlabs.com.au > <mailto:asen...@testlabs.com.au>> > W: http://www.testlabs.com.au <http://www.testlabs.com.au/> > > ---------- Original Message ----------- > From: "Adrian Sender" <asen...@testlabs.com.au > <mailto:asen...@testlabs.com.au>><mailto:asen...@testlabs.com.au > <mailto:asen...@testlabs.com.au>> > To: users@cloudstack.apache.org > <mailto:users@cloudstack.apache.org><mailto:users@cloudstack.apache.org > <mailto:users@cloudstack.apache.org>> > Sent: Wed, 16 Mar 2016 09:59:34 +1000 > Subject: Re: [Where are the snapshot file] > > > > Hi Stravos, > > Jumping in a bit late on this thread...Can you share the exact > management log error. > > We have ran CCP 4.3 with Xenserver, everything is generally ok; most > common snapshot issues are as follows: > > Snapshot failing > ---------------- > > The snapshotting on data volume DATA-3230 Failing. UI shows "Error" > > 2014-08-26 10:51:32,884 DEBUG [o.a.c.s.m.AncientDataMotionStrategy] > > (Work-Job-Executor-6:ctx-274b0a29 job-31179/job-31180 ctx-8511b016) copyAsync > inspecting src type SNAPSHOT copyAsync inspecting dest type SNAPSHOT > 2014-08-26 10:51:33,055 DEBUG [o.a.c.s.m.AncientDataMotionStrategy] > > (Work-Job-Executor-6:ctx-274b0a29 job-31179/job-31180 ctx-8511b016) copy > snasphot failed: java.lang.NullPointerException > 2014-08-26 10:51:33,056 DEBUG [o.a.c.s.m.AncientDataMotionStrategy] > > (Work-Job-Executor-6:ctx-274b0a29 job-31179/job-31180 ctx-8511b016) > copy failed com.cloud.utils.exception.CloudRuntimeException: > > > java.lang.NullPointerException > > > > 1. Show the details of the failing snapshot volume > > mysql> select * from volumes where name=DATA-3230 \G > > 2. get the id of the volume from the above query and delete > snapshots in the 'Ready' state. > > delete from snapshot_store_ref where volume_id=3934 and state='Ready' > > 3. Snapshots will now succeed for volume DATA-3230. > > Snapshot Failing > ---------------- > > You may see the following in the logs: > > 2015-11-17 15:22:16,586 DEBUG [c.c.h.x.r.XenServerStorageProcessor] > (DirectAgent-339:ctx-6465ce91) (logid:9224bc6d) Failed to destroy pbd > SR_BACKEND_FAILURE_40The SR scan failed [opterr=['INTERNAL_ERROR', > 'Db_exn.Uniqueness_constraint_violation("VDI", "uuid", > "e5318c7f-9434-4f1c-8695-3e82a69d2192")']] > > http://support.citrix.com/article/CTX135230 > <http://support.citrix.com/article/CTX135230> > > Note in this example DATA-3701 snapshot is failing, but the > screenshot shows the NFS secondary storage mount for account 29, > DATA-3701 belongs to account 113. So you can go through the tabs > until you find /var/cloud_mount/xxxx/snapshots/113/3701 and unplug > just that volume. > > In this example you will see many attached NFS secondary storage > volumes on the physical xenserver host. > > 1. Get the uuid of the SR from the "General" tab on the NFS storage mount. > > xe sr-list uuid=84142bd7-c19c-4a39-850b-ddd30539d8ed > > [root@qh2-nsp01 ~]# xe sr-list uuid=84142bd7-c19c-4a39-850b-ddd30539d8ed > uuid ( RO) : 84142bd7-c19c-4a39-850b-ddd30539d8ed > name-label ( RW): > 14a68eb4-62aa-4beb-91d5-e5070ecd46a9/var/cloud_mount/b9d64979-0c47- > 31fe-a496-6943e9dfca63/snapshots/29/4469 name-description ( RW): > 14a68eb4-62aa-4beb-91d5-e5070ecd46a9/var/cloud_mount/b9d64979-0c47- > 31fe-a496-6943e9dfca63/snapshots/29/4469 host ( RO): > qh2-nsp01.nsp.nectar.org.au <http://qh2-nsp01.nsp.nectar.org.au/> > type ( RO): file > content-type ( RO): file > > 2. Get the PBD UUID by using the sr-uuid from above > > [root@qh2-nsp01 ~]# xe pbd-list sr-uuid=84142bd7-c19c-4a39-850b-ddd30539d8ed > uuid ( RO) : c15627c7-a968-ac1f-9794-2e3d66c0cec4 > host-uuid ( RO): 14a68eb4-62aa-4beb-91d5-e5070ecd46a9 > sr-uuid ( RO): 84142bd7-c19c-4a39-850b-ddd30539d8ed > device-config (MRO): location: > /var/cloud_mount/b9d64979-0c47-31fe-a496-6943e9dfca63/snapshots/29/4469 > currently-attached ( RO): true > > 3. Unplug the pdb using the uuid from above > > xe pbd-unplug uuid=c15627c7-a968-ac1f-9794-2e3d66c0cec4 > > 4. At this point you should notice the state change to broken and a > red cross through the secondary mount point in Xencenter. > > 5. Finally remove the sr from the host completely using the original > sr uuid. > > xe sr-forget uuid=84142bd7-c19c-4a39-850b-ddd30539d8ed > > 6 Repeat this procedure for any other attached NFS snapshot mounts, > check all xenservers. > > Regards, > Adrian Sender > > ---------- Original Message ----------- > From: Prashant Mishra <prashant.mis...@accelerite.com > <mailto:prashant.mis...@accelerite.com>><mailto:prashant.mis...@accelerite.com > <mailto:prashant.mis...@accelerite.com>> > To: "users@cloudstack.apache.org > <mailto:users@cloudstack.apache.org>"<mailto:users@cloudstack.apache.org > <mailto:users@cloudstack.apache.org>> <users@cloudstack.apache.org > <mailto:users@cloudstack.apache.org>><mailto:users@cloudstack.apache.org > <mailto:users@cloudstack.apache.org>> > Sent: Tue, 15 Mar 2016 12:48:10 +0000 > Subject: Re: [Where are the snapshot file] > > > > There are some discussion related to this please check > https://issues.apache.org/jira/browse/CLOUDSTACK-9297 > <https://issues.apache.org/jira/browse/CLOUDSTACK-9297> > > On 3/15/16, 5:49 PM, "Stavros Konstantaras" <s.konstanta...@uva.nl > <mailto:s.konstanta...@uva.nl>><mailto:s.konstanta...@uva.nl > <mailto:s.konstanta...@uva.nl>> wrote: > > > > Yes indeed. Currently I am fighting with the very well known "Unable to > determine the storage pool of the snapshot² after deleting some snapshots > and creating new ones. > > Is there any workaround for this issue? > > Cheers > Stavros > > ---------------------------- > Stavros Konstantaras > Science faculty Research IT support (FEIOG) > University of Amsterdam, Science Park 904, 1098 XH > > Fingerprint: E5E5 9B19 D1CD 88CD 4763 3465 A8DC 7C92 330F D59A > > > > On 15 Mar 2016, at 13:01, Simon Weller <swel...@ena.com > <mailto:swel...@ena.com>><mailto:swel...@ena.com <mailto:swel...@ena.com>> > wrote: > > Can you post some logs from both the management server and the agent > (on the host) when you try and delete a snapshot? > > I just looked through Jira and there are quite a few issues related to > snapshots. > > - Si > > ________________________________________ > From: Gian Paolo Buono <gianpaolo.bu...@gesca.it > <mailto:gianpaolo.bu...@gesca.it>><mailto:gianpaolo.bu...@gesca.it > <mailto:gianpaolo.bu...@gesca.it>> > Sent: Tuesday, March 15, 2016 6:52 AM > To: users@cloudstack.apache.org > <mailto:users@cloudstack.apache.org><mailto:users@cloudstack.apache.org > <mailto:users@cloudstack.apache.org>> > Subject: Re: [Where are the snapshot file] > > Hi Simon, > > we run Citrix CloudPlatfor 4.3 but we want to migrate to CloudStack > 4.8... > > there is a fix ? > > Best regards > > > > This communication may contain information that is proprietary, > confidential, or exempt from disclosure. If you are not the intended > recipient, please note that any other dissemination, distribution, use > or copying of this communication is strictly prohibited. Anyone who > receives this message in error should notify the sender immediately by > telephone or by return e-mail and delete it from his or her computer. > > On 03/15/2016 12:46 PM, Simon Weller wrote: > > Gian, > > What version of CloudStack are you running? > > I recall (quite a long time ago), there was a bug where volumes and > snapshots weren't getting cleaned up correctly. > If I recall correctly, that was somewhere around 4.2. > > - Si > > ________________________________________ > From: Gian Paolo Buono > <gianpaolo.bu...@gesca.it > <mailto:gianpaolo.bu...@gesca.it>><mailto:gianpaolo.bu...@gesca.it > <mailto:gianpaolo.bu...@gesca.it>><mailto:gianpaolo.bu...@gesca.it > <mailto:gianpaolo.bu...@gesca.it>><mailto:gianpaolo.bu...@gesca.it > <mailto:gianpaolo.bu...@gesca.it>> > Sent: Tuesday, March 15, 2016 6:10 AM > To: users@cloudstack.apache.org > <mailto:users@cloudstack.apache.org><mailto:users@cloudstack.apache.org > <mailto:users@cloudstack.apache.org>><mailto:users@cloudstack.apache.org > <mailto:users@cloudstack.apache.org>><mailto:users@cloudstack.apache.org > <mailto:users@cloudstack.apache.org>> > Subject: Re: [Where are the snapshot file] > > Thank you it's correct :) , but now I have another question.. > > From console of CloudStack I have deleted a snapshot, on the database > the status is changed in Destroyed but the file it's stil on secondary > storage: > > ls -ltr d8736bdb-22c6-46fe-a27e-0ee5dc99a867.vhd > -rw-r--r-- 1 root root 1271296512 Mar 11 04:02 > d8736bdb-22c6-46fe-a27e-0ee5dc99a867.vhd > > Also in snapshots view sometimes the status is allocated...but what > does it mean ? > > > Best Regards > > On 03/15/2016 11:15 AM, Stavros Konstantaras wrote: > > Hi Gian, > > If you run the following query in MySQL and check the column > ³install_path² you will find the location of snapshot file in secondary > storage: > <select * from cloud.snapshot_store_ref;> > > I hope that it helps you. > > Best Regards > Stavros > > > ---------------------------- > Stavros Konstantaras > Science faculty Research IT support (FEIOG) > University of Amsterdam, Science Park 904, 1098 XH > > Fingerprint: E5E5 9B19 D1CD 88CD 4763 3465 A8DC 7C92 330F D59A > > > > On 15 Mar 2016, at 11:11, Gian Paolo Buono > <gianpaolo.bu...@gesca.it > <mailto:gianpaolo.bu...@gesca.it>><mailto:gianpaolo.bu...@gesca.it > <mailto:gianpaolo.bu...@gesca.it>><mailto:gianpaolo.bu...@gesca.it > <mailto:gianpaolo.bu...@gesca.it>><mailto:gianpaolo.bu...@gesca.it > <mailto:gianpaolo.bu...@gesca.it>><mailto:gianpa > olo.bu...@gesca.it><mailto:gianpaolo.bu...@gesca.it><mailto:gianpaolo.bu...@gesca.it><mailto:gianpaolo.bu...@gesca.it> > wrote: > > Hi all, how can I know where are the snapshot file (vhd) on the > secondary storage ? Is there a mysql query ? > > Kind regards > > > > > > > > > > > > > > > > DISCLAIMER > ========== > This e-mail may contain privileged and confidential information > which is the property of Accelerite, a Persistent Systems business. > It is intended only for the use of the individual or entity to which > it is addressed. If you are not the intended recipient, you are not > authorized to read, retain, copy, print, distribute or use this > message. If you have received this communication in error, please > notify the sender and delete all copies of this message. Accelerite, > a Persistent Systems business does not accept any liability for > virus infected mails. > > > ------- End of Original Message ------- > > > ------- End of Original Message -------
