Hi
I can confirm that I am also having this issue on 4.11.1.0. To be honest, this
issue was always present for me as far as I remember.
Kind of an urgent issue to fix to stop people running out of space.
Cheers
----- Original Message -----
> From: "Rafael Weingärtner" <rafaelweingart...@gmail.com>
> To: "users" <users@cloudstack.apache.org>
> Sent: Wednesday, 10 October, 2018 16:24:54
> Subject: Re: VM Snapshot not removed from primary SR
> Well, I am almost sure I have seen a PR fixing something like you describe,
> I thought that it went into 4.11.1.0.However, only with a more in depth
> debugging I would be able to confirm your problem.
>
> Are you seeing any unexpected exception in your log files?
>
> On Wed, Oct 10, 2018 at 11:54 AM Sami Rajala (FAPPS) <sami.raj...@fapps.com>
> wrote:
>
>> Hi,
>>
>> CS running on 4.11.1.0 and does not work.
>>
>> BR
>> -sami
>>
>>
>> 10.10.2018 16.49, "Rafael Weingärtner" <rafaelweingart...@gmail.com>:
>>
>> >If I am not mistaken, this has already been fixed in 4.11.1.0
>> >
>> >On Wed, Oct 10, 2018 at 10:31 AM Sami Rajala <sami.raj...@fapps.com>
>> >wrote:
>> >
>> >> Hi,
>> >>
>> >>
>> >> I have got zero advice for this.
>> >>
>> >> "I have had weird issue since I updated CS from 4.9 to 4.10.
>> >> CS does not delete/remove temporary snapshot from primary SR and I have
>> >> had to removed it by manual time to time, before 30 snapshot limit has
>> >> reached.
>> >> + doesn't CS not update secondary_storege count on snapshot_count table”
>> >>
>> >>
>> >>
>> >>
>> >> Now I have a situation where Xenserver has 19 snapshot on primary SR for
>> >> each VM + resource count for ”secondary_storege” in database increase
>> >>all
>> >> the time per account.
>> >> ACS storage cleaner removed old snapshots from secondary when they
>> >>getting
>> >> old, but thats all, not decrease resource count for ”secondary_storege”
>> >> with value of deleted snapshot ( or not remove snapshot from primary SR
>> >> after snapshot has created and copied to secondary )
>> >>
>> >> I have still delete primary snapshot by manual + give a new value for
>> >> ”secondary_storege” before they reach limit.
>> >>
>> >> has anyone got this kind of issue? any fix
>> >>
>> >> BR
>> >> -sami
>> >>
>> >>
>> >>
>> >> 14.9.2018 12.26, "Sami Rajala (FAPPS)" <sami.raj...@fapps.com>:
>> >>
>> >> >Continue,
>> >> >
>> >> >I created template from VM-s latest snapshot after 3 days ( one parent
>> >>+
>> >> >2
>> >> >child snapshot) - without errors/warning
>> >> >CS create VM from this template ok, but Xen could not start VM
>> >> >
>> >> >so, I return back to situation where every snapshot is full and manual
>> >> >removing snapshot from primary
>> >> >
>> >> >Has anyone any advice?
>> >> >
>> >> >BR
>> >> >-sami
>> >> >
>> >> >
>> >> >
>> >> >13.9.2018 7.56, "Sami Rajala (FAPPS)" <sami.raj...@fapps.com>:
>> >> >
>> >> >>Hello
>> >> >>
>> >> >>I have had weird issue since I updated CS from 4.9 to 4.10.
>> >> >>CS does not delete/remove temporary snapshot from primary SR and I
>> >>have
>> >> >>had to removed it by manual time to time, before 30 snapshot limit has
>> >> >>reached.
>> >> >>+ doesn¹t CS not update secondary_storege count on snapshot_count
>> >>table
>> >> >>
>> >> >>It works fine on 4.9, but stop to work on 4.10 and I have wait
>> >> >>possibility
>> >> >>to update 4.11.1.
>> >> >>Update to 4.11.1 has done now and looks like this not work still
>> >> >>
>> >> >>I removed all snapshot and start over from clean table, snapshot
>> >>policy
>> >> >>is: DAILY, keep 2, delta = 5
>> >> >>There are now 3 snapshot on primary SR and 3 on secondary SR
>> >> >>
>> >> >>The Environment is Xen 7 + CS 4.11.1 + NFS storage
>> >> >>
>> >> >>Is there some other parameter I should look and any other work around
>> >>I
>> >> >>has to do to get it work ?
>> >> >>
>> >> >>Any hits?
>> >> >>
>> >> >>BR
>> >> >>-sami
>> >> >>
>> >> >>VM snapshot Log for last round:
>> >> >>
>> >> >>2018-09-13 06:02:15,983 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>> >> >>(API-Job-Executor-83:ctx-cfb1cd95 job-28020) (logid:4c7a1c38)
>> >>Executing
>> >> >>AsyncJobVO {id:28020, userId: 1, accountId: 7, instanceType: Snapshot,
>> >> >>instanceId: 2656, cmd:
>> >> >>org.apache.cloudstack.api.command.user.snapshot.CreateSnapshotCmd,
>> >> >>cmdInfo:
>> >>
>> >>>>{"policyid":"17","ctxUserId":"1","volumeid":"238","ctxStartEventId":"1"
>> >>>>,"
>> >> >>i
>> >> >>d
>> >> >>":"2656","ctxAccountId":"7"}, cmdVersion: 0, status: IN_PROGRESS,
>> >> >>processStatus: 0, resultCode: 0, result: null, initMsid:
>> >>19873467853209,
>> >> >>completeMsid: null, lastUpdated: null, lastPolled: null, created:
>> >>null}
>> >> >>2018-09-13 06:02:15,991 DEBUG [c.c.u.AccountManagerImpl]
>> >> >>(API-Job-Executor-83:ctx-cfb1cd95 job-28020 ctx-5b8156a6)
>> >> >>(logid:4c7a1c38)
>> >> >>Access to Acct[479c643e-9c84-41fb-9f0a-9bb999893a25-juha] granted to
>> >> >>Acct[479c643e-9c84-41fb-9f0a-9bb999893a25-juha] by DomainChecker
>> >> >>2018-09-13 06:02:16,045 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>> >> >>(API-Job-Executor-83:ctx-cfb1cd95 job-28020 ctx-5b8156a6)
>> >> >>(logid:4c7a1c38)
>> >> >>Sync job-28021 execution on object VmWorkJobQueue.228
>> >> >>2018-09-13 06:02:17,115 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>> >> >>(Work-Job-Executor-53:ctx-3728caef job-28020/job-28021)
>> >>(logid:4c7a1c38)
>> >> >>Executing AsyncJobVO {id:28021, userId: 1, accountId: 7, instanceType:
>> >> >>null, instanceId: null, cmd: com.cloud.vm.VmWorkTakeVolumeSnapshot,
>> >> >>cmdInfo:
>> >>
>> >>>>rO0ABXNyACVjb20uY2xvdWQudm0uVm1Xb3JrVGFrZVZvbHVtZVNuYXBzaG90BL5gG4Li1c8
>> >>>>CA
>> >> >>A
>> >> >>Z
>> >>
>> >>>>aAAthc3luY0JhY2t1cFoACXF1aWVzY2VWbUwADGxvY2F0aW9uVHlwZXQAKUxjb20vY2xvdW
>> >>>>Qv
>> >> >>c
>> >> >>3
>> >>
>> >>>>RvcmFnZS9TbmFwc2hvdCRMb2NhdGlvblR5cGU7TAAIcG9saWN5SWR0ABBMamF2YS9sYW5nL
>> >>>>0x
>> >> >>v
>> >> >>b
>> >>
>> >>>>mc7TAAKc25hcHNob3RJZHEAfgACTAAIdm9sdW1lSWRxAH4AAnhyABNjb20uY2xvdWQudm0u
>> >>>>Vm
>> >> >>1
>> >> >>X
>> >>
>> >>>>b3Jrn5m2VvAlZ2sCAARKAAlhY2NvdW50SWRKAAZ1c2VySWRKAAR2bUlkTAALaGFuZGxlck5
>> >>>>hb
>> >> >>W
>> >> >>V
>> >>
>> >>>>0ABJMamF2YS9sYW5nL1N0cmluZzt4cAAAAAAAAAAHAAAAAAAAAAEAAAAAAAAA5HQAFFZvbH
>> >>>>Vt
>> >> >>Z
>> >> >>U
>> >>
>> >>>>FwaVNlcnZpY2VJbXBsAABwc3IADmphdmEubGFuZy5Mb25nO4vkkMyPI98CAAFKAAV2YWx1Z
>> >>>>Xh
>> >> >>y
>> >> >>A
>> >>
>> >>>>BBqYXZhLmxhbmcuTnVtYmVyhqyVHQuU4IsCAAB4cAAAAAAAAAARc3EAfgAHAAAAAAAACmBz
>> >>>>cQ
>> >> >>B
>> >> >>-
>> >> >>AAcAAAAAAAAA7g, cmdVersion: 0, status: IN_PROGRESS, processStatus: 0,
>> >> >>resultCode: 0, result: null, initMsid: 19873467853209, completeMsid:
>> >> >>null,
>> >> >>lastUpdated: null, lastPolled: null, created: Thu Sep 13 06:02:16 EEST
>> >> >>2018}
>> >> >>2018-09-13 06:02:17,116 DEBUG [c.c.v.VmWorkJobDispatcher]
>> >> >>(Work-Job-Executor-53:ctx-3728caef job-28020/job-28021)
>> >>(logid:4c7a1c38)
>> >> >>Run VM work job: com.cloud.vm.VmWorkTakeVolumeSnapshot for VM 228, job
>> >> >>origin: 28020
>> >> >>2018-09-13 06:02:17,117 DEBUG [c.c.v.VmWorkJobHandlerProxy]
>> >> >>(Work-Job-Executor-53:ctx-3728caef job-28020/job-28021 ctx-e4ed14c2)
>> >> >>(logid:4c7a1c38) Execute VM work job:
>> >>
>> >>>>com.cloud.vm.VmWorkTakeVolumeSnapshot{"volumeId":238,"policyId":17,"sna
>> >>>>ps
>> >> >>h
>> >> >>o
>> >>
>> >>>>tId":2656,"quiesceVm":false,"asyncBackup":false,"userId":1,"accountId":
>> >>>>7,
>> >> >>"
>> >> >>v
>> >> >>mId":228,"handlerName":"VolumeApiServiceImpl"}
>> >> >>2018-09-13 06:02:17,197 DEBUG [c.c.h.o.r.Ovm3HypervisorGuru]
>> >> >>(Work-Job-Executor-53:ctx-3728caef job-28020/job-28021 ctx-e4ed14c2)
>> >> >>(logid:4c7a1c38) getCommandHostDelegation: class
>> >> >>org.apache.cloudstack.storage.command.CreateObjectCommand
>> >> >>2018-09-13 06:02:17,197 DEBUG [c.c.h.XenServerGuru]
>> >> >>(Work-Job-Executor-53:ctx-3728caef job-28020/job-28021 ctx-e4ed14c2)
>> >> >>(logid:4c7a1c38) We are returning the default host to execute commands
>> >> >>because the command is not of Copy type.
>> >> >>2018-09-13 06:02:17,199 DEBUG [c.c.a.t.Request]
>> >> >>(Work-Job-Executor-53:ctx-3728caef job-28020/job-28021 ctx-e4ed14c2)
>> >> >>(logid:4c7a1c38) Seq 128-4630826316843724436: Sending { Cmd , MgmtId:
>> >> >>19873467853209, via: 128(xenserver-15), Ver: v1, Flags: 100011,
>> >>
>> >>>>[{"org.apache.cloudstack.storage.command.CreateObjectCommand":{"data":{
>> >>>>"o
>> >> >>r
>> >> >>g
>> >>
>> >>>>.apache.cloudstack.storage.to.SnapshotObjectTO":{"volume":{"uuid":"969c
>> >>>>79
>> >> >>6
>> >> >>d
>> >>
>> >>>>-072e-4272-adc6-93a492c03ea3","volumeType":"ROOT","dataStore":{"org.apa
>> >>>>ch
>> >> >>e
>> >> >>.
>> >> >>cloudstack.storage.to
>> >> .PrimaryDataStoreTO":{"uuid":"6568d30f-82b6-35a0-bfe
>> >> >>5
>> >> >>-
>> >>
>> >>>>be17c1398ed8","id":2,"poolType":"NetworkFilesystem","host":"10.10.1.33"
>> >>>>,"
>> >> >>p
>> >> >>a
>> >> >>th":"/pool3/primary3","port":2049,"url":"NetworkFilesystem://
>> >> 10.10.1.33/p
>> >> >>o
>> >> >>o
>> >>
>> >>>>l3/primary3/?ROLE=Primary&STOREUUID=6568d30f-82b6-35a0-bfe5-be17c1398ed
>> >>>>8"
>> >> >>,
>> >> >>"
>> >>
>> >>>>isManaged":false}},"name":"ROOT-228","size":21474836480,"path":"3d2a971
>> >>>>6-
>> >> >>8
>> >> >>f
>> >>
>> >>>>a1-497e-8753-941117a0ef50","volumeId":238,"vmName":"i-7-228-VM","accoun
>> >>>>tI
>> >> >>d
>> >> >>"
>> >>
>> >>>>:7,"format":"VHD","provisioningType":"THIN","id":238,"deviceId":0,"hype
>> >>>>rv
>> >> >>i
>> >> >>s
>> >>
>> >>>>orType":"XenServer"},"parentSnapshotPath":"c7b2824a-7402-4f66-b2e6-06ed
>> >>>>45
>> >> >>2
>> >> >>0
>> >> >>72ad","dataStore":{"org.apache.cloudstack.storage.to
>> >> .PrimaryDataStoreTO":
>> >> >>{
>> >> >>"
>> >>
>> >>>>uuid":"6568d30f-82b6-35a0-bfe5-be17c1398ed8","id":2,"poolType":"Network
>> >>>>Fi
>> >> >>l
>> >> >>e
>> >>
>> >>>>system","host":"10.10.1.33","path":"/pool3/primary3","port":2049,"url":
>> >>>>"N
>> >> >>e
>> >> >>t
>> >> >>workFilesystem://
>> >> 10.10.1.33/pool3/primary3/?ROLE=Primary&STOREUUID=6568d3
>> >> >>0
>> >> >>f
>> >>
>> >>>>-82b6-35a0-bfe5-be17c1398ed8","isManaged":false}},"vmName":"i-7-228-VM"
>> >>>>,"
>> >> >>n
>> >> >>a
>> >>
>> >>>>me":"BGI-Maria_ROOT-228_20180913030215","hypervisorType":"XenServer","i
>> >>>>d"
>> >> >>:
>> >> >>2
>> >>
>> >>>>656,"quiescevm":false,"parents":["c7b2824a-7402-4f66-b2e6-06ed452072ad"
>> >>>>],
>> >> >>"
>> >> >>p
>> >> >>hysicalSize":0}},"wait":0}}] }
>> >> >>2018-09-13 06:02:17,199 DEBUG [c.c.a.t.Request]
>> >> >>(Work-Job-Executor-53:ctx-3728caef job-28020/job-28021 ctx-e4ed14c2)
>> >> >>(logid:4c7a1c38) Seq 128-4630826316843724436: Executing: { Cmd ,
>> >>MgmtId:
>> >> >>19873467853209, via: 128(xenserver-15), Ver: v1, Flags: 100011,
>> >>
>> >>>>[{"org.apache.cloudstack.storage.command.CreateObjectCommand":{"data":{
>> >>>>"o
>> >> >>r
>> >> >>g
>> >>
>> >>>>.apache.cloudstack.storage.to.SnapshotObjectTO":{"volume":{"uuid":"969c
>> >>>>79
>> >> >>6
>> >> >>d
>> >>
>> >>>>-072e-4272-adc6-93a492c03ea3","volumeType":"ROOT","dataStore":{"org.apa
>> >>>>ch
>> >> >>e
>> >> >>.
>> >> >>cloudstack.storage.to
>> >> .PrimaryDataStoreTO":{"uuid":"6568d30f-82b6-35a0-bfe
>> >> >>5
>> >> >>-
>> >>
>> >>>>be17c1398ed8","id":2,"poolType":"NetworkFilesystem","host":"10.10.1.33"
>> >>>>,"
>> >> >>p
>> >> >>a
>> >> >>th":"/pool3/primary3","port":2049,"url":"NetworkFilesystem://
>> >> 10.10.1.33/p
>> >> >>o
>> >> >>o
>> >>
>> >>>>l3/primary3/?ROLE=Primary&STOREUUID=6568d30f-82b6-35a0-bfe5-be17c1398ed
>> >>>>8"
>> >> >>,
>> >> >>"
>> >>
>> >>>>isManaged":false}},"name":"ROOT-228","size":21474836480,"path":"3d2a971
>> >>>>6-
>> >> >>8
>> >> >>f
>> >>
>> >>>>a1-497e-8753-941117a0ef50","volumeId":238,"vmName":"i-7-228-VM","accoun
>> >>>>tI
>> >> >>d
>> >> >>"
>> >>
>> >>>>:7,"format":"VHD","provisioningType":"THIN","id":238,"deviceId":0,"hype
>> >>>>rv
>> >> >>i
>> >> >>s
>> >>
>> >>>>orType":"XenServer"},"parentSnapshotPath":"c7b2824a-7402-4f66-b2e6-06ed
>> >>>>45
>> >> >>2
>> >> >>0
>> >> >>72ad","dataStore":{"org.apache.cloudstack.storage.to
>> >> .PrimaryDataStoreTO":
>> >> >>{
>> >> >>"
>> >>
>> >>>>uuid":"6568d30f-82b6-35a0-bfe5-be17c1398ed8","id":2,"poolType":"Network
>> >>>>Fi
>> >> >>l
>> >> >>e
>> >>
>> >>>>system","host":"10.10.1.33","path":"/pool3/primary3","port":2049,"url":
>> >>>>"N
>> >> >>e
>> >> >>t
>> >> >>workFilesystem://
>> >> 10.10.1.33/pool3/primary3/?ROLE=Primary&STOREUUID=6568d3
>> >> >>0
>> >> >>f
>> >>
>> >>>>-82b6-35a0-bfe5-be17c1398ed8","isManaged":false}},"vmName":"i-7-228-VM"
>> >>>>,"
>> >> >>n
>> >> >>a
>> >>
>> >>>>me":"BGI-Maria_ROOT-228_20180913030215","hypervisorType":"XenServer","i
>> >>>>d"
>> >> >>:
>> >> >>2
>> >>
>> >>>>656,"quiescevm":false,"parents":["c7b2824a-7402-4f66-b2e6-06ed452072ad"
>> >>>>],
>> >> >>"
>> >> >>p
>> >> >>hysicalSize":0}},"wait":0}}] }
>> >> >>2018-09-13 06:02:19,815 DEBUG [c.c.a.m.DirectAgentAttache]
>> >> >>(DirectAgent-206:ctx-370f4cd9) (logid:4c7a1c38) Seq
>> >> >>128-4630826316843724436: Response Received:
>> >> >>2018-09-13 06:02:19,815 DEBUG [c.c.a.t.Request]
>> >> >>(DirectAgent-206:ctx-370f4cd9) (logid:4c7a1c38) Seq
>> >> >>128-4630826316843724436: Processing: { Ans: , MgmtId: 19873467853209,
>> >> >>via: 128(xenserver-15), Ver: v1, Flags: 10,
>> >>
>> >>>>[{"org.apache.cloudstack.storage.command.CreateObjectAnswer":{"data":{"
>> >>>>or
>> >> >>g
>> >> >>.
>> >>
>> >>>>apache.cloudstack.storage.to.SnapshotObjectTO":{"path":"e91fb041-779d-4
>> >>>>09
>> >> >>0
>> >> >>-
>> >>
>> >>>>b14c-4f57f4cda54b","id":0,"quiescevm":false,"physicalSize":0}},"result"
>> >>>>:t
>> >> >>r
>> >> >>u
>> >> >>e,"wait":0}}] }
>> >> >>2018-09-13 06:02:19,815 DEBUG [c.c.a.t.Request]
>> >> >>(Work-Job-Executor-53:ctx-3728caef job-28020/job-28021 ctx-e4ed14c2)
>> >> >>(logid:4c7a1c38) Seq 128-4630826316843724436: Received: { Ans: ,
>> >>MgmtId:
>> >> >>19873467853209, via: 128(xenserver-15), Ver: v1, Flags: 10, {
>> >> >>CreateObjectAnswer } }
>> >> >>2018-09-13 06:02:19,934 DEBUG [o.a.c.s.m.AncientDataMotionStrategy]
>> >> >>(Work-Job-Executor-53:ctx-3728caef job-28020/job-28021 ctx-e4ed14c2)
>> >> >>(logid:4c7a1c38) copyAsync inspecting src type SNAPSHOT copyAsync
>> >> >>inspecting dest type SNAPSHOT
>> >> >>2018-09-13 06:02:20,017 ERROR [o.a.c.s.e.DefaultEndPointSelector]
>> >> >>(Work-Job-Executor-53:ctx-3728caef job-28020/job-28021 ctx-e4ed14c2)
>> >> >>(logid:4c7a1c38) IR24 select BACKUPSNAPSHOT from primary to secondary
>> >> >>2656
>> >> >>dest=2656
>> >> >>2018-09-13 06:02:20,023 DEBUG [c.c.h.o.r.Ovm3HypervisorGuru]
>> >> >>(Work-Job-Executor-53:ctx-3728caef job-28020/job-28021 ctx-e4ed14c2)
>> >> >>(logid:4c7a1c38) getCommandHostDelegation: class
>> >> >>org.apache.cloudstack.storage.command.CopyCommand
>> >> >>2018-09-13 06:02:20,023 DEBUG [c.c.h.XenServerGuru]
>> >> >>(Work-Job-Executor-53:ctx-3728caef job-28020/job-28021 ctx-e4ed14c2)
>> >> >>(logid:4c7a1c38) We are returning the default host to execute commands
>> >> >>because the source and destination objects are not NFS type.
>> >> >>2018-09-13 06:02:20,025 DEBUG [c.c.a.t.Request]
>> >> >>(Work-Job-Executor-53:ctx-3728caef job-28020/job-28021 ctx-e4ed14c2)
>> >> >>(logid:4c7a1c38) Seq 130-7345089517264579763: Sending { Cmd , MgmtId:
>> >> >>19873467853209, via: 130(xenserver-81), Ver: v1, Flags: 100111,
>> >>
>> >>>>[{"org.apache.cloudstack.storage.command.CopyCommand":{"srcTO":{"org.ap
>> >>>>ac
>> >> >>h
>> >> >>e
>> >> >>.cloudstack.storage.to
>> >> .SnapshotObjectTO":{"path":"e91fb041-779d-4090-b14c
>> >> >>-
>> >> >>4
>> >>
>> >>>>f57f4cda54b","volume":{"uuid":"969c796d-072e-4272-adc6-93a492c03ea3","v
>> >>>>ol
>> >> >>u
>> >> >>m
>> >>
>> >>>>eType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDat
>> >>>>aS
>> >> >>t
>> >> >>o
>> >>
>> >>>>reTO":{"uuid":"6568d30f-82b6-35a0-bfe5-be17c1398ed8","id":2,"poolType":
>> >>>>"N
>> >> >>e
>> >> >>t
>> >>
>> >>>>workFilesystem","host":"10.10.1.33","path":"/pool3/primary3","port":204
>> >>>>9,
>> >> >>"
>> >> >>u
>> >> >>rl":"NetworkFilesystem://
>> >> 10.10.1.33/pool3/primary3/?ROLE=Primary&STOREUUI
>> >> >>D
>> >> >>=
>> >>
>> >>>>6568d30f-82b6-35a0-bfe5-be17c1398ed8","isManaged":false}},"name":"ROOT-
>> >>>>22
>> >> >>8
>> >> >>"
>> >>
>> >>>>,"size":21474836480,"path":"3d2a9716-8fa1-497e-8753-941117a0ef50","volu
>> >>>>me
>> >> >>I
>> >> >>d
>> >>
>> >>>>":238,"vmName":"i-7-228-VM","accountId":7,"format":"VHD","provisioningT
>> >>>>yp
>> >> >>e
>> >> >>"
>> >>
>> >>>>:"THIN","id":238,"deviceId":0,"hypervisorType":"XenServer"},"parentSnap
>> >>>>sh
>> >> >>o
>> >> >>t
>> >>
>> >>>>Path":"c7b2824a-7402-4f66-b2e6-06ed452072ad","dataStore":{"org.apache.c
>> >>>>lo
>> >> >>u
>> >> >>d
>> >> >>stack.storage.to
>> >> .PrimaryDataStoreTO":{"uuid":"6568d30f-82b6-35a0-bfe5-be1
>> >> >>7
>> >> >>c
>> >>
>> >>>>1398ed8","id":2,"poolType":"NetworkFilesystem","host":"10.10.1.33","pat
>> >>>>h"
>> >> >>:
>> >> >>"
>> >> >>/pool3/primary3","port":2049,"url":"NetworkFilesystem://
>> >> 10.10.1.33/pool3/
>> >> >>p
>> >> >>r
>> >>
>> >>>>imary3/?ROLE=Primary&STOREUUID=6568d30f-82b6-35a0-bfe5-be17c1398ed8","i
>> >>>>sM
>> >> >>a
>> >> >>n
>> >>
>> >>>>aged":false}},"vmName":"i-7-228-VM","name":"BGI-Maria_ROOT-228_20180913
>> >>>>03
>> >> >>0
>> >> >>2
>> >>
>> >>>>15","hypervisorType":"XenServer","id":2656,"quiescevm":false,"parents":
>> >>>>["
>> >> >>c
>> >> >>7
>> >>
>> >>>>b2824a-7402-4f66-b2e6-06ed452072ad"],"physicalSize":0}},"destTO":{"org.
>> >>>>ap
>> >> >>a
>> >> >>c
>> >> >>he.cloudstack.storage.to
>> >> .SnapshotObjectTO":{"path":"snapshots/7/238","vol
>> >> >>u
>> >> >>m
>> >>
>> >>>>e":{"uuid":"969c796d-072e-4272-adc6-93a492c03ea3","volumeType":"ROOT","
>> >>>>da
>> >> >>t
>> >> >>a
>> >> >>Store":{"org.apache.cloudstack.storage.to
>> >> .PrimaryDataStoreTO":{"uuid":"65
>> >> >>6
>> >> >>8
>> >>
>> >>>>d30f-82b6-35a0-bfe5-be17c1398ed8","id":2,"poolType":"NetworkFilesystem"
>> >>>>,"
>> >> >>h
>> >> >>o
>> >>
>> >>>>st":"10.10.1.33","path":"/pool3/primary3","port":2049,"url":"NetworkFil
>> >>>>es
>> >> >>y
>> >> >>s
>> >> >>tem://
>> >> 10.10.1.33/pool3/primary3/?ROLE=Primary&STOREUUID=6568d30f-82b6-35a
>> >> >>0
>> >> >>-
>> >>
>> >>>>bfe5-be17c1398ed8","isManaged":false}},"name":"ROOT-228","size":2147483
>> >>>>64
>> >> >>8
>> >> >>0
>> >>
>> >>>>,"path":"3d2a9716-8fa1-497e-8753-941117a0ef50","volumeId":238,"vmName":
>> >>>>"i
>> >> >>-
>> >> >>7
>> >>
>> >>>>-228-VM","accountId":7,"format":"VHD","provisioningType":"THIN","id":23
>> >>>>8,
>> >> >>"
>> >> >>d
>> >>
>> >>>>eviceId":0,"hypervisorType":"XenServer"},"parentSnapshotPath":"snapshot
>> >>>>s/
>> >> >>7
>> >> >>/
>> >>
>> >>>>238/0c8c6ee3-ce3d-4241-8509-5193d2cff491.vhd","dataStore":{"com.cloud.a
>> >>>>ge
>> >> >>n
>> >> >>t
>> >> >>.api.to.NfsTO":{"_url":"nfs://10.10.1.34/pool8/secondary8
>> >> ","_role":"Image
>> >> >>"
>> >> >>}
>> >>
>> >>>>},"vmName":"i-7-228-VM","name":"BGI-Maria_ROOT-228_20180913030215","hyp
>> >>>>er
>> >> >>v
>> >> >>i
>> >>
>> >>>>sorType":"XenServer","id":2656,"quiescevm":false,"parents":["snapshots/
>> >>>>7/
>> >> >>2
>> >> >>3
>> >>
>> >>>>8/93a42896-0da1-4b82-8985-d28464f1ac4a.vhd","snapshots/7/238/0c8c6ee3-c
>> >>>>e3
>> >> >>d
>> >> >>-
>> >>
>> >>>>4241-8509-5193d2cff491.vhd"],"physicalSize":0}},"executeInSequence":tru
>> >>>>e,
>> >> >>"
>> >> >>o
>> >> >>ptions":{"fullSnapshot":"false"},"options2":{},"wait":28800}}] }
>> >> >>2018-09-13 06:02:20,025 DEBUG [c.c.a.t.Request]
>> >> >>(Work-Job-Executor-53:ctx-3728caef job-28020/job-28021 ctx-e4ed14c2)
>> >> >>(logid:4c7a1c38) Seq 130-7345089517264579763: Executing: { Cmd ,
>> >>MgmtId:
>> >> >>19873467853209, via: 130(xenserver-81), Ver: v1, Flags: 100111,
>> >>
>> >>>>[{"org.apache.cloudstack.storage.command.CopyCommand":{"srcTO":{"org.ap
>> >>>>ac
>> >> >>h
>> >> >>e
>> >> >>.cloudstack.storage.to
>> >> .SnapshotObjectTO":{"path":"e91fb041-779d-4090-b14c
>> >> >>-
>> >> >>4
>> >>
>> >>>>f57f4cda54b","volume":{"uuid":"969c796d-072e-4272-adc6-93a492c03ea3","v
>> >>>>ol
>> >> >>u
>> >> >>m
>> >>
>> >>>>eType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDat
>> >>>>aS
>> >> >>t
>> >> >>o
>> >>
>> >>>>reTO":{"uuid":"6568d30f-82b6-35a0-bfe5-be17c1398ed8","id":2,"poolType":
>> >>>>"N
>> >> >>e
>> >> >>t
>> >>
>> >>>>workFilesystem","host":"10.10.1.33","path":"/pool3/primary3","port":204
>> >>>>9,
>> >> >>"
>> >> >>u
>> >> >>rl":"NetworkFilesystem://
>> >> 10.10.1.33/pool3/primary3/?ROLE=Primary&STOREUUI
>> >> >>D
>> >> >>=
>> >>
>> >>>>6568d30f-82b6-35a0-bfe5-be17c1398ed8","isManaged":false}},"name":"ROOT-
>> >>>>22
>> >> >>8
>> >> >>"
>> >>
>> >>>>,"size":21474836480,"path":"3d2a9716-8fa1-497e-8753-941117a0ef50","volu
>> >>>>me
>> >> >>I
>> >> >>d
>> >>
>> >>>>":238,"vmName":"i-7-228-VM","accountId":7,"format":"VHD","provisioningT
>> >>>>yp
>> >> >>e
>> >> >>"
>> >>
>> >>>>:"THIN","id":238,"deviceId":0,"hypervisorType":"XenServer"},"parentSnap
>> >>>>sh
>> >> >>o
>> >> >>t
>> >>
>> >>>>Path":"c7b2824a-7402-4f66-b2e6-06ed452072ad","dataStore":{"org.apache.c
>> >>>>lo
>> >> >>u
>> >> >>d
>> >> >>stack.storage.to
>> >> .PrimaryDataStoreTO":{"uuid":"6568d30f-82b6-35a0-bfe5-be1
>> >> >>7
>> >> >>c
>> >>
>> >>>>1398ed8","id":2,"poolType":"NetworkFilesystem","host":"10.10.1.33","pat
>> >>>>h"
>> >> >>:
>> >> >>"
>> >> >>/pool3/primary3","port":2049,"url":"NetworkFilesystem://
>> >> 10.10.1.33/pool3/
>> >> >>p
>> >> >>r
>> >>
>> >>>>imary3/?ROLE=Primary&STOREUUID=6568d30f-82b6-35a0-bfe5-be17c1398ed8","i
>> >>>>sM
>> >> >>a
>> >> >>n
>> >>
>> >>>>aged":false}},"vmName":"i-7-228-VM","name":"BGI-Maria_ROOT-228_20180913
>> >>>>03
>> >> >>0
>> >> >>2
>> >>
>> >>>>15","hypervisorType":"XenServer","id":2656,"quiescevm":false,"parents":
>> >>>>["
>> >> >>c
>> >> >>7
>> >>
>> >>>>b2824a-7402-4f66-b2e6-06ed452072ad"],"physicalSize":0}},"destTO":{"org.
>> >>>>ap
>> >> >>a
>> >> >>c
>> >> >>he.cloudstack.storage.to
>> >> .SnapshotObjectTO":{"path":"snapshots/7/238","vol
>> >> >>u
>> >> >>m
>> >>
>> >>>>e":{"uuid":"969c796d-072e-4272-adc6-93a492c03ea3","volumeType":"ROOT","
>> >>>>da
>> >> >>t
>> >> >>a
>> >> >>Store":{"org.apache.cloudstack.storage.to
>> >> .PrimaryDataStoreTO":{"uuid":"65
>> >> >>6
>> >> >>8
>> >>
>> >>>>d30f-82b6-35a0-bfe5-be17c1398ed8","id":2,"poolType":"NetworkFilesystem"
>> >>>>,"
>> >> >>h
>> >> >>o
>> >>
>> >>>>st":"10.10.1.33","path":"/pool3/primary3","port":2049,"url":"NetworkFil
>> >>>>es
>> >> >>y
>> >> >>s
>> >> >>tem://
>> >> 10.10.1.33/pool3/primary3/?ROLE=Primary&STOREUUID=6568d30f-82b6-35a
>> >> >>0
>> >> >>-
>> >>
>> >>>>bfe5-be17c1398ed8","isManaged":false}},"name":"ROOT-228","size":2147483
>> >>>>64
>> >> >>8
>> >> >>0
>> >>
>> >>>>,"path":"3d2a9716-8fa1-497e-8753-941117a0ef50","volumeId":238,"vmName":
>> >>>>"i
>> >> >>-
>> >> >>7
>> >>
>> >>>>-228-VM","accountId":7,"format":"VHD","provisioningType":"THIN","id":23
>> >>>>8,
>> >> >>"
>> >> >>d
>> >>
>> >>>>eviceId":0,"hypervisorType":"XenServer"},"parentSnapshotPath":"snapshot
>> >>>>s/
>> >> >>7
>> >> >>/
>> >>
>> >>>>238/0c8c6ee3-ce3d-4241-8509-5193d2cff491.vhd","dataStore":{"com.cloud.a
>> >>>>ge
>> >> >>n
>> >> >>t
>> >> >>.api.to.NfsTO":{"_url":"nfs://10.10.1.34/pool8/secondary8
>> >> ","_role":"Image
>> >> >>"
>> >> >>}
>> >>
>> >>>>},"vmName":"i-7-228-VM","name":"BGI-Maria_ROOT-228_20180913030215","hyp
>> >>>>er
>> >> >>v
>> >> >>i
>> >>
>> >>>>sorType":"XenServer","id":2656,"quiescevm":false,"parents":["snapshots/
>> >>>>7/
>> >> >>2
>> >> >>3
>> >>
>> >>>>8/93a42896-0da1-4b82-8985-d28464f1ac4a.vhd","snapshots/7/238/0c8c6ee3-c
>> >>>>e3
>> >> >>d
>> >> >>-
>> >>
>> >>>>4241-8509-5193d2cff491.vhd"],"physicalSize":0}},"executeInSequence":tru
>> >>>>e,
>> >> >>"
>> >> >>o
>> >> >>ptions":{"fullSnapshot":"false"},"options2":{},"wait":28800}}] }
>> >> >>2018-09-13 06:04:28,895 DEBUG [c.c.h.x.r.CitrixResourceBase]
>> >> >>(DirectAgent-382:ctx-36b817c8) (logid:4c7a1c38) Host 10.10.1.81
>> >> >>OpaqueRef:f83331df-6634-63fd-6b95-44607a6069b8: Removing SR
>> >> >>2018-09-13 06:04:28,913 DEBUG [c.c.h.x.r.CitrixResourceBase]
>> >> >>(DirectAgent-382:ctx-36b817c8) (logid:4c7a1c38) Host 10.10.1.81
>> >> >>OpaqueRef:05380b03-4cb8-85a7-6a6a-a1595c188f60: Unplugging pbd
>> >> >>2018-09-13 06:04:30,881 DEBUG [c.c.h.x.r.CitrixResourceBase]
>> >> >>(DirectAgent-382:ctx-36b817c8) (logid:4c7a1c38) Host 10.10.1.81
>> >> >>OpaqueRef:f83331df-6634-63fd-6b95-44607a6069b8: Forgetting
>> >> >>2018-09-13 06:04:30,891 DEBUG [c.c.h.x.r.XenServerStorageProcessor]
>> >> >>(DirectAgent-382:ctx-36b817c8) (logid:4c7a1c38) Successfully destroyed
>> >> >>snapshot on volume: 3d2a9716-8fa1-497e-8753-941117a0ef50 execept this
>> >> >>current snapshot e91fb041-779d-4090-b14c-4f57f4cda54b
>> >> >>2018-09-13 06:04:30,891 INFO [c.c.h.x.r.XenServerStorageProcessor]
>> >> >>(DirectAgent-382:ctx-36b817c8) (logid:4c7a1c38) New snapshot details:
>> >>
>> >>>>SnapshotTO[datastore=null|volume=null|pathsnapshots/7/238/0560d536-c945
>> >>>>-4
>> >> >>a
>> >> >>6
>> >> >>1-8c4c-dbaf29f29993.vhd]
>> >> >>2018-09-13 06:04:30,891 INFO [c.c.h.x.r.XenServerStorageProcessor]
>> >> >>(DirectAgent-382:ctx-36b817c8) (logid:4c7a1c38) New snapshot physical
>> >> >>utilization: 2342932992
>> >> >>2018-09-13 06:04:30,891 DEBUG [c.c.a.m.DirectAgentAttache]
>> >> >>(DirectAgent-382:ctx-36b817c8) (logid:4c7a1c38) Seq
>> >> >>130-7345089517264579763: Response Received:
>> >> >>2018-09-13 06:04:30,891 DEBUG [c.c.a.t.Request]
>> >> >>(DirectAgent-382:ctx-36b817c8) (logid:4c7a1c38) Seq
>> >> >>130-7345089517264579763: Processing: { Ans: , MgmtId: 19873467853209,
>> >> >>via: 130(xenserver-81), Ver: v1, Flags: 110,
>> >>
>> >>>>[{"org.apache.cloudstack.storage.command.CopyCmdAnswer":{"newData":{"or
>> >>>>g.
>> >> >>a
>> >> >>p
>> >> >>ache.cloudstack.storage.to
>> >> .SnapshotObjectTO":{"path":"snapshots/7/238/056
>> >> >>0
>> >> >>d
>> >>
>> >>>>536-c945-4a61-8c4c-dbaf29f29993.vhd","parentSnapshotPath":"snapshots/7/
>> >>>>23
>> >> >>8
>> >> >>/
>> >>
>> >>>>0c8c6ee3-ce3d-4241-8509-5193d2cff491.vhd","id":0,"quiescevm":false,"phy
>> >>>>si
>> >> >>c
>> >> >>a
>> >> >>lSize":2342932992}},"result":true,"wait":0}}] }
>> >> >>2018-09-13 06:04:30,891 DEBUG [c.c.a.m.AgentAttache]
>> >> >>(DirectAgent-382:ctx-36b817c8) (logid:4c7a1c38) Seq
>> >> >>130-7345089517264579763: No more commands found
>> >> >>2018-09-13 06:04:30,891 DEBUG [c.c.a.t.Request]
>> >> >>(Work-Job-Executor-53:ctx-3728caef job-28020/job-28021 ctx-e4ed14c2)
>> >> >>(logid:4c7a1c38) Seq 130-7345089517264579763: Received: { Ans: ,
>> >>MgmtId:
>> >> >>19873467853209, via: 130(xenserver-81), Ver: v1, Flags: 110, {
>> >> >>CopyCmdAnswer } }
>> >> >>2018-09-13 06:04:30,957 DEBUG [c.c.r.ResourceLimitManagerImpl]
>> >> >>(Work-Job-Executor-53:ctx-3728caef job-28020/job-28021 ctx-e4ed14c2)
>> >> >>(logid:4c7a1c38) Updating resource Type = secondary_storage count for
>> >> >>Account = 7 Operation = decreasing Amount = 19131903488
>> >> >>2018-09-13 06:04:30,969 DEBUG [c.c.v.VmWorkJobHandlerProxy]
>> >> >>(Work-Job-Executor-53:ctx-3728caef job-28020/job-28021 ctx-e4ed14c2)
>> >> >>(logid:4c7a1c38) Done executing VM work job:
>> >>
>> >>>>com.cloud.vm.VmWorkTakeVolumeSnapshot{"volumeId":238,"policyId":17,"sna
>> >>>>ps
>> >> >>h
>> >> >>o
>> >>
>> >>>>tId":2656,"quiesceVm":false,"asyncBackup":false,"userId":1,"accountId":
>> >>>>7,
>> >> >>"
>> >> >>v
>> >> >>mId":228,"handlerName":"VolumeApiServiceImpl"}
>> >> >>2018-09-13 06:04:30,969 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>> >> >>(Work-Job-Executor-53:ctx-3728caef job-28020/job-28021 ctx-e4ed14c2)
>> >> >>(logid:4c7a1c38) Complete async job-28021, jobStatus: SUCCEEDED,
>> >> >>resultCode: 0, result:
>> >>
>> >>>>rO0ABXNyAA5qYXZhLmxhbmcuTG9uZzuL5JDMjyPfAgABSgAFdmFsdWV4cgAQamF2YS5sYW5
>> >>>>nL
>> >> >>k
>> >> >>5
>> >> >>1bWJlcoaslR0LlOCLAgAAeHAAAAAAAAAKYA
>> >> >>2018-09-13 06:04:30,970 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>> >> >>(Work-Job-Executor-53:ctx-3728caef job-28020/job-28021 ctx-e4ed14c2)
>> >> >>(logid:4c7a1c38) Publish async job-28021 complete on message bus
>> >> >>2018-09-13 06:04:30,970 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>> >> >>(Work-Job-Executor-53:ctx-3728caef job-28020/job-28021 ctx-e4ed14c2)
>> >> >>(logid:4c7a1c38) Wake up jobs related to job-28021
>> >> >>2018-09-13 06:04:30,970 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>> >> >>(Work-Job-Executor-53:ctx-3728caef job-28020/job-28021 ctx-e4ed14c2)
>> >> >>(logid:4c7a1c38) Update db status for job-28021
>> >> >>2018-09-13 06:04:30,971 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>> >> >>(Work-Job-Executor-53:ctx-3728caef job-28020/job-28021 ctx-e4ed14c2)
>> >> >>(logid:4c7a1c38) Wake up jobs joined with job-28021 and disjoin all
>> >> >>subjobs created from job- 28021
>> >> >>2018-09-13 06:04:30,983 DEBUG [c.c.v.VmWorkJobDispatcher]
>> >> >>(Work-Job-Executor-53:ctx-3728caef job-28020/job-28021)
>> >>(logid:4c7a1c38)
>> >> >>Done with run of VM work job: com.cloud.vm.VmWorkTakeVolumeSnapshot
>> >>for
>> >> >>VM
>> >> >>228, job origin: 28020
>> >> >>2018-09-13 06:04:30,983 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>> >> >>(Work-Job-Executor-53:ctx-3728caef job-28020/job-28021)
>> >>(logid:4c7a1c38)
>> >> >>Done executing com.cloud.vm.VmWorkTakeVolumeSnapshot for job-28021
>> >> >>2018-09-13 06:04:30,984 INFO [o.a.c.f.j.i.AsyncJobMonitor]
>> >> >>(Work-Job-Executor-53:ctx-3728caef job-28020/job-28021)
>> >>(logid:4c7a1c38)
>> >> >>Remove job-28021 from job monitoring
>> >> >>2018-09-13 06:04:31,026 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>> >> >>(API-Job-Executor-83:ctx-cfb1cd95 job-28020 ctx-5b8156a6)
>> >> >>(logid:4c7a1c38)
>> >> >>Complete async job-28020, jobStatus: SUCCEEDED, resultCode: 0, result:
>> >>
>> >>>>org.apache.cloudstack.api.response.SnapshotResponse/snapshot/{"id":"3c8
>> >>>>66
>> >> >>2
>> >> >>b
>> >>
>> >>>>2-c76f-4e3a-a928-f76dad820122","account":"juha","domainid":"d3c2420f-75
>> >>>>03
>> >> >>-
>> >> >>4
>> >>
>> >>>>560-a2da-d7e7149ce390","domain":"BGI","snapshottype":"DAILY","volumeid"
>> >>>>:"
>> >> >>9
>> >> >>6
>> >>
>> >>>>9c796d-072e-4272-adc6-93a492c03ea3","volumename":"ROOT-228","volumetype
>> >>>>":
>> >> >>"
>> >> >>R
>> >>
>> >>>>OOT","created":"2018-09-13T06:02:15+0300","name":"BGI-Maria_ROOT-228_20
>> >>>>18
>> >> >>0
>> >> >>9
>> >>
>> >>>>13030215","intervaltype":"DAILY","state":"BackedUp","physicalsize":2342
>> >>>>93
>> >> >>2
>> >> >>9
>> >>
>> >>>>92,"zoneid":"533c8f88-2637-4c0e-8210-e0a355b62d82","tags":[],"revertabl
>> >>>>e"
>> >> >>:
>> >> >>f
>> >>
>> >>>>alse,"ostypeid":"da491b96-3748-11e5-a683-121326fde999","osdisplayname":
>> >>>>"C
>> >> >>e
>> >> >>n
>> >> >>tOS 6.5 (64-bit)","virtualsize":21474836480}
>> >> >>2018-09-13 06:04:31,027 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>> >> >>(API-Job-Executor-83:ctx-cfb1cd95 job-28020 ctx-5b8156a6)
>> >> >>(logid:4c7a1c38)
>> >> >>Publish async job-28020 complete on message bus
>> >> >>2018-09-13 06:04:31,027 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>> >> >>(API-Job-Executor-83:ctx-cfb1cd95 job-28020 ctx-5b8156a6)
>> >> >>(logid:4c7a1c38)
>> >> >>Wake up jobs related to job-28020
>> >> >>2018-09-13 06:04:31,027 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>> >> >>(API-Job-Executor-83:ctx-cfb1cd95 job-28020 ctx-5b8156a6)
>> >> >>(logid:4c7a1c38)
>> >> >>Update db status for job-28020
>> >> >>2018-09-13 06:04:31,028 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>> >> >>(API-Job-Executor-83:ctx-cfb1cd95 job-28020 ctx-5b8156a6)
>> >> >>(logid:4c7a1c38)
>> >> >>Wake up jobs joined with job-28020 and disjoin all subjobs created
>> >>from
>> >> >>job- 28020
>> >> >>2018-09-13 06:04:31,039 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>> >> >>(API-Job-Executor-83:ctx-cfb1cd95 job-28020) (logid:4c7a1c38) Done
>> >> >>executing
>> >> >>org.apache.cloudstack.api.command.user.snapshot.CreateSnapshotCmd for
>> >> >>job-28020
>> >> >>2018-09-13 06:04:31,039 INFO [o.a.c.f.j.i.AsyncJobMonitor]
>> >> >>(API-Job-Executor-83:ctx-cfb1cd95 job-28020) (logid:4c7a1c38) Remove
>> >> >>job-28020 from job monitoring
>> >>
>> >>
>> >>
>> >
>> >--
>> >Rafael Weingärtner
>>
>>
>>
>
> --
> Rafael Weingärtner
