On 9/8/19 5:26 AM, Andrija Panic wrote:
> Maaany release ago, deleting Ceph volume snap, was also only deleting it in
> DB, so the RBD performance become terrible with many tens of (i. e. Hourly)
> snapshots. I'll try to verify this on 4.13 myself, but Wido and the guys
> will know better...
I pinged Gabriel and he's looking into it. He'll get back to it.
Wido
>
> I
>
> On Sat, Sep 7, 2019, 08:34 li jerry <div...@hotmail.com> wrote:
>
>> I found it had nothing to do with storage.cleanup.delay and
>> storage.cleanup.interval.
>>
>>
>>
>> The reason is that when DeleteSnapshot Cmd is executed, because the RBD
>> snapshot does not have Copy to secondary storage, it only changes the
>> database information, and does not enter the main storage to delete the
>> snapshot.
>>
>>
>>
>>
>>
>> Log===========================
>>
>>
>>
>> 2019-09-07 23:27:00,118 DEBUG [c.c.a.ApiServlet]
>> (qtp504527234-17:ctx-2e407b61) (logid:445cbea8) ===START=== 192.168.254.3
>> -- GET
>> command=deleteSnapshot&id=0b50eb7e-4f42-4de7-96c2-1fae137c8c9f&response=json&_=1567869534480
>>
>> 2019-09-07 23:27:00,139 DEBUG [c.c.a.ApiServer]
>> (qtp504527234-17:ctx-2e407b61 ctx-679fd276) (logid:445cbea8) CIDRs from
>> which account 'Acct[2f96c108-9408-11e9-a820-0200582b001a-admin]' is allowed
>> to perform API calls: 0.0.0.0/0,::/0
>>
>> 2019-09-07 23:27:00,204 DEBUG [c.c.a.ApiServer]
>> (qtp504527234-17:ctx-2e407b61 ctx-679fd276) (logid:445cbea8) Retrieved
>> cmdEventType from job info: SNAPSHOT.DELETE
>>
>> 2019-09-07 23:27:00,217 INFO [o.a.c.f.j.i.AsyncJobMonitor]
>> (API-Job-Executor-2:ctx-f0843047 job-1378) (logid:c34a368a) Add job-1378
>> into job monitoring
>>
>> 2019-09-07 23:27:00,219 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (qtp504527234-17:ctx-2e407b61 ctx-679fd276) (logid:445cbea8) submit async
>> job-1378, details: AsyncJobVO {id:1378, userId: 2, accountId: 2,
>> instanceType: Snapshot, instanceId: 13, cmd:
>> org.apache.cloudstack.api.command.user.snapshot.DeleteSnapshotCmd, cmdInfo:
>> {"response":"json","ctxUserId":"2","httpmethod":"GET","ctxStartEventId":"1237","id":"0b50eb7e-4f42-4de7-96c2-1fae137c8c9f","ctxDetails":"{\"interface
>> com.cloud.storage.Snapshot\":\"0b50eb7e-4f42-4de7-96c2-1fae137c8c9f\"}","ctxAccountId":"2","uuid":"0b50eb7e-4f42-4de7-96c2-1fae137c8c9f","cmdEventType":"SNAPSHOT.DELETE","_":"1567869534480"},
>> cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0,
>> result: null, initMsid: 2200502468634, completeMsid: null, lastUpdated:
>> null, lastPolled: null, created: null, removed: null}
>>
>> 2019-09-07 23:27:00,220 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (API-Job-Executor-2:ctx-f0843047 job-1378) (logid:1cee5097) Executing
>> AsyncJobVO {id:1378, userId: 2, accountId: 2, instanceType: Snapshot,
>> instanceId: 13, cmd:
>> org.apache.cloudstack.api.command.user.snapshot.DeleteSnapshotCmd, cmdInfo:
>> {"response":"json","ctxUserId":"2","httpmethod":"GET","ctxStartEventId":"1237","id":"0b50eb7e-4f42-4de7-96c2-1fae137c8c9f","ctxDetails":"{\"interface
>> com.cloud.storage.Snapshot\":\"0b50eb7e-4f42-4de7-96c2-1fae137c8c9f\"}","ctxAccountId":"2","uuid":"0b50eb7e-4f42-4de7-96c2-1fae137c8c9f","cmdEventType":"SNAPSHOT.DELETE","_":"1567869534480"},
>> cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0,
>> result: null, initMsid: 2200502468634, completeMsid: null, lastUpdated:
>> null, lastPolled: null, created: null, removed: null}
>>
>> 2019-09-07 23:27:00,221 DEBUG [c.c.a.ApiServlet]
>> (qtp504527234-17:ctx-2e407b61 ctx-679fd276) (logid:445cbea8) ===END===
>> 192.168.254.3 -- GET
>> command=deleteSnapshot&id=0b50eb7e-4f42-4de7-96c2-1fae137c8c9f&response=json&_=1567869534480
>>
>> 2019-09-07 23:27:00,305 DEBUG [c.c.a.m.ClusteredAgentAttache]
>> (AgentManager-Handler-12:null) (logid:) Seq 1-8660140608456756853: Routing
>> from 2199066247173
>>
>> 2019-09-07 23:27:00,305 DEBUG [o.a.c.s.s.XenserverSnapshotStrategy]
>> (API-Job-Executor-2:ctx-f0843047 job-1378 ctx-f50e25a4) (logid:1cee5097)
>> Can't find snapshot on backup storage, delete it in db
>>
>>
>>
>> -Jerry
>>
>>
>>
>> ________________________________
>> 发件人: Andrija Panic <andrija.pa...@gmail.com>
>> 发送时间: Saturday, September 7, 2019 1:07:19 AM
>> 收件人: users <users@cloudstack.apache.org>
>> 抄送: d...@cloudstack.apache.org <d...@cloudstack.apache.org>
>> 主题: Re: 4.13 rbd snapshot delete failed
>>
>> storage.cleanup.delay
>> storage.cleanup.interval
>>
>> put both to 60 (seconds) and wait for up to 2min - should be deleted just
>> fine...
>>
>> cheers
>>
>> On Fri, 6 Sep 2019 at 18:52, li jerry <div...@hotmail.com> wrote:
>>
>>> Hello All
>>>
>>> When I tested ACS 4.13 KVM + CEPH snapshot, I found that snapshots could
>>> be created and rolled back (using API alone), but deletion could not be
>>> completed.
>>>
>>>
>>>
>>> After executing the deletion API, the snapshot will disappear from the
>>> list Snapshots, but the snapshot on CEPH RBD will not be deleted (rbd
>> snap
>>> list rbd/ac510428-5d09-4e86-9d34-9dfab3715b7c)
>>>
>>>
>>>
>>> Is there any way we can completely delete the snapshot?
>>>
>>> -Jerry
>>>
>>>
>>
>> --
>>
>> Andrija Panić
>>
>
