Maaany release ago, deleting Ceph volume snap, was also only deleting it in
DB, so the RBD performance become terrible with many tens of (i. e. Hourly)
snapshots. I'll try to verify this on 4.13 myself, but Wido and the guys
will know better...
I
On Sat, Sep 7, 2019, 08:34 li jerry <div...@hotmail.com> wrote:
> I found it had nothing to do with storage.cleanup.delay and
> storage.cleanup.interval.
>
>
>
> The reason is that when DeleteSnapshot Cmd is executed, because the RBD
> snapshot does not have Copy to secondary storage, it only changes the
> database information, and does not enter the main storage to delete the
> snapshot.
>
>
>
>
>
> Log===========================
>
>
>
> 2019-09-07 23:27:00,118 DEBUG [c.c.a.ApiServlet]
> (qtp504527234-17:ctx-2e407b61) (logid:445cbea8) ===START=== 192.168.254.3
> -- GET
> command=deleteSnapshot&id=0b50eb7e-4f42-4de7-96c2-1fae137c8c9f&response=json&_=1567869534480
>
> 2019-09-07 23:27:00,139 DEBUG [c.c.a.ApiServer]
> (qtp504527234-17:ctx-2e407b61 ctx-679fd276) (logid:445cbea8) CIDRs from
> which account 'Acct[2f96c108-9408-11e9-a820-0200582b001a-admin]' is allowed
> to perform API calls: 0.0.0.0/0,::/0
>
> 2019-09-07 23:27:00,204 DEBUG [c.c.a.ApiServer]
> (qtp504527234-17:ctx-2e407b61 ctx-679fd276) (logid:445cbea8) Retrieved
> cmdEventType from job info: SNAPSHOT.DELETE
>
> 2019-09-07 23:27:00,217 INFO [o.a.c.f.j.i.AsyncJobMonitor]
> (API-Job-Executor-2:ctx-f0843047 job-1378) (logid:c34a368a) Add job-1378
> into job monitoring
>
> 2019-09-07 23:27:00,219 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> (qtp504527234-17:ctx-2e407b61 ctx-679fd276) (logid:445cbea8) submit async
> job-1378, details: AsyncJobVO {id:1378, userId: 2, accountId: 2,
> instanceType: Snapshot, instanceId: 13, cmd:
> org.apache.cloudstack.api.command.user.snapshot.DeleteSnapshotCmd, cmdInfo:
> {"response":"json","ctxUserId":"2","httpmethod":"GET","ctxStartEventId":"1237","id":"0b50eb7e-4f42-4de7-96c2-1fae137c8c9f","ctxDetails":"{\"interface
> com.cloud.storage.Snapshot\":\"0b50eb7e-4f42-4de7-96c2-1fae137c8c9f\"}","ctxAccountId":"2","uuid":"0b50eb7e-4f42-4de7-96c2-1fae137c8c9f","cmdEventType":"SNAPSHOT.DELETE","_":"1567869534480"},
> cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0,
> result: null, initMsid: 2200502468634, completeMsid: null, lastUpdated:
> null, lastPolled: null, created: null, removed: null}
>
> 2019-09-07 23:27:00,220 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> (API-Job-Executor-2:ctx-f0843047 job-1378) (logid:1cee5097) Executing
> AsyncJobVO {id:1378, userId: 2, accountId: 2, instanceType: Snapshot,
> instanceId: 13, cmd:
> org.apache.cloudstack.api.command.user.snapshot.DeleteSnapshotCmd, cmdInfo:
> {"response":"json","ctxUserId":"2","httpmethod":"GET","ctxStartEventId":"1237","id":"0b50eb7e-4f42-4de7-96c2-1fae137c8c9f","ctxDetails":"{\"interface
> com.cloud.storage.Snapshot\":\"0b50eb7e-4f42-4de7-96c2-1fae137c8c9f\"}","ctxAccountId":"2","uuid":"0b50eb7e-4f42-4de7-96c2-1fae137c8c9f","cmdEventType":"SNAPSHOT.DELETE","_":"1567869534480"},
> cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0,
> result: null, initMsid: 2200502468634, completeMsid: null, lastUpdated:
> null, lastPolled: null, created: null, removed: null}
>
> 2019-09-07 23:27:00,221 DEBUG [c.c.a.ApiServlet]
> (qtp504527234-17:ctx-2e407b61 ctx-679fd276) (logid:445cbea8) ===END===
> 192.168.254.3 -- GET
> command=deleteSnapshot&id=0b50eb7e-4f42-4de7-96c2-1fae137c8c9f&response=json&_=1567869534480
>
> 2019-09-07 23:27:00,305 DEBUG [c.c.a.m.ClusteredAgentAttache]
> (AgentManager-Handler-12:null) (logid:) Seq 1-8660140608456756853: Routing
> from 2199066247173
>
> 2019-09-07 23:27:00,305 DEBUG [o.a.c.s.s.XenserverSnapshotStrategy]
> (API-Job-Executor-2:ctx-f0843047 job-1378 ctx-f50e25a4) (logid:1cee5097)
> Can't find snapshot on backup storage, delete it in db
>
>
>
> -Jerry
>
>
>
> ________________________________
> 发件人: Andrija Panic <andrija.pa...@gmail.com>
> 发送时间: Saturday, September 7, 2019 1:07:19 AM
> 收件人: users <users@cloudstack.apache.org>
> 抄送: d...@cloudstack.apache.org <d...@cloudstack.apache.org>
> 主题: Re: 4.13 rbd snapshot delete failed
>
> storage.cleanup.delay
> storage.cleanup.interval
>
> put both to 60 (seconds) and wait for up to 2min - should be deleted just
> fine...
>
> cheers
>
> On Fri, 6 Sep 2019 at 18:52, li jerry <div...@hotmail.com> wrote:
>
> > Hello All
> >
> > When I tested ACS 4.13 KVM + CEPH snapshot, I found that snapshots could
> > be created and rolled back (using API alone), but deletion could not be
> > completed.
> >
> >
> >
> > After executing the deletion API, the snapshot will disappear from the
> > list Snapshots, but the snapshot on CEPH RBD will not be deleted (rbd
> snap
> > list rbd/ac510428-5d09-4e86-9d34-9dfab3715b7c)
> >
> >
> >
> > Is there any way we can completely delete the snapshot?
> >
> > -Jerry
> >
> >
>
> --
>
> Andrija Panić
>
