Hi, Seems like the sparsify and manual fstrim is doing what it needs to do. When sparsify the image, if image has snapshots let say 3 snapshots, need to wait until it rotates all of them (remove and create with new set instead). I think it reclaims some of it too but I guess it up to free space on that filesystem/volume. Those two commands together really reclaims back after snapshot is rotated.
This article is interesting https://www.ibm.com/docs/en/storage-fusion/2.6?topic=resources-reclaiming-space-target-volumes does both sparsify/fstrim to achieve reclaim. Istvan Szabo Staff Infrastructure Engineer --------------------------------------------------- Agoda Services Co., Ltd. e: istvan.sz...@agoda.com<mailto:istvan.sz...@agoda.com> --------------------------------------------------- ________________________________ From: Ilya Dryomov <idryo...@gmail.com> Sent: Monday, December 4, 2023 6:10 PM To: Szabo, Istvan (Agoda) <istvan.sz...@agoda.com> Cc: Ceph Users <ceph-users@ceph.io> Subject: Re: [ceph-users] Space reclaim doesn't happening in nautilus RBD pool Email received from the internet. If in doubt, don't click any link nor open any attachment ! ________________________________ Hi Istvan, The number of objects in "im" pool (918.34k) doesn't line up with "rbd du" output which says that only 2.2T are provisioned (that would take roughly ~576k objects). This usually occurs when there are object clones caused by previous snapshots -- keep in mind that trimming object clones after a snapshot is removed is an asynchronous process and it can take a while. Just to confirm, what is the output of "rbd info im/root", "rbd snap ls --all im/root", "ceph df" (please recapture) and "ceph osd pool ls detail" (only "im" pool is of interest)? Thanks, Ilya On Fri, Dec 1, 2023 at 5:31 AM Szabo, Istvan (Agoda) <istvan.sz...@agoda.com> wrote: > > Thrash empty. > > Istvan Szabo > Staff Infrastructure Engineer > --------------------------------------------------- > Agoda Services Co., Ltd. > e: istvan.sz...@agoda.com > --------------------------------------------------- > > > > ________________________________ > From: Ilya Dryomov <idryo...@gmail.com> > Sent: Thursday, November 30, 2023 6:27 PM > To: Szabo, Istvan (Agoda) <istvan.sz...@agoda.com> > Cc: Ceph Users <ceph-users@ceph.io> > Subject: Re: [ceph-users] Space reclaim doesn't happening in nautilus RBD pool > > Email received from the internet. If in doubt, don't click any link nor open > any attachment ! > ________________________________ > > On Thu, Nov 30, 2023 at 8:25 AM Szabo, Istvan (Agoda) > <istvan.sz...@agoda.com> wrote: > > > > Hi, > > > > Is there any config on Ceph that block/not perform space reclaim? > > I test on one pool which has only one image 1.8 TiB in used. > > > > > > rbd $p du im/root > > warning: fast-diff map is not enabled for root. operation may be slow. > > NAME PROVISIONED USED > > root 2.2 TiB 1.8 TiB > > > > > > > > I already removed all snaphots and now pool has only one image alone. > > I run both fstrim over the filesystem (XFS) and try rbd sparsify im/root > > (don't know what it is exactly but it mentions to reclaim something) > > It still shows the pool used 6.9 TiB which totally not make sense right? It > > should be up to 3.6 (1.8 * 2) according to its replica? > > Hi Istvan, > > Have you checked RBD trash? > > $ rbd trash ls -p im > > Thanks, > > Ilya > > ________________________________ > > This message is confidential and is for the sole use of the intended > recipient(s). It may also be privileged or otherwise protected by copyright > or other legal rules. If you have received it by mistake please let us know > by reply email and delete it from your system. It is prohibited to copy this > message or disclose its content to anyone. Any confidentiality or privilege > is not waived or lost by any mistaken delivery or unauthorized disclosure of > the message. All messages sent to and from Agoda may be monitored to ensure > compliance with company policies, to protect the company's interests and to > remove potential malware. Electronic messages may be intercepted, amended, > lost or deleted, or contain viruses. _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-le...@ceph.io
