Hi, Thanks all for your quick responses. In my enthusiasm to test I might have been masking the problem, plus not knowing what the output of 'fstrim' should actually show.
Firstly, to answer the question, yes I have the relevant libvirt config, plus have set the correct virtio-scsi settings in the image configuration. I am also not expecting to see anything in userland from a 'df' After re-running the tests from the linked article after an instance was created, it does appear that trimming isn't working, but the number of objects was growing significantly after logging in (almost doubling) and this was masking the effect of trimming. tl:dr - it does actually appear to be working, but thank you for the responses and also confirming that there was no Ceph-specific configuration required. On 29 January 2018 at 12:00, Ric Wheeler <[email protected]> wrote: > I might have missed something in the question. > > Fstrim does not free up space at the user level that you see with a normal > df. > > It is meant to let the block device know about all of the space unused by > the file system. > > Regards, > > Ric > > > On Jan 29, 2018 11:56 AM, "Wido den Hollander" <[email protected]> wrote: > >> >> >> On 01/29/2018 12:29 PM, Nathan Harper wrote: >> >>> Hi, >>> >>> I don't know if this is strictly a Ceph issue, but hoping someone will >>> be able to shed some light. We have an Openstack environment (Ocata) >>> backed onto a Jewel cluster. >>> >>> We recently ran into some issues with full OSDs but couldn't work out >>> what was filling up the pools. >>> >>> It appears that fstrim isn't actually working (or being ignored) within >>> the instances: >>> >>> [centos@with ~]$ df -h >>> Filesystem Size Used Avail Use% Mounted on >>> /dev/sda1 8.0G 1.2G 6.9G 14% / >>> >>> plenty of unused space that can be reclaimed: >>> >>> running fstrim: >>> [centos@with ~]$ sudo fstrim -a -v >>> /: 6.9 GiB (7416942592 bytes) trimmed >>> >>> however, running the command again: >>> >>> [centos@with ~]$ sudo fstrim -a -v >>> /: 6.9 GiB (7416942592 bytes) trimmed >>> >>> >> It depends on the FS you use inside the VM. >> >> iirc XFS will always tell you the total free space of the FS being >> trimmed and ext4 only the actual bytes which are trimmed. >> >> The discard option was enabled following: >>> https://ceph.com/geen-categorie/openstack-and-ceph-rbd-discard/ >>> >>> Have I missed anything at the Ceph layer to enable this? Is there >>> anything I can do to see if the request is making it's way through to the >>> Ceph layer? >>> >>> >> Try using 'rbd du' and see if it makes a difference. >> >> You also want to check if the XML definition of libvirt has unmap enabled >> although I expect it does since you don't see any errors. >> >> Wido >> >> >>> >>> -- >>> *NathanHarper* // IT Systems Lead >>> >>> *e: *[email protected] <mailto:[email protected]> >>> *t*: 0117 906 1104 *m*: 0787 551 0891 *w: *www.cfms.org.uk < >>> http://www.cfms.org.uk/> >>> CFMS Services Ltd // Bristol & Bath Science Park // Dirac Crescent // >>> Emersons Green // Bristol // BS16 7FR >>> CFMS Services Ltd is registered in England and Wales No 05742022 - a >>> subsidiary of CFMS Ltd >>> CFMS Services Ltd registered office // 43 Queens Square // Bristol // >>> BS1 4QP >>> >>> >>> _______________________________________________ >>> ceph-users mailing list >>> [email protected] >>> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com >>> >>> _______________________________________________ >> ceph-users mailing list >> [email protected] >> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com >> > > _______________________________________________ > ceph-users mailing list > [email protected] > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > > -- *Nathan Harper* // IT Systems Lead *e: *[email protected] *t*: 0117 906 1104 *m*: 0787 551 0891 *w: *www.cfms.org.uk CFMS Services Ltd // Bristol & Bath Science Park // Dirac Crescent // Emersons Green // Bristol // BS16 7FR CFMS Services Ltd is registered in England and Wales No 05742022 - a subsidiary of CFMS Ltd CFMS Services Ltd registered office // 43 Queens Square // Bristol // BS1 4QP
_______________________________________________ ceph-users mailing list [email protected] http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
