Re: [ceph-users] Reclaim free space on RBD images that use Bluestore?????

2018-07-25 Thread Sean Bolding 
Thanks. Yes, it turns out this was not an issue with Ceph, but rather an
issue with XenServer. Starting in version 7, Xenserver changed how they
manage LVM by adding a VHD layer on top of it. They did it to handle live
migrations but ironically broke live migrations when using any iSCSI
including iSCSI to Ceph via lrbd. It works just fine with NFS based storage
repositories but not block storage. Doesn't look like they are going to fix
it since they are moving on to using glusterfs instead with an experimental
version of it starting in XenServer 7.5

 

From: ceph-users [mailto:ceph-users-boun...@lists.ceph.com] On Behalf Of
Ronny Aasen
Sent: Monday, July 23, 2018 6:13 PM
To: ceph-users@lists.ceph.com
Subject: Re: [ceph-users] Reclaim free space on RBD images that use
Bluestore?

 

On 23.07.2018 22:18, Sean Bolding wrote:

I have XenServers that connect via iSCSI to Ceph gateway servers that use
lrbd and targetcli. On my ceph cluster the RBD images I create are used as
storage repositories in Xenserver for the virtual machine vdisks. 

 

Whenever I delete a virtual machine, XenServer shows that the repository
size has decreased. This also happens when I mount a virtual drive in
Xenserver as a virtual drive in a Windows guest. If I delete a large file,
such as an exported VM, it shows as deleted and space available. However;
when check in Ceph  using ceph -s or ceph df it still shows the space being
used.

 

I checked everywhere and it seems there was a reference to it here
https://github.com/ceph/ceph/pull/14727 but not sure if a way to trim or
discard freed blocks was ever implemented.

 

The only way I have found is to play musical chairs and move the VMs to
different repositories and then completely remove the old RBD images in
ceph. This is not exactly easy to do.

 

Is there a way to reclaim free space on RBD images that use Bluestore?
What commands do I use and where do I use this from? If such command exist
do I run them on the ceph cluster or do I run them from XenServer? Please
help.

 

 

Sean

 

 

 


I am not familiar with Xen, but it does sounds like you have a rbd mounted
with a filesystem on the xen server.
in that case it is the same as for other filesystems. Deleted files are just
deleted in the file allocation table, and the RBD space is "reclaimed" when
the filesystem zeroes out the now unused blocks. 

in many filesystems you would run the fstrim command to overwrite free'd
blocks with zeroes, optionally mount the fs with the the discard option. 
in xenserver >6.5 this should be a button in xencenter to reclaim freed
space. 


kind regards
Ronny Aasen

 
<http://t.sidekickopen08.com/e1t/o/5/f18dQhb0S7kC8dDMPbW2n0x6l2B9gXrN7sKj6v5
KRN6W56jV0P7dSBj2W5vbH2n6yGGzjf197v5Y04?si=71238989=37da8a12-5013
-4034-91ea-a306cca5c995> 

___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] Reclaim free space on RBD images that use Bluestore?????

2018-07-23 Thread Satish Patel 
Forgive me found this post which solved my issue:

https://www.sebastien-han.fr/blog/2015/02/02/openstack-and-ceph-rbd-discard/

On Mon, Jul 23, 2018 at 11:22 PM, Satish Patel  wrote:
> I have same issue, i just build new Ceph cluster for my Openstack VMs
> workload using rbd and i have created bunch of VM did some dd test to
> create big big file to test performance now i deleted all dd file but
> ceph still showing USED space.
>
> I tried to do from guest VM
>
> [root@c7-vm ~]# sudo fstrim /
> fstrim: /: the discard operation is not supported
>
>
> Can we run fstrim on ceph OSD node? what if i delete my VM in that
> case how do i run fstrim ?
>
> On Mon, Jul 23, 2018 at 6:13 PM, Ronny Aasen  
> wrote:
>> On 23.07.2018 22:18, Sean Bolding wrote:
>>
>> I have XenServers that connect via iSCSI to Ceph gateway servers that use
>> lrbd and targetcli. On my ceph cluster the RBD images I create are used as
>> storage repositories in Xenserver for the virtual machine vdisks.
>>
>>
>>
>> Whenever I delete a virtual machine, XenServer shows that the repository
>> size has decreased. This also happens when I mount a virtual drive in
>> Xenserver as a virtual drive in a Windows guest. If I delete a large file,
>> such as an exported VM, it shows as deleted and space available. However;
>> when check in Ceph  using ceph –s or ceph df it still shows the space being
>> used.
>>
>>
>>
>> I checked everywhere and it seems there was a reference to it here
>> https://github.com/ceph/ceph/pull/14727 but not sure if a way to trim or
>> discard freed blocks was ever implemented.
>>
>>
>>
>> The only way I have found is to play musical chairs and move the VMs to
>> different repositories and then completely remove the old RBD images in
>> ceph. This is not exactly easy to do.
>>
>>
>>
>> Is there a way to reclaim free space on RBD images that use Bluestore?
>> What commands do I use and where do I use this from? If such command exist
>> do I run them on the ceph cluster or do I run them from XenServer? Please
>> help.
>>
>>
>>
>>
>>
>> Sean
>>
>>
>>
>>
>>
>>
>>
>>
>> I am not familiar with Xen, but it does sounds like you have a rbd mounted
>> with a filesystem on the xen server.
>> in that case it is the same as for other filesystems. Deleted files are just
>> deleted in the file allocation table, and the RBD space is "reclaimed" when
>> the filesystem zeroes out the now unused blocks.
>>
>> in many filesystems you would run the fstrim command to overwrite free'd
>> blocks with zeroes, optionally mount the fs with the the discard option.
>> in xenserver >6.5 this should be a button in xencenter to reclaim freed
>> space.
>>
>>
>> kind regards
>> Ronny Aasen
>>
>> ___
>> ceph-users mailing list
>> ceph-users@lists.ceph.com
>> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
>>
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] Reclaim free space on RBD images that use Bluestore?????

2018-07-23 Thread Satish Patel 
I have same issue, i just build new Ceph cluster for my Openstack VMs
workload using rbd and i have created bunch of VM did some dd test to
create big big file to test performance now i deleted all dd file but
ceph still showing USED space.

I tried to do from guest VM

[root@c7-vm ~]# sudo fstrim /
fstrim: /: the discard operation is not supported


Can we run fstrim on ceph OSD node? what if i delete my VM in that
case how do i run fstrim ?

On Mon, Jul 23, 2018 at 6:13 PM, Ronny Aasen  wrote:
> On 23.07.2018 22:18, Sean Bolding wrote:
>
> I have XenServers that connect via iSCSI to Ceph gateway servers that use
> lrbd and targetcli. On my ceph cluster the RBD images I create are used as
> storage repositories in Xenserver for the virtual machine vdisks.
>
>
>
> Whenever I delete a virtual machine, XenServer shows that the repository
> size has decreased. This also happens when I mount a virtual drive in
> Xenserver as a virtual drive in a Windows guest. If I delete a large file,
> such as an exported VM, it shows as deleted and space available. However;
> when check in Ceph  using ceph –s or ceph df it still shows the space being
> used.
>
>
>
> I checked everywhere and it seems there was a reference to it here
> https://github.com/ceph/ceph/pull/14727 but not sure if a way to trim or
> discard freed blocks was ever implemented.
>
>
>
> The only way I have found is to play musical chairs and move the VMs to
> different repositories and then completely remove the old RBD images in
> ceph. This is not exactly easy to do.
>
>
>
> Is there a way to reclaim free space on RBD images that use Bluestore?
> What commands do I use and where do I use this from? If such command exist
> do I run them on the ceph cluster or do I run them from XenServer? Please
> help.
>
>
>
>
>
> Sean
>
>
>
>
>
>
>
>
> I am not familiar with Xen, but it does sounds like you have a rbd mounted
> with a filesystem on the xen server.
> in that case it is the same as for other filesystems. Deleted files are just
> deleted in the file allocation table, and the RBD space is "reclaimed" when
> the filesystem zeroes out the now unused blocks.
>
> in many filesystems you would run the fstrim command to overwrite free'd
> blocks with zeroes, optionally mount the fs with the the discard option.
> in xenserver >6.5 this should be a button in xencenter to reclaim freed
> space.
>
>
> kind regards
> Ronny Aasen
>
> ___
> ceph-users mailing list
> ceph-users@lists.ceph.com
> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
>
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] Reclaim free space on RBD images that use Bluestore?????

2018-07-23 Thread Ronny Aasen 

On 23.07.2018 22:18, Sean Bolding wrote:


I have XenServers that connect via iSCSI to Ceph gateway servers that 
use lrbd and targetcli. On my ceph cluster the RBD images I create are 
used as storage repositories in Xenserver for the virtual machine vdisks.


Whenever I delete a virtual machine, XenServer shows that the 
repository size has decreased. This also happens when I mount a 
virtual drive in Xenserver as a virtual drive in a Windows guest. If I 
delete a large file, such as an exported VM, it shows as deleted and 
space available. However; when check in Ceph  using ceph –s or ceph df 
it still shows the space being used.


I checked everywhere and it seems there was a reference to it here 
https://github.com/ceph/ceph/pull/14727 but not sure if a way to trim 
or discard freed blocks was ever implemented.


The only way I have found is to play musical chairs and move the VMs 
to different repositories and then completely remove the old RBD 
images in ceph. This is not exactly easy to do.


Is there a way to reclaim free space on RBD images that use 
Bluestore? What commands do I use and where do I use this from? If 
such command exist do I run them on the ceph cluster or do I run them 
from XenServer? Please help.


Sean



I am not familiar with Xen, but it does sounds like you have a rbd 
mounted with a filesystem on the xen server.
in that case it is the same as for other filesystems. Deleted files are 
just deleted in the file allocation table, and the RBD space is 
"reclaimed" when the filesystem zeroes out the now unused blocks.


in many filesystems you would run the fstrim command to overwrite free'd 
blocks with zeroes, optionally mount the fs with the the discard option.
in xenserver >6.5 this should be a button in xencenter to reclaim freed 
space.



kind regards
Ronny Aasen
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

[ceph-users] Reclaim free space on RBD images that use Bluestore?????

2018-07-23 Thread Sean Bolding 
I have XenServers that connect via iSCSI to Ceph gateway servers that use
lrbd and targetcli. On my ceph cluster the RBD images I create are used as
storage repositories in Xenserver for the virtual machine vdisks. 

 

Whenever I delete a virtual machine, XenServer shows that the repository
size has decreased. This also happens when I mount a virtual drive in
Xenserver as a virtual drive in a Windows guest. If I delete a large file,
such as an exported VM, it shows as deleted and space available. However;
when check in Ceph  using ceph -s or ceph df it still shows the space being
used.

 

I checked everywhere and it seems there was a reference to it here
https://github.com/ceph/ceph/pull/14727 but not sure if a way to trim or
discard freed blocks was ever implemented.

 

The only way I have found is to play musical chairs and move the VMs to
different repositories and then completely remove the old RBD images in
ceph. This is not exactly easy to do.

 

Is there a way to reclaim free space on RBD images that use Bluestore?
What commands do I use and where do I use this from? If such command exist
do I run them on the ceph cluster or do I run them from XenServer? Please
help.

 

 

Sean

 

 

 

 
 
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com