[ceph-users] ZFS on Ceph (rbd-fuse)

2013-11-29 Thread Charles 'Boyo
Hello all. I have a Ceph cluster using XFS on the OSDs. Btrfs is not available to me at the moment (cluster is running CentOS 6.4 with stock kernel). I intend to maintain a full replica of an active ZFS dataset on the Ceph infrastructure by installing an OpenSolaris KVM guest using rbd-fuse to

Re: [ceph-users] ZFS on Ceph (rbd-fuse)

2013-11-29 Thread John Spray
The trouble with using ZFS copies on top of RBD is that both copies of any particular block might end up on the same OSD. If you have disabled replication in Ceph, then this would mean a single OSD failure could cause data loss. For that reason, it seems it would be better to do the replication

Re: [ceph-users] ZFS on Ceph (rbd-fuse)

2013-11-29 Thread Charles 'Boyo
Thanks for the input John. So I should leave ZFS checksumming on, disable ZFS replicas and rely on Ceph RBD replicas. Is it even sane to use rbd-fuse for this? On a related note, is there any discard/trim support in rbd-fuse? Else I won't ever be able to thin-out the RBD image once it is

Re: [ceph-users] ZFS on Ceph (rbd-fuse)

2013-11-29 Thread James Pearce
On a related note, is there any discard/trim support in rbd-fuse? Apparently so (but not in the kernel module unfortunately). ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] ZFS on Ceph (rbd-fuse)

2013-11-29 Thread Dan van der Ster
On Fri, Nov 29, 2013 at 12:13 PM, Charles 'Boyo charlesb...@gmail.com wrote: That's because qemu-kvm in CentOS 6.4 doesn't support librbd. RedHat just added RBD support in qemu-kvm-rhev in RHEV 6.5. I don't know if that will trickle down to CentOS but you can probably recompile it yourself like

Re: [ceph-users] ZFS on Ceph (rbd-fuse)

2013-11-29 Thread Charles 'Boyo
On a related note, is there any discard/trim support in rbd-fuse? Apparently so (but not in the kernel module unfortunately). Ok, librbd (which is used by the qemu alternative) supports discard, not the rbd kernel module does not. Neither of these are available to me right now. Is rbd-fuse

Re: [ceph-users] ZFS on Ceph (rbd-fuse)

2013-11-29 Thread Charles 'Boyo
That's because qemu-kvm in CentOS 6.4 doesn't support librbd. RedHat just added RBD support in qemu-kvm-rhev in RHEV 6.5. I don't know if that will trickle down to CentOS but you can probably recompile it yourself like we did. https://rhn.redhat.com/errata/RHSA-2013-1754.html