On 25.07.2018 11:20, Mark Adams wrote:
Hi Alwin,
On 25 July 2018 at 07:10, Alwin Antreich <a...@ipnerd.net> wrote:
Hi,
On Wed, Jul 25, 2018, 02:20 Mark Adams <m...@openvs.co.uk> wrote:
Hi All,
I have a proxmox 5.1 + ceph cluster of 3 nodes, each with 12 x WD 10TB
GOLD
drives. Network is 10Gbps on X550-T2, separate network for the ceph
cluster.
Do a rados bench for testing the cluster performance, spinners are not
fast.
This was a typo - I'm actually on 5.2-1. I'll give rados bench a try to see
what it comes back with.
I have 1 VM currently running on this cluster, which is debian stretch
with
a zpool on it. I'm zfs sending in to it, but only getting around ~15MiB/s
write speed. does this sound right? it seems very slow to me.
Never ever use a CoW filesystem on top of another CoW system. This doubles
the writes that need to be made.
Not only that, but when this zfs send is running - I can not do any
parallel sends to any other zfs datasets inside of the same VM. They just
seem to hang, then eventually say "dataset is busy".
Ceph already gives you the possibility of snapshots. You can let PVE do
this through CLI or GUI.
The problem with this is the required features.. I need an HA cluster, and
zfs doesn't support this - so ceph is ideal, however I also need "restore
previous versions" usable inside a file server VM in samba, which ceph
snapshots at the VM layer is no use for.... Unless there is some other
smart way of doing this I don't know about!
I guess my main question is, is there any other config hints to speed this
up whether ceph or in ZFS inside the VM, and is the blocking of other IO
normal with ceph when "max" write speed is being reached? That bit doesn't
seem right to me.
you can expose rbd snapshots in samba using this script.
https://github.com/ksperis/autosnap-rbd-shadow-copy
--
but having ceph-> rbd -> filesystem -> vm mounting filesystem -> samba
export is a long and overly complex chain.
if you mount and export a cephfs filesystem you can remove the need for
a filesystem on a rbd
ceph-> cephfs -> vm with cephfs mounted -> samba export
You can even take it a step further, by using the vfs_ceph samba module
you can translate samba calls to cephfs calls directly
https://fosdem.org/2018/schedule/event/cephfs_gateways/attachments/slides/2636/export/events/attachments/cephfs_gateways/slides/2636/cephfs_samba_and_nfs.pdf
good luck
Ronny Aasen
_______________________________________________
pve-user mailing list
pve-user@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user
