Re: [ceph-users] Create OSD on ZFS Mount (firefly)

2014-11-26 Thread Lindsay Mathieson
On Tue, 25 Nov 2014 03:47:08 PM Eric Eastman wrote: > It has been almost a year since I last tried ZFS, but I had to add to the > ceph.conf file: > >filestore zfs_snap = 1 >journal aio = 0 >journal dio = 0 > > Eric Thanks Eric, I figured it out in the end, though I haven't tried

Re: [ceph-users] Create OSD on ZFS Mount (firefly)

2014-11-26 Thread Eric Eastman
> - Have created ZFS mount: > “/var/lib/ceph/osd/ceph-0” > - followed the instructions at: > http://ceph.com/docs/firefly/rados/operations/add-or-rm-osds/ > failing on the step 4. Initialize the OSD data directory. > ceph-osd -i 0 —mkfs --mkkey > 2014-11-25 22:12:26.563666 7ff12b466780 -1 > fil

Re: [ceph-users] Create OSD on ZFS Mount (firefly)

2014-11-25 Thread Lindsay Mathieson
I've found the cause of the problem - ceph was attempting to create the journal with direct io which zfs doesn't support. I worked round it by disabling journal dio in ceph.config [osd] journal dio = false Dunno if this is a good idea or not or whether there is a better way of doing it :_ On 26

Re: [ceph-users] Create OSD on ZFS Mount (firefly)

2014-11-25 Thread Lindsay Mathieson
Thanks Craig, I had a good read of it - from what I read, the std ceph packages should work with zfs, just not make use of its extra features (writeparallel support), whose performance was not all that good anyway. I did note the "set xattr to sa" comment which gave me a different error:) ceph-os

Re: [ceph-users] Create OSD on ZFS Mount (firefly)

2014-11-25 Thread Craig Lewis
There was a good thread on the mailing list a little while ago. There were several recommendations in that thread, maybe some of them will help. Found it: https://www.mail-archive.com/ceph-users@lists.ceph.com/msg14154.html On Tue, Nov 25, 2014 at 4:16 AM, Lindsay Mathieson < lindsay.mathie...@

[ceph-users] Create OSD on ZFS Mount (firefly)

2014-11-25 Thread Lindsay Mathieson
Testing ceph on top of ZFS (zfsonlinux), kernel driver. - Have created ZFS mount: "/var/lib/ceph/osd/ceph-0" - followed the instructions at: http://ceph.com/docs/firefly/rados/operations/add-or-rm-osds/ failing on the step 4. Initialize the OSD data directory. ceph-osd -i 0 --mkfs --mkkey

[ceph-users] Create OSD on ZFS Mount (firefly)

2014-11-25 Thread Lindsay Mathieson
Testing ceph on top of ZFS (zfsonlinux), kernel driver. - Have created ZFS mount: "/var/lib/ceph/osd/ceph-0" - followed the instructions at: http://ceph.com/docs/firefly/rados/operations/add-or-rm-osds/ failing on the step 4. Initialize the OSD data directory. ceph-osd -i 0 --mkfs --mkkey