On Tue, 18 Sep 2012, Smart Weblications GmbH - Florian Wiessner wrote:
> Am 18.09.2012 04:32, schrieb Sage Weil:
> > On Mon, 17 Sep 2012, Tren Blackburn wrote:
> >> On Mon, Sep 17, 2012 at 5:05 PM, Smart Weblications GmbH - Florian
> >> Wiessner <[email protected]> wrote:
> >>>
> >>> Hi,
> >>>
> >>> i use ceph to provide storage via rbd for our virtualization cluster
> >>> delivering
> >>> KVM based high availability Virtual Machines to my customers. I also use
> >>> it
> >>> as rbd device with ocfs2 on top of it for a 4 node webserver cluster as
> >>> shared
> >>> storage - i do this, because unfortunatelly cephfs is not ready yet ;)
> >>>
> >> Hi Florian;
> >>
> >> When you say "cephfs is not ready yet", what parts about it are not
> >> ready? There are vague rumblings about that in general, but I'd love
> >> to see specific issues. I understand multiple *active* mds's are not
> >> supported, but what other issues are you aware of?
> >
> > Inktank is not yet supporting it because we do not have the QA in place
> > and general hardening that will make us feel comfortable recommending it
> > for customers. That said, it works pretty well for most workloads. In
> > particular, if you stay away from the snapshots and multi-mds, you should
> > be quite stable.
> >
> > The engineering team here is about to do a bit of a pivot and refocus on
> > the file system now that the object store and RBD are in pretty good
> > shape. That will mean both core fs/mds stability and features as well as
> > integration efforts (NFS/CIFS/Hadoop).
> >
> > 'Ready' is in the eye of the beholder. There are a few people using the
> > fs successfully in production, but not too many.
> >
>
> I tried it using multiple mds, because without multiple mds there is no
> redundancy and the single mds will be SPOF. I noticed things like empty
Just to clarify: by multi-mds I mean multiple *active* ceph-mds daemons.
By default if you start a bunch of them they are just standby, ready to
take over in the event of a failure.
> directories which could not be deleted. It said directory not empty, but it
> was
> empty and could not be deleted. I also noticed kernel panic on 3.2 kernels
> using
> kernel ceph client, or crashes with ceph-fuse. It is somewhat unstable so
> that i
> always had to reboot a node after a while of usage for various reasons
> (ceph-fuse crashed and messed up fuse, kernel panic using kernel ceph client,
> unable to delete files/dirs, no fsck for fixing things).
>
> Last time i tried was simple untarring kernel tree in cephfs mountpoint - a
> new
> created cephfs and after 10 minutes there where errors like unable to delete
> dirs etc. Since i do not know how to reset/reformat only the cephfs part
> (without touching rbd!), i stopped testing for now. The last time i tried it i
> lost data - the data was not important and i had backups, but i was feeling
> uncomfortable now with using cephfs...
This is disconcerting. If it is something you are able to reproduce,
sharing those steps with us would be very helpful.
> I also did not have tried btrfs with ceph since 11/2011 again, because of
> losing
> data after reboots when btrfs dies, the btrfs was unmountable and there was no
> fsck so i only could reformat and wait for ceph to rebuild. After a
> powerfailure, no btrfs partitions survived and i lost all test data :/
>
>
> So i think the first thing to be done to cephfs would be to integrate some
> sort
> of fsck and the ability to format only cephfs without losing other rbd
> images/data o rados data...
FWIW it is already possible to create a new fs without touching other
pools with the command
ceph newfs <new metadata pool id> <new data pool id>
where the pool ids are the numeric ids (ceph osd dump | grep ^pool) for
new, empty rados pools.
sage
>
>
>
> --
>
> Mit freundlichen Gr??en,
>
> Florian Wiessner
>
> Smart Weblications GmbH
> Martinsberger Str. 1
> D-95119 Naila
>
> fon.: +49 9282 9638 200
> fax.: +49 9282 9638 205
> 24/7: +49 900 144 000 00 - 0,99 EUR/Min*
> http://www.smart-weblications.de
>
> --
> Sitz der Gesellschaft: Naila
> Gesch?ftsf?hrer: Florian Wiessner
> HRB-Nr.: HRB 3840 Amtsgericht Hof
> *aus dem dt. Festnetz, ggf. abweichende Preise aus dem Mobilfunknetz
>
>
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
