On Sun, 2019-07-28 at 18:20 +0000, Lee Norvall wrote:
> Update to this I found that you cannot create a 2nd files system as yet and
> it is still experimental. So I went down this route:
>
> Added a pool to the existing cephfs and then setfattr -n ceph.dir.layout.pool
> -v SSD-NFS /mnt/cephfs/ssdnfs/ from a ceph-fuse client.
>
> I then nfs mounted from another box. I can see the files and dir etc from the
> nfs client but my issue now is that I do not have permission to write, create
> dir etc. The same goes for the default setup after running the ansible
> playbook even when setting export to no_root_squash. I am missing a chain of
> permission? ganesha-nfs is using admin userid, is this the same as the
> client.admin or is this a user I need to create? Any info appreciated.
>
> Ceph is on CentOS 7 and SELinux is currently off as well.
>
> Copy of the ganesha conf below. Is secType correct or is it missing
> something?
>
> RADOS_URLS {
> ceph_conf = '/etc/ceph/ceph.conf';
> userid = "admin";
> }
> %url rados://cephfs_data/ganesha-export-index
>
> NFSv4 {
> RecoveryBackend = 'rados_kv';
> }
I your earlier email, you mentioned that you had more than one NFS
server, but rados_kv is not safe in a multi-server configuration. The
servers will be competing to store recovery information in the same
objects, and won't honor each others' grace periods/
You may want to explore using "RecoveryBackend = rados_cluster" instead,
which should handle that situation better. See this writeup, for some
guidelines:
https://jtlayton.wordpress.com/2018/12/10/deploying-an-active-active-nfs-cluster-over-cephfs/
Much of this is already automated too if you use k8s+rook.
> RADOS_KV {
> ceph_conf = '/etc/ceph/ceph.conf';
> userid = "admin";
> pool = "cephfs_data";
> }
>
> EXPORT
> {
> Export_id=20133;
> Path = "/";
> Pseudo = /cephfile;
> Access_Type = RW;
> Protocols = 3,4;
> Transports = TCP;
> SecType = sys,krb5,krb5i,krb5p;
> Squash = Root_Squash;
> Attr_Expiration_Time = 0;
>
> FSAL {
> Name = CEPH;
> User_Id = "admin";
> }
>
>
> }
>
>
>
>
>
>
>
>
>
>
> On 28/07/2019 12:11, Lee Norvall wrote:
> > Hi
> >
> > I am using ceph-ansible to deploy and just looking for best way/tips on
> > how to export multiple pools/fs.
> >
> > Ceph: nautilus (14.2.2)
> > NFS-Ganesha v 2.8
> > ceph-ansible stable 4.0
> >
> > I have 3 x osd/NFS gateways running and NFS on the dashboard can see
> > them in the cluster. I have managed to export for cephfs / and mounted
> > it on another box.
> >
> > 1) can I add a new pool/fs to the export under that same NFS gateway
> > cluster, or
> >
> > 2) do I have the to do something like add a new pool to the fs and then
> > setfattr to make the layout /newfs_dir point to /new_pool? does this
> > cause issues and false object count?
> >
> > 3) any other better ways...
> >
> > Rgds
> >
> > Lee
> >
> > _______________________________________________
> > ceph-users mailing list
> > [email protected]
> > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
>
> --
>
>
> Lee Norvall | CEO / Founder
> Mob. +44 (0)7768 201884
> Tel. +44 (0)20 3026 8930
> Web. www.blocz.io
>
> Enterprise Cloud | Private Cloud | Hybrid/Multi Cloud | Cloud Backup
>
>
>
> This e-mail (and any attachment) has been sent from a PC belonging to My Mind
> (Holdings) Limited. If you receive it in error, please tell us by return and
> then delete it from your system; you may not rely on its contents nor
> copy/disclose it to anyone. Opinions, conclusions and statements of intent in
> this e-mail are those of the sender and will not bind My Mind (Holdings)
> Limited unless confirmed by an authorised representative independently of
> this message. We do not accept responsibility for viruses; you must scan for
> these. Please note that e-mails sent to and from blocz IO Limited are
> routinely monitored for record keeping, quality control and training
> purposes, to ensure regulatory compliance and to prevent viruses and
> unauthorised use of our computer systems. My Mind (Holdings) Limited is
> registered in England & Wales under company number 10186410. Registered
> office: 1st Floor Offices, 2a Highfield Road, Ringwood, Hampshire, United
> Kingdom, BH24 1RQ. VAT Registration GB 244
9628 77
> _______________________________________________
> ceph-users mailing list
> [email protected]
> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
--
Jeff Layton <[email protected]>
_______________________________________________
ceph-users mailing list
[email protected]
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
