Re: [ceph-users] [ceph-fuse] Quota size change does not notify another ceph-fuse client.

[John Spray]

On Wed, Mar 15, 2017 at 1:53 AM, yu2xiangyang  wrote:
> Dear cephers,
>
> I met a problem when using ceph-fuse with quota enabled.
>
>  My ceph version is :
>
>  ceph version 10.2.5 (c461ee19ecbc0c5c330aca20f7392c9a00730367) .
>
> I have two ceph-fuse process in two different hosts(node1 and node2).
>
>  One ceph-fuse is mounted with root directory on /mnt/cephfs on node1.
>  [root@node1] ceph-fuse -m mds-host:6789 /mnt/cephfs
>
>  I set quota on a sub directory volumes/nas0 in /mnt/cephfs.
>  [root@node1] setfattr -n ceph.quota.max_bytes -v 1000
> /mnt/cephfs/volumes/nas0
>
> One ceph-fuse is mounted with /volumes/nas0 on node2 with client quota
> enabled.
> [root@node2] ceph-fuse --id yxy -r /volumes/nas0 /shares/share0/
>
>  [root@node2]$ cat /etc/ceph/ceph.client.yxy.keyring
>  [client.yxy] key = AQBmtMdYNPnOJRAAg5t+gkUDmqTpQhZh2VXlWg==
>  caps mds = "allow * path=/volumes/nas0"
> caps mon = "allow *"
> caps osd = "allow *"
>
> [root@node2 ]$ cat /etc/ceph/ceph.conf
> [global] mon_initial_members=x
> mon_host = x
> client quota = true
>
>  DF commond show that ceph-fuse has 94GB capacity just equal to the size we
> set.
>
> ceph-fuse 94G 4G 90G 3% /shares/share0
>
> But when I resize the quota with /volumes/nas0. in node1.
> [root@node1 ~]# setfattr -n ceph.quota.max_bytes -v 700
> /mnt/cephfs/volumes/nas0
>
> On node2 , DF command stilll show 94GB capacity.
> ceph-fuse 94G 4G 90G 3% /shares/share0
>
> One host resizes the quota size and another host does not know the change.
>
>  Is it a problem with ceph-fuse? Appreciate any reply.

There's a known issue that quota updates aren't visible on another
client until it does some IO in the directory:
http://tracker.ceph.com/issues/17939

This probably have the same underlying cause as the statfs (df) data
looking out of date.

I'm looking into it, see ticket for updates.

John


>
> cheers,
> penglaixy
>
> ___
> ceph-users mailing list
> ceph-users@lists.ceph.com
> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
>
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

[ceph-users] [ceph-fuse] Quota size change does not notify another ceph-fuse client.

[yu2xiangyang]

Dear cephers,

I met a problem when using ceph-fuse with quota enabled.

 My ceph version is :

 ceph version 10.2.5 (c461ee19ecbc0c5c330aca20f7392c9a00730367) .

I have two ceph-fuse process in two different hosts(node1 and node2).

 One ceph-fuse is mounted with root directory on /mnt/cephfs on node1.
 [root@node1] ceph-fuse -m mds-host:6789 /mnt/cephfs

 I set quota on a sub directory volumes/nas0 in /mnt/cephfs.
 [root@node1] setfattr -n ceph.quota.max_bytes -v 1000 
/mnt/cephfs/volumes/nas0

One ceph-fuse is mounted with /volumes/nas0 on node2 with client quota enabled.
[root@node2] ceph-fuse --id yxy -r /volumes/nas0 /shares/share0/

 [root@node2]$ cat /etc/ceph/ceph.client.yxy.keyring
 [client.yxy] key = AQBmtMdYNPnOJRAAg5t+gkUDmqTpQhZh2VXlWg==
 caps mds = "allow * path=/volumes/nas0"
caps mon = "allow *"
caps osd = "allow *"

[root@node2 ]$ cat /etc/ceph/ceph.conf
[global] mon_initial_members=x
mon_host = x
client quota = true

 DF commond show that ceph-fuse has 94GB capacity just equal to the size we set.

ceph-fuse 94G 4G 90G 3% /shares/share0

But when I resize the quota with /volumes/nas0. in node1.
[root@node1 ~]# setfattr -n ceph.quota.max_bytes -v 700 
/mnt/cephfs/volumes/nas0

On node2 , DF command stilll show 94GB capacity.
ceph-fuse 94G 4G 90G 3% /shares/share0

One host resizes the quota size and another host does not know the change.

 Is it a problem with ceph-fuse? Appreciate any reply.

cheers,
penglaixy___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] [ceph-fuse] Quota size change does not notify another ceph-fuse client.

[Dan van der Ster]

Hi,

This sounds familiar: http://tracker.ceph.com/issues/17939

I found that you can get the updated quota on node2 by touching the
base dir. In your case:

touch /shares/share0

-- Dan


On Tue, Mar 14, 2017 at 10:52 AM, yu2xiangyang  wrote:
> Dear cephers,
> I met a problem when using ceph-fuse with quota enabled.
>
>My ceph version is :
>ceph version 10.2.5 (c461ee19ecbc0c5c330aca20f7392c9a00730367)
>
> I have two ceph-fuse process in two different hosts(node1 and node2)
> One ceph-fuse is mounted with root directory  on /mnt/cephfs on node1.
> [root@node1] ceph-fuse -m mds-host:6789 /mnt/cephfs
> I set quota on a sub directory volumes/nas0 in /mnt/cephfs.
> [root@node1] setfattr -n ceph.quota.max_bytes -v 1000
> /mnt/cephfs/volumes/nas0
>
> One ceph-fuse is mounted with /volumes/nas0 on node2 with client quota
> enabled.
> [root@node2] ceph-fuse --id yxy  -r /volumes/nas0 /shares/share0/
>
> [root@node2]$ cat /etc/ceph/ceph.client.yxy.keyring
> [client.yxy]
> key = AQBmtMdYNPnOJRAAg5t+gkUDmqTpQhZh2VXlWg==
> caps mds = "allow * path=/volumes/nas0"
> caps mon = "allow *"
> caps osd = "allow *"
>
> [root@node2 ]$ cat /etc/ceph/ceph.conf
> [global]
> mon_initial_members=x
> mon_host = x
> client quota = true
>
> DF commond show that ceph-fuse has  94GB capacity just equal to the size we
> set.
> ceph-fuse 94G  4G 90G 3% /shares/share0
>
>
> But when I resize the quota with /volumes/nas0.
> [root@node1 ~]# setfattr -n ceph.quota.max_bytes -v 700
> /mnt/cephfs/volumes/nas0
>
> On node2 , DF command stilll show 94GB capacity.
> ceph-fuse 94G  4G 90G 3% /shares/share0
>
> One host resizes the quota size and another host  does not know the change.
> Is it  a problem with ceph-fuse?
> Appreciate any reply.
>
> penglaixy
>
>
>
>
>
> ___
> ceph-users mailing list
> ceph-users@lists.ceph.com
> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
>
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

[ceph-users] [ceph-fuse] Quota size change does not notify another ceph-fuse client.

[yu2xiangyang]

Dear cephers,
I met a problem when using ceph-fuse with quota enabled.

   My ceph version is :
   ceph version 10.2.5 (c461ee19ecbc0c5c330aca20f7392c9a00730367)

I have two ceph-fuse process in two different hosts(node1 and node2)
One ceph-fuse is mounted with root directory  on /mnt/cephfs on node1.
[root@node1] ceph-fuse -m mds-host:6789 /mnt/cephfs
I set quota on a sub directory volumes/nas0 in /mnt/cephfs.
[root@node1] setfattr -n ceph.quota.max_bytes -v 1000 
/mnt/cephfs/volumes/nas0

One ceph-fuse is mounted with /volumes/nas0 on node2 with client quota 
enabled.
[root@node2] ceph-fuse --id yxy  -r /volumes/nas0 /shares/share0/
   
[root@node2]$ cat /etc/ceph/ceph.client.yxy.keyring
[client.yxy]   
key = AQBmtMdYNPnOJRAAg5t+gkUDmqTpQhZh2VXlWg==
caps mds = "allow * path=/volumes/nas0"
caps mon = "allow *"
caps osd = "allow *"

[root@node2 ]$ cat /etc/ceph/ceph.conf
[global]
mon_initial_members=x
mon_host = x
client quota = true

DF commond show that ceph-fuse has  94GB capacity just equal to the size we set.
ceph-fuse 94G  4G 90G 3% /shares/share0


But when I resize the quota with /volumes/nas0.
[root@node1 ~]# setfattr -n ceph.quota.max_bytes -v 700 
/mnt/cephfs/volumes/nas0

On node2 , DF command stilll show 94GB capacity.
ceph-fuse 94G  4G 90G 3% /shares/share0

One host resizes the quota size and another host  does not know the change.
Is it  a problem with ceph-fuse?
Appreciate any reply.

penglaixy
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com