Re: [ceph-users] Double-mounting of RBD
Hello, I have a somewhat interesting scenario. I have an RBD of 17TB formatted using XFS. I would like it accessible from two different hosts, one mapped/mounted read-only, and one mapped/mounted as read-write. Both are shared using Samba 4.x. One Samba server gives read-only access to the world for the data. The other gives read-write access to a very limited set of users who occasionally need to add data. However, when testing this, when changes are made to the read-write Samba server the changes don’t seem to be seen by the read-only Samba server. Is there some file system caching going on that will eventually be flushed? I think that this a normal behaviour as your read only filesystem is not aware that some writes occurred. To achieve your goal I think that you should use some clustered filesystem [1] in order that the read-only server know that some writes occurred in the filesystem. [1] https://en.wikipedia.org/wiki/Clustered_file_system Regards, Olivier DELHOMME. ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] Double-mounting of RBD
On Wed, Dec 17, 2014 at 10:31 PM, McNamara, Bradley bradley.mcnam...@seattle.gov wrote: However, when testing this, when changes are made to the read-write Samba server the changes don’t seem to be seen by the read-only Samba server. Is there some file system caching going on that will eventually be flushed? As others have said, the read-only mount doesn't know how to poll the block device to see updates from the read-write mount, so you won't see updates to the data, and in general this is not a safe thing to do. One alternative would be taking a clone of a snapshot of the image, and mounting that read-only -- obviously that data will only be as up-to-date as whenever you did your last snapshot. If the read-only mounts are serving rarely updated files, the administrative overhead of doing the snapshot/remount on data updates might be acceptable. John ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
[ceph-users] Double-mounting of RBD
I have a somewhat interesting scenario. I have an RBD of 17TB formatted using XFS. I would like it accessible from two different hosts, one mapped/mounted read-only, and one mapped/mounted as read-write. Both are shared using Samba 4.x. One Samba server gives read-only access to the world for the data. The other gives read-write access to a very limited set of users who occasionally need to add data. However, when testing this, when changes are made to the read-write Samba server the changes don't seem to be seen by the read-only Samba server. Is there some file system caching going on that will eventually be flushed? Am I living dangerously doing what I have set up? I thought I would avoid most/all potential file system corruption by making sure there is only one read-write access method. Thanks for any answers. Brad ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] Double-mounting of RBD
On Wed, Dec 17, 2014 at 2:31 PM, McNamara, Bradley bradley.mcnam...@seattle.gov wrote: I have a somewhat interesting scenario. I have an RBD of 17TB formatted using XFS. I would like it accessible from two different hosts, one mapped/mounted read-only, and one mapped/mounted as read-write. Both are shared using Samba 4.x. One Samba server gives read-only access to the world for the data. The other gives read-write access to a very limited set of users who occasionally need to add data. However, when testing this, when changes are made to the read-write Samba server the changes don’t seem to be seen by the read-only Samba server. Is there some file system caching going on that will eventually be flushed? Am I living dangerously doing what I have set up? I thought I would avoid most/all potential file system corruption by making sure there is only one read-write access method. Thanks for any answers. Well, you'll avoid corruption by only having one writer, but the other reader is still caching data in-memory that will prevent it from seeing the writes on the disk. Plus I have no idea if mounting xfs read-only actually prevents it from making any writes to the disk; I think some FSes will do stuff like defragment internal data structures in that mode, maybe? -Greg ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] Double-mounting of RBD
On 12/17/2014 03:49 PM, Gregory Farnum wrote: On Wed, Dec 17, 2014 at 2:31 PM, McNamara, Bradley bradley.mcnam...@seattle.gov wrote: I have a somewhat interesting scenario. I have an RBD of 17TB formatted using XFS. I would like it accessible from two different hosts, one mapped/mounted read-only, and one mapped/mounted as read-write. Both are shared using Samba 4.x. One Samba server gives read-only access to the world for the data. The other gives read-write access to a very limited set of users who occasionally need to add data. However, when testing this, when changes are made to the read-write Samba server the changes don’t seem to be seen by the read-only Samba server. Is there some file system caching going on that will eventually be flushed? Am I living dangerously doing what I have set up? I thought I would avoid most/all potential file system corruption by making sure there is only one read-write access method. Thanks for any answers. Well, you'll avoid corruption by only having one writer, but the other reader is still caching data in-memory that will prevent it from seeing the writes on the disk. Plus I have no idea if mounting xfs read-only actually prevents it from making any writes to the disk; I think some FSes will do stuff like defragment internal data structures in that mode, maybe? -Greg FSes mounted read-only still do tend to do things like journal replay, but since the block device is mapped read-only that won't be a problem in this case. ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] Double-mounting of RBD
On Wednesday, December 17, 2014, Josh Durgin josh.dur...@inktank.com wrote: On 12/17/2014 03:49 PM, Gregory Farnum wrote: On Wed, Dec 17, 2014 at 2:31 PM, McNamara, Bradley bradley.mcnam...@seattle.gov wrote: I have a somewhat interesting scenario. I have an RBD of 17TB formatted using XFS. I would like it accessible from two different hosts, one mapped/mounted read-only, and one mapped/mounted as read-write. Both are shared using Samba 4.x. One Samba server gives read-only access to the world for the data. The other gives read-write access to a very limited set of users who occasionally need to add data. However, when testing this, when changes are made to the read-write Samba server the changes don’t seem to be seen by the read-only Samba server. Is there some file system caching going on that will eventually be flushed? Am I living dangerously doing what I have set up? I thought I would avoid most/all potential file system corruption by making sure there is only one read-write access method. Thanks for any answers. Well, you'll avoid corruption by only having one writer, but the other reader is still caching data in-memory that will prevent it from seeing the writes on the disk. Plus I have no idea if mounting xfs read-only actually prevents it from making any writes to the disk; I think some FSes will do stuff like defragment internal data structures in that mode, maybe? -Greg FSes mounted read-only still do tend to do things like journal replay, but since the block device is mapped read-only that won't be a problem in this case. ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com Someone commented that the OS with the readonly mount will still do something potentially damaging to the filesystem at mount time. Something along the lines of replaying the xfs journal and the read write OS being unaware of it. Dig through the ceph mailing list archives. Jake ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
