Thanks, Harshavardhana, Anand for the tips!
I checked out parts of the linux-2.6.34 (the one we are using down here)
knfsd/fuse code. I understood (hopefully, rightly) that when we export a fuse
directory over NFS and specify an fsid, the handle is constructed somewhat like
this:
fh_size (4 bytes) - fh_version and stuff (4 bytes) - fsid, from export parms (4
bytes) - nodeid (8 bytes) - generation number (4 bytes) - parent nodeid (8
bytes) - parent generation (4 bytes).
So, since Anand mentions that nodeid's for glusterfs are just the inode_t
addresses on servers, I can now relate to the fact that the file handles might
not even survive failovers in any and every case, even with the fsid constant.
That's why I was so confused.. I never faced an issue with stale file handles
during failover yet! Maybe something to do with the order in which files were
created on the replica server following heal commencement (our data is quite
static btw) - like, if you malloc identical things on two identical platforms
by running the same executable on each, you get allocations at the exact same
virtual addresses...
However, now that I understand at least in part how this works, Glusterfs NFS
does seem a lot cleaner... Will also try out the Ganesha...
Thanks!
On Tuesday, 24 December 2013 11:04 PM, Harshavardhana
<[email protected]> wrote:
On Tue, Dec 24, 2013 at 8:21 AM, Anirban Ghoshal
<[email protected]> wrote:
> Hi, and Thanks a lot, Anand!
>
> I was initially searching for a good answer to why the glusterfs site lists
> knfsd as NOT compatible with the glusterfs. So, now I know. :)
>
> Funnily enough, we didn't have a problem with the failover during our
> testing. We passed constant fsid's (fsid=xxx) while exporting our mounts and
> NFS mounts on client applications haven't called any of the file handles out
> stale while migrating the NFS service from one server to the other. Not sure
> why this happpens.
Using fsid is just a workaround always used to solve ESTALE on file
handles. The device major/minor numbers are embedded in the NFS file
handle, a problem when an NFS export is failed over or moved to
another node during failover is that these numbers change when the
resource is exported on the new node resulting in client to see a
"Stale NFS file handle" error. We need to make sure the embedded
number stays the same that is where the fsid export option - allowing
us to specify a coherent number across various clients.
GlusterNFS server is way cleaner solution for such consistency.
Another thing would be to take the next step, give a go for
'NFS-Ganesha' and 'GlusterFS' integration?
https://forge.gluster.org/nfs-ganesha-and-glusterfs-integration
http://www.gluster.org/2013/09/gluster-ganesha-nfsv4-initial-impressions/
Cheers
--
Religious confuse piety with mere ritual, the virtuous confuse
regulation with outcomes
_______________________________________________
Gluster-users mailing list
[email protected]
http://supercolony.gluster.org/mailman/listinfo/gluster-users
