I think the nfs reexporting is working again now in trunk, but I had to
make another minor dcache change.
The new pvfs d_delete() function (needed to get the simul stat cases
right) was always returning 1, which means that pvfs was always
encouraging the vfs to uncache our dentries (and therefore inodes) when
it had a safe opportunity.
The kernel nfs daemon however, drops references to inodes sometimes and
expects to be able to retrieve them later just using the inode number.
This doesn't work in PVFS if we no longer have the inode cached, because
an inode number by itself isn't enough to reconstruct the PVFS handle
(the inode is only 32 bits; see iget5 etc.). The normal solution that
many file systems use is to override the default export ops so that you
can tell knfsd to use a better key than the inode number to reacquire
inodes when it needs them. Long story, but I believe this solution
would be somewhat hard to implement in PVFS.
I instead adjusted the d_delete() function so that it only returns 1
when the inode is bad. I tweaked the revalidate inode function to make
certain it marks the inode bad on failure. This should keep the simul
test case working but keep the vfs from uncaching trustworthy PVFS
dentries/inodes that knfsd might want to use again. They entries will
still be revalidated when the time comes.
This isn't ideal, but as far as I can tell it gets us back to the same
nfs re-exporting behavior we had in 2.7.0.
-Phil
Robert Latham wrote:
On Fri, Jan 04, 2008 at 12:16:52PM -0500, Phil Carns wrote:
I think something has changed since 2.7.0 that causes some trouble for
NFS reexporting. If I NFS mount PVFS2 from the 2.7.0 release, things
look fine as far as I can tell. If I NFS mount PVFS2 from a current
trunk build, I get this behavior:
Hey Phil
The nightly builds for both HEAD and the 2.7 branch have been in a bad
state for a bit. System interface and MPI-IO tests pass, but VFS
tests (specifically, dbench) hang. I'm surprised to see iget4/iget5
complaints, but I suspect once the nightlies pass then NFS exporting
will work better too.
I'll probably poke around at it a little bit, but I wanted to go ahead
and record it on the mailing list and see if anyone has suggestions.
There aren't *too* many vfs-related changesets in CVS since the last
"known good" revision. If you're looking for an angle of attack on
this one, maybe see if any of those changesets look suspect to you.
==rob
_______________________________________________
Pvfs2-developers mailing list
[email protected]
http://www.beowulf-underground.org/mailman/listinfo/pvfs2-developers
