Hey Phil, The pvfs2-stat call never actually made it past sys_lookup, so there was no ref.handle to print out. Here is the output:
#pvfs2-stat /mnt/pvfs2/file1.txt PVFS_sys_lookup: No such file or directory (error class: 0) Error stating [/mnt/pvfs2/file1.txt] Bart. On Thu, Oct 7, 2010 at 3:13 PM, Phil Carns <[email protected]> wrote: > Hi Bart, > > Can you run pvfs2-stat on one of the files, and also send along the fs.conf > file? pvfs2-stat might be helpful because it shows the metadata handle > value. We can compare that value to the handle ranges in the conf file to > narrow down whether it is hitting a metadata object that has just been > corrupted somehow, or whether it really is hitting a datafile handle. > > If pvfs2-stat fails to show any output, then maybe you can modify > pvfs2-stat.c to print the value of ref.handle right before the sys_getattr() > call. > > thanks, > -Phil > > > > > On 10/07/2010 01:08 PM, Bart Taylor wrote: > > Hey guys, > > We are having an increasing number of files that cannot be removed on our > 2.6 file systems. When we run the pvfs2-lsplus tool, the output on these > files looks like this: > > [E 15:14:05.798568] Invalid type 2 in readdirplus > ---------- 1 root root 0 1969-12-31 18:00 File1 > [E 15:14:17.712553] Invalid type 2 in readdirplus > ---------- 1 root root 0 1969-12-31 18:00 File2 > [E 15:14:24.799221] Invalid type 2 in readdirplus > [E 15:14:24.799257] Invalid type 2 in readdirplus > [E 15:14:24.799269] Invalid type 2 in readdirplus > ---------- 1 root root 0 1969-12-31 18:00 File3.txt > ---------- 1 root root 0 1969-12-31 18:00 File5.txt > ---------- 1 root root 0 1969-12-31 18:00 File6.txt > > > The "Invalid type 2" message indicates that readdirplus is returning > datafile attributes mixed in with the directory entries. That might explain > why all of the attributes look like default values, but I am not sure why > those files are having problems in the first place. > > This gets noticed when someone tries to update, create, append, etc. the > file. Most operations seem to return "No such file or directory" when trying > to access those files. A standard /bin/ls will return normally, but long > listings fail. pvfs2-ls, pvfs2-viewdist and pvfs2-validate return getattr > failures. pvfs2-rm returns output like this: > > [E 09:42:30.669413] Error: failed removing one or more datafiles associated > with the meta handle 238502937 > [E 09:42:30.669599] WARNING: PVFS_sys_remove() encountered an error which > may lead > to inconsistent state: No such file or directory > [E 09:42:30.669614] WARNING: PVFS2 fsck (if available) may be needed. > Error: An error occurred while removing /mnt/pvfs2/file1.txt > PVFS_sys_remove: No such file or directory (error class: 0) > > Removing these files is a manual process. These are the steps we follow: > - Track down the file(s) that are causing the problems > - pvfs2-stat on the directory where the file resides > - Grab the FSID and handle from the output > - pvfs2-remove-object using the file name, directory handle, and FSID > > As more of these files start appearing, this process is becoming slow and > painful. It would be great if we could sort out why these files are showing > up like they are, but right now I think a utility that could efficiently > remove these files without the legwork would be really helpful. Any idea > what might work based on what we are seeing? > > I am not sure if the problem also exists in 2.8, but it may be related to > this issue mailed in by Jim in September: > http://www.beowulf-underground.org/pipermail/pvfs2-users/2010-September/003186.html > We are experiencing this issue as well. > > Thanks, > Bart. > > > > > > _______________________________________________ > Pvfs2-developers mailing > [email protected]http://www.beowulf-underground.org/mailman/listinfo/pvfs2-developers > > > > _______________________________________________ > Pvfs2-developers mailing list > [email protected] > http://www.beowulf-underground.org/mailman/listinfo/pvfs2-developers > >
_______________________________________________ Pvfs2-developers mailing list [email protected] http://www.beowulf-underground.org/mailman/listinfo/pvfs2-developers
