Hey guys, We have had some problems in the past on 2.6 with file creations leaving bad
files that we cannot delete. Most utilities like ls and rm return "No such file or directory", and pvfs utilities like viewdist, pvfs2-ls, and pvfs2-rm return various errors. We have resorted to looking up the parent handle, the fsid, and filename and using pvfs2-remove-object to delete the entry. But we weren't ever able to intentionally recreate the problem. Recently while testing 2.8, I have been able to reliably trigger a similar scenario where a file creation fails and leaves a garbage entry that cannot be deleted in any of the normal ways requiring the pvfs2-remove-object approach to clean up. The file and various outputs for this case: [r...@client dir]# ls -l 2010.06.10.28050 total 0 ?--------- ? ? ? ? ? File17027 [r...@client dir]# rm 2010.06.10.28050/File17027 rm: cannot lstat `2010.06.10.28050/File17027': No such file or directory [r...@client dir]# rm -rf 2010.06.10.28050 rm: cannot remove directory `2010.06.10.28050': Directory not empty [r...@client dir]# pvfs2-rm 2010.06.10.28050/File17027 Error: An error occurred while removing 2010.06.10.28050/File17027 PVFS_sys_remove: No such file or directory (error class: 0) [r...@client dir]# pvfs2-stat 2010.06.10.28050/File17027 PVFS_sys_lookup: No such file or directory (error class: 0) Error stating [2010.06.10.28050/File17027] [r...@client dir]# pvfs2-viewdist -f 2010.06.10.28050/File17027 PVFS_sys_lookup: No such file or directory (error class: 0) Could not open 2010.06.10.28050/File17027 [r...@client dir]# ls -l 2010.06.10.28050 total 0 ?--------- ? ? ? ? ? File17027 I have included a test script that will spawn off a number of processes, open a bunch of files, write to each of them, then close them. You can tweak the options as you want but using 5 processes and 50,000 files will usually create at least one of these files. Here is an example command: $> ulimit -n 1000000 && ./open-file-limit --num-files=50000 --sleep-time=1 --num-processes=5 --directory=/mnt/pvfs2/ --file-size=1 You may have to do a long listing on any left-over directories to find the file(s). I will give any help I can to help recreate the bad file or find the cause. Until then, is there a better (simpler) way to remove these entries, maybe some sort of utility that doesn't require doing manual handle lookups before getting the file removed? It would ease some support pain if it were simpler to fix. Thanks for your help, Bart.
open-file-limit.c
Description: Binary data
_______________________________________________ Pvfs2-developers mailing list Pvfs2-developers@beowulf-underground.org http://www.beowulf-underground.org/mailman/listinfo/pvfs2-developers
