Nevermind. 1.3.2 fixes the problem. Should have read the ChangeLog more carefully.
-Brian On Friday 20 January 2006 12:05, Brian R. Smith wrote: > Hello again, > > Just to let you know, we've observed this same behavior on another cluster, > with uniform 32-bit Intel Xeon processors, all running > 2.6.14-1.1653_FC4smp. > > Again, thank you again for an help and please let me know if you want any > more information. > > -Brian > > On Thursday 19 January 2006 23:41, Brian R. Smith wrote: > > Hi all, > > > > I'll probably be told to take this up in the user's list, but I've had > > bad experiences with 'user' lists. I figured this time, I'd go straight > > to the guys in-the-know. > > > > I wasn't quite sure how to title this post given the nature of my > > problem. My cluster is made up of some machines which are i386-based, > > running 2.6.13-1.1532_FC4 and 2.6.14-1.1653_FC4smp, and some machines > > which are x86_64-based, running 2.6.11.11smp. I/O and Metaservers are > > running 2.6.13-1.1532_FC4. > > > > Here is what I have found: > > > > Given this example file /scratch/xxx/file where /scratch is our > > PVFS2-1.3.0 volume and an identical file stored on ext3 > > /home/student/x/xxx/file, I get different results from the stat command: > > > > [EMAIL PROTECTED] ~]$ stat /scratch/xxx/file > > File: `/scratch/xxx/file' > > Size: 150263 Blocks: 296 IO Block: 4194304 regular file > > Device: 13h/19d Inode: 1046003 Links: 1 > > Access: (0644/-rw-r--r--) Uid: (1288406/xxx) Gid: ( 100/ users) > > Access: 2006-01-10 16:48:35.000000000 -0500 > > Modify: 2006-01-10 16:48:35.000000000 -0500 > > Change: 2006-01-10 16:48:35.000000000 -0500 > > > > [EMAIL PROTECTED] ~]$ stat /home/student/x/xxx/file > > File: `/home/student/x/xxx/file' > > Size: 150263 Blocks: 304 IO Block: 4096 regular file > > Device: 811h/2065d Inode: 63047415 Links: 1 > > Access: (0644/-rw-r--r--) Uid: (1229806/xxx) Gid: ( 100/ users) > > Access: 2006-01-19 21:24:41.000000000 -0500 > > Modify: 2006-01-19 21:18:02.000000000 -0500 > > Change: 2006-01-19 21:18:02.000000000 -0500 > > > > even though 'diff' shows no differences between the two files. > > > > >From this, I get: > > > > /scratch/xxx/file: Block size = 296 - File size = 147456 > > /home/student/x/xxx/file: Block size = 304 - File size = 150263 > > > > Blocks (stat) Sizes (ls -l) > > 296/304 ~= 147456/150263 > > > > where (296/304 = 0.973584) and (147456/150263 = 0.981319) showing that > > the ratios are roughly equivalent which lead me to investigate the stat, > > cp, and cat commands. Copying the file from PVFS2 causes the file to be > > truncated upon being written to disk. > > > > The 'cp' command must call xstat() or lxstat() to get the number of > > blocks from a particular file, assumedly using that information to know > > "how much" to copy. On the otherhand > > > > cat /scratch/xxx/file > /home/student/x/xxx/file > > > > seems to work perfectly, calling fxstat(). Running 'strings' on 'cp' and > > 'cat' seems to point out a difference in the system calls as 'cat' has a > > single reference to fxstat() while 'cp' references xstat(), fxstat(), > > lxstat(). The stat command only references lxstat() and xstat() so I > > assume that fxstat() must be "safe" for this file at least since 'cat' > > and 'diff' work, but 'cp' and 'stat' are consistently wrong with respect > > to this particular file. > > > > Now, the point of all this is that I am curious as to why PVFS2 is > > recording/reporting false Block sizes in/from the 'inode table' (not sure > > if PVFS2 even has an inode table or if its emulated). Is this a bug, an > > error on my part or has this already been fixed and I probably should > > have asked for a ChangeLog instead? > > > > In summary: > > > > 1) Some files copy cleanly from PVFS2 to other file systems while others > > do not. > > > > 2) The difference in block size reported by stat is proportional to the > > difference in file size reported by 'ls' even though 'cat'ing the file or > > viewing the file while ON PVFS2 shows that all the contents are accounted > > for. > > > > 3) Using the 'cat' command seems to avoid the problem (possibly since it > > avoids making a call to lxstat() or xstat()) but I don't want to push > > this as a fix for our users (nor would any of you, I'd assume). > > > > 4) My PVFS2 I/O and Metaservers are running on AthlonXP's while the > > client in question is running on Opterons. > > > > 5) Files "appear" different using stat when in fact they are the same. > > Copying with 'cp' yields truncated files being written. The difference > > in size of the original file from the copy is proportional to the > > discrepancies observed with 'stat'. > > > > Thanks for any help. > > > > > > -Brian Smith > > > > > > > > > > _______________________________________________ > > Pvfs2-developers mailing list > > [email protected] > > http://www.beowulf-underground.org/mailman/listinfo/pvfs2-developers -- Brian R. Smith HPC Systems Administrator, Research Computing University of South Florida 4202 E Fowler Ave. LIB 608 Tampa, FL. 33620 1(813)974-1467 http://rc.usf.edu _______________________________________________ Pvfs2-developers mailing list [email protected] http://www.beowulf-underground.org/mailman/listinfo/pvfs2-developers
