Hi, Is this with CVS HEAD or with pvfs2-1.5.1? what platform and distro? Can you enable kmod debugging, rerun cp and send the relevant parts of the logs? sudo sh -c "echo 1 > /proc/sys/pvfs2/debug" (for pvfs2-1.5.1) sudo sh -c "echo 32767 > /proc/sys/pvfs2/debug" (for cvs head) thanks! Murali
On Mon, 25 Sep 2006, Robert Olson wrote: > This is weird. I appear to getting permission denied on every other > file to be copied from a local filesystem to the PVFS2 filesystem. > This ring a bell with anyone? > > Thanks, > --bob > > [EMAIL PROTECTED]:/scratch$ cp -r simtab.0917 /disks/pvfs2/olson > cp: cannot create regular file `/disks/pvfs2/olson/simtab.0917/sims. > 10': Permission denied > cp: cannot create regular file `/disks/pvfs2/olson/simtab.0917/sims. > 101': Permission denied > cp: cannot create regular file `/disks/pvfs2/olson/simtab.0917/sims. > 103': Permission denied > cp: cannot create regular file `/disks/pvfs2/olson/simtab.0917/sims. > 105': Permission denied > > > [EMAIL PROTECTED]:/scratch$ ls -l /disks/pvfs2/olson/simtab.0917/ > total 88044 > -rw-r--r-- 1 olson seed-users 37994316 Sep 25 17:26 sims.1 > -rw-r--r-- 1 olson seed-users 30049 Sep 25 17:26 sims.100 > -rw-r--r-- 1 olson seed-users 7226 Sep 25 17:26 sims.102 > -rw-r--r-- 1 olson seed-users 1915643 Sep 25 17:26 sims.104 > -rw-r--r-- 1 olson seed-users 50200576 Sep 25 17:26 sims.106 > > [EMAIL PROTECTED]:/scratch$ for i in simtab.0917/*; do echo $i; /vol/ > pvfs2/bin/pvfs2-cp -t $i /disks/pvfs2/olson/$i; done > simtab.0917/sims.1 > Wrote 37994316 bytes in 0.437222 seconds. 82.873696 MB/seconds > simtab.0917/sims.10 > open: Permission denied > could not open /disks/pvfs2/olson/simtab.0917/sims.10 > Could not open /disks/pvfs2/olson/simtab.0917/sims.10 > simtab.0917/sims.100 > Wrote 30049 bytes in 0.001754 seconds. 16.339859 MB/seconds > simtab.0917/sims.101 > Wrote 18867 bytes in 0.035888 seconds. 0.501365 MB/seconds > simtab.0917/sims.102 > Wrote 7226 bytes in 0.001195 seconds. 5.766959 MB/seconds > simtab.0917/sims.103 > open: Permission denied > could not open /disks/pvfs2/olson/simtab.0917/sims.103 > Could not open /disks/pvfs2/olson/simtab.0917/sims.103 > simtab.0917/sims.104 > Wrote 1915643 bytes in 0.022985 seconds. 79.482314 MB/seconds > simtab.0917/sims.105 > Wrote 113037406 bytes in 3.774026 seconds. 28.563891 MB/seconds > simtab.0917/sims.106 > Wrote 177660236 bytes in 5.191931 seconds. 32.633331 MB/seconds > simtab.0917/sims.107 > open: Permission denied > could not open /disks/pvfs2/olson/simtab.0917/sims.107 > Could not open /disks/pvfs2/olson/simtab.0917/sims.107 > simtab.0917/sims.108 > Wrote 152382253 bytes in 5.079284 seconds. 28.610931 MB/seconds > simtab.0917/sims.109 > open: Permission denied > could not open /disks/pvfs2/olson/simtab.0917/sims.109 > Could not open /disks/pvfs2/olson/simtab.0917/sims.109 > simtab.0917/sims.11 > Wrote 55995114 bytes in 1.893243 seconds. 28.206152 MB/seconds > simtab.0917/sims.110 > open: Permission denied > could not open /disks/pvfs2/olson/simtab.0917/sims.110 > Could not open /disks/pvfs2/olson/simtab.0917/sims.110 > simtab.0917/sims.111 > Wrote 85578997 bytes in 2.864490 seconds. 28.491805 MB/seconds > simtab.0917/sims.112 > open: Permission denied > could not open /disks/pvfs2/olson/simtab.0917/sims.112 > Could not open /disks/pvfs2/olson/simtab.0917/sims.112 > simtab.0917/sims.113 > Wrote 159349826 bytes in 5.469951 seconds. 27.782303 MB/seconds > simtab.0917/sims.114 > open: Permission denied > could not open /disks/pvfs2/olson/simtab.0917/sims.114 > Could not open /disks/pvfs2/olson/simtab.0917/sims.114 > simtab.0917/sims.115 > Wrote 385509 bytes in 0.043475 seconds. 8.456602 MB/seconds > simtab.0917/sims.116 > open: Permission denied > could not open /disks/pvfs2/olson/simtab.0917/sims.116 > Could not open /disks/pvfs2/olson/simtab.0917/sims.116 > simtab.0917/sims.117 > Wrote 125580543 bytes in 4.246360 seconds. 28.203670 MB/seconds > simtab.0917/sims.118 > open: Permission denied > could not open /disks/pvfs2/olson/simtab.0917/sims.118 > Could not open /disks/pvfs2/olson/simtab.0917/sims.118 > simtab.0917/sims.119 > Wrote 74459114 bytes in 2.527315 seconds. 28.096912 MB/seconds > simtab.0917/sims.12 > open: Permission denied > could not open /disks/pvfs2/olson/simtab.0917/sims.12 > Could not open /disks/pvfs2/olson/simtab.0917/sims.12 > [etc] > > > pvfs-statfs is happy: > > [EMAIL PROTECTED]:~$ /vol/pvfs2/bin/pvfs2-statfs -m /mnt/pvfs2 > > aggregate statistics: > --------------------------------------- > > fs_id: 1239905522 > total number of servers (meta and I/O): 4 > handles available (meta and I/O): 5365714538 > handles total (meta and I/O): 5368709115 > bytes available: 1218774892544 > bytes total: 1495224942592 > > NOTE: The aggregate total and available statistics are calculated based > on an algorithm that assumes data will be distributed evenly; thus > the free space is equal to the smallest I/O server capacity > multiplied by the number of I/O servers. If this number seems > unusually small, then check the individual server statistics below > to look for problematic servers. > > meta server statistics: > --------------------------------------- > > server: tcp://bio-ppc-head-4:3334 > RAM bytes total : 4107878400 > RAM bytes free : 13307904 > uptime (seconds) : 4045972 > load averages : 4704 9088 5664 > handles available: 2145227856 > handles total : 2147483646 > bytes available : 310944141312 > bytes total : 373806235648 > mode: serving both metadata and I/O data > > > I/O server statistics: > --------------------------------------- > > server: tcp://bio-ppc-head-4:3334 > RAM bytes total : 4107878400 > RAM bytes free : 13307904 > uptime (seconds) : 4045972 > load averages : 4704 9088 5664 > handles available: 2145227856 > handles total : 2147483646 > bytes available : 310944141312 > bytes total : 373806235648 > mode: serving both metadata and I/O data > > server: tcp://bio-ppc-head-1:3334 > RAM bytes total : 4107878400 > RAM bytes free : 2062213120 > uptime (seconds) : 832980 > load averages : 30624 12352 3872 > handles available: 1073741272 > handles total : 1073741823 > bytes available : 307329638400 > bytes total : 373806235648 > mode: serving only I/O data > > server: tcp://bio-ppc-head-2:3334 > RAM bytes total : 4107878400 > RAM bytes free : 12697600 > uptime (seconds) : 4045977 > load averages : 10592 6720 1792 > handles available: 1073372712 > handles total : 1073741823 > bytes available : 304693723136 > bytes total : 373806235648 > mode: serving only I/O data > > server: tcp://bio-ppc-head-3:3334 > RAM bytes total : 3578347520 > RAM bytes free : 12324864 > uptime (seconds) : 4046045 > load averages : 13056 9376 3136 > handles available: 1073372698 > handles total : 1073741823 > bytes available : 309065900032 > bytes total : 373806235648 > mode: serving only I/O data > > _______________________________________________ > Pvfs2-users mailing list > [email protected] > http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users > > _______________________________________________ Pvfs2-users mailing list [email protected] http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users
