[EMAIL PROTECTED] wrote on Tue, 06 Jun 2006 16:01 -0500:
> Well unfortunately I don't think we can quite yet declare victory. The
> sched_yield() fix allows me to mount the filesystems regularly over
> infiniband and do mkdir's and rm's till my hearts content.
> Unfortunately doing any kind of real work is still causing problems.
>
> Example: I tried untaring a 13MB file into /u/data1 (doing tar -xvzf).
> This takes less than 10 seconds on a local filesystem (even on an nfs
> filesystem). It took nearly 5 minutes on the pvfs2 filesystem! The tar
> started out fine and pvfs2-client and pvfs2-server were both taking up
> 10% of the CPU each. Then these numbers started to climb, and the
> amount of time between each file getting written started to slow at what
> seemed to be an exponential rate. By the time the file had finished
> untarring the system load was around 2.5 and 100% of the CPU was being
> used by pvfs2-server and pvfs2-client. I/O wait was 0 so it wasn't
> waiting on the disk.
>
> So somewhere something still isn't quite right. Just let me know what I
> can do to provide some useful data for you to look at and I'll be more
> than happy to provide it.
No clue. I can't even think where to start here. I did a similar
thing with linux-2.6.12.tar.bz2, 36MB-ish, 1 pvfs2 server:
/tmp (local fs) 14 sec
/pvfs (tcp) 209 sec
/pvfs-ib (ib) 163 sec
It didn't slow down appreciably toward the end, at least as I could
tell by watching the tar verbose output, for any of these tests.
These metadata ops tend to be rather slow using the VFS interface to
pvfs2, you know. But something is funky with your machine.
I think it's best if I get around to doing the event-driven bmi_ib
rather than polling and see if that magically fixes it. Playing
thread scheduling tricks will get us in trouble, as Nathan points
out.
-- Pete
_______________________________________________
Pvfs2-developers mailing list
[email protected]
http://www.beowulf-underground.org/mailman/listinfo/pvfs2-developers
