Pete Wyckoff wrote:
I think we can declare success and chalk it up to weird scheduler
behavior. I'll run some tests myself, then probably stick that
sched_yield in the source, as it should be harmless for more recent
kernels in that location.
The fact that mkdir is eating 10% CPU on the server is due to this
continuous polling thing that's in there. Even when yielding the
processor to the dbpf thread once in a while, the bmi thread will
still busywait for 1 sec hoping to get some more action from the
network. When we switch to event-driven rather than polling, this
should go away (or at least be configurable to go away).
Pete,
Well unfortunately I don't think we can quite yet declare victory. The
sched_yield() fix allows me to mount the filesystems regularly over
infiniband and do mkdir's and rm's till my hearts content.
Unfortunately doing any kind of real work is still causing problems.
Example: I tried untaring a 13MB file into /u/data1 (doing tar -xvzf).
This takes less than 10 seconds on a local filesystem (even on an nfs
filesystem). It took nearly 5 minutes on the pvfs2 filesystem! The tar
started out fine and pvfs2-client and pvfs2-server were both taking up
10% of the CPU each. Then these numbers started to climb, and the
amount of time between each file getting written started to slow at what
seemed to be an exponential rate. By the time the file had finished
untarring the system load was around 2.5 and 100% of the CPU was being
used by pvfs2-server and pvfs2-client. I/O wait was 0 so it wasn't
waiting on the disk.
So somewhere something still isn't quite right. Just let me know what I
can do to provide some useful data for you to look at and I'll be more
than happy to provide it.
Thanks!
-Lee
_______________________________________________
Pvfs2-developers mailing list
[email protected]
http://www.beowulf-underground.org/mailman/listinfo/pvfs2-developers
