Hi David,
I still don't get it: when was the performance good? Same software and
hardware, just some time in the past? Or is there a software change?
The nodes aren't being used for anything else, there are no rogue
processes, and the local file systems are otherwise empty?
Thanks,
Rob
On Jul 1, 2009, at 5:05 PM, David Bonnie wrote:
Rob -
Performance is down across all PVFS2 installations. The benchmark
simply creates files of a random size (between 1 and 25 MB) in a
single folder on the mounted PVFS2 partition, 16 KB at a time. It's
not anywhere near ideal, but it's the workload I'm working with.
Prior to this problem we were getting ~22 MB/s write throughput and
we're down to about 2.5 MB/s for no apparent reason. Reads are down
from about 55 MB/s to 30 MB/s. No hardware has changed and as far
as I can tell no hardware has died either.
- Dave
On Wed, Jul 1, 2009 at 4:00 PM, Rob Ross <[email protected]> wrote:
Do you mean that 2.8.0 is fast and 2.8.1 is slow? Can you describe
the benchmark and how you are doing your measurements?
Rob
On Jul 1, 2009, at 4:43 PM, David Bonnie wrote:
Hello all -
I'm having trouble figuring out a problem with performance
depredation on a simple 10 node cluster. Prior runs on the cluster
(before this problem manifested itself) resulted in bandwidth and
IOPS about 10 times higher on a small file creation workload. Each
node is running as a metadata server and a data server.
The problem is persistent between versions and installations of
PVFS2 2.8.0 and 2.8.1. Rebooting all of the nodes didn't improve
anything. The network connections (simple GigE) showed no errors or
dropped packets. Using different physical disks (both SAS and FC)
didn't improve things. The kernel logs didn't show anything out of
place nor did the pvfs2 server or client logs. It seems like a
network issue but I can't seem to find anything wrong with any of
the connections.
Has anyone seen this kind of problem before? I seem to remember
something on the list before about performance suddenly dropping but
I can't find the message now (of course). Any insight would be
appreciated!
Thanks,
- Dave
_______________________________________________
Pvfs2-developers mailing list
[email protected]
http://www.beowulf-underground.org/mailman/listinfo/pvfs2-developers
_______________________________________________
Pvfs2-developers mailing list
[email protected]
http://www.beowulf-underground.org/mailman/listinfo/pvfs2-developers
