Hi Becky: Thanks for your reply. Here's the code I'm running on the clients:
[root@aeolus ~]# rpm -qa |grep orangefs orangefs-lib-2.8.6-1wsu.x86_64 orangefs-2.8.6-1wsu.x86_64 orangefs-kmod-2.8.6-1wsu.x86_64 And on the servers: [root@pvfs2-io-0-0 ~]# rpm -qa |grep orangefs orangefs-server-2.8.5-1wsu orangefs-2.8.5-1wsu orangefs-lib-2.8.5-1wsu orangefs-doc-2.8.5-1wsu My configs have not changed since the last batch of e-mail on the subject. On the pvfs2 array, I have created a ~10GB file by dd if=/dev/zero of=... trick I get on some clients and cp 10gbtest /dev/null Each client for <=3 gets about 70MB/s Once I add a 4th, they all start dropping transfer speed. With one client, one server shows ~22MB/s, and it goes up until client 3 hits about 70MB/s outbound, client 4 shows no server speed changes, and as mentioned above drops incoming throughput. --Jim On Tue, Oct 16, 2012 at 11:04 AM, Becky Ligon <[email protected]> wrote: > Jim: > > Can you send me your test cases to see if I can duplicate them? > > Which version of OrangeFS are you using? > > Becky > > On Tue, Oct 16, 2012 at 12:17 PM, Jim Kusznir <[email protected]> wrote: >> >> Hi all: >> >> I'm again working on troubleshooting what appears to be poor pvfs2 >> performance on my cluster. Previously, it was suspected that the >> issue was networking, so now I have a 10GbE (Solarflare) network >> solution throughout. Unfortunately, transfer speeds to individual >> clients max out at around 70MB/s (with the servers sending 20-30MB/s) >> for a single concurrent client (i.e., gigabit speeds). When I scale >> that to 3 clients, I see the same performance, but once I add a 4th >> client, the servers max out at about 70-80MB/s and my clients drop to >> about 52MB/s. So it seems that I can only get Gigabit speeds out of >> the system through and through. >> >> What I did notice is a client running at about 70-80MB/s was using up >> about 50% of a core. My server, when it was serving >=3 concurrent >> clients was using 101-107% CPU (eg, one full core plus a little). >> iostat -mx2 reported about 21% utilization on my disks. >> >> I'm suspecting that pvfs2's CPU-intensive portions of the server are >> single-threaded despite how many clients connect, so I'm going to be >> limited there, is that correct? >> >> Any ideas why I can't move data any faster than 80MB/s? >> >> --Jim >> _______________________________________________ >> Pvfs2-users mailing list >> [email protected] >> http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users > > > > > -- > Becky Ligon > OrangeFS Support and Development > Omnibond Systems > Anderson, South Carolina > > _______________________________________________ Pvfs2-users mailing list [email protected] http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users
