Sorry, by PGI, I meant PGFS...an e-mail had come in about PGI while I was typing, and my brain swapped things....
My interconnect between all nodes is 10Gbps ethernet. The main reason is fear of downtime (last time I did an upgrade, the servers went and did their things for a couple hours, and one of them didn't complete successfully, resulting in way too much downtime). command was time cp /mnt/pvfs2/testing/10gbtest /scratch/ network bandwidth was monitored in another window with bwm-ng, showed 11-30MB/s Top was also used for CPU monitoring, and pvfs usage was high. On another system with some user jobs, I was seeing 8-11MB/s network throughput (inbound) and pvfs2-client-core using 66% of a core. I thought that was problematic. --Jim On Thu, Aug 16, 2012 at 10:00 AM, Becky Ligon <[email protected]> wrote: > Hi Jim! > > What is PGI? > > What is your interconnect between compute nodes and I/O servers? > > Is there some reason why you didn't upgrade the I/O servers to 2.8.6? (Just > curious) > > Can you send me the exact commands that you are using for your copy tests? > > Are your users able to access the directories that were inaccessible before > the upgrade? > > What is your linux version now that you've upgraded? Is it still > 2.6.32-220.13.1.el6? > > Are you now able to mount-on-boot? > > Can you attach to the pvfs2-client-core process using gdb to see where it > seems to be spending its time? > > Becky > > On Thu, Aug 16, 2012 at 12:00 PM, Jim Kusznir <[email protected]> wrote: >> >> Hello: >> >> I'm still struggling to get performance from PGI that's close to local >> single-disk I/O speeds. >> >> Presently, on my newly-rebuilt cluster with orangefs 2.8.6 clients / >> 2.8.5 servers (3), I am seeing about 68% CPU core loading from >> pvfs2-client-core, and about 11MB/s throughput on the network. >> >> In other tests, trying to copy a 10gb file from pvfs to local, I maxed >> out at about 30MB/s, with 20MB/s more typical. >> >> I feel I should be seeing significantly better speeds than this...And >> that pvfs should not be so client-cpu-intensive. Suggestions as to >> how to debug? >> >> I'm also getting reports from users that they note a significant >> runtime improvement of their jobs if they switch from using pvfs2 to >> using NFS home directories...I also think that should be wrong. >> >> Thanks! >> --Jim >> _______________________________________________ >> Pvfs2-users mailing list >> [email protected] >> http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users > > > > > -- > Becky Ligon > OrangeFS Support and Development > Omnibond Systems > Anderson, South Carolina > > _______________________________________________ Pvfs2-users mailing list [email protected] http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users
