Thanks for your feedback. Back end storage P2000 G3 which is SAS based - 8 Gbps SAN using 450GB-15K. It is clients requirement to have performanace with Single file using all OST's.
Regards SP --- On Tue, 9/14/10, Fan Yong <[email protected]> wrote: > From: Fan Yong <[email protected]> > Subject: Re: [Lustre-discuss] IOR performance - Need help > To: [email protected] > Date: Tuesday, September 14, 2010, 4:24 PM > On 9/14/10 5:57 PM, satish > patil wrote: > > Hello, > > > > Recently we installed 6 OSS pairs with 8 OST per pair. > Total 48 OST's. Each OST is with 3.7 TB. In all it is 177 TB > file system. Lustre version installed is 1.8.1.1 and > currently using client based on RHEL 5U2 which is 1.6.x. > When running the individual OST test from performance > perspecitve we are able to get around 17.5 GB performance. > Out target is to cross 10 GBPS write performance using > single file w/o -F option avoiding client side cache. > I have reached max to 7.5GB for write performance , but not > going beyond. I tried using stripe count as 48 for a single > file along with default stripe size which is 1MB. But not > able to cross 10 GBPS. > Can you give a detailed description for your system > topology? We have > met customer with more large theory bandwidth, but worse > performance, > because of the unexpected back-end storage performance in > parallel. > > For I/O performance testing, full stripe maybe not the best > choice. > Using single stripe files, and spreading these relative > small files to > all OSTs evenly, maybe give better result. > > Command line used for running the IOR as follow > > /opt/intel/mpi/bin64/mpirun --totalnum=96 > --file=$PBS_NODEFILE --rsh=/usr/bin/ssh -1 --ordered > --verbose -l -machinefile $PBS_NODEFILE -np 96 > /newScratch/IOR/src/C/IOR.mpiio -a MPIIO -b 22G -C -i 3 -k > -t 1m -w -r -R -W -x -N 96 -o > /newScratch/hp.stripeC48/IOR.dat > > > > We have used lustre_config to create the file system. > On the other hand, lustre provides basic I/O performance > utils (under > lustre-iokit). You can use them step by step for the basic > elements > performance (like back-end storage, obdfilter, and > network), which can > help you to locate where the performance issues are. > > > Cheers, > Nasf > > Appriciate your help. > > > > Regards > > SP > > > > > > > > _______________________________________________ > > Lustre-discuss mailing list > > [email protected] > > http://lists.lustre.org/mailman/listinfo/lustre-discuss > _______________________________________________ > Lustre-discuss mailing list > [email protected] > http://lists.lustre.org/mailman/listinfo/lustre-discuss > _______________________________________________ Lustre-discuss mailing list [email protected] http://lists.lustre.org/mailman/listinfo/lustre-discuss
