On 9/14/10 5:57 PM, satish patil wrote: > Hello, > > Recently we installed 6 OSS pairs with 8 OST per pair. Total 48 OST's. Each > OST is with 3.7 TB. In all it is 177 TB file system. Lustre version installed > is 1.8.1.1 and currently using client based on RHEL 5U2 which is 1.6.x. When > running the individual OST test from performance perspecitve we are able to > get around 17.5 GB performance. Out target is to cross 10 GBPS write > performance using single file w/o -F option avoiding client side cache. I > have reached max to 7.5GB for write performance , but not going beyond. I > tried using stripe count as 48 for a single file along with default stripe > size which is 1MB. But not able to cross 10 GBPS. Can you give a detailed description for your system topology? We have met customer with more large theory bandwidth, but worse performance, because of the unexpected back-end storage performance in parallel.
For I/O performance testing, full stripe maybe not the best choice. Using single stripe files, and spreading these relative small files to all OSTs evenly, maybe give better result. > Command line used for running the IOR as follow > /opt/intel/mpi/bin64/mpirun --totalnum=96 --file=$PBS_NODEFILE > --rsh=/usr/bin/ssh -1 --ordered --verbose -l -machinefile $PBS_NODEFILE -np > 96 /newScratch/IOR/src/C/IOR.mpiio -a MPIIO -b 22G -C -i 3 -k -t 1m -w -r -R > -W -x -N 96 -o /newScratch/hp.stripeC48/IOR.dat > > We have used lustre_config to create the file system. On the other hand, lustre provides basic I/O performance utils (under lustre-iokit). You can use them step by step for the basic elements performance (like back-end storage, obdfilter, and network), which can help you to locate where the performance issues are. Cheers, Nasf > Appriciate your help. > > Regards > SP > > > > _______________________________________________ > Lustre-discuss mailing list > [email protected] > http://lists.lustre.org/mailman/listinfo/lustre-discuss _______________________________________________ Lustre-discuss mailing list [email protected] http://lists.lustre.org/mailman/listinfo/lustre-discuss
