Lustre doesn't perform well with these kinds of tiny file i/o and metadata intensive operations. If you search back through the lists this issue has been covered a bunch of times. Small i/o is not efficient for a large filesystem and lustre wasn't designed with this kind of workload in mind. It was designed for a proper HPC workload.
On Jul 17, 2008, at 4:45 AM, Johnlya wrote: > ext3 Filesystem test: > #!/bin/bash > for ((i=0; $i<1000000; i++)) > do > fileid="/home/test/testm/$i"; > touch $fileid > done > > time ./fileid.sh > real 16m44.438s > user 3m0.860s > sys 12m35.920s > > time ls > real 1m54.691s > user 0m13.770s > sys 1m0.220s > > touch:1000000/(16*60+44.438)=995.58 > ls:1000000/(1*60+54.691)=8719.08 > > Lustre 1.6.5 slower than normal file system? > > Add: > The network of Lustre Filesystem is Giga network. > > On Thu, 2008-07-17, at 03:50 PM, Johnlya <[EMAIL PROTECTED]> wrote: >> I test touch and ls command of Lustre on Linux >> 2.6.9-67.0.7.EL_lustre. >> 1.6.5smp (x86, 64bit, mem:4G,two CPU). >> Lustre Filesystem: >> one MDS >> one OSS(one OST) >> I found it some problems: >> With the increase in the number of files, Lustre system performance >> decrease too much. >> 1. fileid.sh >> #!/bin/bash >> for ((i=0; $i<10000; i++)) >> do >> fileid="/mnt/webfile/test/$i"; >> touch $fileid >> done >> time ./fileid.sh >> eal 0m20.493s >> user 0m1.930s >> sys 0m9.340s >> >> time ls >> real 0m2.748s >> user 0m0.050s >> sys 0m0.610s >> touch:10000/20.493=487.97 >> ls:10000/2.748=3639.01 >> 2. >> #!/bin/bash >> for ((i=0; $i<1000000; i++)) >> do >> fileid="/mnt/webfile/test/testm/$i"; >> touch $fileid >> done >> time ./fileid.sh >> real 60m56.666s >> user 3m11.690s >> sys 20m37.190s >> >> real 15m48.615s >> user 0m14.530s >> sys 3m52.960s >> >> touch:1000000/(60*60+56.666)=273.47 >> ls:1000000/(15*60+48.615)=1054.16 >> >> _______________________________________________ >> Lustre-discuss mailing list >> [EMAIL PROTECTED]://lists.lustre.org/mailman/listinfo/lustre-discuss > _______________________________________________ > Lustre-discuss mailing list > [email protected] > http://lists.lustre.org/mailman/listinfo/lustre-discuss _______________________________________________ Lustre-discuss mailing list [email protected] http://lists.lustre.org/mailman/listinfo/lustre-discuss
