On Saturday, September 11, 2010, Andreas Dilger wrote: > On 2010-09-10, at 12:11, Michael Robbert wrote: > > Create performance is a flat line of ~150 files/sec across the board. > > Delete performance is all over the place, but no higher than 3,000 > > files/sec... Then yesterday I was browsing the Lustre Operations Manual > > and found section 33.8 that says Lustre is tested with directories as > > large as 10 million files in a single directory and still get lookups at > > a rate of 5,000 files/sec. That leaves me wondering 2 things. How can we > > get 5,000 files/sec for anything and why is our performance dropping off > > so suddenly at after 20k files? > > > > Here is our setup: > > All IO servers are Dell PowerEdge 2950s. 2 8-core sockets with X5355 @ > > 2.66GHz and 16Gb of RAM. The data is on DDN S2A 9550s with 8+2 RAID > > configuration connected directly with 4Gb Fibre channel. > > Are you using the DDN 9550s for the MDT? That would be a bad > configuration, because they can only be configured with RAID-6, and would > explain why you are seeing such bad performance. For the MDT you always
Unfortunately, we failed to copy the scratch MDT in a reasonable time so far. Copying several hundreds of million files turned out to take ages ;) But I guess Mike did the benchmarks for the other filesystem with an EF3010. > > We have as many as 1.4 million files in a single directory and we now > > have half a billion files that we need to deal with in one way or > > another. Mike, is there a chance you can try which rate acp reports? http://oss.oracle.com/~mason/acp/ Also could you please send me your exact bonnie line or script? We could try to reproduce it on and idle test 9550 with a 6620 for metada (the 6620 is slower for that than the ef3010). Thanks, Bernd -- Bernd Schubert DataDirect Networks _______________________________________________ Lustre-discuss mailing list [email protected] http://lists.lustre.org/mailman/listinfo/lustre-discuss
