Lior, I am curious how you have the 16+2 drives connected to the LSI-9265. It has eight internal ports so I'm guessing you are using some sort of SAS fan out expander? We are thinking of using 3TB drives in a 36 bay enclosure/server so 3 x RAID cards (8+2 RAID6 + 1 spare) seemed like a good fit. It also means the resulting volumes are pretty much bang on the 24TB that Lustre v1.8.6 now supports.
Daire On Tue, Jul 19, 2011 at 7:19 AM, lior amar <[email protected]> wrote: > Hi Daire, > > I am currently building a Lustre system using LSI-2965 (the successor of > the LSI-2960 ) it has 1 GB of RAM and dual core processors. > My experience with the card is quite good (Raid 6 performance of this card > is very good). > > I am using it on my OSS's with the following configuration > > 18 1TB (7200) sas disks in a raid 6 (16 data + 2) with 256KB. > 2 15K sas disks in raid 1 for the journal > > I know the Lustre manual recommends using a stripe unit that will give a > total of 1MB. > But the performance I got when using 64KB stripe units (16*64=1M) were > lower then the case of 256KB. > > The max_sectors_kb is limited to 280 so I set it to 256 > And I tested varios I/O schedulers without getting any difference. > > Raw performance of the raid6 : ~960MB/sec write 1.5GB/sec Read > Lustre performance using the obdfilter-survey (local mode) are also good. I > will email you the results later (not in office now). > > > Please let me know if there are any more details I can provide that might > help you. > > Regards > > --lior > > ----------------------oo--o(:-:)o--oo---------------- > Lior Amar, Ph.D. > Cluster Logic Ltd --> The Art of HPC > www.clusterlogic.net > ---------------------------------------------------------- > >
_______________________________________________ Lustre-discuss mailing list [email protected] http://lists.lustre.org/mailman/listinfo/lustre-discuss
