We are currently evaluating possible commodity hardware candidates suitable for a single OSS with a single OST served to the clients via the IB/RDMA. The goal is to provide the peak performance around 1 GB/sec for large streaming I/O for a single file at the client level, *without* striping. In other words, we want to see if we could build a high performance standalone box which would be acting as a Lustre Head for a couple of clients (obviously, we will have to run also the metadata service on it).
Economically, the most attractive scenario is to use the "storage-in-a-box" element as it allows to save on FC/SCSI cards and external disk enclosures. One such candidate box that we tried had three RAID-6 controllers, with 8 disk modules per controller. The machine is Intel dual core 3 GHz, with 8 GB of RAM. And we are able to get an aggregate disk performance of 300+, 600+, 900+ MB/sec on writes if we run 1,2,3 processes against 1,2,3 distinct logical drives. Now comes the interesting point: if we run a single write process against a striped logical volume built upon the three available drives, we only are able to obtain 750 MB/sec. The writer process eats 100% of CPU, and there is no way to improve this. This behaviour, of course, is perfectly normal, but for us this means that if we would base our OST on this combination of CPU + striped volume, we probably will never be able to spit out more than 750 MB/sec peak i/o perf to the clients. Unless the OST backend service itself is multithreaded! As we do not have at the moment a running Lustre/IB environment to check it, I would appreciate if someone could comment on how OST processes are organized internally. If only one thread is doing i/o towards the the backend ext3 partition, we won't be able to go over 750 MB/sec on such a machine. Otherwise, we could probably grow ip to 900 MB/sec. Thanks ahead for any comment - Andrei.
_______________________________________________ Lustre-discuss mailing list [EMAIL PROTECTED] https://mail.clusterfs.com/mailman/listinfo/lustre-discuss
