On 07/23/2010 10:25 PM, henry...@dell.com wrote: > Hello, > > One of my customer want to set up HPC with thousands of compute nodes. > The parallel file system should have 20GB/s throughput. I am not sure > whether lustre can make it. How many IO nodes needed to achieve this target?
I hate to say "it depends" but, it does in fact depend upon many things. What type of IO is the customer doing; large block sequential spread out over many nodes (parallel IO), or small block random, or a mixture? It is possible to achieve 20GB/s, and quite a bit more, using Lustre. As to whether or not that 20GB/s is meaningful to their code(s), thats a different question. It would be 20GB/s in aggregate, over possibly many compute nodes doing IO. > My assumption is 100 or more IO nodes(rack servers) are needed. Hmmm ... If you can achieve 500+ MB/s per OST, then you would need about 40 OSTs. You can have each OSS handle several OSTs. There are efficiency losses you should be aware of, but 20GB/s using some mechanism to measure this, should be possible with a realistic number of units. Don't forget to count efficiency losses in the design. 100 IO nodes ... I presume you mean OSSes? If your units are slower, then yes, you will need more of them to achieve this performance. You would need to make sure you have a well designed and correctly functional Infiniband infrastructure in addition to the other issues. We've found that Lustre is ... very sensitive ... to the Infiniband implementation. Regards, Joe -- Joseph Landman, Ph.D Founder and CEO Scalable Informatics, Inc. email: land...@scalableinformatics.com web : http://scalableinformatics.com http://scalableinformatics.com/jackrabbit phone: +1 734 786 8423 x121 fax : +1 866 888 3112 cell : +1 734 612 4615 _______________________________________________ Lustre-discuss mailing list Lustre-discuss@lists.lustre.org http://lists.lustre.org/mailman/listinfo/lustre-discuss