Look for the Bull NUMIOA presentation from the recent LUG. The short story is that OST thread pinning is critical to getting good performance. The numbers are something like 3.6GB/s without, and 6.0 GB/s with thread affinity.
Cheers, Andreas On 2011-06-02, at 7:23 PM, Mark Nelson <[email protected]> wrote: > Hi List, > > I was wondering if anyone here has looked at the performance > characteristics of lustre OSSes on dual tylersburg motherboards with > raid controllers split up onto separate IO hubs. I imagine that without > proper pinning of service threads to the right CPUs/IOH and memory pools > this could cause some nasty QPI contention. Is this actually a problem > in practice? Is it possible to pin service threads in a reasonable way > based on which OST is involved? Anyone doing this on purpose to try and > gain more overall PCIE bandwidth? > > I imagine that in general it's probably best to stick with a single > socket single IOH OSS. No pinning to worry about, very direct QPI > setup, consistent performance characteristics, etc. > > Thanks, > Mark > _______________________________________________ > Lustre-discuss mailing list > [email protected] > http://lists.lustre.org/mailman/listinfo/lustre-discuss _______________________________________________ Lustre-discuss mailing list [email protected] http://lists.lustre.org/mailman/listinfo/lustre-discuss
