On Thu, 2011-03-31 at 10:19 -0400, chas williams - CONTRACTOR wrote: > On Wed, 30 Mar 2011 12:00:04 -0400 > David Dillow <[email protected]> wrote: > > > While it is true both are full duplex, there are also setup messages > > flowing in both directions to set up the large transfers. In the past, > > we've certainly seen problems at scale with small messages getting > > blocked behind large bulk traffic on LNET. It would be interesting to > > see how much self-interference is generated when running storage over > > the same HCA as LNET, versus having them on separate NICs -- especially > > i kind of gather that if your clients are doing a mix of i/o (reads and > writes) that this is going to happen regardless. two hcas/ports get a > sort of parallelism to help alleviate some of this congestion. but a > faster port (say 40G) would just as helpful?
That makes sense, but then if we we're doing QDR ports, we'd probably matching that with QDR on the storage as well. If you have DDR on the storage, then using a single QDR HCA would probably work; I'd still want to test a bit before running with it to be sure. -- Dave Dillow National Center for Computational Science Oak Ridge National Laboratory (865) 241-6602 office _______________________________________________ Lustre-discuss mailing list [email protected] http://lists.lustre.org/mailman/listinfo/lustre-discuss
