On Thu, 03 Dec 2009 11:04:16 +0200 Yevgeny Kliteynik <[email protected]> wrote:
> sebastien dugue wrote: > > Hi Yevgeny, > > > > On Thu, 03 Dec 2009 10:01:28 +0200 > > Yevgeny Kliteynik <[email protected]> wrote: > > > >> Sebastien, > >> > >> I noticed that you found the problem in IPoIB child > >> interfaces configuration. Glad that this worked out well. > >> > >> My question is about the note that you left in the issue: > >> > >> " It looks like in 'datagram' mode, the SL weights > >> do not seem to be applied, or maybe this is an > >> artifact of IPoIB in 'datagram mode' " > >> > >> Have you checked that in this mode you do get the right > >> SL for each child interface by shutting off the relevant > >> SL (mapping it to VL15)? > > > > Yes, SL to VL mapping is OK. > > > >> If yes, then what you're saying is that you see that > >> interfaces use the right SL and VL, but you don't see > >> any arbitration between VLs? > > > > Right, whatever the weights I put in the vlarbs tables have absolutely > > no effect when IPoIB is in datagram mode. I don't know if it's > > an arbitration problem (don't think so) or an IPoIB problem. > > > > Could be that due to the 2044 bytes MTU in datagram mode, iperf > > spends much time not doing transfers and fails to provide enough > > data to the interfaces. Don't know. > > > > Once I switched to connected mode, with a 65520 bytes MTU, things > > started to work OK with a much better overall combined bandwidth. > > OK, then "a much better overall combined bandwidth" is an > answer here. VL arbitration kicks in only when you saturate > the link. If you don't, there's no point doing arbitration, > because HW is able so serve any packet that comes w/o the > need to prioritize. Yep, that's the conclusion I came to. Might be interesting to find where the bottleneck is though, that prevents saturating the link. Sébastien. -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
