255 I think I tested with default 0 before, that is send at most one packet before give low priority table the chance according to IBA. It doesn't seem to make a difference though.
Feiyi -----Original Message----- From: Hal Rosenstock [mailto:[EMAIL PROTECTED] Sent: Friday, November 03, 2006 3:51 PM To: Wang, Feiyi Cc: [email protected] Subject: RE: [openib-general] question on QoS support On Fri, 2006-11-03 at 15:43, Wang, Feiyi wrote: > The test is done on two hosts, say A and B. A has 4x SDR (run ib_rdam_bw > as server), B has 4x DDR (run more than one thread of ib_rdma_bw as > clients). The sl2vl table read as: > > smpquery sl2vl 7 > # SL2VL table: Lid 7 > # SL: | 0| 1| 2| 3| 4| 5| 6| 7| 8| 9|10|11|12|13|14|15| > ports: in 0, out 0: | 0| 1| 2| 3| 4| 5| 6| 7| 0| 1| 2| 3| 4| 5| 6| 7| > > smpquery vlarb 7 > # VLArbitration tables: Lid 7 port 0 LowCap 8 HighCap 8 > # Low priority VL Arbitration Table: > VL : |0x0 |0x1 |0x2 |0x3 |0x4 |0x5 |0x6 |0x7 | > WEIGHT: |0x0 |0x0 |0x0 |0x0 |0x0 |0x0 |0x0 |0x0 | > # High priority VL Arbitration Table: > VL : |0x0 |0x1 |0x2 |0x3 |0x4 |0x5 |0x6 |0x7 | > WEIGHT: |0x1 |0x0 |0x8 |0x0 |0x0 |0x0 |0x0 |0x0 | > > Low priority table entries are all zero to skip. > High priority table give VL 0 and VL 2 different weight. > > The SL is specified on command line, one thread with SL 0, the other > thread with SL 2. > > Thanks for looking into this, and let me know if more info is needed. What's the limit of high priority ? -- Hal > Feiyi > > > > -----Original Message----- > From: Hal Rosenstock [mailto:[EMAIL PROTECTED] > Sent: Friday, November 03, 2006 3:27 PM > To: Wang, Feiyi > Cc: [email protected] > Subject: Re: [openib-general] question on QoS support > > On Fri, 2006-11-03 at 15:12, Feiyi Wang wrote: > > In our test at the ORNL - it appears you can "turn off" the traffic by > > giving every VL weight 0. > > A weight of 0 indicates to skip that entry. > > > As soon as you assign non-zero VL weight, > > the traffic starts to flow, however, VL with more weight doesn't have > > expected preference treatment. In other words, traffic shaping didn't > > take place. smpquery vlarb verified the mapping table was there. > > correctly ? > > Is it high or low priority or both ? > > What about SL2VLMapping table ? Is it setup correctly ? > > What's your topology for this ? > > Can you send your SL2VLMapping and VLarbitration configuration ? > > > I believe the scenario described below 'should' be able to generate > > congestion point ... but it would be helpful if someone can elaborate > > a way to "look into" how/if scheduling/arbitration take place. > > The only ways I know would be to look at either the packets on the wire > or what you are doing with multiple streams which seems valid to me. > > Have you read section 7.6.9.2 (p. 189-190) in IBA 1.2 volume 1 to > understand how to configure this ? > > -- Hal > > > Best, > > > > Feiyi > > > > > > On 02 Nov 2006 10:49:04 -0500, Hal Rosenstock <[EMAIL PROTECTED]> > wrote: > > > Hi Oliver, > > > > > > On Thu, 2006-11-02 at 10:20, Oliver wrote: > > > > Hi, Hal - > > > > > > > > > How is this being observed/measured ? > > > > > > > > Host A, B, with 4x DDR both connected to Flextronic switch. > > > > A single process of ibv_read_bw gives about 1415MB /s average > > > > bandwidth. Two concurrent process report 714.45 MB/s each, dead > even. > > > > Now if I bump up one process with a different SL, then I expect to > see > > > > shaping to take place. Please let me if the scenario makes sense. > > > > > > It makes sense. However, if the higher priority traffic does not > fill > > > the scheduling, the low priority can take up the slack so I'm not > sure > > > if this is what you are seeing or something else. > > > > > > It might be interesting to try the same thing at SDR speeds. > > > > > > -- Hal > > > > > > > > Yes, 8 VLs should be supported in your subnet. You can verify > this with > > > > > smpquery portinfo on the HCA port and examine OperVLs assuming > the port > > > > > is ACTIVE. > > > > > > > > yes, I verified the data VL support, it is 8. I will poke for more > > > > info with suggested commands by Sasha. > > > > > > > > > > A related question is, if I modify qos setting in SM, do I > need to > > > > > > restart SA on each hosts for it to see the changes? (I am > hoping not, > > > > > > as I tried in the test, it doesn't seem to make a difference) > > > > > > > > > > Not sure what you mean. SA is tightly coupled with the OpenSM. > Do you > > > > > mean SA client ? The client hosts don't need restarting but did > you > > > > > restart OpenSM with your QoS configuration ? > > > > > > > > I mean client SA. yes, I understand OpenSM needs to be restarted. > > > > > > > > > BTW, which OpenSM are you running ? > > > > > > > > OFED 1.1 based. > > > > > > > > thanks > > > > > > > > - Oliver > > > > > > > _______________________________________________ openib-general mailing list [email protected] http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
