On 10-May-12 4:01 PM, Jeff Squyres wrote:
> Looks ok to me.
>
> FWIW, I'd put EDR in, if you know the values (even though there won't be any
> hardware for a while). That'll just give this code a longer shelf life /
> future-proof-ness...
Great, I'll add EDR too.
There's actually a patch pending to be accepted in kernel
that adds enums instead of all these values. But for now
we only have these numbers.
-- YK
>
> On May 10, 2012, at 8:57 AM, Yevgeny Kliteynik wrote:
>
>> On 10-May-12 3:42 PM, Yevgeny Kliteynik wrote:
>>>
>>> Here are the ActiveSpeed values:
>>>
>>> 1: SDR - 2.5 Gb/s * (8/10 coding)
>>> 2: DDR - 5 Gb/s * (8/10 coding)
>>> 4: QDR - 10 Gb/s * (8/10 coding)
>>> 8: FDR10 - 10 Gb/s * (64/66 coding)
>>> 16: FDR - 14 Gbps * (64/66 coding)
>>> 32: EDR - 25 Gbs * (64/66 coding)
>>>
>>> EDR is not out yet, so we can ignore it for the sake of this patch.
>>> It is also true that *most* of IB cables are 4x.
>>> There are 12x clusters out there (like TACC's Ranger), and sometimes
>>> there are 1x links (mostly because the cable is not perfect or too long).
>>>
>>> So cases of 4 and 8 are not the same for us, because MPI needs actual
>>> *data* rate instead of *bit* rate, which is different in these cases.
>>>
>>> I'll send a patch for review in a minute...
>>>
>>
>> How about this patch? If it looks OK, I'll commit to trunk and open CMR.
>>
>> ndex: btl_openib_component.c
>> ===================================================================
>> --- btl_openib_component.c (revision 26420)
>> +++ btl_openib_component.c (working copy)
>> @@ -810,8 +810,10 @@
>> we have to look up the values corresponding to
>> port->active_speed and port->active_width. These
>> are enums corresponding to the IB spec. Overall
>> - forumula is 80% of the reported speed (to get the
>> - true link speed) times the number of links. */
>> + forumula to get the true link speed is 8/10 or
>> + 64/66 of the reported speed (depends on the coding
>> + that is being used for the particular speed) times
>> + the number of links. */
>> switch (ib_port_attr->active_speed) {
>> case 1:
>> /* 2.5Gbps * 0.8, in megabits */
>> @@ -826,9 +828,13 @@
>> openib_btl->super.btl_bandwidth = 8000;
>> break;
>> case 8:
>> - /* 40.0Gbps * 0.8, in megabits */
>> - openib_btl->super.btl_bandwidth = 32000;
>> + /* FDR10: 10.0Gbps * 64/66, in megabits */
>> + openib_btl->super.btl_bandwidth = 9700;
>> break;
>> + case 16:
>> + /* FDR: 14.0Gbps * 64/66, in megabits */
>> + openib_btl->super.btl_bandwidth = 13600;
>> + break;
>> default:
>> /* Who knows? Declare this port unreachable (do
>> *not* return ERR_VALUE_OF_OUT_OF_BOUNDS; that
>
>
