Hi Or,
Or Gerlitz wrote:
Hi Yevgeny,
I wanted to clarify to/with you some issues re QoS/IPoIB in openSM.
Looking in the documents provided by the ofed-docs package:
from QoS_in_OFED.txt
147
==============================================================================
148 5. IPoIB
149
==============================================================================
151 IPoIB queries the SA for its broadcast group information.
152 It provides the broadcast group SL, MTU, and RATE in every
following
153 PathRecord query performed when a new UDAV is needed by IPoIB.
This is almost all wrong, sorry... the way it works in the Linux ipoib
driver, is the following:
address vectors for unicast traffic, both datagram and connected
modes, are created through the result of an SA path query with this
comp mask <SGID, DGID, NUMB_PATH, TRAFFIC_CLASS, PKEY> where numb_path
is set to one, and the traffic class is the one returned by the SA to
the broadcast group, see below.
address vectors for multicast senders only (clients) are created
through the result of an SA MC member query (join) with this comp mask
<MGID, SGID, PKEY, JOIN_STATE>
address vectors for multicast receivers (that might send as well) are
created through the result of an SA MC member query (join) with this
comp mask <MGID, SGID, PKEY, JOIN_STATE>
For all but the broadcast group, the following bits are also set in
the comp mask
<QKEY, MTU_SELECTOR, MTU, TRAFFIC_CLASS, RATE_SELECTOR, RATE, SL,
FLOW_LABEL, HOP_LIMIT> where they are all derived from the broadcast
group, where in the broadcast case, the SA has assigned values for them.
Bottom line, for path queries the --requested-- SL is not provided to
the SM and there's a difference between the info provided for sender
joins to server joins where only for the latter the driver asks for
specific <SL, RATE, MTU>.
OK. Can you rephrase what you said in two sentences? Something short
to replace the existing (and wrong, as you pointed out) three lines:
151 IPoIB queries the SA for its broadcast group information.
152 It provides the broadcast group SL, MTU, and RATE in every following
153 PathRecord query performed when a new UDAV is needed by IPoIB.
from QoS_management_in_opensm.txt
353 6.1 IPoIB
354 IPoIB query is matched by PKey. Default PKey for IPoIB partition
is 0x7fff, so
355 the following three match rules are equivalent:
356
357 ipoib : <SL>
358 ipoib, 0x7fff : <SL>
359 any, pkey 0x7fff : <SL>
I see, so ipoib is just an acronym for a path query with the default
pkey.
"ipoib" w/o any pkey is an acronym for a path query with the default pkey,
but this rule also tells OpenSM to set the partition's SL to <SL>, and
to set the IPoIB mcast group's SL to <SL>.
Actually, line 358 is wrong (I'll update it). It should be as follows:
357 ipoib : <SL>
358 ipoib, pkey 0x7fff : <SL>
359 any, pkey 0x7fff : <SL>
These three lines are equivalent.
This creates confusion and I am not sure its well defined, at least in
my case when I put my hands on QoS testing, I couldn't guess that
"ipoib" == "pkey 0x7fff". For example, is it correct that "ipoib, pkey
0x8001 : <SL>" is not valid rule?
This rule is also valid, providing that you actually have a partition
with pkey 0x8001 configured in the partition cfg file.
-- Yevgeny
Or.
_______________________________________________
general mailing list
[email protected]
http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general
To unsubscribe, please visit
http://openib.org/mailman/listinfo/openib-general
_______________________________________________
general mailing list
[email protected]
http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general
To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
