Hi Yevgeny, On 16:42 Wed 27 Feb , Yevgeny Kliteynik wrote: > > The following doc describes QoS management in OpenSM. > This doc (named QoS_management_in_OpenSM.txt) has been added to > the OFED docs, along with the QoS_in_OFED.txt. > > I'd like to add this info to OpenSM man pages as well.
Yes, I think that it could be useful to have it under opensm/doc too. > I'm including the text here as is, so it will be easier to follow > possible changes. When those will be done, I'll fix the format to > match the OpenSM man pages and post a patch. > > The only problem is that the whole OpenSM man has ~850 lines, > while this QoS management file has ~500 lines... :) I would suggest to have some basic part (50-100 lines) included in the man page and reference an entire document (under opensm/doc) for more details. > Please review. Looks fine, few tiny nits are below. [snip...] > ============================================================================== > 4. Policy File Syntax Guidelines > ============================================================================== > > - Empty lines are ignored. It is mentioned on the next line too. > - Leading and trailing blanks, as well as empty lines, are ignored, so > the indentation in the example is just for better readability. > - Comments are started with the pound sign (#) and terminated by EOL. > - Any keyword should be the first non-blank in the line, unless it's a > comment. > - Keywords that denote section/subsection start have matching closing > keywords. > - Having a QoS Level named "DEFAULT" is a must - it is applied to PR/MPR > requests that didn't match any of the matching rules. > - Any section/subsection of the policy file is optional. [snip...] > ============================================================================== > 6. Simplified QoS Policy - Details and Examples > ============================================================================== > > Simplified QoS policy match rules are tailored for matching ULPs (or some > application on top of a ULP) PR/MPR requests. This section has a list of > per-ULP (or per-application) match rules and the SL that should be enforced > on the matched PR/MPR query. > > Match rules include: > - Default match rule that is applied to PR/MPR query that didn't match any > of the other match rules > - SDP > - SDP application with a specific target TCP/IP port range > - SRP with a specific target IB port GUID > - RDS > - iSER > - iSER application with a specific target TCP/IP port range > - IPoIB with a default PKey > - IPoIB with a specific PKey > - any ULP/application with a specific Service ID in the PR/MPR query > - any ULP/application with a specific PKey in the PR/MPR query > - any ULP/application with a specific target IB port GUID in the PR/MPR > query > > Since any section of the policy file is optional, as long as basic rules of > the file are kept (such as no referring to nonexisting port group, having > default QoS Level, etc), the simplified policy section (qos-ulps) can serve > as a complete QoS policy file. > The shortest policy file in this case would be as follows: > > qos-ulps > default : 0 #default SL > end-qos-ulps > > It is equivalent to the previous example of the shortest policy file, and > it > is also equivalent to not having policy file at all. > > Below is an example of simplified QoS policy with all the possible > keywords: > > qos-ulps > default : 0 # default SL > sdp, port-num 30000 : 0 # SL for application running on > top > # of SDP when a destination > # TCP/IPport is 30000 > sdp, port-num 10000-20000 : 0 > sdp : 1 # default SL for any other > # application running on top of > SDP > rds : 2 # SL for RDS traffic > iser, port-num 900 : 0 # SL for iSER with a specific > target > # port > iser : 3 # default SL for iSER > ipoib, pkey 0x0001 : 0 # SL for IPoIB on partition with > # pkey 0x0001 > ipoib : 4 # default IPoIB partition, > # pkey=0x7FFF > any, service-id 0x6234 : 6 # match any PR/MPR query with a > # specific Service ID > any, pkey 0x0ABC : 6 # match any PR/MPR query with a > # specific PKey > srp, target-port-guid 0x1234 : 5 # SRP when SRP Target is located > on > # a specified IB port GUID > any, target-port-guid 0x0ABC-0xFFFFF : 6 # match any PR/MPR query > with > # a specific target port GUID > end-qos-ulps Likely I missed this in implementation phase. But isn't it better to have ULPs to match QoS level rather than SL? Or probably both? > Similar to the full policy definition, matching of PR/MPR queries is done > in > order of appearance in the QoS policy file such as the first match takes > precedence, except for the "default" rule, which is applied only if the > query > didn't match any other rule. > > All other sections of the QoS policy file take precedence over the qos-ulps > section. That is, if a policy file has both qos-match-rules and qos-ulps > sections, then any query is matched first against the rules in the > qos-match-rules section, and only if there was no match, the query is > matched > against the rules in qos-ulps section. > > Note that some of these match rules may overlap, so in order to use the > simplified QoS definition effectively, it is important to understand how > each > of the ULPs is matched: > > 6.1 IPoIB > IPoIB query is matched by PKey. Default PKey for IPoIB partition is 0x7fff, > so > the following three match rules are equivalent: > > ipoib : <SL> > ipoib, 0x7fff : <SL> > any, pkey 0x7fff : <SL> > > 6.2 SDP > SDP PR query is matched by Service ID. The Service-ID for SDP is > 0x000000000001PPPP, where PPPP are 4 hex digits holding the remote TCP/IP > Port > Number to connect to. The following two match rules are equivalent: > > sdp : <SL> > any, service-id 0x0000000000010000-0x000000000001ffff : <SL> > > 6.3 RDS > Similar to SDP, RDS PR query is matched by Service ID. The Service ID for > RDS > is 0x000000000106PPPP, where PPPP are 4 hex digits holding the remote > TCP/IP > Port Number to connect to. Default port number for RDS is 0x48CA, which > makes > a default Service-ID 0x00000000010648CA. The following two match rules are > equivalent: > > rds : <SL> > any, service-id 0x00000000010648CA : <SL> > > 6.4 iSER > Similar to RDS, iSER query is matched by Service ID, where the the Service > ID > is also 0x000000000106PPPP. Default port number for iSER is 0x035C, which > makes > a default Service-ID 0x000000000106035C. The following two match rules are > equivalent: > > iser : <SL> > any, service-id 0x000000000106035C : <SL> > > 6.5 SRP > Service ID for SRP varies from storage vendor to vendor, thus SRP query is > matched by the target IB port GUID. The following two match rules are > equivalent: > > srp, target-port-guid 0x1234 : <SL> > any, target-port-guid 0x1234 : <SL> > > Note that any of the above ULPs might contain target port GUID in the PR > query, so in order for these queries not to be recognized by the QoS > manager > as SRP, the SRP match rule (or any match rule that refers to the target > port > guid only) should be placed at the end of the qos-ulps match rules. > > 6.6 MPI > SL for MPI is manually configured by MPI admin. OpenSM is not forcing any > SL > on the MPI traffic, and that's why it is the only ULP that did not appear > in > the qos-ulps section. > > > ============================================================================== > 7. SL2VL Mapping and VL Arbitration > ============================================================================== I think here should be stated that both policies should be merged at some point. [snip...] Sasha _______________________________________________ general mailing list [email protected] http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
