Yevgeny, Lets say I want to run another ULP, say NFS-RDMA. Does this mean by default I will get the SL which is in use by currently running ULP, say SDP? Or the SL is differentiated based on port? And if I want to run different SL level traffic between the same pair of nodes I will need to use different port ID? Look strange for ULPs that have well known port ID...
Thanks, Arkady Kanevsky email: [EMAIL PROTECTED] Network Appliance Inc. phone: 781-768-5395 1601 Trapelo Rd. - Suite 16. Fax: 781-895-1195 Waltham, MA 02451 central phone: 781-768-5300 > -----Original Message----- > From: Yevgeny Kliteynik [mailto:[EMAIL PROTECTED] > Sent: Wednesday, November 14, 2007 3:21 PM > To: Kanevsky, Arkady > Cc: Hal Rosenstock; [email protected] > Subject: Re: [ofa-general] RE: QoS for iSER > > Kanevsky, Arkady wrote: > > what happens when multiple apps runs on the same server? > > I guess that when you say "server" you mean "host" and not > the server from server-client terminology. > > This is what the whole point of QoS is: if the applications > are using the same ULP, they probably would get the same > Service Level, unless they were differentiated by the > administrator is some other way, e.g. they all use SDP, but > connect to different TCP port of the server application. > If the applications are using different ULPs, they will get > Service Level accordingly to the ULPs that they are using. > > -- Yevgeny > > > Arkady Kanevsky email: [EMAIL PROTECTED] > > Network Appliance Inc. phone: 781-768-5395 > > 1601 Trapelo Rd. - Suite 16. Fax: 781-895-1195 > > Waltham, MA 02451 central phone: 781-768-5300 > > > > > >> -----Original Message----- > >> From: Hal Rosenstock [mailto:[EMAIL PROTECTED] > >> Sent: Wednesday, November 14, 2007 8:18 AM > >> To: Yevgeny Kliteynik > >> Cc: [EMAIL PROTECTED]; [email protected] > >> Subject: Re: [ofa-general] RE: QoS for iSER > >> > >> On Wed, 2007-11-14 at 11:11 +0200, Yevgeny Kliteynik wrote: > >>> Hal Rosenstock wrote: > >>>> Or, > >>>> > >>>> On 11/13/07, Or Gerlitz <[EMAIL PROTECTED]> wrote: > >>>>> Yevgeny, > >>>>> > >>>>> iSER (as you can learn from doing a grep) is using the > >> RDMA-CM TCP > >>>>> port space as does RDS. The RDMA-CM signature is > >> something which I > >>>>> am sure exists, you can look on the RDMA-CM IB spec > >> Annex to see if > >>>>> such thing indeed exist or I am wrong. > >>>> Did you really look at the annex for this ? > >>>> > >>>>> The TCP port is the 16 bit port portion of the ip:port address > >>>>> provided by a ULP that uses the RDMA-CM to rdma_resolve_addr(), > >>>>> again the annex explained how the port is embedded into > >> the SID, I > >>>>> don't remember the location within the 64 bit string. > >>>> It's in the low 16 bits (bytes 6-7) of the SID as the > >> annex indicates. > >>>>> Or. > >>>>> > >>>>> -------- Original Message -------- > >>>>> Subject: > >>>>> Re: QoS for iSER > >>>>> Date: > >>>>> Mon, 12 Nov 2007 11:41:43 +0200 > >>>>> From: Yevgeny Kliteynik <[EMAIL PROTECTED]> > >>>>> > >>>>> Hi Erez, > >>>>> > >>>>> Erez Zilber wrote: > >>>>>> to create the SID, the rdma cm combines > >>>>>> > >>>>>> 1) the port space > >>>>> What is the port space for iSER? > >>>>> For SDP it's 0x10000 - 0x1FFFF. > >>>>> For RDS it's 0x1060000 - 0x106FFFF > >> I presume this is just saying RDS uses IP protocol TCP and > there is > >> no well known port (e.g. uses dynamic ports). So how do you know > >> ahead of time which port ? > >> > >>>>> For iSER it's ...? > >>>> These numbers are too large for just "port space". > >>>> > >>>> iSER SID is 0x000000000106035c > >>>> > >>>> in your nomenclature, I guess 0x106035c > >>>> > >>>> 01 says RDMA aware ULP service ID range > >>>> 06 says IP protocol is TCP > >>>> 0x035c (port 860) is the well known TCP port for iSCSI > >>> Thanks, that is just what I needed. > >>> I'm preparing a (very) simplified interface for defining > QoS policy. > >>> I'm adding an additional section in QoS policy file, > where an admin > >>> will be able to configure QoS per ULP or per application > w/o going > >>> into too many details. > >>> Here's the example of what I have in mind: > >>> > >>> qos-ulps > >>> default : 0 #default SL > >>> sdp, port 10000-20000 : 2 > >>> sdp : 0 #default SL for SDP > >>> rds, port 25000 : 2 #SL for RDS when > >> destination port is 25000 > >> > >> Isn't there a chicken and egg problem here with this ? How do you > >> know port 25000 will be assigned "in advance" ? > >> > >>> rds, : 0 #default SL for RDS > >> I don't see how RDS can work separate from other CMA based > protocols > >> which use dynamic ports. > >> > >>> iser *??????* : 4 #SL for iSER > >>> ipoib, pkey 0x0001 : 5 #SL for IPoIB on > >> partition with pkey 0x0001 > >>> ipoib : 6 #default IPoIB > >> partition - pkey=0x7FFF > >> ... > >>> end-qos-ulps > >>> > >>> This syntax is possible only if there are well known facts > >> such as SDP > >>> service ID, in which case admin will be able to just state "sdp: > >>> <sl>", and OpenSM will (internally) generate relevant > matching rule > >>> and QoS level based on this known service ID. > >>> > >>> So back to iSER: > >>> > >>> Can I assume that the target port for iSER will always be > >> 860, hence > >>> the iSER service ID will always be 0x000000000106035c? > >> In terms of iSER, I was only commenting on what the spec > says. I did > >> not verify its operation in terms of the code. > >> Does the code follow the spec ? > >> > >> -- Hal > >> > >>> Or perhaps I can do it similar to SDP, where there is an > option to > >>> specify the port ranges along with the ULP name (SDP): > >>> - if administrator only specifies "iser", I can assume that > >>> the service ID is default 0x000000000106035c > >>> - if administrator only specifies "iser" and ports, OpenSM > >>> will build service ID based on a well known prefix > >>> (0x000000000106pppp) where the last 4 hex digits are target > >>> port number > >>> > >>> Keep in mind that if this doesn't look too flexible and > >> doesn't cover > >>> all the cases, there's always the rest of the QoS policy > >> file with all > >>> the advanced configuration. > >>> > >>> -- Yevgeny > >>> > >>>> -- Hal > >>>> > >>>>>> 2) the rdma cm signature > >>>>> Do you mean something iSER-specific, or just the way the > >> cm builds > >>>>> the service ID out of port space and tcp port? > >>>>> Can you give an example? > >>>>> > >>>>>> 3) the destination tcp port provided to rdma_resolve_addr > >>>>> I guess that tcp port is in the lower 4 nibs of the service ID, > >>>>> similar to SDP. Right? > >>>>> -- Yevgeny > >>>>> > >>>>> _______________________________________________ > >>>>> general mailing list > >>>>> [email protected] > >>>>> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > >>>>> > >>>>> To unsubscribe, please visit > >>>>> http://openib.org/mailman/listinfo/openib-general > >>>>> > >>> _______________________________________________ > >>> general mailing list > >>> [email protected] > >>> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > >>> > >>> To unsubscribe, please visit > >>> http://openib.org/mailman/listinfo/openib-general > >> _______________________________________________ > >> general mailing list > >> [email protected] > >> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > >> > >> To unsubscribe, please visit > >> http://openib.org/mailman/listinfo/openib-general > >> > > > > _______________________________________________ > general mailing list > [email protected] > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit > http://openib.org/mailman/listinfo/openib-general > _______________________________________________ general mailing list [email protected] http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
