Hal Rosenstock wrote:
On Wed, 2007-11-14 at 17:30 +0200, Yevgeny Kliteynik wrote:
Hal Rosenstock wrote:
On Wed, 2007-11-14 at 11:11 +0200, Yevgeny Kliteynik wrote:
Hal Rosenstock wrote:
Or,
On 11/13/07, Or Gerlitz <[EMAIL PROTECTED]> wrote:
Yevgeny,
iSER (as you can learn from doing a grep) is using the RDMA-CM TCP port
space as does RDS. The RDMA-CM signature is something which I am sure
exists, you can look on the RDMA-CM IB spec Annex to see if such thing
indeed exist or I am wrong.
Did you really look at the annex for this ?
The TCP port is the 16 bit port portion of
the
ip:port address provided by a ULP that uses the RDMA-CM to
rdma_resolve_addr(), again the annex explained how the port is embedded
into the SID, I don't remember the location within the 64 bit string.
It's in the low 16 bits (bytes 6-7) of the SID as the annex indicates.
Or.
-------- Original Message --------
Subject:
Re: QoS for iSER
Date:
Mon, 12 Nov 2007 11:41:43 +0200
From: Yevgeny Kliteynik <[EMAIL PROTECTED]>
Hi Erez,
Erez Zilber wrote:
to create the SID, the rdma cm combines
1) the port space
What is the port space for iSER?
For SDP it's 0x10000 - 0x1FFFF.
For RDS it's 0x1060000 - 0x106FFFF
I presume this is just saying RDS uses IP protocol TCP and there is no
well known port (e.g. uses dynamic ports). So how do you know ahead of
time which port ?
See below.
For iSER it's ...?
These numbers are too large for just "port space".
iSER SID is 0x000000000106035c
in your nomenclature, I guess 0x106035c
01 says RDMA aware ULP service ID range
06 says IP protocol is TCP
0x035c (port 860) is the well known TCP port for iSCSI
Thanks, that is just what I needed.
I'm preparing a (very) simplified interface for defining QoS policy.
I'm adding an additional section in QoS policy file, where an admin
will be able to configure QoS per ULP or per application w/o going
into too many details.
Here's the example of what I have in mind:
qos-ulps
default : 0 #default SL
sdp, port 10000-20000 : 2
sdp : 0 #default SL for SDP
rds, port 25000 : 2 #SL for RDS when destination port is
25000
Isn't there a chicken and egg problem here with this ? How do you know
port 25000 will be assigned "in advance" ?
See below.
rds, : 0 #default SL for RDS
I don't see how RDS can work separate from other CMA based protocols
which use dynamic ports.
You're right, it can't.
Moreover, as you know, OpenSM is not aware of the term "protocol"
at all - it just sees Service IDs in PathRecord request, so the
only differentiation it can create is by assigning certain SL to
a certain service ID.
Yes, I could see that you are trying to use protocol as an identifier to
simplify configuration for a ServiceID range.
What I'm trying to do here is to provide a simple way to configure
QoS that will work in *many* cases, not in *all* cases.
I'm giving the following options to differentiate traffic:
- All the SDP traffic
- Selected ports of the SDP traffic
- SRP traffic by providing SRT target guids
- iSER traffic (to a well known TCP port)
- All the RDS traffic
- Selected ports of the RDS traffic
- IPoIB by pkey
- Specific Service IDs
- Specific pkeys
- Specific destination port guids
Understood.
And as you've mentioned, some rules may overlap. For instance,
if the rule for all the RDS traffic will appear before the iSER
rule, then iSER requests will be caught by the RDS rule.
That doesn't sound so good but I don't see a good alternative here other
than for this case to put the iSER rule first.
Right
The other fallback is the
more detailed configuration but RDS falls into the generic range
category which is problematic in terms of this (and can't be
differentiated by ServiceID unlike the other ULPs).
Right again.
If in addition to RDS there are other ULPs in the fabric that fall
in the same range of service ID, then there's no escape from the
rest of the policy file.
For instance, RDS traffic might be differentiated by TCP port range
(service ID range) and target port guids.
But of course, you might find real life examples where nothing helps -
RDS and some other ULP traffic are using same TCP port range, using
the same port guids as source, and using the same guids as target.
In this case I give up - they would end up using the same SL.
-- Yevgeny
-- Hal
There's alway an option to use the rest of the policy file, and
then you can create more complex rules to differentiate applications
and protocols (i.e. by using service ID range AND/OR source port
groups AND/OR destination port groups AND/OR by QoS class.
-- Yevgeny
iser *??????* : 4 #SL for iSER
ipoib, pkey 0x0001 : 5 #SL for IPoIB on partition with pkey
0x0001
ipoib : 6 #default IPoIB partition - pkey=0x7FFF
...
end-qos-ulps
This syntax is possible only if there are well known facts
such as SDP service ID, in which case admin will be able to
just state "sdp: <sl>", and OpenSM will (internally) generate
relevant matching rule and QoS level based on this known
service ID.
So back to iSER:
Can I assume that the target port for iSER will always be 860,
hence the iSER service ID will always be 0x000000000106035c?
In terms of iSER, I was only commenting on what the spec says. I did not
verify its operation in terms of the code. Does the code follow the
spec ?
-- Hal
Or perhaps I can do it similar to SDP, where there is an option
to specify the port ranges along with the ULP name (SDP):
- if administrator only specifies "iser", I can assume that
the service ID is default 0x000000000106035c
- if administrator only specifies "iser" and ports, OpenSM
will build service ID based on a well known prefix
(0x000000000106pppp) where the last 4 hex digits are target
port number
Keep in mind that if this doesn't look too flexible and doesn't
cover all the cases, there's always the rest of the QoS policy
file with all the advanced configuration.
-- Yevgeny
-- Hal
2) the rdma cm signature
Do you mean something iSER-specific, or just the way the cm
builds the service ID out of port space and tcp port?
Can you give an example?
3) the destination tcp port provided to rdma_resolve_addr
I guess that tcp port is in the lower 4 nibs of the service ID,
similar to SDP. Right?
-- Yevgeny
_______________________________________________
general mailing list
[email protected]
http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general
To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
_______________________________________________
general mailing list
[email protected]
http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general
To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
_______________________________________________
general mailing list
[email protected]
http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general
To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
