Re: [PATCH 1/3] rdma_cm: Add support for a new RDMA_PS_LUSTRE Lustre port space
sebastien dugue wrote:
OK, then going with the TCP port space, what we need in OpenSM is a
combination of service id (TCP) _and_ TCP port _and_ target GUID.
I believe that you can have a 'lustre' keyword in opensm qos parser which
stands for the combination of tcp port space + lustre tcp port (maybe it exists
now), so in the policy file this would translate to X,{Z1,Z2,..,Zm} (as was in
your example) and not to X,Y,{Z1,Z2,..,Zm}.
Or.
--
To unsubscribe from this list: send the line unsubscribe linux-rdma in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 1/3] rdma_cm: Add support for a new RDMA_PS_LUSTRE Lustre port space
On Thu, 21 Jan 2010 11:12:08 +0200
Or Gerlitz ogerl...@voltaire.com wrote:
sebastien dugue wrote:
OK, then going with the TCP port space, what we need in OpenSM is a
combination of service id (TCP) _and_ TCP port _and_ target GUID.
I believe that you can have a 'lustre' keyword in opensm qos parser which
stands for the combination of tcp port space + lustre tcp port (maybe it
exists now), so in the policy file this would translate to X,{Z1,Z2,..,Zm}
(as was in your example) and not to X,Y,{Z1,Z2,..,Zm}.
Right, that'd be a nice shortcut. I also think being able to specify both port
and target GUID to the TCP port space might be interesting for other ULPs, not
just Lustre.
Sebastien.
Or.
--
To unsubscribe from this list: send the line unsubscribe linux-rdma in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line unsubscribe linux-rdma in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 1/3] rdma_cm: Add support for a new RDMA_PS_LUSTRE Lustre port space
sebastien dugue wrote: So I guess you need to change the ports used within the new port space -- but then why can't you just stay in the TCP space but change the ports used? No, with the new port space, there's no need to change ports. You only need to specify the target GUIDs. For example: lustre, target-portguid 0x1234,0x1235 : 1 # lustre traffic to MDSs lustre: 2 # default lustre traffic (to OSSs) Hope this helps clarify things a bit. sorry, but it doesn't, as far as I understand there are three possibilities for what the string lustre is being translated to by the opensm QoS logic: (A) lustre port in the TCP port space (B) lustre port space (C) nothing (that is not a service, in the same manner that ipoib just doesn't mean anything to opensm) Assuming C is not the case, then either A or B will yield the same result and as such the new port space buys you nothing. Or. -- To unsubscribe from this list: send the line unsubscribe linux-rdma in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 1/3] rdma_cm: Add support for a new RDMA_PS_LUSTRE Lustre port space
On Wed, 20 Jan 2010 10:03:47 +0200 Or Gerlitz ogerl...@voltaire.com wrote: sebastien dugue wrote: So I guess you need to change the ports used within the new port space -- but then why can't you just stay in the TCP space but change the ports used? No, with the new port space, there's no need to change ports. You only need to specify the target GUIDs. For example: lustre, target-portguid 0x1234,0x1235 : 1 # lustre traffic to MDSs lustre: 2 # default lustre traffic (to OSSs) Hope this helps clarify things a bit. sorry, but it doesn't, as far as I understand there are three possibilities for what the string lustre is being translated to by the opensm QoS logic: (A) lustre port in the TCP port space (B) lustre port space (C) nothing (that is not a service, in the same manner that ipoib just doesn't mean anything to opensm) I'm not sure I'm following you here. With the patches, the 'lustre' keyword is being translated to the lustre port space and only to that port space, never to the TCP port space. So if I'm understanding you correctly, it's B. Assuming C is not the case, then either A or B will yield the same result and as such the new port space buys you nothing. No, because in OpenSM's QoS logic, there's no way to map the TCP port space with specific target GUIDs onto an SL. You have keywords for SDP, SRP, RDS, ISER, ... but not for the TCP port space (or am I missing something?). What B buys you is the ability to map Lustre traffic to a specific target GUID onto an SL. An alternative would be to be able to map the TCP port space to a specific port and specific target GUIDs onto an SL in OpenSM's QoS configuration, but that's currently not possible. Sébastien. -- To unsubscribe from this list: send the line unsubscribe linux-rdma in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 1/3] rdma_cm: Add support for a new RDMA_PS_LUSTRE Lustre port space
sebastien dugue wrote: No, because in OpenSM's QoS logic, there's no way to map the TCP port space with specific target GUIDs onto an SL. You have keywords for SDP, SRP, RDS, ISER, ... but not for the TCP port space (or am I missing something?). going with this, what prevents you from patching opensm qos engine to support the lustre service under the tcp port-space and/or support a combination of service and target port-guid? all in all, first, I don't see what a kernel patch buys you and second, if it buys you something you should be able to gain the same effect with patching open-sm. thinking on this a bit more, since the rules are processed by order wouldn't the following scheme let you achieve the same effect? target-portguid 0x1234,0x1235 : 1 # traffic to MDSs lustre: 2 # default lustre traffic (to OSSs) Or. -- To unsubscribe from this list: send the line unsubscribe linux-rdma in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 1/3] rdma_cm: Add support for a new RDMA_PS_LUSTRE Lustre port space
Hi Roland, On Tue, 19 Jan 2010 17:12:43 -0800 Roland Dreier rdre...@cisco.com wrote: Well, without a specific port space, the default for Lustre is to use the TCP port space so you cannot distinguish Lustre traffic from other traffic using that same port space. I'm still a bit confused. The problem as I understand it is that Lustre always uses the same TCP port, so there's no way to apply different QoS to different types of Lustre traffic. Right, all Lustre servers (MDS and OSS) use the same TCP port, so there's no way to distinguish between the 2 types of traffic. But if we create a new port space and don't change the ports that Lustre uses, then there's still no way to apply different QoS for different Lustre traffic. With the OpenSM patch and the Lustre port space, you can map Lustre traffic to specific target port GUIDs onto a specific SL hence separating the flows. So I guess you need to change the ports used within the new port space -- but then why can't you just stay in the TCP space but change the ports used? No, with the new port space, there's no need to change ports. You only need to specify the target GUIDs. For example: qos-ulps default : 0 # default SL lustre, target-portguid 0x1234,0x1235 : 1 # lustre traffic to MDSs lustre : 2 # default lustre traffic (to OSSs) end-qos-ulps Hope this helps clarify things a bit. Thanks, Sébastien. -- To unsubscribe from this list: send the line unsubscribe linux-rdma in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 1/3] rdma_cm: Add support for a new RDMA_PS_LUSTRE Lustre port space
Hi Sean, On Wed, 13 Jan 2010 08:56:30 -0800 Sean Hefty sean.he...@intel.com wrote: This patch adds a new port space for use by Lustre traffic. This, along with patches to OpenSM and Lustre, allow to define a specific QoS for lustre. In general, I don't like the idea of application port spaces for QoS support. Can't this be done using port numbers in the existing port space? That can be done with port numbers, except that we cannot separate traffic to Lustre MDS and traffic to Lustre OSS as can be done with a target-port-guid. +RDMA_PS_LUSTRE = 0x0153, It looks like this number maps to the VINES protocol. Argh, haven't seen that. From what I understand, the low 8 bit are to be taken from IANA IP protocol ID when bit 8 is 1, but what about when bit 8 is 0 (as is the case with SDP and IPOIB)? Thanks, Sebastien. -- To unsubscribe from this list: send the line unsubscribe linux-rdma in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line unsubscribe linux-rdma in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 1/3] rdma_cm: Add support for a new RDMA_PS_LUSTRE Lustre port space
Hi Roland. On Wed, 13 Jan 2010 09:04:19 -0800 Roland Dreier rdre...@cisco.com wrote: In general, I don't like the idea of application port spaces for QoS support. Can't this be done using port numbers in the existing port space? I agree. If setting QoS requires a kernel patch for each application, I think we've messed up QoS somehow. There needs to be a way to control QoS via configuration, rather than rebuilding. Right, it's possible via configuration only, but you then cannot separate the different Lustre traffics (MDS and OSS for example). Sebastien. -- To unsubscribe from this list: send the line unsubscribe linux-rdma in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 1/3] rdma_cm: Add support for a new RDMA_PS_LUSTRE Lustre port space
sebastien dugue wrote: That can be done with port numbers, except that we cannot separate traffic to Lustre MDS and traffic to Lustre OSS Looking on these patches and going with you for a minute, I don't see how this patch set serves you to assign a different QoS level (e.g SL) to MDS vs OSS related traffic. Can you elaborate on that a bit? Sean Hefty wrote: Can't this be done using port numbers in the existing port space? Indeed, Sebastien what prevents you from using the TCP port space, with one port used for MDS traffic and another port for OSS traffic? how does Lustre get ports to listen on, are they well known or you call bind with port zero and use the port allocated by the rdma-cm? Or. -- To unsubscribe from this list: send the line unsubscribe linux-rdma in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 1/3] rdma_cm: Add support for a new RDMA_PS_LUSTRE Lustre port space
I agree. If setting QoS requires a kernel patch for each application, I think we've messed up QoS somehow. There needs to be a way to control QoS via configuration, rather than rebuilding. Right, it's possible via configuration only, but you then cannot separate the different Lustre traffics (MDS and OSS for example). I guess I don't know enough about Lustre to know why this would be so. How does creating a new port space for Lustre help with this? Why can't one do the same thing in one of the existing port spaces? - R. -- To unsubscribe from this list: send the line unsubscribe linux-rdma in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: [PATCH 1/3] rdma_cm: Add support for a new RDMA_PS_LUSTRE Lustre port space
This patch adds a new port space for use by Lustre traffic. This, along with patches to OpenSM and Lustre, allow to define a specific QoS for lustre. In general, I don't like the idea of application port spaces for QoS support. Can't this be done using port numbers in the existing port space? + RDMA_PS_LUSTRE = 0x0153, It looks like this number maps to the VINES protocol. -- To unsubscribe from this list: send the line unsubscribe linux-rdma in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 1/3] rdma_cm: Add support for a new RDMA_PS_LUSTRE Lustre port space
In general, I don't like the idea of application port spaces for QoS support. Can't this be done using port numbers in the existing port space? I agree. If setting QoS requires a kernel patch for each application, I think we've messed up QoS somehow. There needs to be a way to control QoS via configuration, rather than rebuilding. - R. -- To unsubscribe from this list: send the line unsubscribe linux-rdma in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
