On Sun, Oct 25, 2009 at 01:32:27PM +0200, Or Gerlitz wrote:
> Jason Gunthorpe wrote:
> >So why not have a more general, flexible approach? Isolating ACM from
> >librdmacm by using AF_IB is a good idea, it keeps them seperate and lets
> >ACM and future go where ever. I hope Sean can make it work w
On Sun, Oct 25, 2009 at 01:25:21PM +0200, Or Gerlitz wrote:
> well, you didn't address some of my comments (not the ice-cream
> ones...), which come to say that this wouldn't be inter-operable if for
> one side you convert INET/TCP to IB/IB and for the other side you don't
> (e.g userA/userB us
Jason Gunthorpe wrote:
So why not have a more general, flexible approach? Isolating ACM from librdmacm
by using AF_IB is a good idea, it keeps them seperate and lets ACM and future
go where ever. I hope Sean can make it work with the rdma_getddrinfo idea, that
would completely seperate ACM and
Sean Hefty wrote:
These are the things done today in the kernel wrt IB:
* Map a local or remote IP address to a GID
* If a local address is not given, provide a usable address based on the
destination address
* Acquire a path between the source and destination
* Format the first 36 bytes of priv
>I really tried to follow the thread between you and Jason with quite
>little success, and I am going to give it more tries... in parallel,
>could you help me understand what is the --drive/reasoning-- from your
>perspective to add AF_IB / PS_IB here?
These are the things done today in the kernel
On Thu, Oct 22, 2009 at 05:41:53PM +0200, Or Gerlitz wrote:
> If from other reasons, people want the rdma-cm to support AF_IB and/or
> PS_IB, we can do that as well, but why force doing that behind the cover
> each time ACM is used?!
My view is that ultimately ACM has at best a niche applicatio
Sean Hefty wrote:
okay, I just wanted to make sure that the whole thing (ACM + modified librdmacm
+ modifed rdma-cm) is applicable AND inter-operable for AF_INET / PS_TCP
applications
I do not intend to have any changes that break anything
my question went beyond whether things are going to be
On Tue, Oct 20, 2009 at 01:48:34PM -0700, Sean Hefty wrote:
> >Private data:
> >- AF_IB/PS_TCP - the kernel munges the private data to be compatible
> > with AF_INET/PS_TCP, but otherwise is the same.
> >- AF_IB/PS_IB - the kernel doesn't touch the private data.
>
> I was thinking AF_IB/* - kerne
>Private data:
>- AF_IB/PS_TCP - the kernel munges the private data to be compatible
> with AF_INET/PS_TCP, but otherwise is the same.
>- AF_IB/PS_IB - the kernel doesn't touch the private data.
I was thinking AF_IB/* - kernel doesn't touch the private data, as it lacks the
necessary information.
On Tue, Oct 20, 2009 at 01:05:15PM -0700, Sean Hefty wrote:
> >> I agree. But we can still support AF_IB/PS_TCP by simply assigning
> >> the service ID correctly. rdma_bind_addr only needs to fill in a
> >> service id if one is not given. This should enable
> >
> >'one is not given' == 0 service
>> I agree. But we can still support AF_IB/PS_TCP by simply assigning
>> the service ID correctly. rdma_bind_addr only needs to fill in a
>> service id if one is not given. This should enable
>
>'one is not given' == 0 service ID?
yes
I'm guessing PS_IB would need to assign from the IB CM assi
On Tue, Oct 20, 2009 at 12:13:03PM -0700, Sean Hefty wrote:
> >> rdma_bind_addr fills in the service id according RDMA IP CM service
> >> annex.
> >
> >Hm? If the sockaddr_ib contains the service ID why override it in the
> >kernel?
> >
> >The AF_IB/PS_IB flow in the kernel side must just use strai
>> rdma_bind_addr fills in the service id according RDMA IP CM service
>> annex.
>
>Hm? If the sockaddr_ib contains the service ID why override it in the
>kernel?
>
>The AF_IB/PS_IB flow in the kernel side must just use straight service
>IDs.
I agree. But we can still support AF_IB/PS_TCP by simp
On Tue, Oct 20, 2009 at 11:08:58AM -0700, Sean Hefty wrote:
> >Looking on kernel cma.c format_hdr code it first branches on the address
> >family and next of the port space.
> >
> >Going with your proposed flow, I understand that an app call to
> >rdma_resolve_addr will be broken down to rdma_bind_
>okay, I just wanted to make sure that the whole thing (ACM + modified
>librdmacm + modifed rdma-cm) is applicable AND inter-operable for
>AF_INET / PS_TCP applications.
I do not intend to have any changes that break anything.
>Looking on kernel cma.c format_hdr code it first branches on the addr
Sean Hefty wrote:
From the perspective of IB, the RDMA CM simply defines a specific format to
private data and service ID carried in the IB CM REQ. As long as any use
adheres to that protocol, interoperability won't be an issue.
okay, I just wanted to make sure that the whole thing (ACM + modi
On Fri, Oct 09, 2009 at 01:33:56AM +0200, Or Gerlitz wrote:
> However, I don't manage to follow on your port space discussion with
> Jason. Some apps may have client in user space and server in the
> kernel or vise versa. I wouldn't tie PS_IB or a like with ACM. The ACM
> ARP replacement protocol
>However, I don't manage to follow on your port space discussion with
>Jason. Some apps may have client in user space and server in the
>kernel or vise versa. I wouldn't tie PS_IB or a like with ACM. The ACM
>ARP replacement protocol will reply only if the ip address specified
>in the broadcast req
Jason Gunthorpe wrote:
> If the listening side continues to use the IP mode to listen then I guess the
> client can compute an appropriate service ID, but it seems a bit
> strange for one side to use IP and the other side to use the ACM
> method? I was imagining you'd configure both sides to use
Sean Hefty wrote:
>>When used over IB, the IP address is little more than a qualifier contained
>>within the IB CM REQ private data.
>
> If we added support for AF_GID/AF_IB to the kernel, the rdma_cm could leave
> all
> of the private data carried in the IB CM REQ entirely up to the user. If th
On Thu, Oct 8, 2009 at 1:42 AM, Sean Hefty wrote:
> My intent, which differs from Jason's, was to fully support the existing
> librdmacm interfaces as they are defined.
yes, I agree this is the way to go
> Implementation wise, if the user of the librdmacm calls rdma_resolve_addr
> with a
> src
On Wed, Oct 07, 2009 at 11:46:24PM -0700, Sean Hefty wrote:
> >Yep, not sure how you handle the listening side without port
> >conflicts?? But that doesn't seem to be a huge problem. TBH - since
> >ACM is kinda its own little world, it could just use a seperate
> >service ID space from RDMA CM?
>
>Yep, not sure how you handle the listening side without port
>conflicts?? But that doesn't seem to be a huge problem. TBH - since
>ACM is kinda its own little world, it could just use a seperate
>service ID space from RDMA CM?
I'm not sure how to handle the port space yet. The port space is spec
On Wed, Oct 07, 2009 at 05:38:27PM -0700, Sean Hefty wrote:
> >When used over IB, the IP address is little more than a qualifier contained
> >within the IB CM REQ private data.
>
> If we added support for AF_GID/AF_IB to the kernel, the rdma_cm
> could leave all of the private data carried in the
>When used over IB, the IP address is little more than a qualifier contained
>within the IB CM REQ private data.
If we added support for AF_GID/AF_IB to the kernel, the rdma_cm could leave all
of the private data carried in the IB CM REQ entirely up to the user. If the
user happens to format that
>I understand that rdma_bind_address covers the local device and vlan
>resolutions, but I we should also --keep-- supporting also
>applications that use an explicit source address in rdma_resolve_addr
>or that don't do bind, provide src=NULL to resolve_addr and rely on
>the rdma-cm to use route loo
Sean Hefty wrote:
> From user space, the call sequence does not change. The user calls
> rdma_resolve_addr, rdma_resolve_route, rdma_connect, etc. It is up to the
> librdmacm to perform the resolution. Today, the resolution request is simply
> passed down to the kernel, which restricts how the
On Wed, Oct 07, 2009 at 02:20:29PM -0700, Hefty, Sean wrote:
> >You are trying to make the smallest change possible to work around a
> >performance problem caused by ineffecient abstractions by completely
> >breaking the abstraction.
>
> The performance problem being addressed here is not caused b
>You are trying to make the smallest change possible to work around a
>performance problem caused by ineffecient abstractions by completely
>breaking the abstraction.
The performance problem being addressed here is not caused by an inefficient
abstraction. It's caused by having a single, central
On Wed, Oct 07, 2009 at 12:16:56PM -0700, Sean Hefty wrote:
> >So, I'm just talking about the user space API, the others can be
> >changed as necessary to align with it.
> >
> >This is open source, so choosing a technically better solution over a
> >endlessly backwards compatible solution is done a
>So, I'm just talking about the user space API, the others can be
>changed as necessary to align with it.
>
>This is open source, so choosing a technically better solution over a
>endlessly backwards compatible solution is done all the time and is
>normal, expected, etc. Cost of progress - that is
On Tue, Oct 06, 2009 at 06:20:05PM -0700, Sean Hefty wrote:
> There are 3 interfaces of interest here. The librdmacm API, the rdma_ucm user
> to kernel interface, and the rdma_cm interface. These patches are looking to
> change the rdma_ucm interface. I want to avoid changing the API or behavio
>I'm not suggesting that you implement RMDA CM IP semantics in
>userspace using the IB CM, I'm suggesting you expose the IB CM GID
>semantics through the RDMA CM API exactly as they are. Your IBACM
>would then become an enhanced path resolution module to the RDMA CM,
>much like getaddrinfo is to so
On Tue, Oct 06, 2009 at 03:53:21PM -0700, Sean Hefty wrote:
> >Actually, thinking about it some more, that would be very helpful. As
> >I said before, I have worked on apps using IB CM. The only reason is
> >to have complete control over the addressing. If I could use RDMA CM
> >API in some kind of
>Actually, thinking about it some more, that would be very helpful. As
>I said before, I have worked on apps using IB CM. The only reason is
>to have complete control over the addressing. If I could use RDMA CM
>API in some kind of AF_GID addressing and service ID space, it would
>basically elimina
On Tue, Oct 06, 2009 at 12:05:02PM -0700, Sean Hefty wrote:
> From user space, the call sequence does not change. The user calls
> rdma_resolve_addr, rdma_resolve_route, rdma_connect, etc. It is up to the
> librdmacm to perform the resolution. Today, the resolution request is simply
> passed do
>If I understand correct your suggested changes are to optionally let an
>application to - instead of the following sequence of calls
>
>rdma_resolve_addr / addr resolved event
>rdma_create_qp
>rdma_resolve_route / route resolved event
>rdma_connect / cm events
>
>do
>
>rdma_set_ib_path
>rdma_cre
Sean Hefty wrote:
Provide an option for user's to manually specify the socket address to DGID
mapping on InfiniBand. Currently, all mappings are done using ipoib, and
involve ARP. This will not work across IP subnets, and alternative mechanisms
of resolving the mapping are being explored. Th
Provide an option for user's to manually specify the socket address
to DGID mapping on InfiniBand. Currently, all mappings are done
using ipoib, and involve ARP. This will not work across IP subnets,
and alternative mechanisms of resolving the mapping are being explored.
The latter can be more ef
39 matches
Mail list logo
