Dear networking experts, We are working on adding support for RDMA over IP. A summary description of the project is given in the latter part of this message. Given below are the initial set of interfaces we need from the networking stack. We are bringing up our needs on this discussion board with the hope of getting some advice on how to proceed.
(Some of our needs are satisfied by the netinfo interfaces described in
PSARC 2005/334. They are not shown below.)
(1) Need a way to know all local ip addresses (along with the order of
preference for reaching the destination) through which a given
destination ip address can be reached.
(2) Given a destination ip address, need a way to know the ip address
and MAC address of the next hop
or two separate interfaces
2a) An interface to determine the next hop IP address for reaching a
given destination ip address.
2b) An interface to resolve an IP address into a MAC address;
(3) Allocate a port number
Given an ip address, protocol type (currently TCP), and port
number allocate a port number. If the input port number is zero
allocate an unused port number for the specified ip address (or
for all ip addresses on local RNICs if the ip address is zero).
If input port number is non-zero, allocate the specified port
number for the specified ip address (or for all ip addresses on
local RNICs if the ip address is zero).
(4) If the MTU setting changes, need to know that new MTU setting when
the change occurs.
(We also need to know the current MTU setting. We are able to get this
from net_getmtu(9F) interface described in PSARC 2005/334)
(5) Need a way to tell the networking stack that the NIC is a special NIC
(i.e., RDMA capable NIC).
James Carlson suggested to use mac_callbacks_t->mc_getcapab
interface and define a new flag ( ex MAC_CAPAP_RNIC)
The above is the initial list of interfaces we need. We may need
some more interfaces from the networking stack as we dive deeper into the
project.
Background: RDMA offload project (ROF)
--------------------------------------
ROF project will add support for RDMA over IP. The attached file describes
the high level RDMA stack.
The project will provide two sets of interfaces
1) ROF Transport interfaces (ROF-TI)
for ULP's (Upper Level Protocols) such as SDP (Socket Direct protocol),
RDS (Reliable Datagram Sockets), iSER (ISCSI Extensions for RDMA),
uDAPL (user Direct Access Programming language, used by MPI
(Message Passing Interface)) etc.
2) ROF RNIC Interfaces
for RNIC's (RDMA enabled Network Interface Card) hardware adapter drivers
and for software RNIC drivers.
Hardware RNIC drivers are regular Nemo drivers with RDMA support.
So, they register with Nemo framework and also with ROF framework. All the
regular traffic will be handled using regular network stack and all
the RDMA traffic will be handled by the ROF stack bypassing the regular
network stack.
Software RNIC is just a regular kernel module which implements RDMA
protocols in software and uses regular NIC for sending/receiving RDMA data.
Thanks,
Ramaswamy.
rdma-stack.pdf
Description: Adobe PDF document
_______________________________________________ networking-discuss mailing list [email protected]
