Roland Dreier wrote:
I believe the way forward is to evolve the existing drivers/infiniband
code already in Linux into a drivers/rdma that supports both IB and
RNICs. To be extremely blunt, I believe the RNIC-PI is irrelevant to
the Linux kernel -- no IB vendors will support ripping out a working
midlayer and starting from scratch, and it doesn't make sense to have
two essentially equivalent midlayers in the same kernel.
IMO, any APIs need to evolve out of the implementation. Trying to fit an implementation under an existing API tends to lead to either a poor implementation, or requires changes to the API anyway.
I think a useful path is for someone to implement an RNIC driver and provide feedback on what changes would be required of the Infiniband/core layer to support it.
There needs to be a balance between establishing a good, flexible architecture and how applications or subsystems interact with it. All of this needs to be grounded in implementation experience so there is a healthy does of reality. The problem with the iterative-TTM focused API definitions is they rarely produce something that is focused on the "end game" for a solid interface. They create instant legacy baggage that people are unwilling to shed as time goes forward. This legacy inertia is what has stifled quite a bit of innovation when it comes to software (some hazard that 90% of the software created today is focused primarily on legacy investment protection than really new innovation and when you think about it for a bit of time, you can see that much of this is re-inventing the wheel or to band-aid over a problem and packaging it as something innovative).
So, the problem becomes one of finding balance. The people proposing the various API are people who have implemented various types of solutions so they are coming with both real experience as well as engineering judgement of what is required to get the right infrastructure in place for the "end game" needs. From a practical perspective, one needs to implement these API and see what is really of value and what should be changed. The key is to avoid creating the legacy inertia that ends up reducing fragmenting the interfaces and causes lost productivity, quality problems, etc. So before people write off the various API as unnecessary or as poor quality, there should be some implementations developed and some constructive debate of what features are really of value and what might be deprecated or eliminated. There is no requirement to ever implement all of an API as this is where an iterative approach has value. Implement what is needed to demonstrate the desired value and decide then if it is worthwhile. Just please don't discount it or toss it all out because it isn't all implemented today or you don't like some aspect.
To put a really concrete proposal on the table, I would suggest to
start by extending the current ib_client registration structure
Roland's proposal sounds like a reasonable and fair way to begin. Building an abstraction on top of the existing layers seems secondary to adding support for other RDMA devices.
Getting the right verbs interface is paramount. Whether one does IT API or DAPL on top of that for a given subsystem is secondary.
Ideally, OpenIB and OpenRDMA should be focused on developing the RDMA infrastructure - RDMA verbs, connection management, IB-specific ULP, etc. They should not be focused on the upper subsystems or ULP. Those should be done as separate open source projects and they can decide where best to intersect with the Open* infrastructure. This one-size-fits-all approach for all subsystems to flow through a given interface is simply impractical to execute. It might happen over time but it cannot be force-fit. Let's focus the two efforts on getting as common of an infrastructure as possible. Some propose RNIC PI; other can propose something else if there is desire. RNIC PI has value in that it has focused on the common components between IB and iWARP and left the IB-specific and CM outside of its definition to allow an OS to decide how best to support or to allow IB to do what is needed for its particular usages.
Mike
_______________________________________________ openib-general mailing list [email protected] http://openib.org/mailman/listinfo/openib-general
To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
