shaharf wrote:
It seems to me that the major design approach is to do everything in the kernel but let user mode staff access to the lower levels to enable performance sensitive applications override all kernel layers. Am I right?
The focus is only on kernel components at the moment, plus whatever user-mode support is needed to configure the fabric.
It seems also that within the kernel, the ib interface/verbs (ib_*) is very close to the mthca verbs that are very close to vapi.
The agreement among the IB vendors was the start with VAPI as the base for the development of a new API. But VAPI is little more than verbs as defined by the IB spec.
Infiniband hardware exposes PDs, CQs, QPs, etc, so I think it's natural for these constructs to appear in the software. Abstractions away from IB specific constructs do exist in the form of SDP, ipoib, and DAPL.
It seems to me that the current interfaces evolved to what they are today mainly because of the way IB itself evolved - with a lot of uncertainly and a lot of design holes (not to say "craters"). This forced most of the industry to stick with very straight forward interfaces that were based on Mellanox VAPI.
The verbs interface evolved because IB hardware is expected to expose this sort of functionality. If you examine the layering of the software, ib_mthca and ib_core work together, with ib_core simply de-multiplexing among multiple devices.
I wonder if this is not the right time to come up with much better abstraction - for user mode and for kernel mode.
I believe that we have the correct software layering. At the lowest level you need software that talks directly with the hardware, with support for different hardware devices. Abstractions, such as SDP and DAPL should be above that. It seems that you're just wanting to move the abstraction down lower in the stack.
Do we really want to expose IB related structures such as CQs, QPs, and WQE? Why?
*blinks*
etc.) requirements. I do not know if this requirement will actually realize, but if is will, the SM and maybe also the SMI/GSI agents and the CM will have to significantly change. If this is likely to
Coding requirements that may or may not happen or potential changes in the future is likely to produce nothing usable.
Why should we develop complicated functionality such as RMPP in the kernel when the only few kernel based queries (if any at all) will use them?
I'm not opposed to moving functionality from the kernel to user-space, if it makes sense to do so. Note that TCP is in the kernel, and RMPP is somewhat similar.
very complicated interfaces and therefore much less stable and much more exposed to bugs, they will use 10GE.
Regardless of exposed interfaces, there will still be a need to implement IB management, meaning that the complexity will still be there.
doubt. Yes, it is true that this project is meant to supply HPC code base, but eventually, IB will not survive as HPC interconnect only.
This is a debatable point. I think that IB can survive as an only an HPC interconnect, and that it may have to. Ethernet may never be as good as IB, but that doesn't mean that it won't someday be good enough, especially if it comes for "free" on the motherboard.
_______________________________________________ openib-general mailing list [EMAIL PROTECTED] http://openib.org/mailman/listinfo/openib-general
To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
