> Solaris networking needs special VM support in the following three areas. > > 1) Send > 2) Receive > 3) Jumbo Frames
4) Resource management. Currently any user can consume huge amount of kernel memory in dblks that are accumulated on the stream head and not consumed by the reader. Since the new VM can attribute pages more precisely, it should be possible to attribute memory to a specific user and tie this to resource controls mechanisms which should fail allocations when per-user quotas are exceeded. 5) NUMA optimizations NICs drivers may be smart enough to know what CPU they are interrupting or what CPU is closest to the device memory. They NIC drivers and other network code users should be able to request memory (e.g. dblks) from a specific node. > The current VM provides decent support to achieve zero copy on both send > and receive. The reason networking doesn't use zero copy on send is due > to lack of appropriate interfaces and hopefully this will change with > extended socket interface. Lack of use of zero copy on receive in > networking is due to the fact that it is difficult to implement and > requires HW support which has not been present in the NIC's. Still, it is possible to have 1-copy mechanisms on receive side with software packet classification. > The new VM will maintain current zero copy support (for example segkpm). > However, as is the case today, it is important to note that the new VM > will not be provide support for immutable buffers with asynchronous > semantics. This means networking interfaces for zero copy will have to > provide protection against buffer contents changing before the buffer is > consumed. One of the major issues with 0-copy was the issue of "hold times". Once the buffer is passed up the stack the networking code doesn't have much control on when it will be consumed. This creates a potential for pinning down too much memory. The solution may be tied to the resource management issue above - we can limit the amount of memory that a given user may pin down. > Zero copy on receive will be possible only if the HW supports it. For > example if the HW supports RDMA. A new requirement being imposed by the > new VM is that the driver should support recall capability for the > memory it owns. What does it mean? How can a driver recall some memory that is sent to the application? > Regards, > Rao I think that this is a good time to consider the networking requirements for the VM redesign. One line of thought is - if we are not limited to socket API and are willing to introduce new and better APIs that are more data-copy friendly, what support is needed from VM? For example, if someone decides to implement fbufs, what support is missing? Is it possible to have shared memory between the NIC driver and the application? - Alexander Kolbasov __ http://blogs.sun.com/akolb _______________________________________________ networking-discuss mailing list [email protected]
