On Wed, Nov 12, 2014 at 11:16 AM, Slawa Olhovchenkov <s...@zxy.spb.ru> wrote:
> On Tue, Nov 11, 2014 at 10:13:54PM +0100, Franco Fichtner wrote: > > > Hi Luigi, > > hi all, > > > > so I was running into logistics issues with netmap(4) > > with regard to zero-copy and redirection through pipes: > > working on a load-balancing framework revealed that it > > is very hard to track a packet's origins to later move > > it onward to the respective outgoing interface, be it > > another device or the host stack. > > > > Long story short: user data needs to be stored for the > > packet buffer or slot. > > I think need configurable (by sysctl) space recerved before packet. > This is may be used as user data. Or for insert VLAN/MPLS/QinQ/etc > headers. > > this is yet another requirement: not just metadata but also encapsulation. For the records, the VALE switch does have TSO support (implemented through the VHOST header) so that VMs can pass large segments across a switch and they are properly split when traffic goes to a physical interface or a port that does not support the header. We also support scatter-gather I/O at least on the switch (haven't implemented this feature yet on NICs). But please consider that following this route we end up more or less into the same complications that afflict the standard stack: everything is configurable and decided at runtime, and the code becomes a maze of conditionals or indirect function calls with little chance of optimisations. Also, it's not that one sysctl works for all cases. Different ports typically have different encapsulation sizes, NICs may have alignment constraints (even those who don't suffer if buffers not 64-byte aligned), so you'll end up with scatter-gather I/O or copying anyways. After two years of experience with netmap i am not so sure anymore that zero copy makes much sense, except perhaps for the case of large packets (but i am not so sure about that, either). Apart from benchmarks, if you want to do something useful with the packets you need to read the header, at which point the concerns on having data in cache or not are less significant and the cost of the copy is heavily reduced. Tracking ownership of buffers (which is needed for zero copy) is also expensive even when they are not shared (and we have great trouble in managing the "extra buffers" we recently added to the netmap API to support zero-copy, to the point that I am tempted to remove the feature. cheers luigi _______________________________________________ email@example.com mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"