Andrew Gallatin wrote: > Darren Reed writes: > > Can any of the existing NICs receive a packet into more > > than one buffer simultaneously? > > Myri10GE uses firmware on the NIC, so it can do nearly anything in > theory. > > > To explain this, if I have a NIC with rx rings A and B, > > what I'd like it to do is deliver every packet into both > > rx rings. Is that possible? > > Yes, with firmware modifications, Myri10GE could do this. However, by > DMA'ing things twice, you'd double PCIe bus utilization and increase > the work done by the firmware, limiting performance.
Hmmm. Is it possible to have two descriptors point to the same buffer? Or rather to receive one packet, put it in a single buffer but have two (or more descriptor rings) have a reference to it? And flag the packet as "I am only receiving this because of promiscuous mode"? Besides this being useful for packet capture (where as Seb guessed right, the benefit is to better isolate the workload from the packets being captured), this can also be useful for exclusive stack instances where there are multiple vnics created atop a physical nic that has multiple descriptor rings. How does this latter point come into play? If I have 3 vnics, each belonging to individual zones and part of the same subnet, it stands to reason that each one of them wants to receive the same broadcast packets and possibly the same set of multicast packets. Thus if the NIC can receive one packet but put it on 3 descriptor rings, one for each of 3 zones, there should be a small benefit here in handling broadcast and multicast traffic for all concerned. > > If I can do that, could I program one to put all of the > > packets that I want to receive (broadcast/multicast/unicast) > > on ring A and B but also put any packets that aren't for me > > but that are received because of promiscuous mode also > > on ring B? > > Yes (again, driver and firmware modifications would be required). > This would make a lot more sense. > > Are you trying to get around the packet sniffing performance > limitations discussed earlier where the Solaris stack blindly copies > everything? No. The architecture of OpenSolaris's networking is being driven towards achieving performance through the use of squeues at the lower levels and having dedicated threads (and/or cpus) to handle those squeues. The squeues become the work queues for the task. If the sniffing application is running slow, I'd rather see that pushed down to an individual squeue or descriptor ring where packets can be dropped earlier rather than have them compete for space with normal packets. Most people today who need to pull out snoop or tcpdump or ethereal on OpenSolaris want to use it on a NIC that has IP plumbed on top of it and asking them to unplumb IP so they can use said tools in an efficient manner is out of the question (besides more than likely disrupting the purpose of using the tool in question.) > Speaking of memory copies and packet sniffing, how many memory copies > are typically done in Solaris? It depends on whether or not the NIC is also plumb'd with IPv4 or IPv6. If it is then a duplicate copy of the packet is made. So using snoop on a NIC to see what your NFS traffic is doing is going to have a non trivial impact on the system if it is a busy server. > Is it like BSD, where there is one > copy from the driver's receive buffer into a BPF (equivalent) buffer > where the filtering is done, and yet another copy into userspace is > done for matching packets? No. Today packets that are to be sniffed, lets assume by snoop, must go through a couple of STREAMS pipes. snoop uses both bufmod and pfmod on top of the NIC, so a packet to be sniffed is putnext'd a few times. If we take away the slow pfmod, we're still doing putnext's into bufmod where they're queued up until read (I might be wrong on this, I don't have time to check the code now.) If you're using tcpdump today (or any of the libpcap applications), the packets go up into user space via bufmod before the BPF filter is applied to them. Not exactly efficient. Packet capture/sniffing on OpenSolaris needs to be refreshed and given a whole new breath of life through a new architecture (because we can't just yank out the old one and throw it away. Sigh.) Darren _______________________________________________ networking-discuss mailing list [email protected]
