Sorry for the delay, I wanted to get the week started so I can return to a production level of traffic :-)
On average, I can see between 4 - 5 Gbit/sec and netsniff does not report any loss in the log now. There is around 3% loss in the NIC statistics, visible in both ifpps and ifconfig. > > Thanks for the report and details! > > Can you elaborate on how much you can actually capture? Between 4-5 Gbit/sec > > Have you tuned socket rmem/wmem so you have enough capacities? Not at all - how do I calculate the optimal ratio? Does not netsniff do it automatically? > > Have you tried a smaller ring size e.g. very few MB to reduce > pressure from MMU? Did you see an improvement for SG I/O on the > writeback? I will test smaller ring size in a day - so far a quick experiment with 4MB ring gave me packet loss. SG I/O is indeed better - no packet loss when writing to disk and lower CPU usage, I would say at least 15% of not better. > > There's also a tool called *ifpps* in the netsniff-ng repository, > might be helpful as well. Excellent tool, scales well ,does not lie (hello, iptraf ;-) and shows a lot of useful statistics. Thanks. > > How many instances of netsniff-ng are you running? Is this a > middle box or rather server? If the latter, you could let the > NIC do flow steering (e.g. aRFS) and have e.g. 2 or 4 instances > of netsniff-ng filtering in BPF by queue mapping. I have a single netsniff instance so far. This is a real server with 2x Intel(R) Xeon(R) CPU E5-2620 0 @ 2.00GHz 64GB RAM (each channel got 8GB DIMM) 12x 3TB SATA 7.2k rpm in RAID5 (hardware) The storage subsystem is far from being saturated, I could write easily 1.2GB/sec. Anything special I need to do to enable RFS? I've just started reading about it and understand the idea behind RPS, just need a hint how to use it with netsniff. > > Alternatively, if all else doesn't help, you could also sample > flows in BPF, so you get a portion of the traffic for analysis. > > >> Early results with 'perf' and performance counters indicate that >> packets make it to the card, get written into cache (I think DCA >> writes directly into L1, so on Nehalem and later that's L3, too) of >> core 1 and then core 2 picks that up from L3 and writes to disk. >> >> So far 0 packet loss. >> >> Comments, questions? I am doing everything wrong? >> >> >> >> On Fri, Mar 27, 2015 at 3:35 PM, Mike Reeves <[email protected]> wrote: >>> >>> One thing you can do is make sure you tune your disk writes. I have >>> noticed >>> that I get the best results when pdflush is almost constantly running. >>> There >>> is a ton of info out there about what you can tweak but take a look at >>> vm.dirty_ratio and vm.dirty_background_ratio. >>> >>> You have to play with these settings until you find what works best for >>> your >>> system. >>> >>> Thanks >>> >>> Mike >>> >>> >>> >>> On Mar 27, 2015, at 7:47 AM, Michał Purzyński >>> <[email protected]> >>> wrote: >>> >>> Hello. >>> >>> I went through a massive number of documents regarding the netsniff >>> itself, >>> including reading code of both the application, libpcap and kernel parts, >>> intel manuals for X520 and CPUs. Enough of reading, let's do something. >>> >>> How to tune netsniff-ng for the maximum performance? Question seems >>> trivial, >>> answer is not so. Feel free to correct me. X520 and a few gbit/sec and a >>> NUMA (Sandy Bridge) system is what I have here. >>> >>> For the best cache utilisation, netsniff-ng should be on the same core >>> that >>> card delivers packets to. With IO/AT and DCA we could have something like >>> this: >>> >>> -- packets --> card --> cpu cache --> netsniff-ng >>> >>> It makes sense with DCA, actually. >>> >>> Now, does it mean we want RSS disabled and a single queue pinned to a >>> single >>> core? >>> >>> Won't the number of interrupts (even using interrupt mitigation that's in >>> the ixgbe driver) kill the single core? >>> >>> Alternatively (I have only 2.0Ghz CPU and that's not enough to keep up >>> with >>> the load from interrupts and netsniff-ng together on the same core) - >>> interrupts on core1, netsniff-ng on core2 (we're talking physical cores >>> from >>> the same socket). >>> This will generate L1 and L2 miss most of the time but we still have L3 >>> that's inclusive (and huge). Win? >>> >>> With the kernel parameters "nohz_full=1-23 isolcpus=1-23" scheduler will >>> keep other tasks off the cores (1-23) by setting affinity on the init >>> process and disable ticks on them if there is a single task running >>> (other >>> kernel threads can be set to run on core 0). >>> >>> What gives the best write performance - sg or mmap? >>> >>> -- >>> You received this message because you are subscribed to the Google Groups >>> "netsniff-ng" group. >>> To unsubscribe from this group and stop receiving emails from it, send an >>> email to [email protected]. >>> For more options, visit https://groups.google.com/d/optout. >>> >>> >>> -- >>> You received this message because you are subscribed to the Google Groups >>> "netsniff-ng" group. >>> To unsubscribe from this group and stop receiving emails from it, send an >>> email to [email protected]. >>> For more options, visit https://groups.google.com/d/optout. >> >> > -- You received this message because you are subscribed to the Google Groups "netsniff-ng" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
