Thanks for your replies Vadim ☺
I’ll take advantage of this thread to present you guys with another nightly result I got from Netsniff-NG… I was pushing its’ Recording limits again, on same performance tuned setup, injecting ~5Gbps of FTP traffic for 15+ minutes, back-to-back to VM running Netsniff-NG as follows… netsniff-ng -i ens224 -o TestPacketLoss.pcap -s -J And got the following stats… 310366944 packets incoming (433339 unread on exit) 310367028 packets passed filter 433255 packets failed filter (out of space) 0.1394% packet droprate 984 sec, 690039 usec in total (needless to say, started sniffing before traffic injection started & stopped after ended) 1) What can you suggest to get to 0.000% Packet Loss ? 2) Would bigger Ring Size and/or shorter Kernel Pull Interval expected to help ? 3) I’m using version 0.5.8-rc2 – is that latest & greatest ? Thanks, Asaf. 64bit RHEL 7.2 with 16 x vCores, 64GB RAM, VMXNET3 adapter & 1TB on 10K SAS FC NetApp Storage, Hosted on Dell FC630, with Intel Xeon CPU E5-2670 v3 @ 2.3GHz & Intel X710 10g NIC -----Original Message----- From: Vadim Kochan [mailto:vadi...@gmail.com] Sent: יום ג 06 דצמבר 2016 02:59 To: Shaked, Asaf Cc: netsniff-ng@googlegroups.com; Tobias Klauser; Daniel Borkmann Subject: Re: [netsniff-ng] Few Questions I could not find answer for :) On Tue, Dec 6, 2016 at 2:19 AM, Shaked, Asaf <asaf.sha...@verint.com<mailto:asaf.sha...@verint.com>> wrote: > Hi, > > > > Sorry for not responding earlier, I was buried with work J > > I’ll try to explain in more details what I was trying to accomplish… > > > > > > After using Netsniff-NG to record high throughput traffic > impressively, I was trying it as capture Replayer. > > > > I was trying to replay, in my performance tuned local lab, high > throughput traffic, recorded on remote site, using some expensive equipment. > > > > I received the recording split to multiple pieces, with following details: > (capinfos) > > File type: Wireshark/tcpdump/... - pcap > > File encapsulation: Ethernet > > Packet size limit: file hdr: 9000 bytes > > Number of packets: 6382 k > > File size: 4294 MB > > Data size: 4192 MB > > Capture duration: 5 seconds > > Data byte rate: 824 MBps > > Data bit rate: 6592 Mbps > > Average packet size: 656.97 bytes > > Average packet rate: 1254 kpackets/sec > > Strict time order: True > > > > And I also left MergeCap to run through the night, to have also as > single capture, with following details: > > File type: Wireshark/tcpdump/... - pcap > > File encapsulation: Ethernet > > Packet size limit: file hdr: 9000 bytes > > Number of packets: 1348 M > > File size: 905 GB > > Data size: 883 GB > > Capture duration: 1096 seconds > > Data byte rate: 806 MBps > > Data bit rate: 6451 Mbps > > Average packet size: 655.45 bytes > > Average packet rate: 1230 kpackets/sec > > Strict time order: False > > > > 1) Notice I lost the “Strict time order” in merged file – don’t > understand why :\ > > > > VM I set up for replay is 64bit RHEL 7.2 with 16 x vCores, 64GB RAM, > VMXNET3 adapter & 1TB on 10K SAS FC NetApp Storage, > > Hosted on Dell FC630, with Intel Xeon CPU E5-2670 v3 @ 2.3GHz & Intel > X710 10g NIC > > > > So, the first (optimistic J) thing I tried to do is replay the merged > file using Netsniff-NG… > > netsniff-ng -i <CAP>.pcap -m -o <NIC> -s -J > > > > But I quickly learned that Netsniff-NG loads the entire capture to RAM > before starting to replay J > > > > 2) Is it possible to tell Netsniff-NG not to do that ? so I can replay > a capture much larger than available RAM (as eventually done with > tcpreplay on ~850GB merged) > > As I suppose to have this as a new feature in trafgen then I think we may use the ring size value from the -S,--ring-size option to check if it is filled by for example 80% then it is needed to load the next chunk on the background. > > 3) In addition, if Netsniff-NG would have the ability to receive a list > of captures to replay consecutively without drops in between, > > I would be able to replay my ~4GB multiple pieces, without having to > merge them, nor be unable to play merged because of insufficient RAM. > > I thought of rather simple design in which two files are loaded to > memory concurrently throughout replay… > > Starting with 1&2 à as soon as 1 ends replay, 3 is loaded to memory à > as soon as 2 ends replay, 4 is loaded to memory… and so on. Hm, this sounds interesting, so the pcap files 1st will be sorted by packet timestamps, and only after loaded 1 by one on demand probably ... > > > > So, replaying list of multiple files is one thing, but I also had > issues with replay speed. > > > > As you said Vadim, it seems Netsniff-NG replays any given capture, as > fast as possible. > > > > 4) So I noticed two related things when replayed single ~4GB piece… > > a. Netsniff-NG turned single-threaded all of a sudden & was utilizing > 100% of only a single core > > b. Probably therefore, as fast as possible was not fast enough, as > recorded 5 seconds in ~6.5Gbps, took 30 seconds to replay in ~1.2Gbps > > > > BTW, tcpreplay also started its’ replay at ~1.2Gbps (using --topspeed > flag – see command used below), which gradually dropped to only > ~250Mbps after 4+ hours of replay. > > tcpreplay -i <NIC> --pktlen -t <CAP>.pcap > Well, I am not sure how it would be possible to distribute packets with a strict order between cpus. > > > 5) In other cases, where recording bitrates are more sane J I believe > it would be rather powerful for Netsniff-NG to have tcpreplay speed > related flags/features… > > > > -x, --multiplier=str Modify replay speed to a given multiple > > -p, --pps=num Replay packets at a given packets/sec > > -M, --mbps=str Replay packets at a given Mbps > > -t, --topspeed Replay packets as fast as possible > > > > While the default no parameters is rather original timestamps & bitrate. As I said, I already have changes (which I did not send yet to review) where I added possibility to send packets from single pcap file in trafgen tool, and by default it sends them at the rate considering original packet's timestamps, but rate might be changed via -b option. I am not sure if it is needed to add new options from tcpreply (but may be -x might be useful, not sure about this). > > > Would greatly appreciate your comments regarding any of the points J > > Regards, Vadim Kochan -- This electronic message may contain proprietary and confidential information of Verint Systems Inc., its affiliates and/or subsidiaries. The information is intended to be for the use of the individual(s) or entity(ies) named above. If you are not the intended recipient (or authorized to receive this e-mail for the intended recipient), you may not use, copy, disclose or distribute to anyone this message or any information contained in this message. If you have received this electronic message in error, please notify us by replying to this e-mail. -- You received this message because you are subscribed to the Google Groups "netsniff-ng" group. To unsubscribe from this group and stop receiving emails from it, send an email to netsniff-ng+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
