On Tue, Dec 6, 2016 at 2:19 AM, Shaked, Asaf <asaf.sha...@verint.com> wrote: > Hi, > > > > Sorry for not responding earlier, I was buried with work J > > I’ll try to explain in more details what I was trying to accomplish… > > > > > > After using Netsniff-NG to record high throughput traffic impressively, I > was trying it as capture Replayer. > > > > I was trying to replay, in my performance tuned local lab, high throughput > traffic, recorded on remote site, using some expensive equipment. > > > > I received the recording split to multiple pieces, with following details: > (capinfos) > > File type: Wireshark/tcpdump/... - pcap > > File encapsulation: Ethernet > > Packet size limit: file hdr: 9000 bytes > > Number of packets: 6382 k > > File size: 4294 MB > > Data size: 4192 MB > > Capture duration: 5 seconds > > Data byte rate: 824 MBps > > Data bit rate: 6592 Mbps > > Average packet size: 656.97 bytes > > Average packet rate: 1254 kpackets/sec > > Strict time order: True > > > > And I also left MergeCap to run through the night, to have also as single > capture, with following details: > > File type: Wireshark/tcpdump/... - pcap > > File encapsulation: Ethernet > > Packet size limit: file hdr: 9000 bytes > > Number of packets: 1348 M > > File size: 905 GB > > Data size: 883 GB > > Capture duration: 1096 seconds > > Data byte rate: 806 MBps > > Data bit rate: 6451 Mbps > > Average packet size: 655.45 bytes > > Average packet rate: 1230 kpackets/sec > > Strict time order: False > > > > 1) Notice I lost the “Strict time order” in merged file – don’t > understand why :\ > > > > VM I set up for replay is 64bit RHEL 7.2 with 16 x vCores, 64GB RAM, VMXNET3 > adapter & 1TB on 10K SAS FC NetApp Storage, > > Hosted on Dell FC630, with Intel Xeon CPU E5-2670 v3 @ 2.3GHz & Intel X710 > 10g NIC > > > > So, the first (optimistic J) thing I tried to do is replay the merged file > using Netsniff-NG… > > netsniff-ng -i <CAP>.pcap -m -o <NIC> -s -J > > > > But I quickly learned that Netsniff-NG loads the entire capture to RAM > before starting to replay J > > > > 2) Is it possible to tell Netsniff-NG not to do that ? so I can replay > a capture much larger than available RAM (as eventually done with tcpreplay > on ~850GB merged) > >
As I suppose to have this as a new feature in trafgen then I think we may use the ring size value from the -S,--ring-size option to check if it is filled by for example 80% then it is needed to load the next chunk on the background. > > 3) In addition, if Netsniff-NG would have the ability to receive a list > of captures to replay consecutively without drops in between, > > I would be able to replay my ~4GB multiple pieces, without having to merge > them, nor be unable to play merged because of insufficient RAM. > > I thought of rather simple design in which two files are loaded to memory > concurrently throughout replay… > > Starting with 1&2 à as soon as 1 ends replay, 3 is loaded to memory à as > soon as 2 ends replay, 4 is loaded to memory… and so on. Hm, this sounds interesting, so the pcap files 1st will be sorted by packet timestamps, and only after loaded 1 by one on demand probably ... > > > > So, replaying list of multiple files is one thing, but I also had issues > with replay speed. > > > > As you said Vadim, it seems Netsniff-NG replays any given capture, as fast > as possible. > > > > 4) So I noticed two related things when replayed single ~4GB piece… > > a. Netsniff-NG turned single-threaded all of a sudden & was utilizing > 100% of only a single core > > b. Probably therefore, as fast as possible was not fast enough, as > recorded 5 seconds in ~6.5Gbps, took 30 seconds to replay in ~1.2Gbps > > > > BTW, tcpreplay also started its’ replay at ~1.2Gbps (using --topspeed flag – > see command used below), which gradually dropped to only ~250Mbps after 4+ > hours of replay. > > tcpreplay -i <NIC> --pktlen -t <CAP>.pcap > Well, I am not sure how it would be possible to distribute packets with a strict order between cpus. > > > 5) In other cases, where recording bitrates are more sane J I believe > it would be rather powerful for Netsniff-NG to have tcpreplay speed related > flags/features… > > > > -x, --multiplier=str Modify replay speed to a given multiple > > -p, --pps=num Replay packets at a given packets/sec > > -M, --mbps=str Replay packets at a given Mbps > > -t, --topspeed Replay packets as fast as possible > > > > While the default no parameters is rather original timestamps & bitrate. As I said, I already have changes (which I did not send yet to review) where I added possibility to send packets from single pcap file in trafgen tool, and by default it sends them at the rate considering original packet's timestamps, but rate might be changed via -b option. I am not sure if it is needed to add new options from tcpreply (but may be -x might be useful, not sure about this). > > > Would greatly appreciate your comments regarding any of the points J > > Regards, Vadim Kochan -- You received this message because you are subscribed to the Google Groups "netsniff-ng" group. To unsubscribe from this group and stop receiving emails from it, send an email to netsniff-ng+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
