Hi, Sorry for not responding earlier, I was buried with work ☺ I’ll try to explain in more details what I was trying to accomplish…
After using Netsniff-NG to record high throughput traffic impressively, I was trying it as capture Replayer. I was trying to replay, in my performance tuned local lab, high throughput traffic, recorded on remote site, using some expensive equipment. I received the recording split to multiple pieces, with following details: (capinfos) File type: Wireshark/tcpdump/... - pcap File encapsulation: Ethernet Packet size limit: file hdr: 9000 bytes Number of packets: 6382 k File size: 4294 MB Data size: 4192 MB Capture duration: 5 seconds Data byte rate: 824 MBps Data bit rate: 6592 Mbps Average packet size: 656.97 bytes Average packet rate: 1254 kpackets/sec Strict time order: True And I also left MergeCap to run through the night, to have also as single capture, with following details: File type: Wireshark/tcpdump/... - pcap File encapsulation: Ethernet Packet size limit: file hdr: 9000 bytes Number of packets: 1348 M File size: 905 GB Data size: 883 GB Capture duration: 1096 seconds Data byte rate: 806 MBps Data bit rate: 6451 Mbps Average packet size: 655.45 bytes Average packet rate: 1230 kpackets/sec Strict time order: False 1) Notice I lost the “Strict time order” in merged file – don’t understand why :\ VM I set up for replay is 64bit RHEL 7.2 with 16 x vCores, 64GB RAM, VMXNET3 adapter & 1TB on 10K SAS FC NetApp Storage, Hosted on Dell FC630, with Intel Xeon CPU E5-2670 v3 @ 2.3GHz & Intel X710 10g NIC So, the first (optimistic ☺) thing I tried to do is replay the merged file using Netsniff-NG… netsniff-ng -i <CAP>.pcap -m -o <NIC> -s -J But I quickly learned that Netsniff-NG loads the entire capture to RAM before starting to replay ☺ 2) Is it possible to tell Netsniff-NG not to do that ? so I can replay a capture much larger than available RAM (as eventually done with tcpreplay on ~850GB merged) 3) In addition, if Netsniff-NG would have the ability to receive a list of captures to replay consecutively without drops in between, I would be able to replay my ~4GB multiple pieces, without having to merge them, nor be unable to play merged because of insufficient RAM. I thought of rather simple design in which two files are loaded to memory concurrently throughout replay… Starting with 1&2 --> as soon as 1 ends replay, 3 is loaded to memory --> as soon as 2 ends replay, 4 is loaded to memory… and so on. So, replaying list of multiple files is one thing, but I also had issues with replay speed. As you said Vadim, it seems Netsniff-NG replays any given capture, as fast as possible. 4) So I noticed two related things when replayed single ~4GB piece… a. Netsniff-NG turned single-threaded all of a sudden & was utilizing 100% of only a single core b. Probably therefore, as fast as possible was not fast enough, as recorded 5 seconds in ~6.5Gbps, took 30 seconds to replay in ~1.2Gbps BTW, tcpreplay also started its’ replay at ~1.2Gbps (using --topspeed flag – see command used below), which gradually dropped to only ~250Mbps after 4+ hours of replay. tcpreplay -i <NIC> --pktlen -t <CAP>.pcap 5) In other cases, where recording bitrates are more sane ☺ I believe it would be rather powerful for Netsniff-NG to have tcpreplay speed related flags/features… -x, --multiplier=str Modify replay speed to a given multiple -p, --pps=num Replay packets at a given packets/sec -M, --mbps=str Replay packets at a given Mbps -t, --topspeed Replay packets as fast as possible While the default no parameters is rather original timestamps & bitrate. Would greatly appreciate your comments regarding any of the points ☺ Thanks, Asaf. -----Original Message----- From: Vadim Kochan [mailto:[email protected]] Sent: יום ב 05 דצמבר 2016 11:57 To: Shaked, Asaf Cc: [email protected]; Tobias Klauser; Daniel Borkmann Subject: Re: [netsniff-ng] Few Questions I could not find answer for :) Hi Tobias & Daniel, What do you think is it worth to have it in trafgen with considering original timestamps by default ? Thanks, On Sun, Dec 4, 2016 at 6:28 PM, Vadim Kochan <[email protected]<mailto:[email protected]>> wrote: > On Sun, Dec 4, 2016 at 2:09 PM, Vadim Kochan > <[email protected]<mailto:[email protected]>> wrote: >> On Tue, Nov 22, 2016 at 10:32 AM, Shaked, Asaf >> <[email protected]<mailto:[email protected]>> wrote: >>> Cool... good to see we're on the same page... thanks for the prompt >>> response :) >>> >>> Asaf. >>> >>> -----Original Message----- >>> From: Vadim Kochan [mailto:[email protected]] >>> Sent: יום ג 22 נובמבר 2016 02:00 >>> To: Shaked, Asaf >>> Cc: [email protected]<mailto:[email protected]> >>> Subject: Re: [netsniff-ng] Few Questions I could not find answer for >>> :) >>> >>> On Mon, Nov 21, 2016 at 03:37:48PM +0000, Shaked, Asaf wrote: >>> Hi Asaf, >>> >>>> Hi, >>>> >>>> Just love this toolkit :) >>>> >>>> Few questions, if you may... >>>> >>>> 1) Is it possible to merge pcaps using the toolkit ? >>> As fas as I know - no, but I think it would be good to have such feature >>> via such example: >>> >>> netsniff-ng -i {in_dir} -o {merged_file}.pcap >>> >>>> >>>> 2) Is it possible to replay a list of pcaps consecutively ? >>>> >>> As I know, it is posible only to redirect packets from one device to >>> another, but I really thing that it might be added to trafgen. >>> >>>> 3) Is it possible to control pcaps replay bitrate ? (aiming for >>>> original capture average Bitrate, rather than "as fast as possible") >>>> >>> So if option #2 will be implemented then I think it wold be easy have it in >>> trafgen as there ia already rate limit option. >>> >>>> Thanks, >>>> >>>> Asaf. >>>> >>> >>> Regards, >>> Vadim Kochan >>> >> >> Hi Asaf, >> >> I tried to extend trafgen with ability to send packets from pcap >> file, and you can try it if you able to fetch & build trafgen from: >> >> https://github.com/vkochan/netsniff-ng/tree/trafgen_pcap_send >> >> By default original timestamps are considered when sending traffic >> from pcap, but you can reset sending rate via -b/-t options: >> >> sudo trafgen/trafgen -i /tmp/test.pcap -o lo >> sudo trafgen/trafgen -i /tmp/test.pcap -o lo -b 1pps >> >> After this change will be applied I think I can try to add option to >> netsniff-ng tool to merging pcap files. >> >> Regards, >> Vadim Kochan > > Hm, > > I just realized that netsniff-ng already has pcap -> device sending > ability, but at max rate. > But I am stil think that it is good to have it in trafgen. Lets see if > Tobias/Daniel will agree or not on it ... -- This electronic message may contain proprietary and confidential information of Verint Systems Inc., its affiliates and/or subsidiaries. The information is intended to be for the use of the individual(s) or entity(ies) named above. If you are not the intended recipient (or authorized to receive this e-mail for the intended recipient), you may not use, copy, disclose or distribute to anyone this message or any information contained in this message. If you have received this electronic message in error, please notify us by replying to this e-mail. -- You received this message because you are subscribed to the Google Groups "netsniff-ng" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
