[netsniff-ng] netsniff-ng takes a long time to start up?
Hi,
I'm trying to figure out why netsniff-ng takes a long time to start up
on one of my machines. I'm running the latest git checkout on Debian
unstable (running the Debian 3.11.6 kernel), and when I run:
netsniff-ng --silent -i eth1 -o /dev/null
I see a delay of about 15 seconds before the Running! Hang up with ^C!
message is printed. Looking at netsniff-ng with strace, I see the
following:
[...]
1387508695.193460 setsockopt(3, SOL_PACKET, 0x11 /* PACKET_??? */, [64], 4) = 0
1387508695.193526 setsockopt(3, SOL_PACKET, PACKET_VERSION, [2], 4) = 0
1387508695.193587 getsockopt(3, SOL_PACKET, PACKET_VERSION, [2], [4]) = 0
1387508695.193650 setsockopt(3, SOL_PACKET, PACKET_RX_RING,
\0\0\1\0\346\16\0\0\0\0\1\0\346\16\0\0d\0\0\0\0\0\0\0\0\0\0\0, 28) = 0
1387508709.738025 mmap(NULL, 249954304, PROT_READ|PROT_WRITE,
MAP_SHARED|MAP_POPULATE|MAP_LOCKED, 3, 0) = 0x7f13cba7e000
1387508709.747174 getsockopt(3, SOL_PACKET, PACKET_VERSION, [2], [4]) = 0
1387508709.747347 bind(3, {sa_family=AF_PACKET, proto=0x03, if3,
pkttype=PACKET_HOST, addr(0)={0, }, 20) = 0
[...]
Note that the PACKET_RX_RING setsockopt() call takes about 15 seconds to
complete. During this time I see the netsniff-ng process consuming
about 90% of a CPU, and according to a perf record / perf report run
most of the time appears to be spent in these functions in the kernel:
31.29% netsniff-ng [kernel.kallsyms] [k] get_pageblock_flags_group
26.69% netsniff-ng [kernel.kallsyms] [k] isolate_freepages_block
24.22% netsniff-ng [kernel.kallsyms] [k] isolate_migratepages_range
2.41% netsniff-ng [kernel.kallsyms] [k] compaction_alloc
0.99% netsniff-ng [kernel.kallsyms] [k] __wake_up_bit
0.95% netsniff-ng [kernel.kallsyms] [k] __reset_isolation_suitable
0.85% netsniff-ng [kernel.kallsyms] [k] free_pcppages_bulk
0.80% netsniff-ng [kernel.kallsyms] [k] release_pages
0.76% netsniff-ng [kernel.kallsyms] [k] __pagevec_lru_add_fn
0.62% netsniff-ng [kernel.kallsyms] [k] page_waitqueue
0.61% netsniff-ng [kernel.kallsyms] [k] migrate_pages
0.58% netsniff-ng [kernel.kallsyms] [k] move_to_new_page
0.54% netsniff-ng [kernel.kallsyms] [k] mutex_lock
Help? It seems to be related to the size of the RX ring buffer -- if I
specify -S 1MiB to netsniff-ng it starts up quickly. But I also have
another machine where netsniff-ng starts up quickly with the default
ring buffer size.
I can send the full strace output and perf.data files if needed.
--
Robert Edmonds
--
You received this message because you are subscribed to the Google Groups
netsniff-ng group.
To unsubscribe from this group and stop receiving emails from it, send an email
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
Re: [netsniff-ng] netsniff-ng takes a long time to start up?
On 12/20/2013 04:21 AM, Robert Edmonds wrote:
Hi,
I'm trying to figure out why netsniff-ng takes a long time to start up
on one of my machines. I'm running the latest git checkout on Debian
unstable (running the Debian 3.11.6 kernel), and when I run:
netsniff-ng --silent -i eth1 -o /dev/null
I see a delay of about 15 seconds before the Running! Hang up with ^C!
message is printed. Looking at netsniff-ng with strace, I see the
following:
[...]
1387508695.193460 setsockopt(3, SOL_PACKET, 0x11 /* PACKET_??? */, [64], 4) = 0
1387508695.193526 setsockopt(3, SOL_PACKET, PACKET_VERSION, [2], 4) = 0
1387508695.193587 getsockopt(3, SOL_PACKET, PACKET_VERSION, [2], [4]) = 0
1387508695.193650 setsockopt(3, SOL_PACKET, PACKET_RX_RING,
\0\0\1\0\346\16\0\0\0\0\1\0\346\16\0\0d\0\0\0\0\0\0\0\0\0\0\0, 28) = 0
1387508709.738025 mmap(NULL, 249954304, PROT_READ|PROT_WRITE,
MAP_SHARED|MAP_POPULATE|MAP_LOCKED, 3, 0) = 0x7f13cba7e000
1387508709.747174 getsockopt(3, SOL_PACKET, PACKET_VERSION, [2], [4]) = 0
1387508709.747347 bind(3, {sa_family=AF_PACKET, proto=0x03, if3,
pkttype=PACKET_HOST, addr(0)={0, }, 20) = 0
[...]
Note that the PACKET_RX_RING setsockopt() call takes about 15 seconds to
complete. During this time I see the netsniff-ng process consuming
about 90% of a CPU, and according to a perf record / perf report run
most of the time appears to be spent in these functions in the kernel:
31.29% netsniff-ng [kernel.kallsyms] [k] get_pageblock_flags_group
26.69% netsniff-ng [kernel.kallsyms] [k] isolate_freepages_block
24.22% netsniff-ng [kernel.kallsyms] [k] isolate_migratepages_range
2.41% netsniff-ng [kernel.kallsyms] [k] compaction_alloc
0.99% netsniff-ng [kernel.kallsyms] [k] __wake_up_bit
0.95% netsniff-ng [kernel.kallsyms] [k] __reset_isolation_suitable
0.85% netsniff-ng [kernel.kallsyms] [k] free_pcppages_bulk
0.80% netsniff-ng [kernel.kallsyms] [k] release_pages
0.76% netsniff-ng [kernel.kallsyms] [k] __pagevec_lru_add_fn
0.62% netsniff-ng [kernel.kallsyms] [k] page_waitqueue
0.61% netsniff-ng [kernel.kallsyms] [k] migrate_pages
0.58% netsniff-ng [kernel.kallsyms] [k] move_to_new_page
0.54% netsniff-ng [kernel.kallsyms] [k] mutex_lock
Help? It seems to be related to the size of the RX ring buffer -- if I
specify -S 1MiB to netsniff-ng it starts up quickly. But I also have
another machine where netsniff-ng starts up quickly with the default
ring buffer size.
Thanks for the report. On what kind of hardware are you trying to do that?
I think we need to adjust default buffer allocation anyway, probably you're
perfectly fine with a ring buffer of around 2MB already.
I can send the full strace output and perf.data files if needed.
--
You received this message because you are subscribed to the Google Groups
netsniff-ng group.
To unsubscribe from this group and stop receiving emails from it, send an email
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
Re: [netsniff-ng] netsniff-ng takes a long time to start up?
Daniel Borkmann wrote: Thanks for the report. On what kind of hardware are you trying to do that? Kernel: Linux chase 3.11-1-amd64 #1 SMP Debian 3.11.6-1 (2013-10-27) x86_64 GNU/Linux OS: Debian sid CPU: Intel(R) Xeon(R) CPU E3-1245 v3 @ 3.40GHz Memory: 32 GB Ethernet: Intel Corporation I210 Gigabit Network Connection (rev 03) Motherboard: Supermicro X10SAE I think we need to adjust default buffer allocation anyway, probably you're perfectly fine with a ring buffer of around 2MB already. The funny thing is, I have a similar machine where netsniff-ng starts up instantly. It seems like a kernel issue, so I will try updating the problematic machine's kernel. I can also replicate the issue on the problematic machine with tcpdump by passing a large -B parameter to tcpdump. This is the strace output from the good machine: 1387556106.537430 setsockopt(3, SOL_PACKET, PACKET_RX_RING, \0\0\1\0\346\16\0\0\0\0\1\0\346\16\0\0d\0\0\0\0\0\0\0\0\0\0\0, 28) = 0 1387556106.594919 mmap(NULL, 249954304, PROT_READ|PROT_WRITE, MAP_SHARED|MAP_POPULATE|MAP_LOCKED, 3, 0) = 0x7f5f3bae3000 Kernel: Linux bst 3.11-2-amd64 #1 SMP Debian 3.11.8-1 (2013-11-13) x86_64 GNU/Linux OS: Debian wheezy CPU: Intel(R) Core(TM) i3-4130 CPU @ 3.40GHz Memory: 16 GB Ethernet: Intel Corporation I210 Gigabit Network Connection (rev 03) Motherboard: Supermicro X10SLL-F -- Robert Edmonds -- You received this message because you are subscribed to the Google Groups netsniff-ng group. To unsubscribe from this group and stop receiving emails from it, send an email to netsniff-ng+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: [netsniff-ng] netsniff-ng takes a long time to start up?
On 12/20/2013 07:08 PM, Robert Edmonds wrote: Robert Edmonds wrote: The funny thing is, I have a similar machine where netsniff-ng starts up instantly. It seems like a kernel issue, so I will try updating the problematic machine's kernel. I can also replicate the issue on the problematic machine with tcpdump by passing a large -B parameter to tcpdump. Very interesting. I've updated my machine to this kernel: Linux chase 3.11-2-amd64 #1 SMP Debian 3.11.10-1 (2013-12-04) x86_64 GNU/Linux And the problem has disappeared. Seems like a bug in kernel's VM layer that probably didn't get into stable or so. -- You received this message because you are subscribed to the Google Groups netsniff-ng group. To unsubscribe from this group and stop receiving emails from it, send an email to netsniff-ng+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
[netsniff-ng] packets unread on exit
Hi,
When using netsniff-ng to capture packets to a pcap file, it seems to
not flush any unprocessed packets remaining in the capture ring to the
output file when it receives SIGINT, e.g.:
# netsniff-ng -M -s -i eth1 -o aoeu.pcap
Running! Hang up with ^C!
0 packets incoming (20 unread on exit)
20 packets passed filter
0 packets failed filter (out of space)
0.% packet droprate
6 sec, 589766 usec in total
# ls -l
total 4
-rw-r--r-- 1 root root 24 Dec 20 14:43 aoeu.pcap
# capinfos -c aoeu.pcap
File name: aoeu.pcap
Number of packets: 0
#
I think it would be useful functionality to be able to flush the capture
ring on exit, or maybe periodically (perhaps as an option, at least)
which would match the behavior of other packet capture tools.
I believe the change below periodically flushes the capture ring. Would
this be acceptable for netsniff-ng? If I understand the current code
with its infinite timeout, it allows packets buffered in the capture
ring to potentially remain there indefinitely (e.g., if a traffic
generator attached to the capture interface suddenly goes silent).
diff --git a/netsniff-ng.c b/netsniff-ng.c
index c5966b1..c00078c 100644
--- a/netsniff-ng.c
+++ b/netsniff-ng.c
@@ -981,7 +981,7 @@ static void recv_only_or_dump(struct ctx *ctx)
break;
}
- ret = poll(rx_poll, 1, -1);
+ ret = poll(rx_poll, 1, 1000);
if (unlikely(ret 0)) {
if (errno != EINTR)
panic(Poll failed!\n);
--
Robert Edmonds
--
You received this message because you are subscribed to the Google Groups
netsniff-ng group.
To unsubscribe from this group and stop receiving emails from it, send an email
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
