[netsniff-ng] Re: flowtop: Example of UI tabs interface

[Daniel Borkmann]

On 01/13/2017 02:02 PM, Vadim Kochan wrote:

Hi,

I tried to extend flowtop UI to have some aggregated info via tabbed
panel interface.
Currently there is only 1 new "Process" tab which grouped flows stat info per
unique pid (with summed rates & bytes). Plz let me know if it would be
good to have.


I like that idea! ;) With *_SRC/*_DST you mean in/out?


Of course other kind of tabs might be added too - per user, proto, dst/src ip.

Regards,
Vadim Kochan


--
You received this message because you are subscribed to the Google Groups 
"netsniff-ng" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[netsniff-ng] Re: trafgen: Would be it useful to have pktgen support ?

[Daniel Borkmann]

On 08/07/2016 07:46 PM, Vadim Kochan wrote:

Hi,

I did not ever use Linux pktgen feature, but I just catch the
idea if it would be good to have option to send trafgen protocol
built packet via Linux pktgen ? Theoretically it is possible to create
simple and generic code to generate raw or pktgen packets. But
pktgen's kind of packets will be possible to generate only via protocol
header functions because with it we have all field metadata. Also
when dynamic fields series will be applied it may be possible also
convert them to pktgen's rand/inc commands.

But again let me know if it might be useful.


I think it would be useful, I see value where trafgen would do all the
procfs setup details internally and then runs pktgen as a backend. So the
only thing needed for the user would be to execute trafgen normally via
cmdline.

You won't be able to use all of the trafgen features when selecting pktgen
instead of af_packet as a backend, but I think that's okay.

Thanks,
Daniel

--
You received this message because you are subscribed to the Google Groups 
"netsniff-ng" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[netsniff-ng] Re: [PATCH 1/3] trafgen: Allow to compile without libnl

[Daniel Borkmann]

On 08/03/2016 09:34 AM, Tobias Klauser wrote:
[...]

Form the point of view of compiling nesniff-ng/trafgen there is no
difference of whether an external library "exists" or was explicitely
disabled by the user. I'd really prefer CONFIG_LIBNL for this and I
don't think more fine-grained control (CONFIG_LIBNL_*) will be needed,
as we gain relatively little compared to the amount of #ifdef complexity
we introduce.


Agree, CONFIG_LIBNL is a better choice.

--
You received this message because you are subscribed to the Google Groups 
"netsniff-ng" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[netsniff-ng] Re: flowtop: Flows visual separating

[Daniel Borkmann]

Hi Vadim,

thanks for looking into this, appreciate it!

On 02/20/2016 03:28 PM, Vadim Kochan wrote:

On Sat, Feb 20, 2016 at 1:25 AM, Vadim Kochan  wrote:

Hi,

I tried to come up with visual separating of printed flows as currently
its not easy to identify separate flow entry, so I did some changes and
I am not sure if it looks good so I atached the screenshot.

Regards,


I attached another version of odd & even flows entries style, here I
used cyan & white colors and it seem looks
better as here is no such contrast like in case with black & white
background colors (like in previous example),
also here 'country' color changed to magenta as it looks better on
white & cyan background colors.


Not particularly a fan of these background colors, but I understand
you'd like to improve usability on this. How about making flowtop
look and navigation more like top or htop? Perhaps some of this info
can be collapsed?

Thanks,
Daniel

--
You received this message because you are subscribed to the Google Groups 
"netsniff-ng" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [netsniff-ng] netsniff-ng output file name by date

[Daniel Borkmann]

On 02/19/2016 10:10 AM, Vadim Kochan wrote:

Simply because netsniff-ng does not support custom date-time format
for pcap file name.

But as I said we can extend it in the similar way like tcpdump does.


Agreed, that might be useful.

Thanks,
Daniel

--
You received this message because you are subscribed to the Google Groups 
"netsniff-ng" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [netsniff-ng] Capture every n packets

[Daniel Borkmann]

On 02/18/2016 12:09 PM, Vadim Kochan wrote:

Hi Lorenzo,

On Sat, Feb 6, 2016 at 11:16 PM, Lorenzo Pistone  wrote:

Hi, I would like to suggest a feature. It would be nice to have an option to
capture a packet only once every N packets. I can see that one could
implement it by abusing the fanout-group feature (have one group, one
instance actually capturing and the others with a filter that exclude all
packets, and set the fanout-type to round-robin). I don't know if there's
any better way to implement it with low level API, but such hack may
nevertheless be implemented as an option.


Doing this through round-robin seems not efficient to me.

You can easily write a cBPF (e.g. bpf_asm) or eBPF filter with random packet
sampling, so you only get a fraction of your traffic into AF_PACKET.


Cheers.
Lorenzo

--
You received this message because you are subscribed to the Google Groups
"netsniff-ng" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


Just FYI - Tobias will probably offline till end of this week, I am
sure he will answer to you.

I'd like to help you but I am not so technical good in AF_PACKET, but
this feature sounds like a 'Packet sampling' feature ?
At least it can be implemented for 1 process-only in the while(...)
loop when reading mmap-ed frames from kernel by checking
sniffed number of packets ? For the fan-out group processes it seems
like it needs to check each N packet on the kernel side (eBPF might
help ?) ?

It would be good to hear comments from Daniel too :)

Regards,
Vadim Kochan



--
You received this message because you are subscribed to the Google Groups 
"netsniff-ng" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [netsniff-ng] Replay pcap file on Xenomai kernel in real time

[Daniel Borkmann]

On 02/01/2016 04:51 PM, Tobias Klauser wrote:

On 2016-02-01 at 15:38:28 +0100, Umair Ali  wrote:

[...]

Sorry, but reviewing a random dump of code (that doesn't even compile)
goes beyond the scope of what I'd consider support/help for netsniff-ng.


+1

Ali, I find your request very unreasonable. Tobias was kind enough
to point you to everything you need to look at and if it's for your
university project anyway, then consider the rest a learning process.
Wasting a maintainers precious time like this I consider rude.

Thank you,
Daniel

--
You received this message because you are subscribed to the Google Groups 
"netsniff-ng" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [netsniff-ng] Synchronize file creation between netsniff-ng processes possible?

[Daniel Borkmann]

On 01/27/2016 02:13 PM, e.bengts...@gmail.com wrote:
[...]

Is it possible to add some kind of signal interface for controlling start
of new files instead of relying on incoming events?
I have pulled the source and had a look but don't really know how to
proceed...


If you have some cycles, it would be worth to add timerfd [1] support to
netsniff-ng, so that the timerfd is set to the dump interval we already
pass via cmdline, and we pack that fd along with the pf_packet one into
the poll() syscall, so we get a poll notification from there. We might need
to move update_pcap_next_dump() out of the loop so we can generate a new
dump file. This should reduce the latency for low/mid traffic patterns
quite a bit.

Thanks,
Daniel

  [1] http://man7.org/linux/man-pages/man2/timerfd_create.2.html

--
You received this message because you are subscribed to the Google Groups 
"netsniff-ng" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [netsniff-ng] Re: Shell variables support in trafgen

[Daniel Borkmann]

[ Thanks Vadim! Actually Cc'ing him here. ]

On 12/17/2015 12:38 PM, Vadim Kochan wrote:

On Wednesday, May 13, 2015 at 10:04:28 AM UTC+3, Kenneth Aaron wrote:


Hi,

Is it possible to use shell variables in the configuration file?
The use case I have is a script that sets variables such as IP address (
$SrcIP_Byte1 , $SrcIP_Byte2 , $SrcIP_Byte3 , $SrcIP_Byte4 ) then trafgen is
called using a standard configuration file, but the variables $SrcIP_Byte1
etc are referenced from within the config file itself
The alternative is that the shell script use 'sed' or other means of
manipulating the config file, but variables are so much easier

Thanks


Hi,

I added -D option which allows to .pass C preprocessor macro/define (with
combination of -p,--cpp option) and I think it can be
used similary to what you requested. The changes currently are only in
master branch in official repo. The same option was added
for bpfc.

Regards,
Vadim Kochan


--
You received this message because you are subscribed to the Google Groups 
"netsniff-ng" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [netsniff-ng] Adjusting PPS ?

[Daniel Borkmann]

On 12/14/2015 02:45 PM, Burak Özalp wrote:

Hi everbody,

I recently use trafgen, it is very useful and with your helps, I manage to 
create 6Mpps as you know. My problem is, is it possible to set pps value with 
argument or config file? I need very small values (like 140pps) and i want to 
use trafgen instead of hping, because it deserves it :)

How can i restrict pps value?


Setting a concrete pps value is currently not supported, but you can
specify a interpacket gap that trafgen should wait between packets,
perhaps that helps a bit in reducing your load:

$ trafgen -h
[...]
  -t|--gap Set approx. interpacket gap (s/ms/us/ns, def: 
us)
[...]


Best Regards
Burak Ozalp


--
You received this message because you are subscribed to the Google Groups 
"netsniff-ng" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [netsniff-ng] TCP-SYN Flood Config file

[Daniel Borkmann]

On 11/30/2015 01:02 PM, Burak Özalp wrote:
...

I realized that i use v0.5.8 in CentOS 7. İ tested with v0.6.0 with 12 core and 
finally i can manage to send 6 Mpps without any flags(with Jasper adjusted 
syn_attack config file) !


Nice to hear! Thanks!

--
You received this message because you are subscribed to the Google Groups 
"netsniff-ng" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [netsniff-ng] RPM v.0.6.0 Centos 7

[Daniel Borkmann]

On 11/30/2015 10:21 AM, Burak Özalp wrote:

Hi everyone,

I realized that --qdisc-path option is not available on version 0.5.8-6 whish 
is the latest RPM on CentoOS 7.

Could you please upload the latest RPM (v0.6.0) for CentOS 7.


I think you need to open a bug against RHEL7 (there should be a netsniff-ng
component to file a bug against) at : https://bugzilla.redhat.com/

This would also allow RH folks to check whether RHEL7 kernel still needs
backporting of the direct xmit mode patches.

Note: In case you only want --qdisc-path, then trafgen versions that don't
have this option, will use it by default. Since more recent trafgen versions
use direct xmit mode by default, --qdisc-path was an option to fallback to
the previous behavior.

But would be good in any case, if the RHEL7 package gets an update anyway.

Thanks,
Daniel

--
You received this message because you are subscribed to the Google Groups 
"netsniff-ng" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [netsniff-ng] TCP-SYN Flood Config file

[Daniel Borkmann]

On 11/27/2015 08:57 AM, Burak Özalp wrote:

Hi everyone;

First of all, you've done great job with netsniff-ng!Congrulations!
I want to use trafgen to simulate an syn_flood attack. In --help, i saw that

"  trafgen --dev eth0 --conf tcp_syn.cfg -u `id -u bob` -g `id -g bob`"

command. Could you please share tcp_syn.cfg file?


Did you try:

  trafgen -e > tcp_syn.cfg

It has a built-in example for getting started. You can test it with:

  trafgen -e | trafgen -i - -o eth0 --cpp -n 1

If you want to analyse the packet on TX (a la wireshark), you need to:

  trafgen -e | trafgen -i - -o eth0 --qdisc-path --cpp -n 1

Cheers,
Daniel

--
You received this message because you are subscribed to the Google Groups 
"netsniff-ng" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [netsniff-ng] TCP-SYN Flood Config file

[Daniel Borkmann]

On 11/27/2015 11:01 AM, Burak Özalp wrote:

Thank you! I edited the data part and at most i could send 1.800.000 packets 
per second. Does it possible to increase the pps ? I think that netsniff-ng is 
a good alternative to netmap, especially about tcp_syn flood attacks.


Hmm, likely depends on your network tuning and hardware, people have
reported to get ~6Mpps on TCP SYNs:

https://www.mail-archive.com/netdev@vger.kernel.org/msg87192.html

What kernel/settings do you have for fine-tuning?


On 27-11-2015 11:13, Daniel Borkmann wrote:

On 11/27/2015 08:57 AM, Burak Özalp wrote:

Hi everyone;

First of all, you've done great job with netsniff-ng!Congrulations!
I want to use trafgen to simulate an syn_flood attack. In --help, i saw that

"  trafgen --dev eth0 --conf tcp_syn.cfg -u `id -u bob` -g `id -g bob`"

command. Could you please share tcp_syn.cfg file?


Did you try:

  trafgen -e > tcp_syn.cfg

It has a built-in example for getting started. You can test it with:

  trafgen -e | trafgen -i - -o eth0 --cpp -n 1

If you want to analyse the packet on TX (a la wireshark), you need to:

  trafgen -e | trafgen -i - -o eth0 --qdisc-path --cpp -n 1

Cheers,
Daniel




--
You received this message because you are subscribed to the Google Groups 
"netsniff-ng" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [netsniff-ng] Re: [PATCH 2/2] trafgen: Allow to build packet from command line

[Daniel Borkmann]

On 11/09/2015 12:26 PM, vkochan wrote:
...

If you conceptually agree with this idea, I probably may re-work series
and add separate '-p, --pkt' option for the specify packet language from
command line, or additionally send this as separate patch.


Can you elaborate on "specify packet language"?

With this patch set, we support specifying a packet from a file or via cmdline.
Both end up being pushed through the *same* lexer/parser, I think that is good.

What we could end up doing on top of this is to further simplify the 
configuration
on top of this. Say a user just wants to push a TCP SYN packet (or DNS request, 
or
...), for example, with random source addresses to a fixed destination address.

Then I think, it would be good to have a means of "templates" for such things
that would expand/rewrite internally into the lower level trafgen representation
we have right now.

--
You received this message because you are subscribed to the Google Groups 
"netsniff-ng" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [netsniff-ng] Re: [PATCH] build: Check for libnl-route

[Daniel Borkmann]

On 11/06/2015 05:06 PM, Vadim Kochan wrote:

On Fri, Nov 06, 2015 at 05:05:38PM +0100, Tobias Klauser wrote:

On 2015-11-06 at 16:53:02 +0100, Vadim Kochan  wrote:

On Fri, Nov 06, 2015 at 04:36:08PM +0100, Tobias Klauser wrote:

On 2015-11-06 at 15:46:16 +0100, Vadim Kochan  wrote:

libnl-route is used in netsniff-ng tool for dump nlmsg flags.

Signed-off-by: Vadim Kochan 


Good catch! Applied, thanks.


I think it would be good to make (if it is possible) to make libnl
dependency optionally. As I remember is is used mostly for creating mon
interfaces, so may be it is better to do just panic if the config for
libnl is not defined by configure script, so it will be needed to wrap
libnl callers with #ifdef's where panic will be invoked.


Besides the nlmsg dissector it's used for rfmon support, i.e. this would
mean making making --rfraw in netsniff-ng and trafgen compile time
dependent. I would much prefer to avoid this, unless there's a strong
reason for it.

Do you see a use case for an netsniff-ng without libnl to justify this
change? Given also, that all major distributions I know of provide libnl
and it has a rather small footprint...


I am just thinking in a way to make a less dependencies if to use
trafgen/netsniff-ng on the system where there is not needed to install libnl ...
Or may be provide some defines which may turn on/off code which depends
on such libs.


Tobias has a good point. I mean, if it's just/mostly for the sake of setting
of setting up rfmon device, how big would be the hassle of implementing this
feature in a *clean* way entirely w/o using libnl? (Not sure if it's worth the
effort though ... but if you really need this, I'm preferring this way over
some more ifdef stuff with partial feature availability.)

Thanks,
Daniel

--
You received this message because you are subscribed to the Google Groups 
"netsniff-ng" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [netsniff-ng] Stats differences between ifpps and netsniff-ng.

[Daniel Borkmann]

On 10/13/2015 11:37 AM, tmartin...@gmail.com wrote:

Hello,

I'm currently trying to do some stats (max pps, max bps, etc) on a company 
network traffic.

My setup is quite simple: all traffic going through the "external" interface of 
a firewall is mirrored to a network interface on a server.

So I used netsniff-ng and ifpps at the same time on the server (approximatively 
during 10 hours).


Here is the commands used:
- netsniff-ng
netsniff-ng -i eth3 -o /pcaps/ -s --prefix datacenter. --verbose --ring-size 
64MiB --interval 1min --mmap -f /root/headers_only.bpfc

/root/headers_only.bpfc contains filters to only dump headers:
# cat /root/headers_only.bpfc
{ 0x20, 0, 0, 0xf034 },
{ 0x16, 0, 0, 0x },


- ifpps
ifpps -d eth3 -c -l > ifpps.csv


So I decide to take a look to the maximum traffic got with ifpps:
# grep "^[0-9]" /pcap/stats/ifpps.csv | cut -d' ' -f2 | sort -n | tail -1
30262932 (bytes)

It's corresponding to 12:19:25 UTC time.


When I take a look to the maximum traffic within the netsniff-ng dumps around 
the same time I got a lot less traffic:
# tcpstat -r /pcaps/datacenter.1444652352.pcap 1 | awk '{print $5}' | cut -d'=' 
-f2 | sort -n | tail -1
121340528 (bps) ~= 15167566 bytes


Yes, that would be expected. The reason is that ifpps dumps the stats from the 
NIC
while tcpstat looks at the pcap file, where you only dump the headers. So when 
the
BPF filter truncates the packet, the new, truncated length is written to the 
pcap
file.


Almost the same with tshark:
$ tshark -q -nr /pcaps/datacenter.1444652352.pcap -t ad -z io,stat,1 | grep 
2015- | awk '{print $7}' | sort -n | tail -1
16297450 (bytes)



Same "issue" with pps and without bpfc filter for netsniff-ng.

So, am I missing something? Is it an expected behavior?


Any help would be really appreciate!

Thanks!

Thomas


My system:
netsniff-ng 0.5.8 (Debian package: 0.5.8-2)
Debian Jessie 64 bits (3.16.0-4-amd64)
eth3's module: tg3



--
You received this message because you are subscribed to the Google Groups 
"netsniff-ng" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [netsniff-ng] Netsniff-ng Packet Capture with Intervals

[Daniel Borkmann]

On 08/07/2015 12:01 AM, Stefano Pirrello wrote:

I took another look at my server and saw the process is hung up again.  See
below on how many captures were successful but eventually they stopped
generating.

~/captures$ ls -ltr
total 3297540
-rwxr--r-- 1 n3tus3r n3tus3r  6225 Aug  3 23:06 acloudshark-upload.py
-rw-r--r-- 1 rootroot151763085 Aug  6 11:59 NOR-1438829948.pcap
-rw--- 1 n3tus3r n3tus3r   170 Aug  6 12:00 nohup.out
-rw-r--r-- 1 rootroot614790736 Aug  6 12:15 NOR-1438876806.pcap
-rw-r--r-- 1 rootroot531106907 Aug  6 12:30 NOR-1438877706.pcap
-rw-r--r-- 1 rootroot469131877 Aug  6 12:45 NOR-1438878606.pcap
-rw-r--r-- 1 rootroot447301234 Aug  6 13:00 NOR-1438879506.pcap
-rw-r--r-- 1 rootroot536482188 Aug  6 13:15 NOR-1438880406.pcap
-rw-r--r-- 1 rootroot502705750 Aug  6 13:30 NOR-1438881306.pcap
-rw-r--r-- 1 rootroot123361242 Aug  6 13:33 NOR-1438882206.pcap


You can also see how the process is still running:

$ sudo ps -ef | grep netsniff
[sudo] password for n3tus3r:
root   885   618  0 12:00 pts/000:00:00 sudo nohup netsniff-ng --in
bond0 --out . --prefix NOR- --interval 15min -s -H -f net 10.10.192.0/18
root   886   885 22 12:00 pts/001:22:08 netsniff-ng --in bond0
--out . --prefix NOR- --interval 15min -s -H -f net 10.10.192.0/18
n3tus3r   1326  1297  0 18:00 pts/100:00:00 grep --color=auto netsniff


Hmm, is there any additional debugging information where it could be hung?
Would strace give any details?

--
You received this message because you are subscribed to the Google Groups 
netsniff-ng group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [netsniff-ng] [PATCH] flowtop: Fix collector stuck while flush IPv6 flows

[Daniel Borkmann]

On 07/27/2015 12:20 AM, Vadim Kochan wrote:

On Sun, Jul 26, 2015 at 09:57:04PM +0300, Vadim Kochan wrote:

On Sun, Jul 26, 2015 at 08:39:04PM +0200, Daniel Borkmann wrote:

On 07/25/2015 06:09 PM, Vadim Kochan wrote:

From: Vadim Kochan vadi...@gmail.com

Seems it was caused by specifying all netfilter groups
when flushing connections.

Used separated nfct instance w/o netfilter groups to
flush ipv4/ipv6 connections.

More info can be fetched from the issue item on github:

https://github.com/netsniff-ng/netsniff-ng/issues/145

Signed-off-by: Vadim Kochan vadi...@gmail.com


Applied, thanks.


BTW is it really needs to flush connections ?
Because counters will be cleared too, and they will not reflect
the real data flow through the connection ?

Regards,


Seems I understand, this is for catch existing connections through the
event-driven way, but I assume it will be possible too if dump everything
once to save the counters, and then do a polling.


Yes, I agree it's a bit suboptimal currently.

--
You received this message because you are subscribed to the Google Groups 
netsniff-ng group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [netsniff-ng] [PATCH] flowtop: Use prev next vars in flow_list_find_prev_id(...)

[Daniel Borkmann]

On 07/25/2015 07:22 PM, Vadim Kochan wrote:

From: Vadim Kochan vadi...@gmail.com

Replace n  tmp variables to more understandable prev  next.

Signed-off-by: Vadim Kochan vadi...@gmail.com


Applied.

--
You received this message because you are subscribed to the Google Groups 
netsniff-ng group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [netsniff-ng] [PATCH] ifpps: Cleanup screen on panic

[Daniel Borkmann]

On 07/26/2015 02:17 PM, Vadim Kochan wrote:

From: Vadim Kochan vadi...@gmail.com

Clean up ncurses screen when panic was caused.
There will be limitation that error message
will be printed on stderr after ncurses cleanup
so it will be needed now to use redirect to error file:

 ifpps -d dev 2 /tmp/err

to see error message.

Signed-off-by: Vadim Kochan vadi...@gmail.com


Also applied, thank you!

--
You received this message because you are subscribed to the Google Groups 
netsniff-ng group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [netsniff-ng] [PATCH] ifpps: Fix panic when dev name contains _

[Daniel Borkmann]

On 07/26/2015 02:01 PM, Vadim Kochan wrote:

From: Vadim Kochan vadi...@gmail.com

Add match device name with _ in /proc/net/dev.

Signed-off-by: Vadim Kochan vadi...@gmail.com


Applied, thanks!

--
You received this message because you are subscribed to the Google Groups 
netsniff-ng group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [netsniff-ng] [PATCH 0/3] flowtop: Convert int - bool in several funcs

[Daniel Borkmann]

On 07/25/2015 06:56 PM, Vadim Kochan wrote:

This series converts int - bool in several functions:

- flow_entry_get_extended_is_dns(...)
- get_port_inode(..)
- presenter_get_port(..)


Series applied, thanks.

--
You received this message because you are subscribed to the Google Groups 
netsniff-ng group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [netsniff-ng] [PATCH] flowtop: Fix collector stuck while flush IPv6 flows

[Daniel Borkmann]

On 07/25/2015 06:09 PM, Vadim Kochan wrote:

From: Vadim Kochan vadi...@gmail.com

Seems it was caused by specifying all netfilter groups
when flushing connections.

Used separated nfct instance w/o netfilter groups to
flush ipv4/ipv6 connections.

More info can be fetched from the issue item on github:

https://github.com/netsniff-ng/netsniff-ng/issues/145

Signed-off-by: Vadim Kochan vadi...@gmail.com


Applied, thanks.

--
You received this message because you are subscribed to the Google Groups 
netsniff-ng group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[netsniff-ng] Re: [PATCH] flowtop: Check dst port when resolve process id

[Daniel Borkmann]

On 07/23/2015 08:15 AM, Vadim Kochan wrote:
...

Sorry this might be buggy if dst port from #1 connection will be the
same as src port from connectoin #2, so it needs to do checking
separately for src  dst in case if src lookup was failed.


Ok, dropped.

--
You received this message because you are subscribed to the Google Groups 
netsniff-ng group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [netsniff-ng] [PATCH 0/2] flowtop: Revert cond lock wait before screen initializing

[Daniel Borkmann]

On 07/18/2015 01:12 PM, Vadim Kochan wrote:

This series reverts unnecesseary cond lock when presenter waits for collector
to finish its initializing part before do screen initializing, it was added
only to do not panic collector when screen is initialized with ncurses so after 
this
shell will be colored.

Now screen is cleaned up in panic handler so cond lock is not needed anymore.


Series applied, thanks!

--
You received this message because you are subscribed to the Google Groups 
netsniff-ng group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [netsniff-ng] [PATCH 0/3] sysctl: Use new sysctl set/get helpers

[Daniel Borkmann]

On 07/20/2015 10:12 AM, Vadim Kochan wrote:

On Mon, Jul 20, 2015 at 10:09:52AM +0200, Daniel Borkmann wrote:

On 07/19/2015 01:41 PM, Vadim Kochan wrote:

Replace direct access to /proc/sys params by new added helpers from
sysctl.c module.


Looks good, applied. Thanks for following up on this!


Thanks, I am thinking about automatically revert sysctl params to old
values on the exit, seems it needs to use atexit(..) and some rework of
existing panic handler logic.


Sounds good to me.

--
You received this message because you are subscribed to the Google Groups 
netsniff-ng group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [netsniff-ng] netsniff-ng dropping packets at 80Mbps

[Daniel Borkmann]

On 07/20/2015 02:06 PM, Unnikannan Nair, Jishnu wrote:

Hi
I'm running the latest version 0.5.9, and im not running it in silent mode as 
of now just to see the dropped packets information when the capture is 
complete. I intend to run it as silent when all issues are solved :)


Well, dumping everything through the dissector code and expecting it
to serve the packet as fast as possible and clear up the ring slot
again certainly seems a false expectation. Besides, even if you terminate
in silent mode, it's giving you stats:

netsniff-ng -i any -s
Running! Hang up with ^C!

  35  packets incoming (0 unread on exit)
  35  packets passed filter
   0  packets failed filter (out of space)
  0.% packet droprate
   6  sec, 178111 usec in total
...

Cheers,
Daniel

--
You received this message because you are subscribed to the Google Groups 
netsniff-ng group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [netsniff-ng] netsniff-ng dropping packets at 80Mbps

[Daniel Borkmann]

On 07/20/2015 01:45 PM, Unnikannan Nair, Jishnu wrote:

Hi
I'm trying to build a network sniffer for UDP multicast streams using 
netsniff-ng. I have deployed two linux systems (Ubuntu server 14.04 LTS) on an 
ESXi one has the sniffer and other has tcpreplay sending packets at needed 
Mbps. The configuration is:
Sniffer:

* Intel i7 3770 3.4GHz

* 2 cores added to the sniffer VM

* 16GB RAM



The sniffer job is  as follows :

Sudo netsniff-ng -I eth1 -out=/share/job1/ --prefix=job1_  --interval 2MiB -ring-size 
3GiB -prio-high -f dst net 224.10.10 -user 1000 -group 1000 -b 1


Hmm, what version do you use? Have you tried a more recent version?

Also, I fail to see that you're invoking it in 'silent' mode (-s).





The tcp replay job is as follows

tcpreplay -i eth1 -M 80Mbps -loop 100 -preload-pcap -sleep-accel=1  
/home/caps/*.pcap



Tcpreplay sends 10 pcap files , 100 times i.e. total packets send is 10487400 @ 
80.77Mbps but netsniff-ng received only 504813



TCPREPLAY:

Actual: 10487400 packets (5229757800 bytes) sent in 481.67 seconds.
Rated: 10857554.0 bps, 82.84 Mbps, 21773.00 pps

Statistics for network device: eth1

 Attempted packets: 10487400

 Successful packets:10487400

 Failed packets:0

 Retried packets (ENOBUFS): 0

 Retried packets (EAGAIN):  0



NETSNIFF_NG:

504813  packets incoming (9982587 unread on exit)

5929469  packets passed filter

4557931  packets failed filter (out of space)

43.4610% packet droprate

559  sec, 715362 usec in total

Cannot set NIC flags!



I have increased the kernel RX Cache to 1GB in /etc/sysctl.conf, but it did not have any 
effect. Could someone give me some information as to how to fix this?? Also the last line 
Cannot set NIC Flag what does that mean?? Is that an error?? My target is to 
run two instance of netsniff on eth1 and eth2 with different filters and bind them on CPU 
1 and CPU 2 later on.



Regards

Jishnu



Jishnu Unnikannan Nair | Systems Engineer
NOV Rig Systems
Systems  Controls | Drilling data Center
Lagerveien 8| 4033 Stavanger, Norway
T   +475.181.8181
M +473.819.4208
E   jishnu.unnikannanna...@nov.com
nov.comhttp://www.nov.com/
Connect with us on Facebookhttps://www.facebook.com/NationalOilwellVarco | 
LinkedInhttp://www.linkedin.com/company/national-oilwell-varco?trk=tyahtrkInfo=tarId%3A1405455811306%2Ctas%3Anational+oilwell%2Cidx%3A2-1-4
 | Twitterhttps://twitter.com/NOVGlobal
The information contained in this transmission is for the personal and 
confidential use of the individual or entity to which it is addressed. If the 
reader is not the intended recipient, you are hereby notified that any review, 
dissemination, or copying of this communication is strictly prohibited. If you 
have received this transmission in error, please notify the sender immediately.




--
You received this message because you are subscribed to the Google Groups 
netsniff-ng group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [netsniff-ng] netsniff-ng dropping packets at 80Mbps

[Daniel Borkmann]

On 07/20/2015 02:21 PM, Unnikannan Nair, Jishnu wrote:

So are you saying that it's impossible to dump pcap files at this rate??


Please, that's not what I wrote in my email.

I wrote that you should use -s.

--
You received this message because you are subscribed to the Google Groups 
netsniff-ng group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [netsniff-ng] netsniff-ng dropping packets at 80Mbps

[Daniel Borkmann]

On 07/20/2015 04:58 PM, Unnikannan Nair, Jishnu wrote:

Hi
I have double checked, it seems if I run with -s option with the application saving 
files on to a folder --out /share/capture/job/ will not give any statistics up on 
stopping it using  ^C.
And it is giving a Cannot set NIC Flags message at the end. So I'm not able 
to check if the sniffer has captured all the packets. Could you please help me with this.


Could you pull/recompile the latest repo? We don't dump the actual
error, which we should do to easier identify issues. If you cannot
recompile, the alternative is to strace into it, so we can see what
error you get that lets netsniff-ng bail out?

Thanks,
Daniel

--
You received this message because you are subscribed to the Google Groups 
netsniff-ng group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [netsniff-ng]

[Daniel Borkmann]

On 07/14/2015 04:57 PM, Vadim Kochan wrote:

vadi...@gmail.com
Bcc:
Subject: Re: [netsniff-ng] wrong number of packets filtered in
Reply-To:
In-Reply-To: ac7d7957a7715142afeae26281c9be9851d99...@srveurmbx06.nov.com

On Mon, Jul 13, 2015 at 02:27:22PM +, Unnikannan Nair, Jishnu wrote:

Hi
I'm using netsniff-ng for sniffing UDP packets on a network. For testign the 
performance I tried to send packets using iperf with bandwidth 100M and length 
250. When the capture was completed iperf says it send 28000 packets sent but 
netsniff-ng shows a lesser value and it also shows 0% packet dropped. Could you 
please help me solve this confusion on whats happening. I'm more intereted in 
checking if the sniffer can handle upto 100Mbits/s data rate on the network


Normally, recording 100Mbit to pcap should not be an issue (unless you have 
some weak
embedded system, etc) ... do you use things like GSO? How big is a packet that 
you've
captured (does it match the 250b)? As Vadmin said, please provide some more 
details.


Regards
Jishnu

--
You received this message because you are subscribed to the Google Groups 
netsniff-ng group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


Hi,

I am not an expert in netsniff-ng, but would you please give more
details about your setup (how do you run netsniff-ng, HW info, stats
info, examples of output iperf  netsniff-ng, etc ...).

Regards,
Vadim Kochan



--
You received this message because you are subscribed to the Google Groups 
netsniff-ng group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[netsniff-ng] Re: flowtop: ping for fixes ...

[Daniel Borkmann]

On 07/13/2015 10:24 AM, Vadim Kochan wrote:

Hi,

After 1 week I decided to ping for some flowtop related fixes:
http://article.gmane.org/gmane.linux.network.netsniff-ng/973
http://article.gmane.org/gmane.linux.network.netsniff-ng/974


Sorry for the long delay. I've applied both fixes and am going over
the traffic counter series now. Thanks for your patience!


and series about traffic counters:
http://article.gmane.org/gmane.linux.network.netsniff-ng/978

Regards,



--
You received this message because you are subscribed to the Google Groups 
netsniff-ng group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [netsniff-ng] [PATCH] flowtop: Fix flows disappearing

[Daniel Borkmann]

On 07/04/2015 09:18 PM, Vadim Kochan wrote:

From: Vadim Kochan vadi...@gmail.com

While removing flow which is pointed by 'head' then
head is set to NULL and all the list disappears, so fixed
by set removing flow next entry to list 'head'.

Signed-off-by: Vadim Kochan vadi...@gmail.com


Applied, thanks!

--
You received this message because you are subscribed to the Google Groups 
netsniff-ng group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [netsniff-ng] [PATCH 0/5] flowtop: Add traffic accounting dump

[Daniel Borkmann]

On 07/13/2015 12:09 PM, Vadim Kochan wrote:

On Mon, Jul 13, 2015 at 12:31:39PM +0300, Vadim Kochan wrote:

...

If I understood correctly, you asked to do not use separated sysctl
module unless it will not be used in other places ? But these funcs can
be used statically in flowtop.c so they then can be used later while
sysctl refactoring ?


Yes, would be good if we introduce such library bits, to also make
consistent use of it across the project.


OK I squashed sysctl commit with traffic counters dump commit and will
resend series v2.


Great, thanks!

Btw, I think it would also be useful to have flowtop dump/list the current
data set to the test console (like conntrack -L), so that an admin could do
a one time snapshot dump and do some more analysis on the data afterwards.

Thoughts? ;)

Cheers,
Daniel

--
You received this message because you are subscribed to the Google Groups 
netsniff-ng group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [netsniff-ng] [RFC v2 00/10] trafgen: Build protocol packet via command line

[Daniel Borkmann]

On 06/29/2015 02:58 AM, Vadim Kochan wrote:

Hi,

This is the 1-st try version of how I see the protocol generation API for the
trafgen util as mz replacement (at least for better performance).

I am sending this just because to get your feedback about conceptual idea,
and as soon as I got some basic working version I decided to share the
code just to get know if I am moving in the right direction.

Added high-level command line protocol packet building intreface,
which allows to specify protocol parameters to build the header and
payload.

Each protocol is represented by proto_gen struct which is responsible
only for providing field info (size, data) by name to trafgen's
low level packet generation layer.

All packet generation routine is performed by the generic code in
trafgen.c which parses the command line, obtains proto name, param=value
list and calls the specific protocol handler to get protocol field info
by name, so the TX routine remains the same.

The command line syntax looks like:

 trafgen/trafgen --dev lo eth da = AA:BB:CC:DD:EE:FF sa=11:22:33:44:55:66, 
arp op=rep tip=192.168.1.1 -n 1

so the first is proto name and after there are param value pairs which
are separated by space, in case if there are multiple protocols
specified - their should be separated by , after last param value of
the previous protocol.

I think the picture will be more clear after adding IP protocol with checksum
handling.


First of all, thanks for working on this, Vadim! I like seeing something like
this integrated after we've resolved all outstanding issues. I'll certainly
make trafgen also easier to use.

Before digging into the very details, I have a couple of high-level comments/
thoughts. All the manual string parsing you are doing, isn't it easier to just
extend the flex/bison files with the related protocol information?

I.e. I was thinking of 1) make the current configuration syntax also available
for the direct command line interface, and after that 2) extend the flex/bison
parser with L2, L3, etc information in a similar syntax as you did above (e.g.
multiple packets could also here be defined with separator { ... }, if no 
separator
is provided, we assume a single packet). This would give the flexibility of 
having
a mz-like cmdline syntax and at the same time one could also use it in the
config file. Do you see any major obstacles with that?

Regarding the default values, f.e. if we've specified only L3 information (e.g.
IPv4), but no L2 information, we should look up src/dst mac based on the output
interface resp. the neighbor cache. We should be careful with broadcasts, i.e.
if no other information is available for determining a dst, only then we should
use broadcast (f.e. if only L2 was specified w/o a dst mac, etc); in all other
cases we should try hard to resolve all needed information from the kernel.

Anything I've missed, Tobias? :)

Thanks again,
Daniel

--
You received this message because you are subscribed to the Google Groups 
netsniff-ng group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [netsniff-ng] netsniff-ng: Incorrect packet count #73: May be closed ?

[Daniel Borkmann]

[ Cc'ing Jon ]

On 06/29/2015 11:32 AM, Vadim Kochan wrote:

Hi,

Since the possible fix was comitted  there was no response from #jonship
may be it can be closed ?

https://github.com/netsniff-ng/netsniff-ng/issues/73

Regards,



--
You received this message because you are subscribed to the Google Groups 
netsniff-ng group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [netsniff-ng] mausezahn: TX fast path ... or may be rewrite everything ?

[Daniel Borkmann]

On 06/21/2015 07:02 PM, Vadim Kochan wrote:
...

Not sure if it is necessary to use libnet by trafgen as it is used by mz
to generate packets ? It will add the new dependency, but seems it can
simplify generation of the proto packets.


Yes sure, we shouldn't depend on it, I think some small helper functions
inside the netsniff-ng tree could do the job, too.

--
You received this message because you are subscribed to the Google Groups 
netsniff-ng group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [netsniff-ng] mausezahn: TX fast path ... or may be rewrite everything ?

[Daniel Borkmann]

On 06/22/2015 03:37 PM, Vadim Kochan wrote:
...

Well, anyway I am trying to implement basic cmdline mz-like proto
generation API with few layered protocols: eth, ip, udp for trafgen
utility.

If I success with this I will share the patch series (I will try to
better split it for easier review).


Looking forward to review!

--
You received this message because you are subscribed to the Google Groups 
netsniff-ng group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [netsniff-ng] mausezahn: TX fast path ... or may be rewrite everything ?

[Daniel Borkmann]

On 06/19/2015 10:46 AM, Vadim Kochan wrote:
...

OK here is another (may be better) design.

Here is an example of the cfg script:

{
 eth
 {
 dst: eth_addr(AA:BB:CC:DD:EE:FF)
 proto: ipv4,
 }
 ip
 {
 src: ip_addr(192.168.1.1)
 dst: ip_addr(192.168.1.2)
 }
 tcp
 {
 flags: syn | ack
 }
}

Needs some refactoring of trafgen code  existing cfg funcs which allows to use 
 manipulate of
current packet offset variable.

Each proto should be registered via struct proto_gen:

struct proto_field {
 int id;
 uint32_t offset;
 uint32_t len;
};

struct proto_value {
 void *value;
 uint32_t len;
};

struct proto_gen {
 struct cfg_ctx *ctx;

 /* get field name info - id, len, offset */
 struct proto_field *field_parse(struct proto_gen *prot, char *name);

 /* indicates that this field was filled */
 void field_set(struct proto_gen *proto, struct proto_field *field);

 /* get info about some proto defined const value */
 struct proto_value *value_get(struct proto_gen *proto, struct proto_field 
*field, char *name);

 /* fill the rest unfilled fields */
 void fill(struct proto_gen *prot, uint8_t *hdr);

 /* finish after whole packet was built */
 void finish(struct proto_gen *prot, uint8_t *data, int len);
}

Each proto hdr field can be filled by specifying field name separated by ':' and
macro/func which will fill the bytes.The field may be filled with any
existing trafgen macro (const(), rnd() - the field len should be
considered). Also some additional macro/funcs should be added - ipv4_addr, 
eth_addr, etc.

The parser will lookup  keep struct proto_gen proto_ptr when the proto section
was entered (when '#proto {' was parsed), and parser can call the
proto_ptr-field_parse to obtain the field information by name (id,
offset, len) then change the current packet offset  call the macro/func
which will fill the field. Then parser should call proto_ptr-field_set
to let know to the proto_gen which field was filled. At the end parser
will call proto_ptr-fill to fill the rest unfilled fields with some
default values  fill the csum fields if needed.

While filling the default fields values the proto_gen should know some
trafgen's context info - interface id, and may be some other info which may
help to construct such unfilled fields like dst/src Ethernet addresses or
src/dst IP addresses.

Also it is possible that each proto_gen may have some own defined values like
flag names which can be evaluated via proto_gen-value_get where field pointer
may be specified. It might be useful if these values will be a part of 
expressions
like: syn | ack.

At the end the parser will walk over each proto from higher layer to do a last 
call
proto_gen-finish where each proto may do some calculations and fill some 
unfilled fields.
Meanwhile I am not sure if it will be needed.

Well this is very conceptually and I will try to prepare better version which
would consider the existing trafgen code  cfg syntax parser.


Ok, how would all of this tie together with flex and bison that we
use for trafgen as parser generator? It looks on a first glance that
this design would be outside of that scope?

Cheers,
Daniel

--
You received this message because you are subscribed to the Google Groups 
netsniff-ng group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [netsniff-ng] [PATCH 0/2 v2] Add suport for Linux cooked header sniff dissect

[Daniel Borkmann]

On 06/20/2015 12:38 AM, Vadim Kochan wrote:
...

Regarding cleanups ... what do you think about change pcap_pkthdr_t to:

typedef struct {
union {
struct pcap_pkthdr  ppo;
struct pcap_pkthdr_ns   ppn;
struct pcap_pkthdr_ll   ppo_ll;
struct pcap_pkthdr_ns_llppn_ll;
struct pcap_pkthdr_kuz  ppk;
struct pcap_pkthdr_bkm  ppb;
uint8_t raw;
} h;
pcap_type_t type;
} pcap_pkthdr_t;

which will allow to remove 'type' argument from I/O  other pcap functions ... 
and
makes calling of these functions shorter ...


Seems okay with me, but depends on how the actual code looks eventually. ;)

--
You received this message because you are subscribed to the Google Groups 
netsniff-ng group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [netsniff-ng] [PATCH 0/2 v2] Add suport for Linux cooked header sniff dissect

[Daniel Borkmann]

On 06/19/2015 11:37 PM, Vadim Kochan wrote:
...

Never applied patches from emails, I tried to do in mutt by saving as
attachments or mbox, I even cut till 'diff' line but with no luck to
apply this. May be you can suggest some work flow with mutt for
patches via email ?


Ok, so here's both as an attachment, also your dissector on top,
which works fine for me.

(I usually save the whole source and git am it.)

--
You received this message because you are subscribed to the Google Groups 
netsniff-ng group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
From 045e9395ca9c2971ab7f7a886f5d6b4cf679c7fd Mon Sep 17 00:00:00 2001
Message-Id: 045e9395ca9c2971ab7f7a886f5d6b4cf679c7fd.1434749532.git.dan...@iogearbox.net
From: Daniel Borkmann dan...@iogearbox.net
Date: Fri, 19 Jun 2015 00:30:31 +0200
Subject: [PATCH 1/2] pcap_io: add cooked mode support

Originally submitted by Vadim in a different form, he wrote:

  Use Linux cooked header for Netlink interface automatically or
  as replacement of L2 header if --cooked option is specified:

http://www.tcpdump.org/linktypes/LINKTYPE_LINUX_SLL.html

  'Cooked headers' makes sense to use for default or nsec pcap
  types which does not contain protocol info.

  Added new LINKTYPE_LINUX_SLL which indicates pcap file with
  Linux cooked header as L2 layer header. This pcap file is
  compatible with Wireshark's cooked header  vice-versa.

Signed-off-by: Vadim Kochan vadi...@gmail.com
Signed-off-by: Daniel Borkmann dan...@iogearbox.net
---
 netsniff-ng.c |  17 ++
 pcap_io.h | 192 +++---
 2 files changed, 172 insertions(+), 37 deletions(-)

diff --git a/netsniff-ng.c b/netsniff-ng.c
index 0a9c620..e593b9d 100644
--- a/netsniff-ng.c
+++ b/netsniff-ng.c
@@ -1519,6 +1519,23 @@ int main(int argc, char **argv)
 
 		if (!ctx.link_type)
 			ctx.link_type = pcap_dev_to_linktype(ctx.device_in);
+		if (link_has_sll_hdr(ctx.link_type)) {
+			switch (ctx.magic) {
+			case ORIGINAL_TCPDUMP_MAGIC:
+ctx.magic = ORIGINAL_TCPDUMP_MAGIC_LL;
+break;
+			case NSEC_TCPDUMP_MAGIC:
+ctx.magic = NSEC_TCPDUMP_MAGIC_LL;
+break;
+			case ___constant_swab32(ORIGINAL_TCPDUMP_MAGIC):
+ctx.magic = ___constant_swab32(ORIGINAL_TCPDUMP_MAGIC_LL);
+break;
+			case ___constant_swab32(NSEC_TCPDUMP_MAGIC):
+ctx.magic = ___constant_swab32(NSEC_TCPDUMP_MAGIC_LL);
+break;
+			}
+		}
+
 
 		if (!ctx.device_out) {
 			ctx.dump = 0;
diff --git a/pcap_io.h b/pcap_io.h
index 497e453..5beccf9 100644
--- a/pcap_io.h
+++ b/pcap_io.h
@@ -27,6 +27,8 @@
 #define TCPDUMP_MAGIC0xa1b2c3d4
 #define ORIGINAL_TCPDUMP_MAGIC			TCPDUMP_MAGIC
 #define NSEC_TCPDUMP_MAGIC			0xa1b23c4d
+#define ORIGINAL_TCPDUMP_MAGIC_LL		0xb1b2c3d4	/* Internal dummy just for mapping */
+#define NSEC_TCPDUMP_MAGIC_LL			0xb1b23c4d	/* Internal dummy just for mapping */
 #define KUZNETZOV_TCPDUMP_MAGIC			0xa1b2cd34
 #define BORKMANN_TCPDUMP_MAGIC			0xa1e2cb12
 
@@ -78,6 +80,20 @@ struct pcap_pkthdr_ns {
 	uint32_t len;
 };
 
+struct pcap_pkthdr_ll {
+	struct pcap_timeval ts;
+	uint32_t caplen;
+	uint32_t len;
+	struct pcap_ll ll;
+};
+
+struct pcap_pkthdr_ns_ll {
+	struct pcap_timeval_ns ts;
+	uint32_t caplen;
+	uint32_t len;
+	struct pcap_ll ll;
+};
+
 struct pcap_pkthdr_kuz {
 	struct pcap_timeval ts;
 	uint32_t caplen;
@@ -99,21 +115,27 @@ struct pcap_pkthdr_bkm {
 };
 
 typedef union {
-	struct pcap_pkthdr	ppo;
-	struct pcap_pkthdr_ns	ppn;
-	struct pcap_pkthdr_kuz	ppk;
-	struct pcap_pkthdr_bkm	ppb;
-	uint8_t			raw;
+	struct pcap_pkthdr		ppo;
+	struct pcap_pkthdr_ns		ppn;
+	struct pcap_pkthdr_ll		ppo_ll;
+	struct pcap_pkthdr_ns_ll	ppn_ll;
+	struct pcap_pkthdr_kuz		ppk;
+	struct pcap_pkthdr_bkm		ppb;
+	uint8_traw;
 } pcap_pkthdr_t;
 
 enum pcap_type {
 	DEFAULT		  =	ORIGINAL_TCPDUMP_MAGIC,
 	NSEC		  =	NSEC_TCPDUMP_MAGIC,
+	DEFAULT_LL	  =	ORIGINAL_TCPDUMP_MAGIC_LL,
+	NSEC_LL		  =	NSEC_TCPDUMP_MAGIC_LL,
 	KUZNETZOV	  =	KUZNETZOV_TCPDUMP_MAGIC,
 	BORKMANN	  =	BORKMANN_TCPDUMP_MAGIC,
 
 	DEFAULT_SWAPPED	  =	___constant_swab32(ORIGINAL_TCPDUMP_MAGIC),
 	NSEC_SWAPPED	  =	___constant_swab32(NSEC_TCPDUMP_MAGIC),
+	DEFAULT_LL_SWAPPED =	___constant_swab32(ORIGINAL_TCPDUMP_MAGIC_LL),
+	NSEC_LL_SWAPPED	  =	___constant_swab32(NSEC_TCPDUMP_MAGIC_LL),
 	KUZNETZOV_SWAPPED =	___constant_swab32(KUZNETZOV_TCPDUMP_MAGIC),
 	BORKMANN_SWAPPED  =	___constant_swab32(BORKMANN_TCPDUMP_MAGIC),
 };
@@ -244,6 +266,19 @@ static inline int pcap_devtype_to_linktype(int dev_type)
 	}
 }
 
+static inline bool link_has_sll_hdr(uint32_t link_type)
+{
+	switch (link_type) {
+	case LINKTYPE_NETLINK:
+	case LINKTYPE_LINUX_SLL:
+	case ___constant_swab32(LINKTYPE_NETLINK):
+	case ___constant_swab32(LINKTYPE_LINUX_SLL):
+		return true;
+	default:
+		return false;
+	}
+}
+
 static inline int pcap_dev_to_linktype(const char *ifname)
 {
 	return

Re: [netsniff-ng] [PATCH 0/2 v2] Add suport for Linux cooked header sniff dissect

[Daniel Borkmann]

On 06/18/2015 11:49 AM, Daniel Borkmann wrote:

On 06/18/2015 11:42 AM, Vadim Kochan wrote:

On Thu, Jun 18, 2015 at 10:45:29AM +0200, Daniel Borkmann wrote:

I've started splitting this into smaller digestible chunks,
couple of more evenings and it should be done from my side.
Thanks for your patience.


But implementation now is OK in the last original patch series v2 ?


The only missing piece from your first patch I need to go through
is the pcap i/o parts. I wanted to check if two new dummy types
are possible (where we'd have transparent mapping - so it would be
semantically the same as this patch), whether they result in a
smaller code diff and would have lesser impact on the fast path.
If that's the case, I'd go for that, if not I will take the current
remaining piece. Will let you know.


Okay, here it goes. I tested this with capturing from netsniff-ng,
reading via Wireshark and capturing from Wireshark and reading via
netsniff-ng.

Seems fine, please double check it.

All in all this should have less overhead and result in smaller
code diff. Apart from that, the pcap_io.h might need some cleanups
anyway.

[PATCH] pcap_io: add cooked mode support

Originally submitted by Vadim in a different form, he wrote:

  Use Linux cooked header for Netlink interface automatically or
  as replacement of L2 header if --cooked option is specified:

http://www.tcpdump.org/linktypes/LINKTYPE_LINUX_SLL.html

  'Cooked headers' makes sense to use for default or nsec pcap
  types which does not contain protocol info.

  Added new LINKTYPE_LINUX_SLL which indicates pcap file with
  Linux cooked header as L2 layer header. This pcap file is
  compatible with Wireshark's cooked header  vice-versa.

Signed-off-by: Vadim Kochan vadi...@gmail.com
Signed-off-by: Daniel Borkmann dan...@iogearbox.net
---
 netsniff-ng.c |  17 ++
 pcap_io.h | 192 +++---
 2 files changed, 172 insertions(+), 37 deletions(-)

diff --git a/netsniff-ng.c b/netsniff-ng.c
index 0a9c620..e593b9d 100644
--- a/netsniff-ng.c
+++ b/netsniff-ng.c
@@ -1519,6 +1519,23 @@ int main(int argc, char **argv)

if (!ctx.link_type)
ctx.link_type = pcap_dev_to_linktype(ctx.device_in);
+   if (link_has_sll_hdr(ctx.link_type)) {
+   switch (ctx.magic) {
+   case ORIGINAL_TCPDUMP_MAGIC:
+   ctx.magic = ORIGINAL_TCPDUMP_MAGIC_LL;
+   break;
+   case NSEC_TCPDUMP_MAGIC:
+   ctx.magic = NSEC_TCPDUMP_MAGIC_LL;
+   break;
+   case ___constant_swab32(ORIGINAL_TCPDUMP_MAGIC):
+   ctx.magic = 
___constant_swab32(ORIGINAL_TCPDUMP_MAGIC_LL);
+   break;
+   case ___constant_swab32(NSEC_TCPDUMP_MAGIC):
+   ctx.magic = 
___constant_swab32(NSEC_TCPDUMP_MAGIC_LL);
+   break;
+   }
+   }
+

if (!ctx.device_out) {
ctx.dump = 0;
diff --git a/pcap_io.h b/pcap_io.h
index 497e453..5beccf9 100644
--- a/pcap_io.h
+++ b/pcap_io.h
@@ -27,6 +27,8 @@
 #define TCPDUMP_MAGIC  0xa1b2c3d4
 #define ORIGINAL_TCPDUMP_MAGIC TCPDUMP_MAGIC
 #define NSEC_TCPDUMP_MAGIC 0xa1b23c4d
+#define ORIGINAL_TCPDUMP_MAGIC_LL  0xb1b2c3d4  /* Internal 
dummy just for mapping */
+#define NSEC_TCPDUMP_MAGIC_LL  0xb1b23c4d  /* Internal 
dummy just for mapping */
 #define KUZNETZOV_TCPDUMP_MAGIC0xa1b2cd34
 #define BORKMANN_TCPDUMP_MAGIC 0xa1e2cb12

@@ -78,6 +80,20 @@ struct pcap_pkthdr_ns {
uint32_t len;
 };

+struct pcap_pkthdr_ll {
+   struct pcap_timeval ts;
+   uint32_t caplen;
+   uint32_t len;
+   struct pcap_ll ll;
+};
+
+struct pcap_pkthdr_ns_ll {
+   struct pcap_timeval_ns ts;
+   uint32_t caplen;
+   uint32_t len;
+   struct pcap_ll ll;
+};
+
 struct pcap_pkthdr_kuz {
struct pcap_timeval ts;
uint32_t caplen;
@@ -99,21 +115,27 @@ struct pcap_pkthdr_bkm {
 };

 typedef union {
-   struct pcap_pkthdr  ppo;
-   struct pcap_pkthdr_ns   ppn;
-   struct pcap_pkthdr_kuz  ppk;
-   struct pcap_pkthdr_bkm  ppb;
-   uint8_t raw;
+   struct pcap_pkthdr  ppo;
+   struct pcap_pkthdr_ns   ppn;
+   struct pcap_pkthdr_ll   ppo_ll;
+   struct pcap_pkthdr_ns_llppn_ll;
+   struct pcap_pkthdr_kuz  ppk;
+   struct pcap_pkthdr_bkm  ppb;
+   uint8_t raw;
 } pcap_pkthdr_t;

 enum pcap_type {
DEFAULT   = ORIGINAL_TCPDUMP_MAGIC,
NSEC  = NSEC_TCPDUMP_MAGIC

Re: [netsniff-ng] [PATCH 0/2 v2] Add suport for Linux cooked header sniff dissect

[Daniel Borkmann]

On 06/19/2015 10:58 PM, Vadim Kochan wrote:

On Fri, Jun 19, 2015 at 10:47:49PM +0200, Daniel Borkmann wrote:

...

@@ -737,6 +837,24 @@ static inline void pcap_validate_header(const struct 
pcap_filehdr *hdr)
if (unlikely(hdr-version_minor != PCAP_VERSION_MINOR) 
 ___constant_swab16(hdr-version_minor) != 
PCAP_VERSION_MINOR)
panic(This file has an invalid pcap minor version (must be 
%d)\n, PCAP_VERSION_MINOR);
+
+   /* Remap to internal *_LL types in case of LINKTYPE_LINUX_SLL. */
+   if (linktype == LINKTYPE_LINUX_SLL) {

should not link_has_sll_hdr() be used here ?


Why? At this point, when reading a pcap file, basic validation has already
been done and linktype is in host endianess. In netsniff-ng, we write out
the pcap header as LINKTYPE_LINUX_SLL in case of Netlink.

In case some other source doesn't use LINKTYPE_LINUX_SLL but LINKTYPE_NETLINK,
we actually need not to assume cooked format.

--
You received this message because you are subscribed to the Google Groups 
netsniff-ng group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [netsniff-ng] mausezahn: TX fast path ... or may be rewrite everything ?

[Daniel Borkmann]

On 06/18/2015 03:24 PM, Vadim Kochan wrote:

On Thu, Jun 18, 2015 at 11:07:12AM +0200, Daniel Borkmann wrote:

On 06/18/2015 10:57 AM, Vadim Kochan wrote:
...

Not yet, but I will try keep thinking about this, and will let know if I
will have some real ideas regarding implementation.


Sounds good!


Some 1st though which I came up is may be to start extend current
trafgen generation api set:

{
 eth
 {
 src(),
dst(),
proto(),
 }
 ,
 ip
 {
 ver(),
src(),
dst(),
opt(0x1, 20),
opt(0x2, 30),
 },
 tcp
 {
 dport(),
sport(),
flags(),
 },
}


Can you elaborate on your idea, what would this config do? I.e. if src()
etc is empty.

Thanks,
Daniel

--
You received this message because you are subscribed to the Google Groups 
netsniff-ng group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [netsniff-ng] [PATCH 0/2 v2] Add suport for Linux cooked header sniff dissect

[Daniel Borkmann]

I've started splitting this into smaller digestible chunks,
couple of more evenings and it should be done from my side.
Thanks for your patience.

--
You received this message because you are subscribed to the Google Groups 
netsniff-ng group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [netsniff-ng] mausezahn: TX fast path ... or may be rewrite everything ?

[Daniel Borkmann]

On 06/18/2015 01:30 AM, Vadim Kochan wrote:

Hi,

I tried to look over mz tool and try to start adding TX fast path as it
is done in trafgen but I am not sure it will be easy, for me it seems the
code is not easy to change.

I have no some objective arguments but I feel that may be it would be
good to start something like pktgen-ng or start writing mz from empty
page ?

What do you think ?


I think in general, it would be useful to 'merge' mz into trafgen.
No doubt it's quite some work, but that would allow us to have a
clean, fresh implementation at some point. By 'merge', I mean to
have mostly compatible syntax/features, but the underlying
implementation would be from scratch. I.e. to let trafgen have an
interactive mode, so you can have it as a shell on an appliance,
and for quick tests, the direct cmdline mode from mz. Do you have
any plans?

Thanks,
Daniel

--
You received this message because you are subscribed to the Google Groups 
netsniff-ng group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [netsniff-ng] mausezahn: TX fast path ... or may be rewrite everything ?

[Daniel Borkmann]

On 06/18/2015 10:57 AM, Vadim Kochan wrote:
...

Not yet, but I will try keep thinking about this, and will let know if I
will have some real ideas regarding implementation.


Sounds good!

--
You received this message because you are subscribed to the Google Groups 
netsniff-ng group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [netsniff-ng] [PATCH 2/3] netsniff-ng: Add support of Linux cooked header

[Daniel Borkmann]

On 06/13/2015 02:30 PM, Vadim Kochan wrote:

From: Vadim Kochan vadi...@gmail.com

Use Linux cooked header for Netlink interface automatically
of as replavement of L2 header by specifiyng --cooked option.

 http://www.tcpdump.org/linktypes/LINKTYPE_LINUX_SLL.html

cooked header makes sense to use for default or nsec pcap types
which does not contain protocol info.

Added new LINKTYPE_LINUX_SLL which indicates pcap file with
Linux cooked header as L2 layer header.This pcap file is
compatible with Wireshark's cooked header  vice-versa.

Signed-off-by: Vadim Kochan vadi...@gmail.com


First, patch 3/3 looks okay. Will apply it once we have this
part resolved.

Then, I've worked through this patch finally. I had it applied
locally to fix up some minor things (i.e. some places don't have
proper Linux kernel coding style, etc), but still have some
thoughts here:

Anyway, what I don't yet like is that we need to tough every pcap
backend layer. I.e. the pcap i/o layer was designed in mind that
we could easily add other formats without having to touch mm/sg/rw
methods. Have you tested all of these with all corner cases?

I.e. why not letting  pcap_get_hdr_length() handle the additional
sll size in order to avoid putting upper layer logic into the
backend section (it should be decoupled from this, also for
performance reasons)?

That also leaves me wondering, if it would be easier/possible to
add two internal/pseudp pcap magic types that just are pcap-us + sll
and pcap-ns + sll format, and since this number is unknown to others,
we'd need an automatic detection, that would remap the default to
the cooked type magics. Perhaps that could shrink the extra fast-path
logic we need to handle this (it would just need to have some extra
logic with the pcap file header itself, the rest might only be the
mapping of pkt-hdr, but no extra backend changes would be needed)?

What do you think?

Thanks,
Daniel

...

diff --git a/pcap_mm.c b/pcap_mm.c
index f7b248e..b4bbae4 100644
--- a/pcap_mm.c
+++ b/pcap_mm.c
@@ -56,8 +56,18 @@ static ssize_t pcap_mm_write(int fd, pcap_pkthdr_t *phdr, 
enum pcap_type type,
if ((off_t) (ptr_va_curr - ptr_va_start) + hdrsize + len  map_size)
__pcap_mmap_write_need_remap(fd);

-   fmemcpy(ptr_va_curr, phdr-raw, hdrsize);
+   fmemcpy(ptr_va_curr, phdr-h.raw, hdrsize);
ptr_va_curr += hdrsize;
+
+   if (pcap_pkthdr_has_ll(phdr, type)) {
+   size_t ll_len = sizeof(struct pcap_pkthdr_ll);
+
+   fmemcpy(ptr_va_curr, phdr-ll_hdr, ll_len);
+   ptr_va_curr += ll_len;
+   len -= ll_len;
+   hdrsize += ll_len;
+   }
+
fmemcpy(ptr_va_curr, packet, len);
ptr_va_curr += len;

@@ -72,8 +82,22 @@ static ssize_t pcap_mm_read(int fd __maybe_unused, 
pcap_pkthdr_t *phdr,
if (unlikely((off_t) (ptr_va_curr + hdrsize - ptr_va_start)  (off_t) 
map_size))
return -EIO;

-   fmemcpy(phdr-raw, ptr_va_curr, hdrsize);
+   fmemcpy(phdr-h.raw, ptr_va_curr, hdrsize);
ptr_va_curr += hdrsize;
+
+   if (pcap_pkthdr_has_ll(phdr, type)) {
+   size_t ll_len = sizeof(struct pcap_pkthdr_ll);
+
+   if (unlikely((off_t) (ptr_va_curr + ll_len - ptr_va_start) 
+   (off_t) map_size))
+   return -EIO;
+
+   fmemcpy(phdr-ll_hdr, ptr_va_curr, ll_len);
+   ptr_va_curr += hdrsize;
+   hdrsize += ll_len;
+   len -= ll_len;
+   }
+
hdrlen = pcap_get_length(phdr, type);

if (unlikely((off_t) (ptr_va_curr + hdrlen - ptr_va_start)  (off_t) 
map_size))
diff --git a/pcap_rw.c b/pcap_rw.c
index b6be922..651cef7 100644
--- a/pcap_rw.c
+++ b/pcap_rw.c
@@ -21,12 +21,25 @@ static ssize_t pcap_rw_write(int fd, pcap_pkthdr_t *phdr, 
enum pcap_type type,
 const uint8_t *packet, size_t len)
  {
ssize_t ret, hdrsize = pcap_get_hdr_length(phdr, type), hdrlen = 0;
+   ssize_t ll_len = 0;

-   ret = write_or_die(fd, phdr-raw, hdrsize);
+   ret = write_or_die(fd, phdr-h.raw, hdrsize);
if (unlikely(ret != hdrsize))
panic(Failed to write pkt header!\n);

+   if (pcap_pkthdr_has_ll(phdr, type)) {
+   ll_len = sizeof(struct pcap_pkthdr_ll);
+   int ret_ll = write_or_die(fd, (uint8_t *)phdr-ll_hdr, ll_len);
+
+   if (unlikely(ret_ll != sizeof(phdr-ll_hdr)))
+   panic(Failed to write pkt header!\n);
+   }
+
hdrlen = pcap_get_length(phdr, type);
+   hdrlen  -= ll_len;
+   len -= ll_len;
+   hdrsize += ll_len;
+
if (unlikely(hdrlen != (ssize_t) len))
return -EINVAL;

@@ -41,12 +54,25 @@ static ssize_t pcap_rw_read(int fd, pcap_pkthdr_t *phdr, 
enum pcap_type type,
uint8_t *packet, size_t len)
  {

Re: [netsniff-ng] [PATCH 1/3] dev: Add device string convertions (addr, dev type)

[Daniel Borkmann]

On 06/13/2015 02:30 PM, Vadim Kochan wrote:

From: Vadim Kochan vadi...@gmail.com

Move device string convertions funcs (device_type2str, device_addr2str)
from proto_nlmsg.c to dev.c to use them in other modules.

Signed-off-by: Vadim Kochan vadi...@gmail.com


Patch 1/3 applied with minor modifications, thanks!

--
You received this message because you are subscribed to the Google Groups 
netsniff-ng group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [netsniff-ng] slow packet rate on a pair of ports

[Daniel Borkmann]

On 06/06/2015 01:05 AM, Perf Tech wrote:

I have a PC with  3 NICs (eth0 is Intel 82579LM NIC, eth1 is 82574L), OS is
ubuntu 12.04 server.
I connected eth0 and eth1 with a ethernet cable and both ports are up.

Trying to find out how fast trafgen(0.5.9-rc4) can send packets on this
system.

# ./trafgen --dev eth0 --conf tmp2.txt  --cpp
  4 packets to schedule
292 bytes in total
Running! Hang up with ^C!


I am monitoring ifconfig eth0 with a monitoring program that I have used
many years, the issue is,  trafgen is only sending
pcap at a rate 129000 packets/sec with data rate of 79Mbps.   I had
expected it to be a long higher.


What kernel and NIC driver do you use? Did you do any particular tuning
settings?

E.g. a starting point could be:

http://netoptimizer.blogspot.ch/2014/09/network-setup-for-accurate-nanosec.html

Also, if your config uses checksumming, it's done in trafgen in software,
so if you want to test a simple minimal config, you could use a static buffer
as a start.



Did I run the command wrong?

Thanks



--
You received this message because you are subscribed to the Google Groups 
netsniff-ng group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Fwd: [netsniff-ng] error when running trafgen for the first time.

[Daniel Borkmann]

On 06/05/2015 09:58 AM, Vadim Kochan wrote:
...

May be you should add --cpp option ?

$./trafgen --dev eth0 --conf tmp2.txt --rand --gap 1000us --cpp


Yes, that's correct. Thanks for the quick reply, Vadim!

--
You received this message because you are subscribed to the Google Groups 
netsniff-ng group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [netsniff-ng] [PATCH] netsniff-ng nlmsg: Dissect rtnl route type messages

[Daniel Borkmann]

On 06/04/2015 09:34 PM, Vadim Kochan wrote:

From: Vadim Kochan vadi...@gmail.com

Dump RTnetlink routing related info with attributes.

Signed-off-by: Vadim Kochan vadi...@gmail.com


Applied, thanks Vadim.

Please in future don't reuse the exact same patch subject resp.
commit message from some of your earlier work, since people going
through the git log might otherwise not identify the right commit
immediately if they are debugging, etc.

--
You received this message because you are subscribed to the Google Groups 
netsniff-ng group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Fwd: [netsniff-ng] error when running trafgen for the first time.

[Daniel Borkmann]

On 06/06/2015 12:16 AM, Vadim Kochan wrote:
...

Do you have cpp installed on your machine?


I think in that case there could be a message like:
sh: cpp: command not found
Failed to invoke C preprocessor!


Yes, agreed. Want to send a patch? ;)

--
You received this message because you are subscribed to the Google Groups 
netsniff-ng group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [netsniff-ng] [PATCH] netsniff-ng: Add support of pcap cooked header

[Daniel Borkmann]

On 06/02/2015 10:44 AM, Vadim Kochan wrote:

On Tue, Jun 02, 2015 at 10:36:12AM +0200, Daniel Borkmann wrote:

On 06/01/2015 05:43 PM, Vadim Kochan wrote:

On Mon, Jun 01, 2015 at 05:14:10PM +0200, Daniel Borkmann wrote:

On 06/01/2015 04:23 PM, Vadim Kochan wrote:
...

I checked that in this case Ethernet  Netlink frames will be mixed in the
pcap file, but netsniff-ng tries to identify all the frames as Ethernet,
and in this case frames is not wrapped with cooked header.


Right, so for that use case, it wouldn't work currently. I'm thinking that
in case of 'any' device *and* if there's at least one device present at the
time we bind the socket to any that would have netlink type, then we would
need to turn on cooked mode generally, so that in such environments Ethernet
and Netlink would have both cooked headers.

I think that would be useful to adapt the patch towards covering such use
cases as well since people might also like to correlate i.e. Netlink events
together with network packets. Could you please respin the patch towards
supporting this, too? Then I think, it should be ready to go in.

...


Then I think link type can be set as LINKTYPE_LINUX_SLL in the pcap
header. May be it would be useful to use LINKTYPE_LINUX_SLL by default
if 'any' was specified or only if the Netlink device present in UP state ?


I'm thinking that the following might be useful for the pcap writer:
Have an option --cooked that generally adds cooked header (no matter
what the underlying device is, so could be 'any' as well). Whenever
we bound our socket specifically to a nlmon device, then we enable
that option transparently in the case of default/ns pcap.


So, the solution might be - use cooked header automatically if only
nlmon device was specified or when --cooked option was specified ?


Yes, I would go for that as a start.

--
You received this message because you are subscribed to the Google Groups 
netsniff-ng group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [netsniff-ng] curvetun logfile?

[Daniel Borkmann]

Hi Benjamin,

Sorry for the late reply, this slipped through. :/

On 04/23/2015 05:21 PM, Benjamin Henrion wrote:

Hi,

Would it be possible to have an option to curvetun to log to a logfile?

Curvetun sends to syslog, but on my system I want to avoid having a
syslog daemon.

I have made an init script and /etc/default/curvetun for booting it on
debian wheezy, if you are interested.


If generic, we could add that to the repository.


On that topic, I don't know if you could accept a debian/ dir for
debian packaging.


For debian packaging related improvements, I'd rather be happy if you
work towards upstreaming that directly into Debian (Cc'ing Kartik, who
is the Debian maintainer).

Wrt your question:

In curvetun's daemonize(), we hijack stdout and stderr to redirect
normal printf() output into syslog in daemon mode. You could specify
an option to have an alternative log file, I wouldn't mind. Could you
send a patch?

Thanks,
Daniel


The debian parallel repo:

https://anonscm.debian.org/git/collab-maint/netsniff-ng.git

Does not have the latest releases.

Best,

--
Benjamin Henrion bhenrion at ffii.org
FFII Brussels - +32-484-566109 - +32-2-4148403
In July 2005, after several failed attempts to legalise software
patents in Europe, the patent establishment changed its strategy.
Instead of explicitly seeking to sanction the patentability of
software, they are now seeking to create a central European patent
court, which would establish and enforce patentability rules in their
favor, without any possibility of correction by competing courts or
democratically elected legislators.



--
You received this message because you are subscribed to the Google Groups 
netsniff-ng group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [netsniff-ng] [RFC] netsniff-ng: Add support of pcap cooked header

[Daniel Borkmann]

On 05/10/2015 02:37 PM, Vadim Kochan wrote:

From: Vadim Kochan vadi...@gmail.com

Since Netlink messages are written in default pcap format there is no
way to identify Netlink family by socket protocol number, so
use pcap cooked header for Netlink messages as described here:

http://www.tcpdump.org/linktypes/LINKTYPE_NETLINK.html

Signed-off-by: Vadim Kochan vadi...@gmail.com


Thanks for looking into this, Vadim!

One preliminary question: you would only set that for netlink or
also for other protocols?

The basic issue is that normal pcap hdr in front of every captured
packet looses this information for the dissector, but alternative
formats such as kuznet/netsniff-ng format still have it preserved,
but are not supported by tools like wireshark.

Ok. Thinking out loud, what if we add another pcap type as an option
to choose?

--
You received this message because you are subscribed to the Google Groups 
netsniff-ng group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [netsniff-ng] multiple instances of netsniff-ng with AF_PACKET hash fanout

[Daniel Borkmann]

On 05/07/2015 02:32 AM, Michał Purzyński wrote:

I used 3.13 for testing. It's in Ubuntu as HWE stack.


So, following commit was added v3.0-rc4-846-gdc99f60 ...

commit dc99f600698dcac69b8f56dda9a8a00d645c5ffc
Author: David S. Miller da...@davemloft.net
Date:   Tue Jul 5 01:45:05 2011 -0700

packet: Add fanout support.

... but would be great if you have the chance to try something more recent
as Michal pointed out.


On May 7, 2015 2:29 AM, arse...@gmail.com wrote:


Hi all,


The qm won't work in your case since your kernel is too old


$ uname -a
Linux 3.2.0-74-generic #109-Ubuntu SMP Tue Dec 9 16:45:49 UTC 2014 x86_64
x86_64 x86_64 GNU/Linux

Do I need newer kernel ? 3.2.0-82-generic is available.


The only issue (as I don't know what traffic you're seeing) could be

that  your rxhash always falls into fanout member with that prefix,
strange.


I would need to run more isolated test because I capture gigabytes of 

data, but I can tell by size of capture files that I don't get all data 
since

my 1 minute files are lot smaller than single file I get when capturing
w/o fanout feature.



Does the above example work for you on loopback? If you have two

instances  in the same

group outputting to a normal pcap file with rnd, do you both files get 

written?

For rnd and qm, I get Cannot set fanout ring mode! when trying to use Lo
or physical NIC eth5 in my case.


Ok, I see. Since your kernel doesn't support that.


For hash mode results in only last instance file being written. Tried lo
and eth5.


And that does also happen for rr/lb mode (round robin), right? Even if you
remove the --out and --silent, etc?

One terminal:  netsniff-ng --fanout-group 1 --fanout-type rr --in lo
Another:   netsniff-ng --fanout-group 1 --fanout-type rr --in lo

And then ping 127.0.0.1 ? In any case, I'd recommend trying out a newer
kernel (yours is roughly 4 years old).

Cheers,
Daniel

--
You received this message because you are subscribed to the Google Groups 
netsniff-ng group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [netsniff-ng] multiple instances of netsniff-ng with AF_PACKET hash fanout

[Daniel Borkmann]

On 05/07/2015 04:47 PM, arse...@gmail.com wrote:
...

I am very sorry but it looks like this was working all the time at least in 
couple of modes but I messed up ls command and didn't see other files !


Ok, no problem. It's good that we now have it verified from a couple of people
that it works fine. :))

Cheers  thanks,
Daniel

--
You received this message because you are subscribed to the Google Groups 
netsniff-ng group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [netsniff-ng] multiple instances of netsniff-ng with AF_PACKET hash fanout

[Daniel Borkmann]

Sorry for the late answer.

On 05/01/2015 02:53 PM, arse...@gmail.com wrote:
...

Interesting, lets say if you use --fanout-type lb, would that make a difference?

This should round robin each packet between these 3 processes, so you should
definitely see something unless we have a bug. ;) Let me know, so we can look
further ...


no change if using --fanout-type lb. I see all 3 instances run and show similar 
CPU % in top, but only last file seems to be on my SSD.
BTW, --fanout-type qm returns $ Cannot set fanout ring mode!, that's why I 
switched to type cpu. ( considering I use irq affinity script to lock Rx queues to 
certain CPUs and also I lock netsniff-ng, I guess it doesn't matter which one I use cpu 
or qm )


The qm won't work in your case since your kernel is too old.

So I was just using the following:

netsniff-ng --fanout-group 1 --fanout-type rnd --fanout-opts defrag --ring-size 
128MiB --bind-cpu 0 --notouch-irq --silent \
--in lo --out /tmp/netcapture/ --prefix a. --interval 60sec  

netsniff-ng --fanout-group 1 --fanout-type rnd --fanout-opts defrag --ring-size 
128MiB --bind-cpu 1 --notouch-irq --silent \
--in lo --out /tmp/netcapture/ --prefix b. --interval 60sec  

netsniff-ng --fanout-group 1 --fanout-type rnd --fanout-opts defrag --ring-size 
128MiB --bind-cpu 2 --notouch-irq --silent \
--in lo --out /tmp/netcapture/ --prefix c. --interval 60sec  

And doing a ping 127.0.0.1, gives me pcap files in /tmp/netcapture/ with all 
three {a,b,c}.
prefixes; plus after 60sec they start a new one.

I've also tried two instances dumping to two different files with various 
fanout-types,
they seem to work as expected. That means that fanout in general seems to 
function.

The only issue (as I don't know what traffic you're seeing) could be that your 
rxhash
always falls into fanout member with that prefix, strange.

Does the above example work for you on loopback? If you have two instances in 
the same
group outputting to a normal pcap file with rnd, do you both files get written?

--
You received this message because you are subscribed to the Google Groups 
netsniff-ng group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [netsniff-ng] [PATCH] netsniff-ng: Do not use O_NOATIME when read pcap

[Daniel Borkmann]

On 05/06/2015 07:36 PM, Vadim Kochan wrote:

From: Vadim Kochan vadi...@gmail.com

It fixes the case when user made pcap file in sudo
mode but after it should still use sudo to read it
because of setting O_NOATIME option requires higher
privileges.

Signed-off-by: Vadim Kochan vadi...@gmail.com
---
  netsniff-ng.c | 2 +-
  1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/netsniff-ng.c b/netsniff-ng.c
index dfab81a..d426af6 100644
--- a/netsniff-ng.c
+++ b/netsniff-ng.c
@@ -570,7 +570,7 @@ static void read_pcap(struct ctx *ctx)
if (ctx-pcap == PCAP_OPS_MM)
ctx-pcap = PCAP_OPS_SG;
} else {
-   fd = open_or_die(ctx-device_in, O_RDONLY | O_LARGEFILE | 
O_NOATIME);
+   fd = open_or_die(ctx-device_in, O_RDONLY | O_LARGEFILE);


Okay, I see, thanks for looking into this, Vadim!

Could you rather make the test optional? I.e. if normal open(2) with O_NOATIME
fails due to bad permissions, fall back to open_or_die(ctx-device_in, O_RDONLY 
|
O_LARGEFILE). I think that would be better.

Thanks,
Daniel

--
You received this message because you are subscribed to the Google Groups 
netsniff-ng group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [netsniff-ng] multiple instances of netsniff-ng with AF_PACKET hash fanout

[Daniel Borkmann]

On 05/06/2015 11:52 PM, Daniel Borkmann wrote:

Sorry for the late answer.

On 05/01/2015 02:53 PM, arse...@gmail.com wrote:
...

Interesting, lets say if you use --fanout-type lb, would that make a difference?

This should round robin each packet between these 3 processes, so you should
definitely see something unless we have a bug. ;) Let me know, so we can look
further ...


no change if using --fanout-type lb. I see all 3 instances run and show similar 
CPU % in top, but only last file seems to be on my SSD.
BTW, --fanout-type qm returns $ Cannot set fanout ring mode!, that's why I 
switched to type cpu. ( considering I use irq affinity script to lock Rx queues to 
certain CPUs and also I lock netsniff-ng, I guess it doesn't matter which one I use cpu 
or qm )


The qm won't work in your case since your kernel is too old.

So I was just using the following:

netsniff-ng --fanout-group 1 --fanout-type rnd --fanout-opts defrag --ring-size 
128MiB --bind-cpu 0 --notouch-irq --silent \
 --in lo --out /tmp/netcapture/ --prefix a. --interval 60sec  

netsniff-ng --fanout-group 1 --fanout-type rnd --fanout-opts defrag --ring-size 
128MiB --bind-cpu 1 --notouch-irq --silent \
 --in lo --out /tmp/netcapture/ --prefix b. --interval 60sec  

netsniff-ng --fanout-group 1 --fanout-type rnd --fanout-opts defrag --ring-size 
128MiB --bind-cpu 2 --notouch-irq --silent \
 --in lo --out /tmp/netcapture/ --prefix c. --interval 60sec  

And doing a ping 127.0.0.1, gives me pcap files in /tmp/netcapture/ with all 
three {a,b,c}.
prefixes; plus after 60sec they start a new one.

I've also tried two instances dumping to two different files with various 
fanout-types,
they seem to work as expected. That means that fanout in general seems to 
function.

The only issue (as I don't know what traffic you're seeing) could be that your 
rxhash
always falls into fanout member with that prefix, strange.

Does the above example work for you on loopback? If you have two instances in 
the same
group outputting to a normal pcap file with rnd, do you both files get written?


Also fanout-type by hash on physical interface looks correct in my case, f.e.:

netsniff-ng --fanout-group 2 --fanout-type hash --fanout-opts defrag 
--ring-size 128MiB --bind-cpu 0 --notouch-irq --silent \
--in wlp2s0b1 --out /tmp/netcapture/ --prefix a. --interval 60sec  


netsniff-ng --fanout-group 2 --fanout-type hash --fanout-opts defrag 
--ring-size 128MiB --bind-cpu 1 --notouch-irq --silent \
--in wlp2s0b1 --out /tmp/netcapture/ --prefix b. --interval 60sec  


netsniff-ng --fanout-group 2 --fanout-type hash --fanout-opts defrag 
--ring-size 128MiB --bind-cpu 2 --notouch-irq --silent \
--in wlp2s0b1 --out /tmp/netcapture/ --prefix c. --interval 60sec  


E.g. if I'm running ping 8.8.8.8, then that /only/ lands with some other stuff 
in c's pcap,
which is as expected.

Cheers,
Daniel

--
You received this message because you are subscribed to the Google Groups 
netsniff-ng group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [netsniff-ng] multiple instances of netsniff-ng with AF_PACKET hash fanout

[Daniel Borkmann]

On 05/07/2015 12:14 AM, Michał Purzyński wrote:

That were my results as well.

It would be worthwhile to investigate how the hash is generated in the
kernel. I remember, somewhere in the original patch that introduced
this functionality, developers decided to let the hardware generate
rxhash and pass it to user space to avoid L1 cache miss.

As we all know, 82599 hashing is no symmetric. But in my testing both
a-b and b-a traffic goes correctly to the same cluster. Maybe the
hardware hash is not used at all?


Depends, for example, in ixgbe which I have a couple of, you can see in
the driver that in ixgbe_process_skb_fields() the rxhash can be copied
from hw ring descriptor into skb_set_hash() [1] (only l3). Quite a number
of drivers support offload for that.

At latest, if your nic doesn't support l4 hash, then in pf_packet when
doing the fanout, skb_get_hash() [2] gets called to build one with the
kernel's flow dissector in sw; l4 in order to get a bit more entropy.

That's why queue mapping (qm) can be less costly in some cases, for example.

In any case, it also doesn't really matter if 'hash' is used, even 'rr' or
'rnd' should populate other pcaps, which I find strange that apparently it
was not the case by Ivan. Works from my side. Would be good if you could
clarify the questions from the other mail, Ivan.


I will dig deeper.


  [1] 
http://lingrok.org/xref/linux-net-next/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c#1593
  [2] http://lingrok.org/xref/linux-net-next/net/packet/af_packet.c#1285

--
You received this message because you are subscribed to the Google Groups 
netsniff-ng group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [netsniff-ng] [PATCH] netsniff-ng: Fix capturing extra packets from other dev

[Daniel Borkmann]

On 05/05/2015 12:10 PM, Daniel Borkmann wrote:

On 05/05/2015 11:26 AM, Tobias Klauser wrote:

On 2015-05-02 at 22:54:50 +0200, Vadim Kochan vadi...@gmail.com wrote:

On Sat, May 02, 2015 at 07:00:53PM +0200, Tobias Klauser wrote:

On 2015-04-29 at 21:18:24 +0200, Vadim Kochan vadi...@gmail.com wrote:

From: Vadim Kochan vadi...@gmail.com

It might be related to the issue #73 noticed by

Jon Schipp jonsch...@gmail.com

where he pointed that netsniff-ng captures some extra packets.

I observed this issue when I captured few Ethernet frames from
wireless device while I was sniffing Netlink monitor device (!!!),
especially under high load traffic (HD Video).

...

The better fix would just be to consolidate pf_socket() and pf_tx_socket()
and allocate both with socket(PF_PACKET, SOCK_RAW, 0). That way, we
should also be able to avoid the synchronize_net() barrier (as po-running
is false at bind time).

If we change pf_socket(), the only thing we need to make sure is that
all such sockets do a proper bind() call before starting to capture
packets, so that we don't break anything.


Vadim, feel free to follow-up on that with a fix based on above suggestion,
as you've already investigated and proposed a first patch.

Thanks,
Daniel

--
You received this message because you are subscribed to the Google Groups 
netsniff-ng group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [netsniff-ng] netsniff-ng: wireshark does not understand pcap file with Netlink frames

[Daniel Borkmann]

On 05/05/2015 12:59 PM, Vadim Kochan wrote:

Wireshark does not understand netsniff-ng's pcap file with Netlink
frames, I assume thats because W-shark expects that each Netlink frame
should have additional header on-top described here:

http://www.tcpdump.org/linktypes/LINKTYPE_NETLINK.html

it shows this is a Netlink type link but can't dissect Netlink frames.

Meanwhile I do not have a fix for this yet. Don't know if it is important
for this release.


Well, tcpdump has that type (nlmon) registered so far, that's more
important. ;)

Cheers,
Daniel

--
You received this message because you are subscribed to the Google Groups 
netsniff-ng group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [netsniff-ng] multiple instances of netsniff-ng with AF_PACKET hash fanout

[Daniel Borkmann]

Hi Ivan,

On 04/30/2015 05:28 PM, arse...@gmail.com wrote:

Hi all,

I have been using netsniff-ng for some time now and am very excited about 
packet fanout feature.


Cool, great to hear! :)


Have one AF_PACKET hash fanout functionality related question if somebody has 
time to comment :

how can I get 3 or more netsniff-ng instances in one fanout-group output into 1 
single PCAP file ?


You below command-line invocation looks good to me. Letting all processes
write into one single pcap file at one, I'm afraid, won't work. There are
various reasons, i.e. it would corrupt the pcap file as there's no
synchronization between the processes to write a single packet atomically
into the pcap.

You also wouldn't want to do that. ;) Because assume if such a possibility
would exist, then the bottleneck becomes easily the write to disc on that
single file.

You rather want to have parallelism all the way to the hardware in the best
case. If you need to merge file, there could f.e. be a background process
grabbing individual pcap files and merge them based on the time-stamps into
a single one, e.g. mergecap:

  https://www.wireshark.org/docs/wsug_html_chunked/AppToolsmergecap.html

Hope that helps,

Thanks,
Daniel


So far, I have tried to start 3 instances with :

sudo nohup /usr/local/sbin/netsniff-ng --fanout-group 1 --fanout-type hash --mmap --ring-size 
256MiB --bind-cpu 18 --silent --in eth5 --out /mnt/sdb1/netcapture/ --prefix 
eth5. --interval 60sec  
sudo nohup /usr/local/sbin/netsniff-ng --fanout-group 1 --fanout-type hash --mmap --ring-size 
256MiB --bind-cpu 20 --silent --in eth5 --out /mnt/sdb1/netcapture/ --prefix 
eth5. --interval 60sec  
sudo nohup /usr/local/sbin/netsniff-ng --fanout-group 1 --fanout-type hash --mmap --ring-size 
256MiB --bind-cpu 22 --silent --in eth5 --out /mnt/sdb1/netcapture/ --prefix 
eth5. --interval 60sec  

However, since interval isn't exactly 60 seconds, after 1-2 days I end up with 
separate output files, like :

-rw-r--r-- 1 root root  135M Apr 30 14:44 
/mnt/sdb1/netcapture/eth5.1430405040.pcap
-rw-r--r-- 1 root root  125M Apr 30 14:44 
/mnt/sdb1/netcapture/eth5.1430405041.pcap
$ tcpslice /mnt/sdb1/netcapture/eth5.1430404980.pcap -t
/mnt/sdb1/netcapture/eth5.1430404980.pcap   2015y04m30d14h43m00s733651u 
2015y04m30d14h44m00s742344u
$ tcpslice /mnt/sdb1/netcapture/eth5.1430404981.pcap -t
/mnt/sdb1/netcapture/eth5.1430404981.pcap   2015y04m30d14h43m01s118241u 
2015y04m30d14h44m01s138441u

Am I doing something wrong the way I start instances, is there different way to 
start 3 instances to write into single output pcap file ?

Also, I was wondering if there are any plans to make command line ability to 
start multiple instances using something like one command line with --bind-cpu 
18,20,22 and one --out file, which would trigger 3 netsniff-ng instances while 
output goes into single output pcap file ? ( SolarCapture/SolarFlare uses that 
approach with multiple capture cores and one writeout core )

Let me know if you need more details.

Best Regards

Ivan



--
You received this message because you are subscribed to the Google Groups 
netsniff-ng group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [netsniff-ng] netsniff-ng: Do we need to set IO prio when do only read from pcap ?

[Daniel Borkmann]

On 04/22/2015 05:09 PM, Vadim Kochan wrote:
...

Sure, I will try to fix it, really I dont have a fix yet. The issue goes
from pcap ops in init one function, where IO prio is set, the first
think which came up in my mind is to have separate pcap ops for read
only where set IO prio will be not called ... but I am not sure if it is
correct.


I think we have two options:

1) Don't panic on error, but perhaps just throw a warning to the user
   instead in case of insufficient permissions.

2) Change init_once_pcap() into void (*init_once_pcap)(bool enforce_prio);
   and push down the enforcement into the various functions, and inside
   read_pcap(), you'd set __pcap_io-init_once_pcap(false). And in case
   the enforcement is not set, we just move on silently.

Perhaps 2nd option is cleaner?

--
You received this message because you are subscribed to the Google Groups 
netsniff-ng group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [netsniff-ng] [PATCH] netsniff-ng: Dont set IO prio when reading pcap file

[Daniel Borkmann]

On 04/22/2015 06:26 PM, Vadim Kochan wrote:

From: Vadim Kochan vadi...@gmail.com

It allows to read pcap file for users who have no
permissions to set process IO prio.

Signed-off-by: Vadim Kochan vadi...@gmail.com


I'm okay with that, read_pcap() is slow-path anyway.

Thanks

--
You received this message because you are subscribed to the Google Groups 
netsniff-ng group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [netsniff-ng] netsniff-ng: mac80211: Better print layout for HT Capabilities element ?

[Daniel Borkmann]

On 04/22/2015 11:50 PM, Vadim Kochan wrote:
...

I think that HT Capabilities element info should be showed in more structured 
view so
it will be more readable what do you think ?

Does someone have a better option ?


Yes, I'm fine with that.

--
You received this message because you are subscribed to the Google Groups 
netsniff-ng group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [netsniff-ng] [Re: multiple instances of netsniff-ng with AF_PACKET hash fanout] Initial implementation of a multiprocess functionality.

[Daniel Borkmann]

Hi Michal,

On 04/14/2015 09:53 AM, Daniel Borkmann wrote:

Thanks Michal, looks almost ready to go in!


Okay, I just fixed everything up and pushed it out under your
authorship:

  
https://github.com/netsniff-ng/netsniff-ng/commit/f00d4d54f28c0374cc57e6ca07dd648d7684c69c

We can still fix things up later on if needed.

Cheers  thanks,
Daniel

--
You received this message because you are subscribed to the Google Groups 
netsniff-ng group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [netsniff-ng] [PATCH] mac80211: Check existing of generated monX device

[Daniel Borkmann]

Hi Vadim,

On 04/17/2015 09:04 PM, Vadim Kochan wrote:

From: Vadim Kochan vadi...@gmail.com

Fixed case when netsniff fails if there is already existing monX device
while generating.

Signed-off-by: Vadim Kochan vadi...@gmail.com

...

+int device_ifindex(const char *ifname)
+{
+   int index = device_ifindex_get(ifname);
+   if (unlikely(index = 0))


This test should be  0 only as ifindex 0 would mean to
capture on any device.

So starting netsniff-ng w/o any arguments would not work
anymore otherwise, fix up:

  
https://github.com/netsniff-ng/netsniff-ng/commit/42ca7e42aa76ee52499ae82370d11d044e811f35


+   panic(Cannot get ifindex from device!\n);
+
+   return index;
+}


Cheers,
Daniel

--
You received this message because you are subscribed to the Google Groups 
netsniff-ng group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [netsniff-ng] [Re: multiple instances of netsniff-ng with AF_PACKET hash fanout] Initial implementation of a multiprocess functionality.

[Daniel Borkmann]

On 04/21/2015 01:13 PM, Michał Purzyński wrote:

Thanks a lot, I was trying to find a time for it over the weekend,
turned out to be difficult so decided to work on it next weekend :-)


No problem. ;) Thanks for looking into this!


I'll focus on some good documentation next, showing how to use it and when, etc.


I think that would be useful. If you think putting this into an extra
file e.g. called SCALING or somewhere appropriate into the netsniff-ng
man-page, I'm totally fine with it.

I think one option would be to make a description on all the tuning knobs
and the new fanout option to provide a small guide for users with such
demand.

Cheers  thanks,
Daniel

--
You received this message because you are subscribed to the Google Groups 
netsniff-ng group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [netsniff-ng] [Re: multiple instances of netsniff-ng with AF_PACKET hash fanout] Initial implementation of a multiprocess functionality.

[Daniel Borkmann]

Thanks Michal, looks almost ready to go in!

Other than Tobias' excellent feedback, some minor things from my side:
On 04/14/2015 01:10 AM, Michal Purzynski wrote:

---
  netsniff-ng.c | 28 +---
  ring_rx.c | 17 -
  ring_rx.h |  2 +-
  3 files changed, 42 insertions(+), 5 deletions(-)

diff --git a/netsniff-ng.c b/netsniff-ng.c
index dfb99bb..c603919 100644
--- a/netsniff-ng.c
+++ b/netsniff-ng.c
@@ -60,12 +60,13 @@ struct ctx {
bool randomize, promiscuous, enforce, jumbo, dump_bpf, hwtimestamp, 
verbose;
enum pcap_ops_groups pcap; enum dump_mode dump_mode;
uid_t uid; gid_t gid; uint32_t link_type, magic;
+uint32_t cluster_id, cluster_type;


I know you have changed the naming from the command line site,
but please also rename this internally, e.g. fanout_group, fanout_type.


  };

  static volatile sig_atomic_t sigint = 0;
  static volatile bool next_dump = false;

-static const char *short_options = 
d:i:o:rf:MNJt:S:k:n:b:HQmcsqXlvhF:RGAP:Vu:g:T:DBU;
+static const char *short_options = 
d:i:o:rf:MNJt:S:k:n:b:HQmcsqXlvhF:RGAP:Vu:g:T:DBU:C:K:;
  static const struct option long_options[] = {
{dev,   required_argument,  NULL, 'd'},
{in,required_argument,  NULL, 'i'},
@@ -81,6 +82,8 @@ static const struct option long_options[] = {
{user,  required_argument,  NULL, 'u'},
{group, required_argument,  NULL, 'g'},
{magic, required_argument,  NULL, 'T'},
+   {fanout-group,  required_argument,  NULL, 'C'},
+   {fanout-type,required_argument, NULL, 'K'},


Looks like we run out of short option chars, I guess C/K is fine.


{rand,  no_argument,NULL, 'r'},
{rfraw, no_argument,NULL, 'R'},
{mmap,  no_argument,NULL, 'm'},

...

diff --git a/ring_rx.c b/ring_rx.c
index 8ad64d1..c97dd2d 100644
--- a/ring_rx.c
+++ b/ring_rx.c
@@ -209,9 +209,23 @@ static void alloc_rx_ring_frames(int sock, struct ring 
*ring)
  rx_ring_get_size(ring, v3));
  }

+void create_cluster(int sock, uint32_t cluster_id, uint32_t cluster_mode)


create_fanout_instance() or something along that line.


+{
+uint32_t cluster_option = 0;
+int ret = 0;
+
+if (cluster_mode == PACKET_FANOUT_HASH)
+cluster_mode = PACKET_FANOUT_HASH | PACKET_FANOUT_FLAG_DEFRAG;
+else
+cluster_option = (cluster_id | (cluster_mode  16));
+ret = setsockopt(sock, SOL_PACKET, PACKET_FANOUT,(void *)cluster_option, 
sizeof(cluster_option));
+if (ret  0)
+panic(Cannot set fanout ring mode!\n);
+}
+
  void ring_rx_setup(struct ring *ring, int sock, size_t size, int ifindex,
   struct pollfd *poll, bool v3, bool jumbo_support,
-  bool verbose)
+  bool verbose, uint32_t cluster_id, uint32_t cluster_type)
  {
fmemset(ring, 0, sizeof(*ring));
setup_rx_ring_layout(sock, ring, size, jumbo_support, v3);
@@ -220,6 +234,7 @@ void ring_rx_setup(struct ring *ring, int sock, size_t 
size, int ifindex,
alloc_rx_ring_frames(sock, ring);
bind_ring_generic(sock, ring, ifindex, false);
prepare_polling(sock, poll);
+create_cluster(sock, cluster_id, cluster_type);


As Tobias mentioned, this should only be enabled if the user specifies
at least fanout_group over the command line. fanout_type, if not specified,
could be on default hash.


  }

  void sock_rx_net_stats(int sock, unsigned long seen)
diff --git a/ring_rx.h b/ring_rx.h
index edd0feb..ae2cd76 100644
--- a/ring_rx.h
+++ b/ring_rx.h
@@ -13,7 +13,7 @@

  extern void ring_rx_setup(struct ring *ring, int sock, size_t size, int 
ifindex,
  struct pollfd *poll, bool v3, bool jumbo_support,
- bool verbose);
+ bool verbose, uint32_t cluster_id, uint32_t 
cluster_type);
  extern void destroy_rx_ring(int sock, struct ring *ring);
  extern void sock_rx_net_stats(int sock, unsigned long seen);




--
You received this message because you are subscribed to the Google Groups 
netsniff-ng group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [netsniff-ng] multiple instances of netsniff-ng with AF_PACKET hash fanout

[Daniel Borkmann]

On 04/11/2015 06:59 PM, Michał Purzyński wrote:

OK, try one. I'm ready to accept heavy artillery fire ;-) Man, it
takes a while to find a free letter for getopt.


Two new parameters were added:
-C cluster id with integer that represents the socket fanout group
identifier and must be shared between all processes in the group
-K hash/lb/cpu/rnd - the type of fanout. The only really useful here
is hash because it preserves flows. If it is choosen, kernel side
defragmentation is enabled as well (fragments would have different
hash).


I'd prefer if we name it fanout-type and fanout-group perhaps,
that way it's clear that we mean the packet-sockets fanout mechanism.
What do you think?

Other than that, there's a couple of more fanout disciplines:

  http://lingrok.org/xref/linux-net-next/net/packet/af_packet.c#1371

So it would be: hash/lb/cpu/rnd/qm/roll

I think we might need to describe the exact meaning of that in the
--help option. But I'm totally fine if we do that as follow-up.


Now, kernel does not allow to choose what we are hashing on, and it
seems to be 4-tuple.


With QM (queue-mapping), you can just use the NICs HW queue flow
steering for the fanout group distribution.


I tested it with lb and hash cluster types and everything worked. The


That's great.


lb cluster type is useless (as is anything that is not hash but
given how advanced the nesniff-ng software is, someone might find it
useful and it's just a few lines more.


Sure, if we add it, we might also want to give the user a choice. I
think that hash/cpu/qm and roll (== moves to the next fanout socket
after the first's queue is full) might be useful.

--
You received this message because you are subscribed to the Google Groups 
netsniff-ng group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [netsniff-ng] trafgen seems broken on kernel 3.19

[Daniel Borkmann]

On 03/23/2015 11:37 AM, Vadim Kochan wrote:

On Mon, Mar 23, 2015 at 11:25:26AM +0100, Lorenzo Pistone wrote:

...

Thats what I got on 3.18 with the same cfg file:

$ trafgen/trafgen -c ~/trafgen.cfg -o wlp3s0 -n 1
  4 packets to schedule
168 bytes in total
Running! Hang up with ^C!


0 packets outgoing
0 bytes outgoing
0 sec, 0 usec on CPU0 (0 packets)
0 sec, 0 usec on CPU1 (0 packets)
0 sec, 0 usec on CPU2 (0 packets)
0 sec, 0 usec on CPU3 (0 packets)

Seems packet was not sent from trafgen ?


If you increase -n e.g. to 64 or omit -n entirely, does that work?
Looks like a trafgen bug.

--
You received this message because you are subscribed to the Google Groups 
netsniff-ng group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [netsniff-ng] trafgen seems broken on kernel 3.19

[Daniel Borkmann]

Hi Lorenzo,

On 03/22/2015 03:13 PM, Lorenzo Pistone wrote:

Hi,
I'm trying to send UDP packets with zero length withthis simple configuration 
on trafgen:

{
   # --- ethernet header ---
   0xbe, 0x15, 0x1d, 0x12, 0x1c, 0x57,  # mac destination
   0xfa, 0x16, 0x3e, 0xa0, 0x5d, 0x18,  # mac source
   const16(0x0800), # protocol
   # --- ip header ---
   # ipv4 version (4-bit) + ihl (4-bit), tos
   0b01000101, 0,
   # ipv4 total len
   const16(28),
   # id (note: runtime dynamic random)
   drnd(2),
   # ipv4 3-bit flags + 13-bit fragment offset
   # 001 = more fragments
   0b0100, 0,
   64, # ttl
   17, # proto udp
   # dynamic ip checksum (note: offsets are zero indexed)
   csumip(14, 33),
   92, 222, 69, 15, # source ip
   85, 214, 106, 103, # dest ip
   # --- udp header ---
   # as this is a fragment the below stuff does not matter too much
   const16(48054), # src port
   const16(28785), # dst port
   const16(8),# udp length
   # udp checksum can be dyn calc via csumudp(offset ip, offset tcp)
   # which is csumudp(14, 34), but for udp its allowed to be zero
   csumudp(14, 34),
}

I can send these packets on lo, but trafgen fails to send anything on real 
devices. I used both the TX_RING and sendto method.

 I've tried with iwlwifi and virtio. mausezahn on the contrary works. I 
checked with strace and there does not seem to be any
 call that returns an error.

Thanks for the bug report.

Can you provide some more debug information?

How do you invoke trafgen? strace -f? Something suspicious in dmesg? ip link?

Cheers,
Daniel

--
You received this message because you are subscribed to the Google Groups 
netsniff-ng group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [netsniff-ng] trafgen seems broken on kernel 3.19

[Daniel Borkmann]

On 03/23/2015 03:39 PM, Lorenzo Pistone wrote:

I'm checking with netsniff-ng, I use the same interface with which I'm sending 
this email


Ok, well that doesn't work. If you emit packets with trafgen, it
uses by default a path that bypasses the traffic control layer.

If you really want to see with netsniff-ng or tcpdump what has been
emitted, you need to run trafgen with --qdisc-path .

For stress testing you could use ifpps to display current stats,
but those are rather global ones.

--
You received this message because you are subscribed to the Google Groups 
netsniff-ng group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [netsniff-ng] trafgen seems broken on kernel 3.19

[Daniel Borkmann]

On 03/23/2015 03:24 PM, Lorenzo Pistone wrote:

I don't get the same message (obviously) with not -n but -t, however still no 
packet gets out.


Well, with -t you'll be using sendto(), so that's expected.

So let me ask a stupid question, assuming you don't use -n and therefore
not hit this rounding bug, and you still cannot send a packet out ... is
your link from the device actually up (ip link)?

Are you perhaps checking with tcpdump if a packet has been sent out?


Il 23/03/2015 15:23, Lorenzo Pistone ha scritto:

So I tried now without -n, still absolutely no packet out on real devices (contrary to 
lo), but when interrupting I get Cannot destroy the TX_RING: Device or resource 
busy! for each CPU (so 4 times in my case). Looks like a start.

Il 23/03/2015 10:27, Daniel Borkmann ha scritto:

Hi Lorenzo,

On 03/22/2015 03:13 PM, Lorenzo Pistone wrote:

Hi,
I'm trying to send UDP packets with zero length withthis simple configuration 
on trafgen:

{
   # --- ethernet header ---
   0xbe, 0x15, 0x1d, 0x12, 0x1c, 0x57,  # mac destination
   0xfa, 0x16, 0x3e, 0xa0, 0x5d, 0x18,  # mac source
   const16(0x0800), # protocol
   # --- ip header ---
   # ipv4 version (4-bit) + ihl (4-bit), tos
   0b01000101, 0,
   # ipv4 total len
   const16(28),
   # id (note: runtime dynamic random)
   drnd(2),
   # ipv4 3-bit flags + 13-bit fragment offset
   # 001 = more fragments
   0b0100, 0,
   64, # ttl
   17, # proto udp
   # dynamic ip checksum (note: offsets are zero indexed)
   csumip(14, 33),
   92, 222, 69, 15, # source ip
   85, 214, 106, 103, # dest ip
   # --- udp header ---
   # as this is a fragment the below stuff does not matter too much
   const16(48054), # src port
   const16(28785), # dst port
   const16(8),# udp length
   # udp checksum can be dyn calc via csumudp(offset ip, offset tcp)
   # which is csumudp(14, 34), but for udp its allowed to be zero
   csumudp(14, 34),
}

I can send these packets on lo, but trafgen fails to send anything on real 
devices. I used both the TX_RING and sendto method.

 I've tried with iwlwifi and virtio. mausezahn on the contrary works. I 
checked with strace and there does not seem to be any
 call that returns an error.

Thanks for the bug report.

Can you provide some more debug information?

How do you invoke trafgen? strace -f? Something suspicious in dmesg? ip link?

Cheers,
Daniel






--
You received this message because you are subscribed to the Google Groups 
netsniff-ng group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [netsniff-ng] [PATCH] netsniff-ng: Move proto dissectors to separate dir

[Daniel Borkmann]

On 01/28/2015 11:15 PM, Vadim Kochan wrote:

From: Vadim Kochan vadi...@gmail.com

There might be more proto dissectors which
will make root src directory huge, hence move
them to the separate 'dissect' dir.

Signed-off-by: Vadim Kochan vadi...@gmail.com


For now I'd prefer them actually where they are, this also
makes it easier for some distros to cherry-pick individual
patches for backporting.

Thanks,
Daniel

--
You received this message because you are subscribed to the Google Groups 
netsniff-ng group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [netsniff-ng] tcpdump capture delays and dropped packets

[Daniel Borkmann]

On 07/18/2014 03:49 PM, Tobias Klauser wrote:

On 2014-07-18 at 15:09:55 +0200, 'peter' via netsniff-ng 
netsniff-ng@googlegroups.com wrote:

hi,


im looking for a way to automate packet generation for traffic
shaper testing.
found a testfile online and modified it a bit, only
source/destination ip are valid for the interface ...the rest is
from the sourcefile so there are wrong mac addresses and so on.
for traffic shaping testing (HFSC) i only need a mix of small / big
packets that i can capture and graph.
its udp, its 322bytes long (according to trafgen output).


when i run:
trafgen --cpp --dev wlan0 --conf packets/322bytes -t 10ms -n 10

and:
tcpdump -i wlan0 port 


it shows 10 packets, each about 10ms apart from another but it takes
some seconds before the packets show up in tcpdump...why is that?


You are running this on the same machine? Note that trafgen (at
least recent versions) bypass the qdisc layer completely and thus
you *won't* be able to see the traffic from that looped back into
packet sockets (and thus libpcap/tcpdump).

Have you tried running tcpdump on a different machine (e.g. the
receiving end)?


Any specific reason you're using tcpdump instead of netsniff-ng? ;-)


running the same command with -n 100 i get:
31 packets captured
100 packets received by filter
69 packets dropped by kernel

when aborting tcpdump with ctrl+c.
why would the linux kernel drop packets? since they are 10ms apart
and 322bytes long it should manage?
it not like i send 1000 packets without time between sends or am i
hitting some limitation?


What does the trafgen config file look like? What kernel version are you
using? What driver for wlan0? What version of
netsniff-ng/trafgen/tcpdump?

Without this information it's gonna be rather difficult to pinpoint the
problem...

Cheers
Tobias



--
You received this message because you are subscribed to the Google Groups 
netsniff-ng group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [netsniff-ng] refine delay in mausezahn, or provide a pps/bitrate option

[Daniel Borkmann]

On 05/07/2014 02:51 PM, Lorenzo Pistone wrote:

Hello,
the delay (-d) option is not satisfactory with low delays. The unavoidable 
imprecision of the sleep time screws the actual pps a lot (see 
http://www.martani.net/2011/07/nanosleep-usleep-and-sleep-precision.html). I 
believe also that nanosleep would just sweep the problem under the carpet.
A proper fix would be to check what time it is after resuming from sleep, and 
then send packets according to time elapsed. Or one can implement busy waiting 
for low usleep times, which is my current dirty hack:


Sorry for the very late answer.

Could you submit this as a proper patch (or Github merge request)?

Thanks a lot.


diff --git a/staging/mz.h b/staging/mz.h
index cad091f..b77f6db 100644
--- a/staging/mz.h
+++ b/staging/mz.h
@@ -82,7 +82,20 @@ static inline void verbose_l2(const char *format,
...)
  #define MZ_DEFAULT_CONFIG_PATH /etc/mausezahn/   // see also
mz_default_config_path below
  #define MZ_DEFAULT_LOG_PATH /var/log/mausezahn/  // see also
mz_default_log_path below

-#define SLEEP usleep   // The sleep function to use.
Consider 'nanosleep' in future.
+static inline void mz_sleep(unsigned long long delay)
+{
+struct timespec t;
+if (delay  1000 || clock_gettime(CLOCK_MONOTONIC, t))
usleep(delay);
+else
+{
+#define tmicro (t.tv_nsec / 1000 + t.tv_sec * 100ULL)
+unsigned long long end = tmicro + delay;
+while (clock_gettime(CLOCK_MONOTONIC, t), tmicro  end)
sched_yield();
+#undef tmicro
+}
+}
+
+#define SLEEP mz_sleep   // The sleep function to use.
Consider 'nanosleep' in future.
  #define DEFAULT_DELAY 0
  #define PCAP_READ_TIMEOUT_MSEC 1   // The read timeout for
pcap_open_live()
  #define MZ_MAX_DEVICES 10  // Max number of network
devices supported



--
You received this message because you are subscribed to the Google Groups 
netsniff-ng group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [netsniff-ng] Need help Regarding mausezahn tool

[Daniel Borkmann]

On 03/29/2014 07:32 PM, mukul joshi wrote:

Hi,

I am working on one project where I am using mausezahn for packet
generation. I want to print the generated packet buffer (i.e. the whole
buffer which is delivered to the network interface) I want to provide this
packet buffer to another process which is running in the same system
through inter process communication mechanism. Is there any function
supported in mausezahn which will give me the pointer to packet buffer.


Not really, there's no such IPC included. If performance is not a concern,
you could capture the transmitted packet from there perhaps.

--
You received this message because you are subscribed to the Google Groups 
netsniff-ng group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [netsniff-ng] netsniff-ng

[Daniel Borkmann]

On 03/17/2014 06:21 AM, rukanth sameera wrote:

Hi

Hello, I am rukanth. I am very happy to say i like your netsniff-ng toolkit
and it is very useful to me. I have to ask question that is what are the
compatible hardware for netsniff-ng.


Generally, you need a Linux box and the more recent your kernel,
the better. So, the only requirement is that your hardware must
be supported by Linux. In terms of what hardware is good, I can
recommend you to go through the excellent article at [1][2] as
an example, which applies to most capturing applications.

 [1] http://www.ntop.org/pf_ring/not-all-servers-are-alike-with-dna/
 [2] http://www.ntop.org/pf_ring/not-all-servers-are-alike-with-dna-part-2/


Thank you very much.

Rukanth


--
You received this message because you are subscribed to the Google Groups 
netsniff-ng group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[netsniff-ng] [announce] netsniff-ng.git upstream url

[Daniel Borkmann]

We have decided to move the netsniff-ng upstream Git
repository to a new Github netsniff-ng organization:

Web:
  https://github.com/netsniff-ng/netsniff-ng

Git:
  git://github.com/netsniff-ng/netsniff-ng.git

Please update your Git origin to this location.

Thanks !

--
You received this message because you are subscribed to the Google Groups 
netsniff-ng group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [netsniff-ng] Almost kernel panic

[Daniel Borkmann]

On 02/17/2014 01:44 PM, Daniel Borkmann wrote:

On 02/16/2014 04:13 PM, Daniel Borkmann wrote:

On 02/16/2014 03:15 PM, Lorenzo Pistone wrote:

On 02/16/2014 02:00 PM, Daniel Borkmann wrote:


It's a kernel bug in the VM subsystem, dealing with transparent hugepages.

A patch was submitted but is so far not yet included, I'll ask for a status
update on lkml so that we get this sorted out.

Meanwhile you can compile your kernel with THP turned off.


Thanks for the info, I found the thread and I will follow it.
Does disabling thp at boot time or through sysctl work for you? It doesn't for 
me, and I'm not such a fan of kernel recompiling as pastime. Decently recent 
libpcap use the same mmap mechanism, but I see that you have the problem in 
allocating the TX buffer, which libpcap doesn't do, right?


It should actually also work on rx mmap'ed buffer, I'd expect libpcap
might not have this issue as it doesn't call m{un}lock() on THPs.

I just tested and resubmitted the proposed fix to lkml for inclusion;
kernel since 3.11 are affected.


Ok, patch just got queued for kernel + stable inclusion, so it should
hit your distro soon. ;)

--
You received this message because you are subscribed to the Google Groups 
netsniff-ng group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [netsniff-ng] netsniff-ng vs tshark

[Daniel Borkmann]

Hi Olivier,

On 02/11/2014 05:33 PM, Olivier Marce wrote:

Hi guys
thanks a lot for this splendid toolkit.

I got a strange (for me) behavior that I would like to share with you.
Platform Ubuntu 12.10

I have a WiFi interface named wlan5 and IP@ 192.168.1.5 that I ping from 
another machine. MAC@ of this interface is 60:67:20:b0:b5:48
I setup a monitor virtual interface mon5 (iw dev wlan5 interface mon5 mode 
monitor)

Then, I do not have the same results with netsniff-ng (0.5.8-rc5)  and tshark 
1.8.2 :


# netsniff-ng --in wlan5 -f ether dst 60:67:20:b0:b5:48


Would it work for you to use something like ...

# netsniff-ng --in wlan5 --rfraw -f ether dst 60:67:20:b0:b5:48

?

Usually --rfraw would be the option to enable monitor mode.


Running! Hang up with ^C!

 wlan5 98 1392136256s.25182523ns
  [ Eth MAC (b8:a3:86:96:a5:0d = 60:67:20:b0:b5:48), Proto (0x0800, IPv
  [snip]

but :

# netsniff-ng --in mon5 -f ether dst 60:67:20:b0:b5:48
Running! Hang up with ^C!
(... nothing...)

while :

# tshark -i mon5 -f ether dst 60:67:20:b0:b5:48
Capturing on mon5
   0.00  192.168.1.1 - 192.168.1.5  ICMP 134 Echo (ping) request 
id=0x0b7c, seq=838/17923, ttl=64

Is there a reason why netsniff-ng does not capture packet on monitor virtual 
interface ?

Best regards



Thanks,

Daniel

--
You received this message because you are subscribed to the Google Groups 
netsniff-ng group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [netsniff-ng] trafgen: Flushing TX_RING failed: No such device or address!

[Daniel Borkmann]

On 12/23/2013 08:06 PM, Robert Edmonds wrote:

Hi,

I'm seeing the error message Flushing TX_RING failed: No such device or
address! when I try to run trafgen on Debian kernel 3.12.6, but not on 3.11.
Here is the full trafgen output:


A PF_PACKET stable patch for the kernel is on it's way to address that issue.
I expect the next stable kernel or the one after that to contain this. In case
you need it earlier than that, feel free to backport [1] to your kernel.

 [1] 
https://git.kernel.org/cgit/linux/kernel/git/davem/net-next.git/commit/?id=66e56cd46b93ef407c60adcac62cf33b06119d50


 # trafgen -V -i ./trafgen.cfg -o eth1 -P 1 -n 1000
 [0] pkt
  len 85 cnts 0 rnds 6
  payload 00 25 90 86 bd d6 00 25 90 d9 dd 81 08 00 45 00 00 47 00 00 00 00 
40 11 00 00 c0 a8 00 01 c0 a8 00 35 00 00 00 35 00 33 00 00 00 00 01 00 00 01 
00 00 00 00 00 01 03 77 77 77 06 67 6f 6f 67 6c 65 03 63 6f 6d 00 00 01 00 01 
00 00 29 10 00 00 00 00 00 00 00
  rnd0 off 18
  rnd1 off 19
  rnd2 off 34
  rnd3 off 35
  rnd4 off 42
  rnd5 off 43
  1 packets to schedule
 85 bytes in total
 Running! Hang up with ^C!

 No kernel support for PACKET_QDISC_BYPASS (kernel  3.14?)
 TX,V2: 238.41 MiB, 122064 Frames, each 2048 Byte allocated
 Flushing TX_RING failed: No such device or address!

And the relevant strace output:

bind(3, {sa_family=AF_PACKET, proto=, if3, pkttype=PACKET_HOST, addr(0)={0, 
}, 20) = 0
setgid(0)   = 0
setuid(0)   = 0
setitimer(ITIMER_REAL, {it_interval={0, 10}, it_value={0, 10}}, NULL) = 0
--- SIGALRM (Alarm clock) @ 0 (0) ---
sendto(3, NULL, 0, MSG_DONTWAIT, NULL, 0) = -1 ENXIO (No such device or address)



--
You received this message because you are subscribed to the Google Groups 
netsniff-ng group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [netsniff-ng] netsniff-ng takes a long time to start up?

[Daniel Borkmann]

On 12/20/2013 04:21 AM, Robert Edmonds wrote:

Hi,

I'm trying to figure out why netsniff-ng takes a long time to start up
on one of my machines.  I'm running the latest git checkout on Debian
unstable (running the Debian 3.11.6 kernel), and when I run:

netsniff-ng --silent -i eth1 -o /dev/null

I see a delay of about 15 seconds before the Running! Hang up with ^C!
message is printed.  Looking at netsniff-ng with strace, I see the
following:

[...]
1387508695.193460 setsockopt(3, SOL_PACKET, 0x11 /* PACKET_??? */, [64], 4) = 0
1387508695.193526 setsockopt(3, SOL_PACKET, PACKET_VERSION, [2], 4) = 0
1387508695.193587 getsockopt(3, SOL_PACKET, PACKET_VERSION, [2], [4]) = 0
1387508695.193650 setsockopt(3, SOL_PACKET, PACKET_RX_RING, 
\0\0\1\0\346\16\0\0\0\0\1\0\346\16\0\0d\0\0\0\0\0\0\0\0\0\0\0, 28) = 0
1387508709.738025 mmap(NULL, 249954304, PROT_READ|PROT_WRITE, 
MAP_SHARED|MAP_POPULATE|MAP_LOCKED, 3, 0) = 0x7f13cba7e000
1387508709.747174 getsockopt(3, SOL_PACKET, PACKET_VERSION, [2], [4]) = 0
1387508709.747347 bind(3, {sa_family=AF_PACKET, proto=0x03, if3, 
pkttype=PACKET_HOST, addr(0)={0, }, 20) = 0
[...]

Note that the PACKET_RX_RING setsockopt() call takes about 15 seconds to
complete.  During this time I see the netsniff-ng process consuming
about 90% of a CPU, and according to a perf record / perf report run
most of the time appears to be spent in these functions in the kernel:

  31.29%  netsniff-ng  [kernel.kallsyms]  [k] get_pageblock_flags_group
  26.69%  netsniff-ng  [kernel.kallsyms]  [k] isolate_freepages_block
  24.22%  netsniff-ng  [kernel.kallsyms]  [k] isolate_migratepages_range
   2.41%  netsniff-ng  [kernel.kallsyms]  [k] compaction_alloc
   0.99%  netsniff-ng  [kernel.kallsyms]  [k] __wake_up_bit
   0.95%  netsniff-ng  [kernel.kallsyms]  [k] __reset_isolation_suitable
   0.85%  netsniff-ng  [kernel.kallsyms]  [k] free_pcppages_bulk
   0.80%  netsniff-ng  [kernel.kallsyms]  [k] release_pages
   0.76%  netsniff-ng  [kernel.kallsyms]  [k] __pagevec_lru_add_fn
   0.62%  netsniff-ng  [kernel.kallsyms]  [k] page_waitqueue
   0.61%  netsniff-ng  [kernel.kallsyms]  [k] migrate_pages
   0.58%  netsniff-ng  [kernel.kallsyms]  [k] move_to_new_page
   0.54%  netsniff-ng  [kernel.kallsyms]  [k] mutex_lock

Help?  It seems to be related to the size of the RX ring buffer -- if I
specify -S 1MiB to netsniff-ng it starts up quickly.  But I also have
another machine where netsniff-ng starts up quickly with the default
ring buffer size.


Thanks for the report. On what kind of hardware are you trying to do that?

I think we need to adjust default buffer allocation anyway, probably you're
perfectly fine with a ring buffer of around 2MB already.


I can send the full strace output and perf.data files if needed.



--
You received this message because you are subscribed to the Google Groups 
netsniff-ng group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [netsniff-ng] netsniff-ng takes a long time to start up?

[Daniel Borkmann]

On 12/20/2013 07:08 PM, Robert Edmonds wrote:

Robert Edmonds wrote:

The funny thing is, I have a similar machine where netsniff-ng starts up
instantly.  It seems like a kernel issue, so I will try updating the
problematic machine's kernel.  I can also replicate the issue on the
problematic machine with tcpdump by passing a large -B parameter to
tcpdump.


Very interesting.  I've updated my machine to this kernel:

Linux chase 3.11-2-amd64 #1 SMP Debian 3.11.10-1 (2013-12-04) x86_64 GNU/Linux

And the problem has disappeared.


Seems like a bug in kernel's VM layer that probably didn't get into
stable or so.

--
You received this message because you are subscribed to the Google Groups 
netsniff-ng group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [netsniff-ng] Cross-compiling netsniff-ng with Mausezahn

[Daniel Borkmann]

On 12/17/2013 10:34 AM, Drasko DRASKOVIC wrote:

Hi all,
I am trying to compile netsniff-ng with Mausezahn under Yocto (Open
Embedded),

So far I have this output of configure:
...
[!] The following tools will *not* be built: curvetun flowtop astraceroute
mausezahn
[*] The following tools will be built: netsniff-ng trafgen ifpps bpfc
...

Why these package will *not* be built, and how to force their compilation?

Output of ./confure run in Yocto is presented below.

Best regards,
Drasko



+ do_configure

+ ./configure

./configure: line 21: [: too many arguments

[*] Checking pkg-config ...
[YES]

[*] Checking ccache ...
[NO]

[*] Checking flex ...
[YES]

[*] Checking bison ...
[YES]

[*] Checking nacl ...
[NO]

[*] Checking libnl ...
[YES]

[*] Checking tpacket_v2 ...
+ cd
/home/drasko/yocto/build/tmp/work/armv7ahf-vfp-neon-poky-linux-gnueabi/netsniff-ng-git-r0/git
+ sstate_create_package

+ cd
/home/drasko/yocto/build/tmp/work/armv7ahf-vfp-neon-poky-linux-gnueabi/netsniff-ng-git-r0/sstate-build-populate-lic/
++ mktemp
/home/drasko/yocto/build/sstate-cache/af/sstate-netsniff-ng-armv7ahf-vfp-neon-poky-linux-gnueabi-git-r0-armv7ahf-vfp-neon-2-af9f0408718c53f8993de471628ad9bd_populate-lic.tgz.

+
TFILE=/home/drasko/yocto/build/sstate-cache/af/sstate-netsniff-ng-armv7ahf-vfp-neon-poky-linux-gnueabi-git-r0-armv7ahf-vfp-neon-2-af9f0408718c53f8993de471628ad9bd_populate-lic.tgz.5c6aTDix

++ ls -A

+ '[' netsniff-ng ']'

+ tar -czf
/home/drasko/yocto/build/sstate-cache/af/sstate-netsniff-ng-armv7ahf-vfp-neon-poky-linux-gnueabi-git-r0-armv7ahf-vfp-neon-2-af9f0408718c53f8993de471628ad9bd_populate-lic.tgz.5c6aTDix
netsniff-ng

+ chmod 0664
/home/drasko/yocto/build/sstate-cache/af/sstate-netsniff-ng-armv7ahf-vfp-neon-poky-linux-gnueabi-git-r0-armv7ahf-vfp-neon-2-af9f0408718c53f8993de471628ad9bd_populate-lic.tgz.5c6aTDix

+ mv
/home/drasko/yocto/build/sstate-cache/af/sstate-netsniff-ng-armv7ahf-vfp-neon-poky-linux-gnueabi-git-r0-armv7ahf-vfp-neon-2-af9f0408718c53f8993de471628ad9bd_populate-lic.tgz.5c6aTDix
/home/drasko/yocto/build/sstate-cache/af/sstate-netsniff-ng-armv7ahf-vfp-neon-poky-linux-gnueabi-git-r0-armv7ahf-vfp-neon-2-af9f0408718c53f8993de471628ad9bd_populate-lic.tgz

+ cd
/home/drasko/yocto/build/tmp/work/armv7ahf-vfp-neon-poky-linux-gnueabi/netsniff-ng-git-r0

+ rm -rf
/home/drasko/yocto/build/tmp/work/armv7ahf-vfp-neon-poky-linux-gnueabi/netsniff-ng-git-r0/sstate-build-populate-lic/


[YES]

[*] Checking tpacket_v3 ...
[YES]

[*] Checking libnetfilter-conntrack ...
[NO]

[*] Checking ncurses ...
[YES]

[*] Checking libGeoIP ...
[NO]

[*] Checking libz ...
[YES]

[*] Checking liburcu ...
[YES]

[*] Checking libpcap ...
[YES]

[*] Checking hw timestamping ...
[YES]

[*] Checking libcli ...
[NO]

[*] Checking libnet ...
[NO]

[*] Generating config.h ...

[!] Some libraries or header definitions are missing or too old. Thus
 certain tools will not be built (see below). Please refer to the
 INSTALL file for the libraries needed to build the complete
 netsniff-ng toolkit.

[!] The NaCl crypto library is currently not present on your system or
 could not be found. Either install it from your distro or build it
 manually using 'make nacl' and make sure that the NACL_INC_DIR and
 NACL_LIB_DIR environment variables are set appropriately.
[!] The following tools will *not* be built: curvetun flowtop astraceroute
mausezahn


For curvetun, you need libnacl, which the script detected you have not.

For mausezahn libnet, libcli.

For flowtop, you need libgeoip and libnetfilter-conntrack.

For astraceroute you need to install libgeoip.


[*] The following tools will be built: netsniff-ng trafgen ifpps bpfc
[!] There were errors in the configure script. Please check the file
 config.log for details.



--
You received this message because you are subscribed to the Google Groups 
netsniff-ng group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [netsniff-ng] Re: Netsniff-NG in Applied NSM Book

[Daniel Borkmann]

On 12/17/2013 06:31 AM, Jon Schipp wrote:

Also, page 108 has a small section on installing and using ifpps.



That's awesome, thanks for the great news!


On Thu, Dec 12, 2013 at 11:21 PM, Jon Schipp jonsch...@gmail.com wrote:


FYI:

The Applied NSM book was released today [1]. From the table of contents
there's a section on netsniff-ng for full content packet capture .[2]

[1] http://www.appliednsm.com/applied-nsm-released/
[2]
http://www.amazon.com/Applied-Network-Security-Monitoring-Collection/dp/0124172083/

Thanks
--
Jon Schipp,
jonschipp.com, sickbits.net







--
You received this message because you are subscribed to the Google Groups 
netsniff-ng group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [netsniff-ng] netsniff-ng and ERSPAN

[Daniel Borkmann]

On 10/22/2013 11:36 AM, Doug Burks wrote:

Hi Daniel,

I'm referring to running netsniff-ng, having it accept ERSPAN data,
and write the decapsulated data to a pcap file.


So far there are no such plans, as we do not treat what we get from the
kernel differently. We can certainly open a ticket to not forget about
this idea.


Thanks,
Doug

On Tue, Oct 22, 2013 at 4:23 AM, Daniel Borkmann dbork...@redhat.com wrote:

On 10/21/2013 05:00 PM, Doug Burks wrote:


Hello all,

Have you considered implementing support for decoding ERSPAN?  Looks
like gulp and snort currently support this:



Are you referring to the packet dissector or to store the decapsulated
data to a pcap file?



http://staff.washington.edu/corey/gulp/
http://blog.snort.org/2013/07/snort-295-is-now-available.html

Thanks for your consideration!









--
You received this message because you are subscribed to the Google Groups 
netsniff-ng group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [netsniff-ng] netsniff-ng and ERSPAN

[Daniel Borkmann]

On 10/21/2013 05:00 PM, Doug Burks wrote:

Hello all,

Have you considered implementing support for decoding ERSPAN?  Looks
like gulp and snort currently support this:


Are you referring to the packet dissector or to store the decapsulated
data to a pcap file?


http://staff.washington.edu/corey/gulp/
http://blog.snort.org/2013/07/snort-295-is-now-available.html

Thanks for your consideration!



--
You received this message because you are subscribed to the Google Groups 
netsniff-ng group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [netsniff-ng] bpfc-vhdl

[Daniel Borkmann]

Sorry for the late answer,

On 09/19/2013 09:12 PM, rometor...@gmail.com wrote:

Hi All,

I am looking on implementing HW filters using bpfc.


Very cool! BPF engine offloading into a NIC might be interesting!


This is a process outline

1) Generate BPF file in human readable format - How to use BPFC to generate 
human readable format(labels etc.)..?


What do you define as human readable here? Is the bpfc asm-like language
sufficient as a starting point for doing the programming, or are you thinking
of something C-like?


2) parse and check for (if any backward jumps and correct the sequence)


Done in bpfc and the Linux kernel.


3) Generate VHDL state machine

Also,i was not able to generate JIT or C o/p using bpfc..any help is more than 
welcome


BPF JIT compiler is in the Linux kernel itself. I'm not sure if we're talking
about various things here? :)

--
You received this message because you are subscribed to the Google Groups 
netsniff-ng group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [netsniff-ng] Make error: installing netsniff-ng-0.571 on ubuntu 9.10 Karmic Kaola

[Daniel Borkmann]

On 09/23/2013 02:58 PM, Daniel Borkmann wrote:

On 09/23/2013 06:26 AM, onubogu Last Nameokey wrote:


Hi group members,
Please can someone help me debug this cmake and make error. I am trying to 
install netsniff-ng-0.571 on ubuntu 9.10 Karmic Kaola.I have downloaded 
(manually and using apt-get) all the packages and their dependencies but I am 
still getting this error. I look forward to hear from someone soon.


Have you tried the Git version?

git clone git://github.com/borkmann/netsniff-ng.git
cd netsniff-ng
./configure
make [...]


With that at least, you can avoid building curvetun as it seems your
linux version does not support UDP corking ...

--
You received this message because you are subscribed to the Google Groups 
netsniff-ng group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [netsniff-ng] BPF Options/ Writing IP Packet Header Information to Output File and Practical Usage Question

[Daniel Borkmann]

On 09/04/2013 08:22 PM, Daniel Martin wrote:

Hey Daniel,

Thanks for your reply.  I did compile your example

bpfc foo  bar

Which did return
cat bar
{ 0x20, 0, 0, 0xf034 },
{ 0x16, 0, 0, 0x },


You need 3.10 or higher. This was developed during netfilter workshop,
and merged in on 2013-05-01 (net-next pull).

--
You received this message because you are subscribed to the Google Groups 
netsniff-ng group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [netsniff-ng] gencfg: Added rfc2544 configs

[Daniel Borkmann]

On 08/21/2013 09:52 PM, Jon Schipp wrote:

I added the configurations outlined in Section 9.1 of RFC2544,
Benchmarking Methodology for Network Interconnect Devices to
gencfg [1].  Currently, it will write each configuration to a file in the
current working directory.

$ ./gencfg -G rfc2544 -s 10.1.1.1 -d 10.1.1.2 -m de:ad:be:ef:00:00 -M
00:0c:29:8d:4d:a2
$ ls *.cfg
64.cfg ..1518.cfg
$ trafgen --in 64.cfg --out eth1 --verbose

Hopefully, this will make it easier to perform quick benchmarks.


Indeed, that is very helpful! Thanks Jon!


[1] https://github.com/jonschipp/gencfg

Thanks


--
You received this message because you are subscribed to the Google Groups 
netsniff-ng group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[netsniff-ng] Re: BPF Compilation

[Daniel Borkmann]

On 08/22/2013 07:10 PM, Jon Schipp wrote:

Are BPF filters compiled for each packet that reaches a filter?
I assumed they were compiled once and then applied until the sniffing
application process ends.


BPF filters are attached to the kernel's socket representation. If no JIT is
active, the instructions you have passed from user-space are run inside a
minimal virtual machine. If you use the kernel's JIT compiler, then arch
specific opcodes are emitted at the time the filter is being attached to the
socket (so only done once), and the kernel jumps to the JIT image instead of
running the interpreter for each packet. When the application ends, the socket
is destroyed, therefore also the JIT image attached to your socket.

BPF filters are used in various code, sec-comp, PTP, packetraw sockets and so
on. When you have pf_packet sockets then the packet is forwarded to each 
protocol
handler and the filter is being run there.


A simple benchmark https://lwn.net/Articles/437986/ shows a 50ns savings
for each invocation of a simple filter - that may seem small, but, when
multiplied by the number of packets going through a system, that difference
can add up quickly. [1]

[1] https://lwn.net/Articles/437981/


--
You received this message because you are subscribed to the Google Groups 
netsniff-ng group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [netsniff-ng] Using a time-based interval in netsniff-ng crashes at the end of the first interval

[Daniel Borkmann]

On 08/16/2013 04:48 PM, branchnetconsult...@gmail.com wrote:

I built netsniff-ng 0.5.8-rc2 from git just last night on a 64bit Ubuntu
12.04.2 LTS box.

When I specify a time-based interval, netsniff-ng records for the full
interval but then crashes with a Poll failed! error before starting a 2nd
pcap file.  Like this:

root@server:~# netsniff-ng --in eth1 --out dump -s --interval 30s
Running! Hang up with ^C!

Poll failed!
root@server:~# ll dump
total 152064
drwxr-xr-x  2 root root  4096 Aug 16 10:27 ./
drwx-- 57 root root 12288 Aug 16 10:26 ../
-rw-r--r--  1 root root 155690405 Aug 16 10:27 dump-1376663235.pcap

This does not occur with a traffic volume interval like --interval 1MiB

I googled about for netsniff-ng and Poll failed! but it appears this
may have cropped up recently, since nothing turned up in my digging.


Thanks for reporting! I've just pushed a fix upstream to the Git repository.

Could you test it on your side?


Kevin


--
You received this message because you are subscribed to the Google Groups 
netsniff-ng group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [netsniff-ng] Re: Can netsniff-ng create pcap file by size

[Daniel Borkmann]

On 08/16/2013 09:54 PM, Allen Ting wrote:

Thanks for the answers.

I tried to limit the pcap file size to 100MB by setting the option to
be  --interval
100MiB, however, I saw most of the pcap files were created at the size
around 170MB to 200MB. Is this the expected behavior?


Is this reproducible with the latest netsniff-ng version from Git?

I remember there was an issue with TPACKET_V3 as accounting seems to
be an issue there that we recently fixed.

--
You received this message because you are subscribed to the Google Groups 
netsniff-ng group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.