[netsniff-ng] Re: [PATCH] trafgen: Fix documentation for seqinc and seqdec

[Tobias Klauser]

On 2021-12-07 at 20:55:10 +0100, Sean Anderson  wrote:
> These functions take the "times" argument as the second parameter, not
> the third.
> 
> Fixes: 00e83a5 ("man: trafgen: finish syntax section")
> Signed-off-by: Sean Anderson 

Applied, thank you!

-- 
You received this message because you are subscribed to the Google Groups 
"netsniff-ng" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/netsniff-ng/20211209141533.64jrsos3lrmothw5%40distanz.ch.

[netsniff-ng] Re: [PATCH] trafgen.8: fix typo in example packet

[Tobias Klauser]

On 2021-09-02 at 13:01:10 +0200, Baruch Siach  wrote:
> First MAC address is destination, but the second is source.
> 
> Signed-off-by: Baruch Siach 

Applied, thanks.

-- 
You received this message because you are subscribed to the Google Groups 
"netsniff-ng" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/netsniff-ng/20210902130816.3ewzgl5n7b727wxd%40distanz.ch.

[netsniff-ng] Re: [PATCH] Detect libpcap dependencies using pkg-config

[Tobias Klauser]

On 2021-09-02 at 11:56:00 +0200, Baruch Siach  wrote:
> When building statically the link command line must include all
> dependencies of all libraries. libpcap can optionally depend on libnl.
> mausezahn can't build statically in this case.
> 
> Use pkg-config in configure and in the link command to construct the
> library flags we need to link with libpcap.
> 
> Signed-off-by: Baruch Siach 

Applied, thanks.

-- 
You received this message because you are subscribed to the Google Groups 
"netsniff-ng" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/netsniff-ng/20210902130803.m7mhr2vj5fn7dsrd%40distanz.ch.

[netsniff-ng] [ANNOUNCE] netsniff-ng v0.6.8

[Tobias Klauser]

It is our pleasure to announce the release of netsniff-ng 0.6.8!

The summary of changes and the short log of all changes since v0.6.7 can
be found below and also on github at

  https://github.com/netsniff-ng/netsniff-ng/releases/tag/v0.6.8

Thanks to all contributors for this release.

Happy packet sniffing!

---

netsniff-ng 0.6.8 (Flutternozzle) has been released to the public.

It can be fetched via Git:

   git clone git://github.com/netsniff-ng/netsniff-ng.git
   git checkout v0.6.8

Or via HTTP from one of our mirrors:

   http://pub.netsniff-ng.org/netsniff-ng/netsniff-ng-0.6.8.tar.gz
   http://mirror.distanz.ch/netsniff-ng/netsniff-ng-0.6.8.tar.gz
   http://github.com/netsniff-ng/netsniff-ng/archive/v0.6.8.tar.gz

The release can be verified via Git (see README):

   git tag -v v0.6.8

Major high-level changes since the last release (v0.6.7) are:

   Fix mausezahn build with GCC 10.

Contributions since last release:

 11  Tobias Klauser
  2  uno20001
  1  Rosen Penev

Git changelog since last release:

Tobias Klauser (11):
  mausezahn: remove unused MZ_SIZE_LONG_INT
  astraceroute.8: use more sensitive terminology
  trafgen: remove write-only variable pkt
  AUTHORS: update
  mausezahn: make needlessly global variables static
  mausezahn: move variable definitions cli.h to cli.c
  mausezahn: move variable definitions from mops.h to mops.c
  mausezahn: move variable definitions from llist.h to llist.c
  mausezahn: move variable definitions from mz.h to mausezahn.c
  all: change bug report instructions in copyright message
  netsniff-ng v0.6.8

uno20001 (2):
  astraceroute: change type of variables from int to bool in struct ctx
  astraceroute: make some panic() messages more verbose

Rosen Penev (1):
  cookie: add sys/types header

-- 
You received this message because you are subscribed to the Google Groups 
"netsniff-ng" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/netsniff-ng/2021015817.q5v6537avj754xdb%40distanz.ch.

[netsniff-ng] [ANNOUNCE] netsniff-ng v0.6.7

[Tobias Klauser]

It is our pleasure to announce the release of netsniff-ng 0.6.7!

The summary of changes and the short log of all changes since v0.6.7 can
be found below and also on github at

  https://github.com/netsniff-ng/netsniff-ng/releases/tag/v0.6.7

Thanks to all contributors for this release:

  Nathaniel Ferguson, Petr Machata, uno20001, Michael R. Torres, Joachim
  Nilsson, Benoît Ganne

Happy packet sniffing!

---

netsniff-ng 0.6.7 (Polygon Window) has been released to the public.

It can be fetched via Git:

   git clone git://github.com/netsniff-ng/netsniff-ng.git
   git checkout v0.6.7

Or via HTTP from one of our mirrors:

   http://pub.netsniff-ng.org/netsniff-ng/netsniff-ng-0.6.7.tar.gz
   http://mirror.distanz.ch/netsniff-ng/netsniff-ng-0.6.7.tar.gz
   http://github.com/netsniff-ng/netsniff-ng/archive/v0.6.7.tar.gz

The release can be verified via Git (see README):

   git tag -v v0.6.7

Major high-level changes since the last release (v0.6.6) are:

   *** BLURB HERE (specific bits) ***

Contributions since last release:

  5  Tobias Klauser
  4  Nathaniel Ferguson
  2  Petr Machata
  1  uno20001
  1  Michael R. Torres
  1  Michael R Torres
  1  Joachim Nilsson
  1  Benoît Ganne

Git changelog since last release:

Tobias Klauser (5):
  AUTHORS: update
  AUTHORS: update
  AUTHORS: update
  AUTHORS: update
  netsniff-ng v0.6.7

Nathaniel Ferguson (4):
  proto_ipv4: Make netsniff detect and properly handle some malformed 
packets
  proto_lldp: Check inet_ntop() return values
  proto_lldp: fix pointer increment in LLDP_TLV_SYSTEM_CAP case
  proto_lldp: prevent accidental out of bounds memory access

Michael R Torres (2):
  mz: Fix accidental assignment in conditional statement Corrects the 
accidental assignment of _c_ to 'c' or 'p' due to a missing equals sign. This 
enables the proper display of the missing argument error message for all 
relevant options.
  mz: Zero memory allocated for new automops element

Petr Machata (2):
  mausezahn.8: Document -r
  mausezahn: Support -R to set packet priority

Benoît Ganne (1):
  ifpps: fix iface stat parsing if uppercase

Joachim Nilsson (1):
  trafgen: reset errno before calling sscanf in str2mac

uno20001 (1):
  astraceroute: reorder function declarations

-- 
You received this message because you are subscribed to the Google Groups 
"netsniff-ng" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/netsniff-ng/20200504133847.6ef5ob2df7qo5wdv%40distanz.ch.

Re: [netsniff-ng] [PATCH 2/2] mausezahn: Support -R to set packet priority

[Tobias Klauser]

On 2019-10-01 at 18:40:11 +0200, Petr Machata  wrote:
> Add a command line option -R to specify SO_PRIORITY socket option. This
> then sets priority of the generated SKBs, which is handy for testing Qdiscs
> and other priority-dependent functionality.
> 
> Signed-off-by: Petr Machata 

Both patches applied, thanks Petr!

-- 
You received this message because you are subscribed to the Google Groups 
"netsniff-ng" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/netsniff-ng/20191004082455.ghkroji76h2pkbos%40distanz.ch.

[netsniff-ng] [ANNOUNCE] netsniff-ng v0.6.6

[Tobias Klauser]

It is our pleasure to announce the release of netsniff-ng 0.6.6!

The summary of changes and the short log of all changes since v0.6.6 can
be found below and also on github at

  https://github.com/netsniff-ng/netsniff-ng/releases/tag/v0.6.6

Thanks to all contributors for this release:

  Mandar Gokhale, Whang Choi, uno20001, Quentin Chateau, Kartik Mistry,
  Jaroslav Škarvada, Daniel Roggow

Happy packet sniffing!

---

netsniff-ng 0.6.6 (Syro) has been released to the public.

It can be fetched via Git:

   git clone git://github.com/netsniff-ng/netsniff-ng.git
   git checkout v0.6.6

Or via HTTP from one of our mirrors:

   http://pub.netsniff-ng.org/netsniff-ng/netsniff-ng-0.6.6.tar.gz
   http://mirror.distanz.ch/netsniff-ng/netsniff-ng-0.6.6.tar.gz
   http://github.com/netsniff-ng/netsniff-ng/archive/v0.6.6.tar.gz

The release can be verified via Git (see README):

   git tag -v v0.6.6

Major high-level changes since the last release (v0.6.5) are:

   *** BLURB HERE (specific bits) ***

Contributions since last release:

 14  Tobias Klauser
  2  Mandar Gokhale
  1  Whang Choi
  1  uno20001
  1  Quentin Chateau
  1  Kartik Mistry
  1  Jaroslav Škarvada
  1  Daniel Roggow

Git changelog since last release:

Tobias Klauser (14):
  build: use system libsodium by default
  AUTHORS: add Whang Choi
  zsh: netsniff-ng: add completion for --overwrite
  netsniff-ng: store default prefix in ctx
  docs: add netsniff-ng website source for GitHub pages rendering
  docs: remove again, moved to the netsniff-ng.github.io
  contrib: move auxiliary scripts to own directory
  AUTHORS: single list of contributors
  AUTHORS: auto-generate from commit log
  oui.conf: update from upstream
  AUTHORS: update authors
  AUTHORS: update
  AUTHORS: regenerate and correct Jon's name
  netsniff-ng v0.6.6

Mandar Gokhale (2):
  mz: Add error handling for mismatched address families
  astraceroute: Fix for reading mirrors from file

Daniel Roggow (1):
  Build system: Use GZIP_ENV instead of GZIP

Jaroslav Škarvada (1):
  trafgen: fixed '--in -' to work again with STDIN

Kartik Mistry (1):
  Fix manpage warnings

Quentin Chateau (1):
  trafgen: fix -t 0 option to use sendto

Whang Choi (1):
  netsniff-ng: implement rotating capture files

uno20001 (1):
  astraceroute: checksum calculation for ICMP and TCP

-- 
You received this message because you are subscribed to the Google Groups 
"netsniff-ng" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/netsniff-ng/20190509074642.d4vqbwbsnnhwao6x%40distanz.ch.
For more options, visit https://groups.google.com/d/optout.

[netsniff-ng] [ANNOUNCE] netsniff-ng v0.6.5

[Tobias Klauser]

It is our pleasure to announce the release of netsniff-ng 0.6.5!

There hasn't been a tree freeze this time. There weren't any critical
changes and the overall number of changes is small. I'll keep the
release procedure that way as long as the inflow of new changes an
issues remains that low, i.e. there will be a new release every once in
a while once enough changes have accumulated.

The summary of changes and the short log of all changes since v0.6.4 can
be found below and also on github at

  https://github.com/netsniff-ng/netsniff-ng/releases/tag/v0.6.5

Thanks to all contributors for this release:

  Daniel Roberson, Markus Amend, Matteo Croce, Nick Grauel, Petr Machata,
  Radoslav Bodo, Ondřej Gajdušek

Happy packet sniffing!

---

netsniff-ng 0.6.5 (Spiral Staircase) has been released to the public.

It can be fetched via Git:

   git clone git://github.com/netsniff-ng/netsniff-ng.git
   git checkout v0.6.5

Or via HTTP from one of our mirrors:

   http://pub.netsniff-ng.org/netsniff-ng/netsniff-ng-0.6.5.tar.gz
   http://mirror.distanz.ch/netsniff-ng/netsniff-ng-0.6.5.tar.gz
   http://github.com/netsniff-ng/netsniff-ng/archive/v0.6.5.tar.gz

The release can be verified via Git (see README):

   git tag -v v0.6.5

Contributions since last release:

 16  Tobias Klauser
  2  Daniel Roberson
  1  Radoslav Bodo
  1  Petr Machata
  1  Nick Grauel
  1  Matteo Croce
  1  Markus Amend

Git changelog since last release:

Tobias Klauser (16):
  bpf: don't use builtin memset/memcpy in bpf_parse_rules
  ring: use xzmalloc_aligned
  geoip: store GeoIP files in $(PREFIX)/share by default
  trafgen: don't close dev_out and dev_in in parent process
  trafgen: fix signedness warning in cleanup_packets
  all: drop fmem{cpy,set}
  make: consider $DESTDIR when creating $DATDIR
  trafgen: support dumping IPv6 protocol header command
  AUTHORS: add Daniel Roberson
  man: netsniff-ng: document time formats for -o/--out
  AUTHORS: add Petr Machata
  man: reformat all man pages
  AUTHORS: add Nick Grauel
  AUTHORS: add Radoslav Bodo
  AUTHORS: add Matteo Croce
  netsniff-ng v0.6.5

Daniel Roberson (2):
  mausezahn: fix strtok() segfault if s or m are missing
  netsniff-ng: add date format strings to --out

Markus Amend (1):
  netsniff-ng: add DCCP support

Matteo Croce (1):
  mausezahn: improve random mac address generation

Nick Grauel (1):
  mausezahn: Restore handling of raw hex string passed in on command line

Petr Machata (1):
  mausezahn: Fix IPv6 address comparison

Radoslav Bodo (1):
  trafgen: support ICMPv6 checksums

-- 
You received this message because you are subscribed to the Google Groups 
"netsniff-ng" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [netsniff-ng] [PATCH] mausezahn: Fix IPv6 address comparison

[Tobias Klauser]

On 2018-05-14 at 23:59:59 +0200, Petr Machata  wrote:
> CMP_INT evaluates its arguments more than once, and thus passing a
> post-incremented pointer as an argument causes double increments and
> hence buffer overruns. This can be observed by erratic behavior of IPv6
> address ranges. Fix by moving the increment to loop header.
> 
> Signed-off-by: Petr Machata 

Applied, thank you!

-- 
You received this message because you are subscribed to the Google Groups 
"netsniff-ng" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[netsniff-ng] [ANNOUNCE] netsniff-ng v0.6.4

[Tobias Klauser]

It is our pleasure to announce the release of netsniff-ng 0.6.4 (and
sorry about the delay).

The summary of changes and the short log of all changes since v0.6.3 can
be found below and also on github at

  https://github.com/netsniff-ng/netsniff-ng/releases/tag/v0.6.4

Please note that there might be still an issue with flowtop segfaulting
under certain circumstances, please see the issue at

  https://github.com/netsniff-ng/netsniff-ng/issues/183

for details and please report any reproducers you encounter there.

Thanks to all contributors for this release:

  Vadim Kochan, Baruch Siach, Paolo Abeni, Jaroslav Škarvada, Jia
  Zhouyang, Eduardo Miravalls Sierra, Sami Farin, @archey

This release also means that the tree is again open for new features.

Happy packet sniffing!

---

netsniff-ng 0.6.4 (Spiral Staircase) has been released to the public.

It can be fetched via Git:

   git clone git://github.com/netsniff-ng/netsniff-ng.git
   git checkout v0.6.4

Or via HTTP from one of our mirrors:

   http://pub.netsniff-ng.org/netsniff-ng/netsniff-ng-0.6.4.tar.gz
   http://mirror.distanz.ch/netsniff-ng/netsniff-ng-0.6.4.tar.gz
   http://github.com/netsniff-ng/netsniff-ng/archive/v0.6.4.tar.gz

The release can be verified via Git (see README):

   git tag -v v0.6.4

Major high-level changes since the last release (v0.6.3) are:

1. DNS header generation in trafgen (Vadim Kochan)

2. Support for dumping proto headers in trafgen (Vadim Kochan)

3. Fixes for dinc()/ddec() to properly respect min/max range and avoid
   counter corruption (Paolo Abeni)

4. Fixes for NULL pointer dereferences and resource leaks in trafgen
   (Tobias Klauser) and mausezahn (Jia Zhouyang)

5. Build fixes for various compiler versions, libc implementations and
   distributions (Baruch Siach, Jaroslav Škarvada, Tobias Klauser)

Contributions since last release:

 22  Tobias Klauser
 15  Vadim Kochan
  2  Paolo Abeni
  2  Baruch Siach
  1  Jia Zhouyang
  1  Jaroslav Škarvada

Git changelog since last release:

Tobias Klauser (22):
  netsniff-ng: remove unnecessary zeroing of packet counters in init_ctx()
  make: use sed instead of perl to extract lex/yacc prefix
  proc: include headers to get definitions for ino_t and pid_t
  geoip: adjust geoip{4,6}_{city,region}_name prototypes for !HAVE_GEOIP
  all: use  headers instead of  where possible
  built_in: don't redefine memcpy/memset
  bpfc: disable NLS in the parser
  trafgen: disable NLS in the parser
  AUTHORS: add Baruch Siach
  trafgen: gracefully handle ENOBUFS on tx ring teardown
  netsniff-ng: fix --bind-cpu option in example command line
  AUTHOR: add Zhouyang Jia
  dev: only calculate wireless bitrate if necessary
  ifpps: use uint32_t instead of u32
  link: use uint32_t instead of u32
  ifpps: fix unintendet assignment
  build: check for fopencookie() in configure
  trafgen: fix resource leaks
  trafgen: fix NULL pointer dereference in -i option parsing
  astraceroute: use switch instead of lookup table for short proto id
  flowtop: change tab title for process tab
  netsniff-ng v0.6.4

Vadim Kochan (15):
  trafgen: parser: Use proto_field_set_xxx where it is possible
  flowtop: Improve and unify up/down scrolling
  trafgen: proto: Allow to set field with variable length
  str: Add function for converting string into DNS name
  trafgen: l7: Add DNS header generation API
  trafgen: parser: Add syntax to generate DNS header
  flowtop: Move out stats fields from flow & proc entry
  trafgen: Allow to generate packets to output pcap file
  trafgen: Fix output pcap file name length trimming
  trafgen: Delegate creation of rfraw to dev_io API
  trafgen: Get packet from proto_hdr if possible
  trafgen: dev_io: Change read/write to specify struct packet *
  trafgen: Dump proto headers in *.cfg format
  flowtop: Use RCU flow deletion from process entry
  flowtop: Fix use-after-free on filter reload

Baruch Siach (2):
  flowtop: take PKG_CONFIG into account for libnetfilter_conntrack
  proc.h: add missing headers

Paolo Abeni (2):
  trafgen: fix packet socket initialization with multiple CPUs
  trafgen: fix dinc()/ddec() modifiers

Jaroslav Škarvada (1):
  staging: compilation fix with new gcc

Jia Zhouyang (1):
  mausezahn: fix segmentation fault

-- 
You received this message because you are subscribed to the Google Groups 
"netsniff-ng" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[netsniff-ng] Re: [PATCH] flowtop: Fix use-after-free on filter reload

[Tobias Klauser]

On 2017-12-29 at 10:17:09 +0100, Vadim Kochan <vadi...@gmail.com> wrote:
> On Tue, Dec 19, 2017 at 12:30 PM, Tobias Klauser <tklau...@distanz.ch>
> wrote:
> 
> > On 2017-12-19 at 11:24:40 +0100, Vadim Kochan <vadi...@gmail.com> wrote:
> > > May it possible that you tried flowtop compiled without the fix ?
> >
> > No, I made sure to have the patch applied and recompiled flowtop. I can
> > still quite reliably reproduce the issue and flowtop sometimes even
> > segfaults on startup before displaying anything.
> >
> > > On Tue, Dec 19, 2017 at 12:18 PM, Vadim Kochan <vadi...@gmail.com>
> > wrote:
> > >
> > > > Thats really strange, because before this patch I really easy triggered
> > > > the issue, but
> > > > now I cant.
> > > >
> > > > On Tue, Dec 19, 2017 at 11:12 AM, Tobias Klauser <tklau...@distanz.ch>
> > > > wrote:
> > > >
> > > >> On 2017-12-18 at 23:38:18 +0100, Vadim Kochan <vadi...@gmail.com>
> > wrote:
> > > >> > There is missing logic which removes flown entry from
> > > >> > related proc's entry while destroying global flows list on
> > > >> > filter reloading, hence add common __flow_list_del_entry which
> > > >> > handles this logic for both cases - when ct destroyed or filter
> > > >> > changed.
> > > >> >
> > > >> > This is a 2nd fix for issue #183.
> > > >>
> > > >> Thanks for the patch. While it is certainly correct, it unfortunately
> > > >> still doesn't fix #183 properly. I can still trigger a segfault by
> > > >> repeatedly enabling/disabling TCP, UDP and ICMP flows ('T', 'U' or 'I'
> > > >> key).
> > > >>
> > > >
> > > >
> >
> 
> Hi Tobias,
> 
> Looks like https://github.com/netsniff-ng/netsniff-ng/issues/183
> is not reproducible, do you still see issues with flowtop ?

The issue is still reproducible for me on latest master. Unfortunately
not in gdb though.

But since the bug does appear to occur less often (at least you and the
original reporter can no longer reproduce), I think we should still go
ahead with the release.

Thanks
Tobias

-- 
You received this message because you are subscribed to the Google Groups 
"netsniff-ng" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[netsniff-ng] Re: [PATCH] flowtop: Fix use-after-free on filter reload

[Tobias Klauser]

On 2017-12-19 at 11:24:40 +0100, Vadim Kochan <vadi...@gmail.com> wrote:
> May it possible that you tried flowtop compiled without the fix ?

No, I made sure to have the patch applied and recompiled flowtop. I can
still quite reliably reproduce the issue and flowtop sometimes even
segfaults on startup before displaying anything.

> On Tue, Dec 19, 2017 at 12:18 PM, Vadim Kochan <vadi...@gmail.com> wrote:
> 
> > Thats really strange, because before this patch I really easy triggered
> > the issue, but
> > now I cant.
> >
> > On Tue, Dec 19, 2017 at 11:12 AM, Tobias Klauser <tklau...@distanz.ch>
> > wrote:
> >
> >> On 2017-12-18 at 23:38:18 +0100, Vadim Kochan <vadi...@gmail.com> wrote:
> >> > There is missing logic which removes flown entry from
> >> > related proc's entry while destroying global flows list on
> >> > filter reloading, hence add common __flow_list_del_entry which
> >> > handles this logic for both cases - when ct destroyed or filter
> >> > changed.
> >> >
> >> > This is a 2nd fix for issue #183.
> >>
> >> Thanks for the patch. While it is certainly correct, it unfortunately
> >> still doesn't fix #183 properly. I can still trigger a segfault by
> >> repeatedly enabling/disabling TCP, UDP and ICMP flows ('T', 'U' or 'I'
> >> key).
> >>
> >
> >

-- 
You received this message because you are subscribed to the Google Groups 
"netsniff-ng" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[netsniff-ng] Re: [PATCH] flowtop: Fix use-after-free on filter reload

[Tobias Klauser]

On 2017-12-18 at 23:38:18 +0100, Vadim Kochan  wrote:
> There is missing logic which removes flown entry from
> related proc's entry while destroying global flows list on
> filter reloading, hence add common __flow_list_del_entry which
> handles this logic for both cases - when ct destroyed or filter
> changed.
> 
> This is a 2nd fix for issue #183.

Thanks for the patch. While it is certainly correct, it unfortunately
still doesn't fix #183 properly. I can still trigger a segfault by
repeatedly enabling/disabling TCP, UDP and ICMP flows ('T', 'U' or 'I'
key).

-- 
You received this message because you are subscribed to the Google Groups 
"netsniff-ng" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[netsniff-ng] Re: [PATCH] flowtop: Use RCU flow deletion from process entry

[Tobias Klauser]

On 2017-12-17 at 22:25:40 +0100, Vadim Kochan  wrote:
> Use cds_list_del_rcu for safer deletion flow from the process flow
> list to prevent possible use-after-free by UI thread when it is
> refreshing the processes.
> 
> It may fix the #183 issue.
> 
> Signed-off-by: Vadim Kochan 

Applied, thank you Vadim!

-- 
You received this message because you are subscribed to the Google Groups 
"netsniff-ng" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [netsniff-ng] Re: [ANNOUNCE] Preparing for netsniff-ng release v0.6.4

[Tobias Klauser]

Hi Vadim

On 2017-12-15 at 02:12:20 +0100, Vadim Kochan <vadi...@gmail.com> wrote:
> Looks like its data-race issue, can't reproduce it yet, *BUT*, looking
> into the code it looks like I missed to remove flow entry from the
> proc_entry->flows
> list while flow entry is removing & freeing from the global list, plz give
> me few more days for testing & fixing
> if its possible.

Thanks a lot for the follow up. Will hold off the release until next
week then.

Thanks!

> Regards,
> Vadim Kochan
> 
> On Thu, Dec 14, 2017 at 2:47 PM, Tobias Klauser <tklau...@distanz.ch> wrote:
> 
> > On 2017-12-11 at 09:08:15 +0100, Tobias Klauser <tklau...@distanz.ch>
> > wrote:
> > > On 2017-12-08 at 18:30:24 +0100, Vadim Kochan <vadi...@gmail.com> wrote:
> > > > Hi All,
> > > >
> > > > I just noticed there some bug reports, I will look on them on weekend,
> > I
> > > > assume it is better
> > > > to wait with release unless isues will be fixed ?
> > >
> > > Yes, in particular https://github.com/netsniff-ng/netsniff-ng/issues/183
> > > worries me a bit. Would be nice to get it fixed before the release. If
> > > we're not able to find a fix within 2-3 days, I'd say we could still do
> > > a release with the current state (as the bug already seems to be present
> > > in 0.6.3 anyway) and do another release once we find a fix.
> > >
> > > Thanks for looking into this!
> >
> > Vadim, any progress? If not, I'd mention this as a known issue in the
> > release notes for now and still go forward with the release (we can
> > still do a point release if we find a fix for it soon).
> >
> > As for the release, I'd like to pull in PR 184 [1] which fixes an
> > obvious bug.
> >
> > [1] https://github.com/netsniff-ng/netsniff-ng/pull/184
> >
> > Tobias
> >

-- 
You received this message because you are subscribed to the Google Groups 
"netsniff-ng" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [netsniff-ng] Re: [ANNOUNCE] Preparing for netsniff-ng release v0.6.4

[Tobias Klauser]

On 2017-12-11 at 09:08:15 +0100, Tobias Klauser <tklau...@distanz.ch> wrote:
> On 2017-12-08 at 18:30:24 +0100, Vadim Kochan <vadi...@gmail.com> wrote:
> > Hi All,
> > 
> > I just noticed there some bug reports, I will look on them on weekend, I
> > assume it is better
> > to wait with release unless isues will be fixed ?
> 
> Yes, in particular https://github.com/netsniff-ng/netsniff-ng/issues/183
> worries me a bit. Would be nice to get it fixed before the release. If
> we're not able to find a fix within 2-3 days, I'd say we could still do
> a release with the current state (as the bug already seems to be present
> in 0.6.3 anyway) and do another release once we find a fix.
> 
> Thanks for looking into this!

Vadim, any progress? If not, I'd mention this as a known issue in the
release notes for now and still go forward with the release (we can
still do a point release if we find a fix for it soon).

As for the release, I'd like to pull in PR 184 [1] which fixes an
obvious bug.

[1] https://github.com/netsniff-ng/netsniff-ng/pull/184

Tobias

-- 
You received this message because you are subscribed to the Google Groups 
"netsniff-ng" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [netsniff-ng] Re: [ANNOUNCE] Preparing for netsniff-ng release v0.6.4

[Tobias Klauser]

On 2017-12-08 at 18:30:24 +0100, Vadim Kochan  wrote:
> Hi All,
> 
> I just noticed there some bug reports, I will look on them on weekend, I
> assume it is better
> to wait with release unless isues will be fixed ?

Yes, in particular https://github.com/netsniff-ng/netsniff-ng/issues/183
worries me a bit. Would be nice to get it fixed before the release. If
we're not able to find a fix within 2-3 days, I'd say we could still do
a release with the current state (as the bug already seems to be present
in 0.6.3 anyway) and do another release once we find a fix.

Thanks for looking into this!

Tobias

-- 
You received this message because you are subscribed to the Google Groups 
"netsniff-ng" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[netsniff-ng] [ANNOUNCE] Preparing for netsniff-ng release v0.6.4

[Tobias Klauser]

The last release was in April and there have been quite a few changes
since which are worth another release.

Thus, the tree [1] is now closed for new features as of today. This
means only bug fixes and small, non-intrusive fixes/cleanups will now be
considered for inclusion in v0.6.4. The final release is planned in two
weeks time. Afterwards the tree is open again for new features and more
experimental changes.

  [1] https://github.com/netsniff-ng/netsniff-ng

Everybody who's interested in the netsniff-ng toolkit is kindly invited
to give the current git HEAD some good testing, report issues [2] or
send patches [3].

  [2] https://github.com/netsniff-ng/netsniff-ng/issues
  [3] https://github.com/netsniff-ng/netsniff-ng/blob/master/SubmittingPatches

Please see the shortlog below for an overview of the fixes and features
that were added since v0.6.3.

Thanks to everybody who contributed to the upcoming release so far:
Baruch Siach, Jaroslav Škarvada, Jia Zhouyang, Paolo Abeni, and Vadim Kochan.

Baruch Siach (2):
  flowtop: take PKG_CONFIG into account for libnetfilter_conntrack
  proc.h: add missing headers

Jaroslav Škarvada (1):
  staging: compilation fix with new gcc

Jia Zhouyang (1):
  mausezahn: fix segmentation fault

Paolo Abeni (1):
  trafgen: fix packet socket initialization with multiple CPUs

Tobias Klauser (20):
  netsniff-ng: remove unnecessary zeroing of packet counters in init_ctx()
  make: use sed instead of perl to extract lex/yacc prefix
  proc: include headers to get definitions for ino_t and pid_t
  geoip: adjust geoip{4,6}_{city,region}_name prototypes for !HAVE_GEOIP
  all: use  headers instead of  where possible
  built_in: don't redefine memcpy/memset
  bpfc: disable NLS in the parser
  trafgen: disable NLS in the parser
  AUTHORS: add Baruch Siach
  trafgen: gracefully handle ENOBUFS on tx ring teardown
  netsniff-ng: fix --bind-cpu option in example command line
  AUTHOR: add Zhouyang Jia
  dev: only calculate wireless bitrate if necessary
  ifpps: use uint32_t instead of u32
  link: use uint32_t instead of u32
  ifpps: fix unintendet assignment
  build: check for fopencookie() in configure
  trafgen: fix resource leaks
  trafgen: fix NULL pointer dereference in -i option parsing
  astraceroute: use switch instead of lookup table for short proto id

Vadim Kochan (13):
  trafgen: parser: Use proto_field_set_xxx where it is possible
  flowtop: Improve and unify up/down scrolling
  trafgen: proto: Allow to set field with variable length
  str: Add function for converting string into DNS name
  trafgen: l7: Add DNS header generation API
  trafgen: parser: Add syntax to generate DNS header
  flowtop: Move out stats fields from flow & proc entry
  trafgen: Allow to generate packets to output pcap file
  trafgen: Fix output pcap file name length trimming
  trafgen: Delegate creation of rfraw to dev_io API
  trafgen: Get packet from proto_hdr if possible
  trafgen: dev_io: Change read/write to specify struct packet *
  trafgen: Dump proto headers in *.cfg format

-- 
You received this message because you are subscribed to the Google Groups 
"netsniff-ng" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [netsniff-ng] Re: [PATCH] trafgen: fix packet socket initialization with multiple CPUs

[Tobias Klauser]

On 2017-09-14 at 10:13:17 +0200, Paolo Abeni  wrote:
> On Thu, 2017-09-14 at 10:40 +0300, Vadim Kochan wrote:
> > Thanks Paolo! Shame on me, I did not test it properly :(
> 
> No problem at all, I guess. After all, bugs are the only feature shared
> by all kind of software ;-)

Indeed :) I certainly also should have tested the fix more thoroughly.

> I added the 'Fixes' tag following the kernel guidelines for patch
> submission, to help 3rd party packager which can't rebase the their
> version too often.

This is very much appreciated.

Cheers
Tobias

-- 
You received this message because you are subscribed to the Google Groups 
"netsniff-ng" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [netsniff-ng] [PATCH] trafgen: fix packet socket initialization with multiple CPUs

[Tobias Klauser]

On 2017-09-13 at 17:54:52 +0200, Paolo Abeni  wrote:
> The commit 78c13b71e196 ("trafgen: Allow to generate packets
> to output pcap file") introduced a regression when output is
> a network device and multiple CPU are in use: the packet
> socket is created before fork() and thus the socket is shared
> among all the processes: all of them except the first will
> fail while setting the tx_ring.
> 
> Fix it splitting the io open() helper in a create() op,
> called before forking, and the open() op called by each process.
> 
> Fixes: 78c13b71e196 ("trafgen: Allow to generate packets to output pcap file")
> Signed-off-by: Paolo Abeni 

Applied, thanks a lot Paolo!

-- 
You received this message because you are subscribed to the Google Groups 
"netsniff-ng" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[netsniff-ng] Re: Question on Curvetun

[Tobias Klauser]

Hi

On 2017-09-06 at 21:33:39 +0200, sandman  wrote:
> Hi Tobias
> 
> Thanks for your work on curvetun! I just exploring it and I wanted to check
> with you on potential usage. Would help me a great deal if you can help
> with me with following queries.

In general, please ask these kinds of questions on the netsniff-ng
mailing list https://groups.google.com/forum/#!forum/netsniff-ng
That way you're more likely to get your question answered by someone who
might have already done something similar and other people will also
benefit from the answers.

I Cc'ed my reply to the list.

> My use case:
> 
> I am looking at building a lightweight packet forwarder (much like rpcapd
> from wireshark/winpcap suite) but with end to end encryption. Basically a
> soft network tap using which I can capture packets on a production machine
> and send them out securely to another machine and analyze them for
> anomalies.
> 
> After having ruled out rpcapd due to instability and lack of encryption. I
> am currently evaluating between tinc and curvetun to act as secure tunnel
> over which I can ship captured packets.
> 
> 
> 1. How does curvetun compare to tinc (or openvpn for that matter) on
> performance front? Any high level ideas here? On performance, do you think
> my approach will fly or I should take something like rpcapd and add
> encryption on top of that?

I haven't used tinc or looked at it in depth, so I cannot really say
much about how it compares w.r.t. performance. I'd suggest, you just try
it out with a small test setup to get a high level picture.

If performance is of concern you might also want to look at Wireguard
[1], which is an in-kernel VPN implementation designed for performance
and ease-of-use. Though, it is not yet in the mainline kernel AFIAK.

  [1] https://www.wireguard.com

> 2. As you can see, I will be transferring packets from N production servers
> to 1 analysis server, is this use case supported? I think it is.

Yes, this is supported by curvetun. The analysis server would run
curvetun in server mode and the N production servers would each run
curvetun in client mode.

> 3. Any ready to use docker images of curvetun you can point to would be
> great too.

There is a docker image for the netsniff-ng toolkit from the OpenNSM
group on docker hub [2]. It doesn't seem to contain curvetun though, but
you might want to send them a pull request [3] to add it ;)

  [2] https://hub.docker.com/r/opennsm/netsniff-ng/
  [3] https://github.com/open-nsm/ContainNSM

Hope that helps
Tobias

-- 
You received this message because you are subscribed to the Google Groups 
"netsniff-ng" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[netsniff-ng] Re: [PATCH v2 0/3] trafgen: Add dump of proto headers into *.cfg format

[Tobias Klauser]

On 2017-07-29 at 11:46:06 +0200, Vadim Kochan  wrote:
> Added trafgen_dump.c module which dumps headers from packet
> in .cfg format. Packet is dumped if -o .cfg was specified,
> it might be useful to specify *.pcap file as input and convert it
> into .cfg file to edit proto fields in human readable format.
> 
> To make it possible several main changes were added:
> 
> 1) packet id is embedded into struct packet.id, and
>it is updated on each realloc_packet()
> 
> 2) Added new struct proto_hdr.get_next_proto callback
>to make possible apply fields of next header.
> 
> 3) Added new dev_io ops for writting packets into .cfg file,
>to re-use common dev_io mechsnism for packets dumping.
> 
> 4) Changed dev_io ops read/write to specify struct packet * instead of
>buf & count.
> 
> 5) Updated trafgen_proto.c to obtain packet from the header if possible to
>do not depend on last packet, which is not right way to get related 
> packet.
> 
> Before dump the default ETH_PROTO fields are applied as first header and
> then next proto_hdr is identified via .get_next_proto(...) callback.
> 
> Meanwhile only eth, arp, vlan, ip4, udp, & tcp protos can be dissected
> into *.cfg format.
> 
> v2:
> 1) Missed local patch
> 
> Vadim Kochan (3):
>   trafgen: Get packet from proto_hdr if possible
>   trafgen: dev_io: Change read/write to specify struct packet *
>   trafgen: Dump proto headers in *.cfg format

Series applied, thanks Vadim!

-- 
You received this message because you are subscribed to the Google Groups 
"netsniff-ng" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[netsniff-ng] Re: [PATCH v2] trafgen: Delegate creation of rfraw to dev_io API

[Tobias Klauser]

On 2017-07-17 at 08:06:36 +0200, Vadim Kochan  wrote:
> Simplify a bit of creation rfraw device by delegating it to the dev_io
> API, also in case the output device is pcap file the --rfraw option
> sets the link type to ieee80211 radio tap.
> 
> Signed-off-by: Vadim Kochan 

Applied, thanks!

-- 
You received this message because you are subscribed to the Google Groups 
"netsniff-ng" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[netsniff-ng] Re: [PATCH v2] trafgen: Fix output pcap file name length trimming

[Tobias Klauser]

On 2017-06-12 at 22:55:37 +0200, Vadim Kochan  wrote:
> Trim output name to IFNAMSIZ only if the output is a networking device,
> otherwise the following error occured if output name is greater then IFNAMSIZ:
> 
>   ~/src/netsniff-ng$ trafgen -n 1 '{ udp() }' -o /tmp/xx.pcap
>   No networking device or pcap file: /tmp/xx
>   Failed to open output device

Signed-off-by line is missing, I added it and applied the patch. Thanks.

-- 
You received this message because you are subscribed to the Google Groups 
"netsniff-ng" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[netsniff-ng] Re: [PATCH] trafgen: Delegate creation of rfraw to dev_io API

[Tobias Klauser]

On 2017-06-13 at 06:13:58 +0200, Vadim Kochan  wrote:
> Simplify a bit of creation rfraw device by delegating it to the dev_io
> API, also in case the output device is pcap file the --rfraw option
> sets the link type to ieee80211 radio tap.
> 
> Signed-off-by: Vadim Kochan 
> ---
>  trafgen.8 |  3 ++-
>  trafgen.c | 25 +
>  trafgen_dev.c | 36 +++-
>  trafgen_dev.h |  4 +++-
>  4 files changed, 49 insertions(+), 19 deletions(-)
> 

[...]

> diff --git a/trafgen_dev.c b/trafgen_dev.c
> index cd99a0c..46fb897 100644
> --- a/trafgen_dev.c
> +++ b/trafgen_dev.c
> @@ -16,6 +16,8 @@
>  #include "xmalloc.h"
>  #include "pcap_io.h"
>  #include "built_in.h"
> +#include "mac80211.h"
> +#include "linktype.h"
>  #include "trafgen_dev.h"
>  
>  static int dev_pcap_open(struct dev_io *dev, const char *name, enum 
> dev_io_mode_t mode)
> @@ -164,9 +166,35 @@ static int dev_net_write(struct dev_io *dev, const 
> uint8_t *buf, size_t len)
>   return sendto(dev->fd, buf, len, 0, (struct sockaddr *) , 
> sizeof(saddr));
>  }
>  
> +static int dev_net_set_link_type(struct dev_io *dev, int link_type)
> +{
> + if (link_type != LINKTYPE_IEEE802_11 && link_type != 
> LINKTYPE_IEEE802_11_RADIOTAP)
> + return 0;
> +
> + dev->trans = xstrdup(dev->name);
> + xfree(dev->name);
> +
> + enter_rfmon_mac80211(dev->trans, >name);
> + dev->ifindex = __device_ifindex(dev->name);
> + dev->dev_type = device_type(dev->name);
> + sleep(0);

Is this sleep really needed? It was already there in the original code,
but I guess it can be omitted...

-- 
You received this message because you are subscribed to the Google Groups 
"netsniff-ng" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[netsniff-ng] Re: [PATCH v2] trafgen: Allow to generate packets to output pcap file

[Tobias Klauser]

On 2017-06-07 at 21:24:52 +0200, Vadim Kochan  wrote:
> Add trafgen_dev.c module which provides generic way of
> reading and writing packets to/from networking device or a pcap file.
> 
> Also allow to handle output pcap file via '-o, --out, --dev' option.
> 
> It might be useful in future for testing some link protocols which is
> not easy to capture (e.g. wlan packets) w/o having some special setup.
> 
> Signed-off-by: Vadim Kochan 

Applied, thanks!

-- 
You received this message because you are subscribed to the Google Groups 
"netsniff-ng" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[netsniff-ng] Re: [PATCH] trafgen: Allow to generate packets to output pcap file

[Tobias Klauser]

On 2017-06-06 at 22:30:36 +0200, Vadim Kochan  wrote:
> Add trafgen_dev.c module which provides generic way of
> reading and writing packets to/from networking device or a pcap file.
> 
> Also allow to handle output pcap file via '-o, --out, --dev' option.
> 
> It might be useful in future for testing some link protocols which is
> not easy to capture (e.g. wlan packets) w/o having some special setup.

Looks good in general, a few minor nits below.

> Signed-off-by: Vadim Kochan 
> ---
>  trafgen.8|   5 +-
>  trafgen.c| 102 +
>  trafgen/Makefile |   1 +
>  trafgen_dev.c| 263 
> +++
>  trafgen_dev.h|  49 +++
>  trafgen_l2.c |   4 +
>  trafgen_l3.c |   8 +-
>  trafgen_proto.c  |  50 ++-
>  trafgen_proto.h  |   6 +-
>  9 files changed, 396 insertions(+), 92 deletions(-)
>  create mode 100644 trafgen_dev.c
>  create mode 100644 trafgen_dev.h
> 
> diff --git a/trafgen.8 b/trafgen.8
> index fd9788a..50deacf 100644
> --- a/trafgen.8
> +++ b/trafgen.8
> @@ -74,8 +74,9 @@ It is also possible to specify PCAP file with .pcap 
> extension via -i,--in option
>  by default packets will be sent at rate considering timestamp from PCAP file 
> which
>  might be reset via -b/-t options.
>  .PP
> -.SS -o , -d , --out , --dev 
> -Defines the outgoing networking device such as eth0, wlan0 and others.
> +.SS -o , -d , --out , --dev 
> +Defines the outgoing networking device such as eth0, wlan0 and others or
> +a pcap file.
>  .PP
>  .SS -p, --cpp
>  Pass the packet configuration to the C preprocessor before reading it into
> diff --git a/trafgen.c b/trafgen.c
> index b25760f..cc54f9e 100644
> --- a/trafgen.c
> +++ b/trafgen.c
> @@ -57,6 +57,7 @@
>  #include "csum.h"
>  #include "trafgen_proto.h"
>  #include "pcap_io.h"
> +#include "trafgen_dev.h"
>  
>  enum shaper_type {
>   SHAPER_NONE,
> @@ -79,6 +80,8 @@ struct shaper {
>  struct ctx {
>   bool rand, rfraw, jumbo_support, verbose, smoke_test, enforce, 
> qdisc_path;
>   size_t reserve_size;
> + struct dev_io *dev_out;
> + struct dev_io *dev_in;
>   unsigned long num;
>   unsigned int cpus;
>   uid_t uid; gid_t gid;
> @@ -145,7 +148,6 @@ static const char *copyright = "Please report bugs to 
>    "This is free software: you are free to change and redistribute it.\n"
>   "There is NO WARRANTY, to the extent permitted by law.";
>  
> -static int sock;
>  static struct cpu_stats *stats;
>  static unsigned int seed;
>  
> @@ -664,11 +666,6 @@ static void xmit_slowpath_or_die(struct ctx *ctx, 
> unsigned int cpu, unsigned lon
>   unsigned long num = 1, i = 0;
>   struct timeval start, end, diff;
>   unsigned long long tx_bytes = 0, tx_packets = 0;
> - struct sockaddr_ll saddr = {
> - .sll_family = PF_PACKET,
> - .sll_halen = ETH_ALEN,
> - .sll_ifindex = device_ifindex(ctx->device),
> - };
>  
>   if (ctx->num > 0)
>   num = ctx->num;
> @@ -688,8 +685,7 @@ static void xmit_slowpath_or_die(struct ctx *ctx, 
> unsigned int cpu, unsigned lon
>   while (likely(sigint == 0 && num > 0 && plen > 0)) {
>   packet_apply_dyn_elements(i);
>  retry:
> - ret = sendto(sock, packets[i].payload, packets[i].len, 0,
> -  (struct sockaddr *) , sizeof(saddr));
> + ret = dev_io_write(ctx->dev_out, packets[i].payload, 
> packets[i].len);
>   if (unlikely(ret < 0)) {
>   if (errno == ENOBUFS) {
>   sched_yield();
> @@ -745,15 +741,16 @@ retry:
>  
>  static void xmit_fastpath_or_die(struct ctx *ctx, unsigned int cpu, unsigned 
> long orig_num)
>  {
> - int ifindex = device_ifindex(ctx->device);
> + int ifindex = dev_io_ifindex_get(ctx->dev_out);
>   uint8_t *out = NULL;
>   unsigned int it = 0;
>   unsigned long num = 1, i = 0;
> - size_t size = ring_size(ctx->device, ctx->reserve_size);
> + size_t size = ring_size(dev_io_name_get(ctx->dev_out), 
> ctx->reserve_size);
>   struct ring tx_ring;
>   struct frame_map *hdr;
>   struct timeval start, end, diff;
>   unsigned long long tx_bytes = 0, tx_packets = 0;
> + int sock = dev_io_fd_get(ctx->dev_out);
>  
>   set_sock_prio(sock, 512);
>  
> @@ -938,69 +935,37 @@ static void xmit_packet_precheck(struct ctx *ctx, 
> unsigned int cpu)
>   }
>  }
>  
> -static void pcap_load_packets(const char *path)
> +static void pcap_load_packets(struct dev_io *dev)
>  {
> - const struct pcap_file_ops *pcap_io = pcap_ops[PCAP_OPS_SG];
> - uint32_t link_type, magic;
> - pcap_pkthdr_t phdr;
> + struct timespec tstamp;
>   size_t buf_len;
>   uint8_t *buf;
> - int ret;
> - int fd;
> -
> - fd = open(path, O_RDONLY | O_LARGEFILE | 

[netsniff-ng] Re: [PATCH] flowtop: Improve and unify up/down scrolling

[Tobias Klauser]

On 2017-05-27 at 17:30:08 +0200, Vadim Kochan  wrote:
> Move scrolling logic to the ui.c module, it requires to have
> some data iteration provided in flowtop.c and delegated to ui.c part.
> 
> So approach is that now flowtop provides 2 additional callbacks for:
> 
> 1) Iterate over flows/procs list
> 2) Draw flow/proc on each iteration which is controlled from ui.c
> 
> it allows to unify scrolling logic and delegate it to the ui.c, in the
> future it should allow to easy handle press event on selected row and
> drow some additional information, or draw a cursor line per selected
> row.
> 
> Also fixed case when down scrolling was bigger that printed rows, not
> it is handled by ui part.
> 
> Signed-off-by: Vadim Kochan 

Applied, thank you.

-- 
You received this message because you are subscribed to the Google Groups 
"netsniff-ng" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [netsniff-ng] [PATCH v2] pcap_io.h: fix if_arp.h musl libc/kernel headers conflict

[Tobias Klauser]

On 2017-04-16 at 13:34:00 +0200, Baruch Siach  wrote:
> Use libc provided arp definitions to avoid build failure with musl libc:
> 
> In file included from pcap_io.h:19:0,
>  from trafgen.c:59:
> .../usr/x86_64-buildroot-linux-musl/sysroot/usr/include/linux/if_arp.h:113:8: 
> error: redefinition of ‘struct arpreq’
>  struct arpreq {
> ^
> In file included from 
> .../usr/x86_64-buildroot-linux-musl/sysroot/usr/include/netinet/if_ether.h:111:0,
>  from 
> .../usr/x86_64-buildroot-linux-musl/sysroot/usr/include/net/ethernet.h:10,
>  from trafgen.c:23:
> .../usr/x86_64-buildroot-linux-musl/sysroot/usr/include/net/if_arp.h:99:8: 
> note: originally defined here
>  struct arpreq {
> ^
> 
> Add a local definition of the ARPHRD_CAN macro that glibc does not provide.
> 
> Signed-off-by: Baruch Siach 

Before I saw your patch, I already applied a simiar patch fixing up some
additional includes in commit b25a51fa5915 ("all: use  headers
instead of  where possible"). Sorry about that!

-- 
You received this message because you are subscribed to the Google Groups 
"netsniff-ng" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [netsniff-ng] [PATCH 1/2] proc.h: add missing headers

[Tobias Klauser]

Hi Baruch

On 2017-04-16 at 07:22:20 +0200, Baruch Siach  wrote:
> ino_t and pid_t require stat.h and types.h, respectively. Fixes the following
> build failure with musl libc:
> 
> In file included from cpp.c:7:0:
> proc.h:11:31: error: unknown type name ‘ino_t’
>  extern int proc_find_by_inode(ino_t ino, char *cmdline, size_t len, pid_t 
> *pid);
>^
> proc.h:11:69: error: unknown type name ‘pid_t’
>  extern int proc_find_by_inode(ino_t ino, char *cmdline, size_t len, pid_t 
> *pid);
>  ^
> proc.h:12:25: error: unknown type name ‘pid_t’
>  extern bool proc_exists(pid_t pid);
>  ^
> 
> Signed-off-by: Baruch Siach 

I already applied a similar patch in commit a9f4431e0a20 ("proc: include
headers to get definitions for ino_t and pid_t") but missed to add
sys/stat.h, so I'll apply a modified version of your patch.

Thanks!

-- 
You received this message because you are subscribed to the Google Groups 
"netsniff-ng" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [netsniff-ng] [PATCH] flowtop: take PKG_CONFIG into account for libnetfilter_conntrack

[Tobias Klauser]

Hi Baruch

On 2017-04-16 at 06:49:27 +0200, Baruch Siach  wrote:
> Use $PKG_CONFIG to determine the linker flags for libnetfilter_conntrack. This
> fixes static link failure like the following:
> 
>   LD  flowtop
> .../usr/x86_64-buildroot-linux-musl/sysroot/usr/lib/../lib64/libnetfilter_conntrack.a(main.o):
>  In function `nfct_open_nfnl':
> main.c:(.text+0x52): undefined reference to `nfnl_subsys_open'
> main.c:(.text+0x69): undefined reference to `nfnl_subsys_close'
> main.c:(.text+0x87): undefined reference to `nfnl_subsys_open'
> main.c:(.text+0xa3): undefined reference to `nfnl_subsys_close'
> .../usr/x86_64-buildroot-linux-musl/sysroot/usr/lib/../lib64/libnetfilter_conntrack.a(main.o):
>  In function `nfct_open':
> main.c:(.text+0xc9): undefined reference to `nfnl_open'
> main.c:(.text+0xf0): undefined reference to `nfnl_close'
> ...
> 
> Signed-off-by: Baruch Siach 

Sorry for taking so long to reply.

Patch now applied, thanks!

-- 
You received this message because you are subscribed to the Google Groups 
"netsniff-ng" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[netsniff-ng] [ANNOUNCE] netsniff-ng v0.6.3 released

[Tobias Klauser]

It is our pleasure to announce the release of netsniff-ng 0.6.3.

The summary of changes and the short log of all changes since v0.6.2 can
be found below and also on github at

  https://github.com/netsniff-ng/netsniff-ng/releases/tag/v0.6.3

Thanks to all contributors for this release:

  Vadim Kochan, Mandar Gokhale, Ken Wu, Jaroslav Škarvada, and
  @jamieparfet.

This release also means that the tree is again open for new features.

Happy packet sniffing and generating!

---

netsniff-ng 0.6.3 (Mookid) has been released to the public.

It can be fetched via Git:

   git clone git://github.com/netsniff-ng/netsniff-ng.git
   git checkout v0.6.3

Or via HTTP from one of our mirrors:

   http://pub.netsniff-ng.org/netsniff-ng/netsniff-ng-0.6.3.tar.gz
   http://mirror.distanz.ch/netsniff-ng/netsniff-ng-0.6.3.tar.gz
   http://github.com/netsniff-ng/netsniff-ng/archive/v0.6.3.tar.gz

The release can be verified via Git (see README):

   git tag -v v0.6.3

Major high-level changes since the last release (v0.6.2) are:

1) Allow to specify packet offsets in trafgen packet header functions.
   By Vadim Kochan.

2) Allow trafgen to send packets from pcap file. By Vadim Kochan.

3) Support trafgen interface without IP address in packet header
   functions. By Ken Wu.

4) Various other improvements and cleanups in the packet header
   functions of trafgen. By Vadim Kochan.

5) Tab based user interface for flowtop. Introduced a tab for flow
   statistics per PID. By Vadim Kochan.

6) Fixed buffer overflow in device_addr2str() causing Linux cooked
   header dissector to crash. Reported by @jamieparfet, fixed by Tobias
   Klauser.

7) Fixed build due to missing GENL_ID_GENERATE if compiling against
   kernel headers >= 4.10 (see issue #171). Reported by Jaroslav Škarvada,
   fixed by Tobias Klauser.

Contributions since last release:

 24  Tobias Klauser
 20  Vadim Kochan
  4  Mandar Gokhale
  1  Ken Wu

Git changelog since last release:

Tobias Klauser (24):
  AUTHORS: Add Mandar Gokhale
  build: configure: clearify missing defs warning
  build: configure: rewrap missing defs warning
  AUTHORS: Add Ken Wu
  all: Remove unused longindex parameter to getopt_long()
  netsniff-ng: proto_ipv4: Move sockaddr_in declarations to narrower scope
  Revert "mausezahn: Add error handling for mismatched address families"
  bpfc: make verbose parameter boolean
  sock: change type of verbose parameter to set_sock_qdisc_bypass()
  zsh: trafgen: Add missing option -q/--qdisc-path to zsh completion
  trafgen: proto: Zero out newly allocated struct packet
  trafgen: proto: Add missing brace
  bpfc: Remove unnecessary prototype for compile_filter()
  bpf: Fix confusing panic() message
  trafgen: proto: Zero out the correct packet slot
  str: Avoid trailing space in string returng by argv2str()
  gittattributes: Exclude dotfiles from created archives
  trafgen: parser: Add terminating ';' to mpls_expr and icmpv6_proto rules
  list: Remove cds_list_* wrappers
  make: Fix spelling yaac -> yacc
  build: Don't show echo commands in verbose mode
  dev: Fix buffer overflow in device_addr2str()
  netsniff-ng: nlmsg: Drop dissection of GENL_ID_GENERATE type
  netsniff-ng v0.6.3

Vadim Kochan (20):
  trafgen: parser: Fix undefined ETH_P_802AD on 2.6.x Linux
  flowtop: Move & refactor walk_processes() to proc.c
  trafgen: parser: Rename field_expr -> field_value_expr rule
  trafgen: proto: Change __proto_field_set_bytes(...) to take struct 
proto_field
  trafgen: proto: Rename proto_field_xxx -> proto_hdr_field_xxx
  trafgen: proto: Add proto field only setters/getters
  trafgen: parser: Parse IPv6 address by strict match pattern
  trafgen: parser: Allow to set function at field offset
  man: trafgen: Add short description about field offset usage
  flowtop: Minimize delay via halfdelay(1) function
  flowtop: Replace single linked list by list_head from list.h
  flowtop: Add tab control to switch between tables
  flowtop: Add process UI tab entry
  trafgen: l3: Fix checksum for UDP/TCP protos
  geoip: Fix memory leak when using GeoIPRecord
  trafgen: l3: Make possible to send frames via tun device
  pcap_io: Add function to get packet timestamp
  trafgen: Allow send packets from pcap file
  trafgen: man: Add description with pcap file for -i, --in option
  trafgen: parser: Rename bytes -> mac

Mandar Gokhale (4):
  mausezahn: Typofix & grammar change
  man: mausezahn: Add -6 option to manpage
  mausezahn: Use all-nodes link local address for IPv6
  mausezahn: Add error handling for mismatched address families

Ken Wu (1):
  trafgen: l3: Support interface without IP address

-- 
You received this message because you are subscribed to the Google Groups 
"netsniff-ng" group.
To unsubscribe from this gr

[netsniff-ng] [ANNOUNCE] Tree closed for new features until v0.6.3

[Tobias Klauser]

It has again been quite some time since the last release and I'd like to
prepare for the release of netsniff-ng 0.6.3.

Thus, the tree [1] is now closed for new features as of today. This
means only bug fixes and small, non-intrusive fixes/cleanups will now be
considered for inclusion in v0.6.3. The final release is planned in two
weeks time. Afterwards the tree is open again for new features and more
experimental changes.

[1] https://github.com/netsniff-ng/netsniff-ng

Everybody who's interested in the netsniff-ng toolkit is kindly invited
to give the current git HEAD some good testing, report issues [2] or
send patches [3].

[2] https://github.com/netsniff-ng/netsniff-ng/issues
[3] https://github.com/netsniff-ng/netsniff-ng/blob/master/SubmittingPatches

Please see the shortlog below for an overview of the fixes and features
that were added since v0.6.2.

Thanks to everybody who contributed to the upcoming release so far:
Vadim Kochan, Mandar Gokhale, Ken Wu, Jaroslav Škarvada, @jamieparfet.

---

Ken Wu (1):
  trafgen: l3: Support interface without IP address

Mandar Gokhale (4):
  mausezahn: Typofix & grammar change
  man: mausezahn: Add -6 option to manpage
  mausezahn: Use all-nodes link local address for IPv6
  mausezahn: Add error handling for mismatched address families

Tobias Klauser (23):
  AUTHORS: Add Mandar Gokhale
  build: configure: clearify missing defs warning
  build: configure: rewrap missing defs warning
  AUTHORS: Add Ken Wu
  all: Remove unused longindex parameter to getopt_long()
  netsniff-ng: proto_ipv4: Move sockaddr_in declarations to narrower scope
  Revert "mausezahn: Add error handling for mismatched address families"
  bpfc: make verbose parameter boolean
  sock: change type of verbose parameter to set_sock_qdisc_bypass()
  zsh: trafgen: Add missing option -q/--qdisc-path to zsh completion
  trafgen: proto: Zero out newly allocated struct packet
  trafgen: proto: Add missing brace
  bpfc: Remove unnecessary prototype for compile_filter()
  bpf: Fix confusing panic() message
  trafgen: proto: Zero out the correct packet slot
  str: Avoid trailing space in string returng by argv2str()
  gittattributes: Exclude dotfiles from created archives
  trafgen: parser: Add terminating ';' to mpls_expr and icmpv6_proto rules
  list: Remove cds_list_* wrappers
  make: Fix spelling yaac -> yacc
  build: Don't show echo commands in verbose mode
  dev: Fix buffer overflow in device_addr2str()
  netsniff-ng: nlmsg: Drop dissection of GENL_ID_GENERATE type

Vadim Kochan (20):
  trafgen: parser: Fix undefined ETH_P_802AD on 2.6.x Linux
  flowtop: Move & refactor walk_processes() to proc.c
  trafgen: parser: Rename field_expr -> field_value_expr rule
  trafgen: proto: Change __proto_field_set_bytes(...) to take struct 
proto_field
  trafgen: proto: Rename proto_field_xxx -> proto_hdr_field_xxx
  trafgen: proto: Add proto field only setters/getters
  trafgen: parser: Parse IPv6 address by strict match pattern
  trafgen: parser: Allow to set function at field offset
  man: trafgen: Add short description about field offset usage
  flowtop: Minimize delay via halfdelay(1) function
  flowtop: Replace single linked list by list_head from list.h
  flowtop: Add tab control to switch between tables
  flowtop: Add process UI tab entry
  trafgen: l3: Fix checksum for UDP/TCP protos
  geoip: Fix memory leak when using GeoIPRecord
  trafgen: l3: Make possible to send frames via tun device
  pcap_io: Add function to get packet timestamp
  trafgen: Allow send packets from pcap file
  trafgen: man: Add description with pcap file for -i, --in option
  trafgen: parser: Rename bytes -> mac

-- 
You received this message because you are subscribed to the Google Groups 
"netsniff-ng" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[netsniff-ng] Re: [RFC 6/7] trafgen: l7: Add DNS header generation API

[Tobias Klauser]

On 2017-03-14 at 20:15:45 +0100, Vadim Kochan <vadi...@gmail.com> wrote:
> On Tue, Mar 14, 2017 at 12:21 PM, Tobias Klauser <tklau...@distanz.ch> wrote:
> > On 2017-01-30 at 09:33:29 +0100, Vadim Kochan <vadi...@gmail.com> wrote:
> >> Add trafgen_l7.c module with DNS proto header generation with
> >> support of filling DNS query/answer/authority/additional sections
> >> as sub headers.
> >>
> >> Introcuded new concept as 'sub header' which is needed to easy handle
> >> DNS sections which might be added on-demand, and to simplify using
> >> sub-header as regular header with a fields, offset, etc. There is a
> >> parent header which contains array of pointers of sub-headers, and the
> >> array is ordered as they are located in the parent header. The
> >> sub-headers mostly encapsulated by the parent header which 'knows'
> >> the semantic of them. The new proto_hdr->push_sub_header(...) callback
> >> was added to tell the parent header to push the sub-header's fields,
> >> sub-header also may have proto_ops which must be filled by the parent.
> >> This sub-header concept might be used in the future if it will be needed
> >> to support DHCP, WLAN headers.
> >>
> >> There are 4 kinds of DNS sub-headers - query, answer, authority,
> >> additional. 'id' of each sub-header is used to only differentiate these
> >> types of sections. These sections have strict order inside DNS header,
> >> and there was added the proto_hdr_move_sub_header(...) to sort them in
> >> required order.
> >
> > Might be a bit of a naive question: But wouldn't it be possible to
> > enforce the sub-header order through the parser (i.e. only allow
> > trafgen scripts which specify the respective sections in the right
> > order? This would safe us from doing the whole header sorting/moving
> > dance which looks a bit sacry to me (memmove of payload especially).
> >
> 
> Well, I was thinking about this and decided to do so because:
> 
> 1) I really think that there might be in the future libtrafgen
> which will handle sub-headers
>  automatically (if it is needed by protocol header) both for
> the application and trafgen tool (this
>  reason might be not so strict).

Well, that's a bit hypothetical ;) And a don't quite see how this
prevents us from going with the easier, less scary solution for now ;)

> 2) sub-headers might be a generic way for other protocol headers
> like DHCP, WLAN.

We could enforce sub-header order there too, no?

> Yes, the logic which does sub-headers moving is a bit scary as you
> said, not sure if I can re-write it
> to simplify it, will try.

Ok, let's try that. I'm not strictly against the current solution and I
see that there might be cases in the future where this would be
benefitial.  I'd just like to prevent us from introducing a lot of code
with growing complexity which could be prevented if we were a bit more
restrictive in terms of what the user can do.

Thanks
Tobias

-- 
You received this message because you are subscribed to the Google Groups 
"netsniff-ng" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[netsniff-ng] Re: [RFC 6/7] trafgen: l7: Add DNS header generation API

[Tobias Klauser]

On 2017-01-30 at 09:33:29 +0100, Vadim Kochan  wrote:
> Add trafgen_l7.c module with DNS proto header generation with
> support of filling DNS query/answer/authority/additional sections
> as sub headers.
> 
> Introcuded new concept as 'sub header' which is needed to easy handle
> DNS sections which might be added on-demand, and to simplify using
> sub-header as regular header with a fields, offset, etc. There is a
> parent header which contains array of pointers of sub-headers, and the
> array is ordered as they are located in the parent header. The
> sub-headers mostly encapsulated by the parent header which 'knows'
> the semantic of them. The new proto_hdr->push_sub_header(...) callback
> was added to tell the parent header to push the sub-header's fields,
> sub-header also may have proto_ops which must be filled by the parent.
> This sub-header concept might be used in the future if it will be needed
> to support DHCP, WLAN headers.
> 
> There are 4 kinds of DNS sub-headers - query, answer, authority,
> additional. 'id' of each sub-header is used to only differentiate these
> types of sections. These sections have strict order inside DNS header,
> and there was added the proto_hdr_move_sub_header(...) to sort them in
> required order.

Might be a bit of a naive question: But wouldn't it be possible to
enforce the sub-header order through the parser (i.e. only allow
trafgen scripts which specify the respective sections in the right
order? This would safe us from doing the whole header sorting/moving
dance which looks a bit sacry to me (memmove of payload especially).

A few minor comments inline below.

> Actually there are only 2 proto_hdr's which describes 4 DNS sections -
> query & rrecord, because rrecord covers another 3 - answer, auhority,
> additional which have the same layout.
> 
> Signed-off-by: Vadim Kochan 
> ---
>  trafgen/Makefile |   1 +
>  trafgen_l4.c |  32 ++
>  trafgen_l7.c | 175 
> +++
>  trafgen_l7.h |  45 ++
>  trafgen_proto.c  | 128 
>  trafgen_proto.h  |  18 +-
>  6 files changed, 398 insertions(+), 1 deletion(-)
>  create mode 100644 trafgen_l7.c
>  create mode 100644 trafgen_l7.h
> 
> diff --git a/trafgen/Makefile b/trafgen/Makefile
> index 876ed93..95a31e0 100644
> --- a/trafgen/Makefile
> +++ b/trafgen/Makefile
> @@ -25,6 +25,7 @@ trafgen-objs =  xmalloc.o \
>   trafgen_l2.o \
>   trafgen_l3.o \
>   trafgen_l4.o \
> + trafgen_l7.o \
>   trafgen_lexer.yy.o \
>   trafgen_parser.tab.o \
>   trafgen.o
> diff --git a/trafgen_l4.c b/trafgen_l4.c
> index 5a694b3..198d622 100644
> --- a/trafgen_l4.c
> +++ b/trafgen_l4.c
> @@ -80,6 +80,21 @@ static void udp_packet_finish(struct proto_hdr *hdr)
>   udp_csum_update(hdr);
>  }
>  
> +static void udp_set_next_proto(struct proto_hdr *hdr, enum proto_id pid)
> +{
> + uint16_t dport;
> +
> + switch (pid) {
> + case PROTO_DNS:
> + dport = 53;
> + break;
> + default:
> + bug();
> + }
> +
> + proto_hdr_field_set_default_be16(hdr, UDP_DPORT, dport);
> +}
> +
>  static const struct proto_ops udp_proto_ops = {
>   .id = PROTO_UDP,
>   .layer  = PROTO_L4,
> @@ -87,6 +102,7 @@ static const struct proto_ops udp_proto_ops = {
>   .packet_update  = udp_csum_update,
>   .packet_finish  = udp_packet_finish,
>   .field_changed  = udp_field_changed,
> + .set_next_proto = udp_set_next_proto,
>  };
>  
>  static struct proto_field tcp_fields[] = {
> @@ -160,6 +176,21 @@ static void tcp_csum_update(struct proto_hdr *hdr)
>   hdr->is_csum_valid = true;
>  }
>  
> +static void tcp_set_next_proto(struct proto_hdr *hdr, enum proto_id pid)
> +{
> + uint16_t dport;
> +
> + switch (pid) {
> + case PROTO_DNS:
> + dport = 53;
> + break;
> + default:
> + bug();
> + }
> +
> + proto_hdr_field_set_default_be16(hdr, TCP_DPORT, dport);
> +}
> +
>  static const struct proto_ops tcp_proto_ops = {
>   .id = PROTO_TCP,
>   .layer  = PROTO_L4,
> @@ -167,6 +198,7 @@ static const struct proto_ops tcp_proto_ops = {
>   .packet_update  = tcp_csum_update,
>   .packet_finish  = tcp_csum_update,
>   .field_changed  = tcp_field_changed,
> + .set_next_proto = tcp_set_next_proto,
>  };
>  
>  static struct proto_field icmpv4_fields[] = {
> diff --git a/trafgen_l7.c b/trafgen_l7.c
> new file mode 100644
> index 000..1e82ccb
> --- /dev/null
> +++ b/trafgen_l7.c
> @@ -0,0 +1,175 @@
> +/*
> + * netsniff-ng - the packet sniffing beast
> + * Subject to the GPL, version 2.
> + */
> +
> +#include 
> +
> +#include "str.h"
> +#include "xmalloc.h"
> +#include "built_in.h"
> +#include "trafgen_l7.h"
> +#include "trafgen_proto.h"
> +
> 

[netsniff-ng] Re: [RFC 0/7] Add DNS proto header support

[Tobias Klauser]

Hi Vadim

On 2017-02-21 at 16:35:12 +0100, Vadim Kochan  wrote:
> On Mon, Jan 30, 2017 at 10:33 AM, Vadim Kochan  wrote:
[...]
> > Vadim Kochan (7):
> >   trafgen: parser: Rename bytes -> mac
> >   trafgen: proto: Add 'len' parameter to *_set_bytes(...) functions
> >   trafgen: proto: Allow to set field with variable length
> >   trafgen: parser: Use proto_field_set_xxx where it is possible
> >   str: Add function for converting string into DNS name
> >   trafgen: l7: Add DNS header generation API
> >   trafgen: parser: Add syntax to generate DNS header
> >
> >  str.c|  37 +
> >  str.h|   1 +
> >  trafgen/Makefile |   1 +
> >  trafgen_l2.c |   6 +-
> >  trafgen_l4.c |  32 
> >  trafgen_l7.c | 175 +
> >  trafgen_l7.h |  45 +++
> >  trafgen_lexer.l  |  26 ++-
> >  trafgen_parser.y | 216 ---
> >  trafgen_proto.c  | 231 
> > +--
> >  trafgen_proto.h  |  23 +-
> >  11 files changed, 750 insertions(+), 43 deletions(-)
> >  create mode 100644 trafgen_l7.c
> >  create mode 100644 trafgen_l7.h
> >
> > --
> > 2.11.0
> >
> 
> Hi Tobias,
> 
> I am sorry for the reminder, just want clarify if you will continue to
> review this.

Sorry, I'm quite busy at the moment but haven't forgotten about your
series. I'll finish reviewing it once time allows...

Thanks
Tobias

-- 
You received this message because you are subscribed to the Google Groups 
"netsniff-ng" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[netsniff-ng] Re: [RFC 4/7] trafgen: parser: Use proto_field_set_xxx where it is possible

[Tobias Klauser]

On 2017-01-30 at 09:33:27 +0100, Vadim Kochan  wrote:
> Use proto_field_set_xxx(field, ...)  instead of
> proto_hdr_field_set_xxx(hdr, fid, ...) to be more generic and do not
> depent on 'hdr' variable.
> 
> Signed-off-by: Vadim Kochan 

This does not strictly need to be part of this series but the patch in
itself makes sense. Care to resend it separately based on master?

-- 
You received this message because you are subscribed to the Google Groups 
"netsniff-ng" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[netsniff-ng] Re: [RFC 1/7] trafgen: parser: Rename bytes -> mac

[Tobias Klauser]

On 2017-01-30 at 09:33:24 +0100, Vadim Kochan  wrote:
> Rename  token member to  as it is used only for MAC
> address parsing, for dynamic sized bytes array we have an .
> Signed-off-by: Vadim Kochan 

Applied, as this is not directly related to the rest of the series.

Thanks.

-- 
You received this message because you are subscribed to the Google Groups 
"netsniff-ng" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[netsniff-ng] Re: [RFC 2/7] trafgen: proto: Add 'len' parameter to *_set_bytes(...) functions

[Tobias Klauser]

On 2017-01-30 at 09:33:25 +0100, Vadim Kochan  wrote:
> Add 'len' parameter to *_set_bytes(...) functoins to have better
> control over it.

While this is certainly a good idea (and you are also going to use it in
the following patch), the current patch doesn't use the passed len
parameter, so it's rather pointless standalone. I suggest you also use
it in __proto_field_set_bytes to check against field->len.

-- 
You received this message because you are subscribed to the Google Groups 
"netsniff-ng" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[netsniff-ng] Re: [PATCH v2] trafgen: l3: Make possible to send frames via tun device

[Tobias Klauser]

On 2017-02-06 at 17:23:11 +0100, Vadim Kochan  wrote:
> tun interface does not have Ethernet header so lets push Ethernet
> header only if device supports this.
> 
> Signed-off-by: Vadim Kochan 

Applied, thank you.

-- 
You received this message because you are subscribed to the Google Groups 
"netsniff-ng" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[netsniff-ng] Re: [PATCH] geoip: Fix memory leak when using GeoIPRecord

[Tobias Klauser]

On 2017-02-04 at 10:56:14 +0100, Vadim Kochan  wrote:
> GeoIP_record_by_ipnum{,_v6} returns allocated pointer to
> GeoIPRecord with allocated city, region & postal_code which is
> not freed after the call.
> 
> Fixed by xstrdup-ing required GeoIPRecord member (city/region) and
> after calling GeoIPRecord_delete to free the geoip record.
> 
> Ofcourse it is needed to also free obtained city/region in netsniff-ng,
> astraceroute & flowtop tools.
> 
> Fixes: #169
> Signed-off-by: Vadim Kochan 

Applied, thanks.

-- 
You received this message because you are subscribed to the Google Groups 
"netsniff-ng" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[netsniff-ng] Re: [PATCH 2/3] trafgen: Allow send packets from pcap file

[Tobias Klauser]

On 2017-01-31 at 23:31:35 +0100, Vadim Kochan  wrote:
> Add ability to send packets from pcap file if it has
> ".pcap" extension via "-i,--in" option.
> 
> By default packet sending is delayed considering original
> packets timestamps if no rate or delay is specified via -b/-t options.
> 
> Signed-off-by: Vadim Kochan 
> ---
>  trafgen.c| 155 
> +--
>  trafgen/Makefile |   4 ++
>  trafgen_conf.h   |   3 ++
>  trafgen_parser.y |   2 +-
>  4 files changed, 137 insertions(+), 27 deletions(-)
> 
> diff --git a/trafgen.c b/trafgen.c
> index 524b260..c0d77a3 100644
> --- a/trafgen.c
> +++ b/trafgen.c
> @@ -5,6 +5,8 @@
>   * Subject to the GPL, version 2.
>   */
>  
> +#define _GNU_SOURCE
> +
>  #include 
>  #include 
>  #include 
> @@ -16,7 +18,6 @@
>  #include 
>  #include 
>  #include 
> -#include 
>  #include 
>  #include 
>  #include 
> @@ -55,6 +56,7 @@
>  #include "ring_tx.h"
>  #include "csum.h"
>  #include "trafgen_proto.h"
> +#include "pcap_io.h"
>  
>  #ifndef timeval_to_timespec
>  #define timeval_to_timespec(tv, ts) { \
> @@ -65,14 +67,17 @@
>  
>  enum shaper_type {
>   SHAPER_NONE,
> + SHAPER_DELAY,
>   SHAPER_PKTS,
>   SHAPER_BYTES,
> + SHAPER_TSTAMP,
>  };
>  
>  struct shaper {
>   enum shaper_type type;
>   unsigned long long sent;
>   unsigned long long rate;
> + struct timeval tstamp;
>   struct timeval start;
>   struct timeval end;
>   struct timespec delay;
> @@ -88,6 +93,7 @@ struct ctx {
>   struct sockaddr_in dest;
>   struct shaper sh;
>   char *packet_str;
> + char *pcap_in;
>  };
>  
>  struct cpu_stats {
> @@ -556,15 +562,12 @@ static int xmit_smoke_probe(int icmp_sock, struct ctx 
> *ctx)
>  
>  static bool shaper_is_set(struct shaper *sh)
>  {
> - if ((sh->delay.tv_sec | sh->delay.tv_nsec) > 0)
> - return true;
> -
> - return sh->type != SHAPER_NONE;
> +   return sh->type != SHAPER_NONE;
>  }
>  
>  static void shaper_init(struct shaper *sh)
>  {
> - if (sh->type == SHAPER_NONE)
> + if (sh->type == SHAPER_NONE || sh->type == SHAPER_DELAY)
>   return;
>  
>   memset(>delay, 0, sizeof(struct timespec));
> @@ -574,6 +577,12 @@ static void shaper_init(struct shaper *sh)
>  
>  static void shaper_set_delay(struct shaper *sh, time_t sec, long int ns)
>  {
> + if (!(sec | ns)) {
> + sh->type = SHAPER_NONE;
> + return;
> + }
> +
> + sh->type = SHAPER_DELAY;
>   sh->delay.tv_sec = sec;
>   sh->delay.tv_nsec = ns;
>  }
> @@ -586,23 +595,48 @@ static void shaper_set_rate(struct shaper *sh, unsigned 
> long long rate,
>   sh->type = type;
>  }
>  
> -static void shaper_delay(struct shaper *sh, unsigned long pkt_len)
> +static void shaper_set_tstamp(struct shaper *sh, struct timeval *tv)
>  {
> - if (sh->type != SHAPER_NONE)
> + sh->tstamp.tv_sec = tv->tv_sec;
> + sh->tstamp.tv_usec = tv->tv_usec;
> +}
> +
> +static void shaper_delay(struct shaper *sh, struct packet *pkt)
> +{
> + if (sh->type == SHAPER_BYTES || sh->type == SHAPER_PKTS) {
> + unsigned long pkt_len = pkt->len;
> +
>   sh->sent += sh->type == SHAPER_BYTES ? pkt_len : 1;
>  
> - if (sh->sent >= sh->rate && sh->rate > 0) {
> - struct timeval delay_us;
> - struct timeval time_sent;
> - struct timeval time_1s = { .tv_sec = 1 };
> + if (sh->sent >= sh->rate && sh->rate > 0) {
> + struct timeval delay_us;
> + struct timeval time_sent;
> + struct timeval time_1s = { .tv_sec = 1 };
> +
> + bug_on(gettimeofday(>end, NULL));
> + timersub(>end, >start, _sent);
> +
> + if (timercmp(_1s, _sent, > )) {
> + timersub(_1s, _sent, _us);
> + timeval_to_timespec(_us, >delay);
> + }
> + }
> + } else if (sh->type == SHAPER_TSTAMP) {
> + struct timeval pkt_diff;
> + struct timeval diff;
>  
>   bug_on(gettimeofday(>end, NULL));
> - timersub(>end, >start, _sent);
> + timersub(>end, >start, );
> + timersub(>tstamp, >tstamp, _diff);
> +
> + if (timercmp(, _diff, <)) {
> + struct timeval delay;
>  
> - if (timercmp(_1s, _sent, > )) {
> - timersub(_1s, _sent, _us);
> - timeval_to_timespec(_us, >delay);
> + timersub(_diff, , );
> + timeval_to_timespec(, >delay);
>   }
> +
> + memcpy(>tstamp, >tstamp, sizeof(sh->tstamp));
>   }
>  
>   if ((sh->delay.tv_sec | sh->delay.tv_nsec) > 0) {
> @@ -698,7 +732,7 @@ retry:
>   num--;
>  
>   if 

[netsniff-ng] Re: [PATCH 1/3] pcap_io: Add function to get packet timestamp

[Tobias Klauser]

On 2017-01-31 at 23:31:34 +0100, Vadim Kochan  wrote:
> Add pcap_get_tstamp(...) function to get packet's timestamp considering
> different packet types & bytes order.
> 
> Signed-off-by: Vadim Kochan 
> ---
>  pcap_io.h | 53 +
>  1 file changed, 53 insertions(+)
> 
> diff --git a/pcap_io.h b/pcap_io.h
> index 4e41362..7bf5fe6 100644
> --- a/pcap_io.h
> +++ b/pcap_io.h
> @@ -373,6 +373,59 @@ static inline void pcap_set_length(pcap_pkthdr_t *phdr, 
> enum pcap_type type, u32
>   }
>  }
>  
> +static inline void pcap_get_tstamp(pcap_pkthdr_t *phdr, enum pcap_type type,
> +struct timeval *tv)

This should take a struct timespec in order to retain nanosecond
precision.

> +{
> + switch (type) {
> + case DEFAULT:
> + case DEFAULT_LL:
> + tv->tv_sec = phdr->ppo.ts.tv_sec;
> + tv->tv_usec = phdr->ppo.ts.tv_usec;
> + break;
> +
> + case DEFAULT_SWAPPED:
> + case DEFAULT_LL_SWAPPED:
> + tv->tv_sec = ___constant_swab32(phdr->ppo.ts.tv_sec);
> + tv->tv_usec = ___constant_swab32(phdr->ppo.ts.tv_usec);
> + break;
> +
> + case NSEC:
> + case NSEC_LL:
> + tv->tv_sec = phdr->ppn.ts.tv_sec;
> + tv->tv_usec = phdr->ppn.ts.tv_nsec / 1000;
> + break;
> +
> + case NSEC_SWAPPED:
> + case NSEC_LL_SWAPPED:
> + tv->tv_sec = ___constant_swab32(phdr->ppn.ts.tv_sec);
> + tv->tv_usec = ___constant_swab32(phdr->ppn.ts.tv_nsec) / 1000;
> + break;
> +
> + case KUZNETZOV:
> + tv->tv_sec = phdr->ppk.ts.tv_sec;
> + tv->tv_usec = phdr->ppk.ts.tv_usec;
> + break;
> +
> + case KUZNETZOV_SWAPPED:
> + tv->tv_sec = ___constant_swab32(phdr->ppk.ts.tv_sec);
> + tv->tv_usec = ___constant_swab32(phdr->ppk.ts.tv_usec);
> + break;
> +
> + case BORKMANN:
> + tv->tv_sec = phdr->ppb.ts.tv_sec;
> + tv->tv_usec = phdr->ppb.ts.tv_nsec / 1000;
> + break;
> +
> + case BORKMANN_SWAPPED:
> + tv->tv_sec = ___constant_swab32(phdr->ppb.ts.tv_sec);
> + tv->tv_usec = ___constant_swab32(phdr->ppb.ts.tv_nsec) / 1000;
> + break;
> +
> + default:
> + bug();
> + }
> +}
> +
>  static inline u32 pcap_get_hdr_length(pcap_pkthdr_t *phdr, enum pcap_type 
> type)
>  {
>   switch (type) {
> -- 
> 2.11.0
> 

-- 
You received this message because you are subscribed to the Google Groups 
"netsniff-ng" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[netsniff-ng] Re: [PATCH] trafgen: l3: Fix checksum for UDP/TCP protos

[Tobias Klauser]

On 2017-01-25 at 13:34:54 +0100, Vadim Kochan <vadi...@gmail.com> wrote:
> On Wed, Jan 25, 2017 at 2:24 PM, Tobias Klauser <tklau...@distanz.ch> wrote:
> > On 2017-01-23 at 22:49:22 +0100, Vadim Kochan <vadi...@gmail.com> wrote:
> >> While fixing the issue with getting of IPv4 address from device,
> >> the setting of default src IPv4/IPv6 addresses was moved from
> >> hdr->header_init(...) callback to hdr->packet_finish(...), but
> >> packet_finish(...) is called in the following order:
> >>
> >> udp_hdr->packet_finish() - UDP csum calculation over IPv4/6 pseudo 
> >> header
> >> ip4_hdr->packet_finish() - setting default src IPv4 address from dev
> >> ...
> >>
> >> So src IPv4/6 address will be set after UDP/TCP csum calculation which
> >> is wrong, so fixed issue by moving it to the hdr->header_init(...) stage
> >> as it was before the c4e07d5142c8.
> >>
> >> Fixes: c4e07d5142c8 ("trafgen: l3: Support interface without IP address")
> >> Signed-off-by: Vadim Kochan <vadi...@gmail.com>
> >
> > Applied, thanks Vadim.
> 
> BTW,
> We really need some sub-folder with scapy tests to check
> trafgen/mausezahn packet's generation.

Sure, feel free to send patches.

-- 
You received this message because you are subscribed to the Google Groups 
"netsniff-ng" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[netsniff-ng] Re: [PATCH] trafgen: l3: Fix checksum for UDP/TCP protos

[Tobias Klauser]

On 2017-01-23 at 22:49:22 +0100, Vadim Kochan  wrote:
> While fixing the issue with getting of IPv4 address from device,
> the setting of default src IPv4/IPv6 addresses was moved from
> hdr->header_init(...) callback to hdr->packet_finish(...), but
> packet_finish(...) is called in the following order:
> 
> udp_hdr->packet_finish() - UDP csum calculation over IPv4/6 pseudo header
> ip4_hdr->packet_finish() - setting default src IPv4 address from dev
> ...
> 
> So src IPv4/6 address will be set after UDP/TCP csum calculation which
> is wrong, so fixed issue by moving it to the hdr->header_init(...) stage
> as it was before the c4e07d5142c8.
> 
> Fixes: c4e07d5142c8 ("trafgen: l3: Support interface without IP address")
> Signed-off-by: Vadim Kochan 

Applied, thanks Vadim.

-- 
You received this message because you are subscribed to the Google Groups 
"netsniff-ng" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[netsniff-ng] Re: [PATCH v2 0/2] flowtop: Add UI tab control for process stats

[Tobias Klauser]

On 2017-01-19 at 00:09:02 +0100, Vadim Kochan  wrote:
> Add UI tab control to switch between tabbed windows for dirrent flows table
> statistics info per pid/peer/port.
> 
> The tab entries are switched by TAB keypress.
> 
> Introduced proc_entry which holds aggregated related flows statistics 
> (rate/pkts/bytes),
> beside this changed flow_entry to point to proc_entry pid & name info.
> 
> Added UI tab for per pid flows info - flows count, bytes, rates. The 
> bytes/pkts are collected
> during the process living (but proc_entry is not removed till at lest one 
> flow points to it),
> the rates info refreshes each 1 second per proc_entry.
> 
> Re-added list.h which contains just missing cds_list_{next,prev,last}_entry 
> functions which
> are needed for navigation over UI tab entries.
> 
> v2:
> 1) Constify ui_table.delim & ui_col.name
> 2) Rename list.h -> urcu-list-compat.h and include urcu/list.h here.
> 3) Add Signed-off-by into 2nd commit.
> 4) In flow_entry_find_process(...) use proc_entry->name member instead
> of local procname string.
> 5) Rename proc_exist -> proc_exists
> 6) Rename proc_entry.procnnum -> pid
> 7) Rename proc_entry.procname -> name
> 
> Vadim Kochan (2):
>   flowtop: Add tab control to switch between tables
>   flowtop: Add process UI tab entry

Series applied, thanks.

-- 
You received this message because you are subscribed to the Google Groups 
"netsniff-ng" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[netsniff-ng] Re: [PATCH 2/2] flowtop: Add process UI tab entry

[Tobias Klauser]

On 2017-01-18 at 14:00:25 +0100, Vadim Kochan  wrote:
> Add process UI tab entry to show flows statistics per pid.
> 
> Also changed flow_entry which now has pointer to new struct proc_entry
> object which contains process related info.
> 
> On each 1 second refresh proc_entry is checked if it exists by checking
> /proc/ path, and is deleted if there is no any flows related to it
> (flows_count is 0), if the process exists then dst & src rates info is
> zeroed and summed from the all related flows which are in the
> proc_entry->flows list.
> 
> The bytes & pkts amount info is collected during all the time process
> exists.

A Signed-off-by line is missing here ;)

A few minor remarks below.

> ---
>  flowtop.c | 348 
> +-
>  proc.c|  11 ++
>  proc.h|   2 +
>  3 files changed, 314 insertions(+), 47 deletions(-)
> 
> diff --git a/flowtop.c b/flowtop.c
> index 78ac253..f6ec4a6 100644
> --- a/flowtop.c
> +++ b/flowtop.c
> @@ -52,7 +52,25 @@
>  #define USEC_PER_SEC 100L
>  #endif
>  
> +struct proc_entry {
> + struct cds_list_head entry;
> + struct cds_list_head flows;
> + struct rcu_head rcu;
> +
> + struct timeval last_update;
> + unsigned int procnum;
> + char procname[256];
> + uint64_t pkts_src, bytes_src;
> + uint64_t pkts_dst, bytes_dst;
> + double rate_bytes_src;
> + double rate_bytes_dst;
> + double rate_pkts_src;
> + double rate_pkts_dst;
> + int flows_count;
> +};
> +
>  struct flow_entry {
> + struct cds_list_head proc_head;
>   struct cds_list_head entry;
>   struct rcu_head rcu;
>  
> @@ -69,9 +87,8 @@ struct flow_entry {
>   char country_code_src[4], country_code_dst[4];
>   char city_src[128], city_dst[128];
>   char rev_dns_src[256], rev_dns_dst[256];
> - char procname[256];
> + struct proc_entry *proc;
>   int inode;
> - unsigned int procnum;
>   bool is_visible;
>   struct nf_conntrack *ct;
>   struct timeval last_update;
> @@ -85,6 +102,10 @@ struct flow_list {
>   struct cds_list_head head;
>  };
>  
> +struct proc_list {
> + struct cds_list_head head;
> +};
> +
>  enum flow_direction {
>   FLOW_DIR_SRC,
>   FLOW_DIR_DST,
> @@ -129,6 +150,7 @@ static volatile bool do_reload_flows;
>  static volatile bool is_flow_collecting;
>  static volatile sig_atomic_t sigint = 0;
>  static int what = INCLUDE_IPV4 | INCLUDE_IPV6 | INCLUDE_TCP;
> +static struct proc_list proc_list;
>  static struct flow_list flow_list;
>  static struct sysctl_params_ctx sysctl = { -1, -1 };
>  
> @@ -156,10 +178,22 @@ enum tbl_flow_col {
>   TBL_FLOW_RATE,
>  };
>  
> +enum tbl_proc_col {
> + TBL_PROC_NAME,
> + TBL_PROC_PID,
> + TBL_PROC_FLOWS,
> + TBL_PROC_BYTES_SRC,
> + TBL_PROC_RATE_SRC,
> + TBL_PROC_BYTES_DST,
> + TBL_PROC_RATE_DST,
> +};
> +
>  static struct ui_table flows_tbl;
> +static struct ui_table procs_tbl;
>  
>  enum tab_entry {
>   TAB_FLOWS,
> + TAB_PROCS,
>  };
>  
>  static const char *short_options = "vhTUsDIS46ut:nGb";
> @@ -432,6 +466,11 @@ static int flow_list_del_entry(struct flow_list *fl, 
> const struct nf_conntrack *
>  
>   n = flow_list_find_id(fl, nfct_get_attr_u32(ct, ATTR_ID));
>   if (n) {
> + if (n->proc) {
> + cds_list_del(>proc_head);
> + n->proc->flows_count--;
> + }
> +
>   cds_list_del_rcu(>entry);
>   call_rcu(>rcu, flow_entry_xfree_rcu);
>   }
> @@ -449,22 +488,71 @@ static void flow_list_destroy(struct flow_list *fl)
>   }
>  }
>  
> +static void proc_list_init(struct proc_list *proc_list)
> +{
> + CDS_INIT_LIST_HEAD(_list->head);
> +}
> +
> +static struct proc_entry *proc_list_new_entry(unsigned int pid, const char 
> *name)
> +{
> + struct proc_entry *proc;
> +
> + cds_list_for_each_entry(proc, _list.head, entry) {
> + if (proc->procnum && proc->procnum == pid)
> + return proc;
> + }
> +
> + proc = xzmalloc(sizeof(*proc));
> +
> + strlcpy(proc->procname, name, sizeof(proc->procname));
> + bug_on(gettimeofday(>last_update, NULL));
> + CDS_INIT_LIST_HEAD(>flows);
> + proc->procnum = pid;
> +
> + cds_list_add_tail(>entry, _list.head);
> +
> + return proc;
> +}
> +
> +static void proc_entry_xfree_rcu(struct rcu_head *head)
> +{
> + struct proc_entry *p = container_of(head, struct proc_entry, rcu);
> +
> + xfree(p);
> +}
> +
> +static void proc_list_destroy(struct proc_list *pl)
> +{
> + struct proc_entry *p, *tmp;
> +
> + cds_list_for_each_entry_safe(p, tmp, >head, entry) {
> + cds_list_del_rcu(>entry);
> + call_rcu(>rcu, proc_entry_xfree_rcu);
> + }
> +}
> +
>  static void flow_entry_find_process(struct flow_entry *n)
>  {
> + char procname[256];
>   char cmdline[512];
>   pid_t pid;
>   int ret;
>  

[netsniff-ng] Re: [PATCH 1/2] flowtop: Add tab control to switch between tables

[Tobias Klauser]

On 2017-01-18 at 14:00:24 +0100, Vadim Kochan  wrote:
> Add ui_tab API to create ui tab control to switch between
> different ui tables which may contain different aggregated
> info per unique pid/port/proto/dst/src.
> 
> Meanwhile there is only 1 ui tab entry for flows table.
> 
> Added some missing cds_list_{next,prev,last}_entry functions
> into list.h header.
> 
> Signed-off-by: Vadim Kochan 

Some small remarks below, otherwise looks good from a first sight.

> ---
>  flowtop.c | 26 +++--
>  list.h| 19 +
>  ui.c  | 95 
> ++-
>  ui.h  | 30 
>  4 files changed, 166 insertions(+), 4 deletions(-)
>  create mode 100644 list.h
> 
> diff --git a/flowtop.c b/flowtop.c
> index b2d6546..78ac253 100644
> --- a/flowtop.c
> +++ b/flowtop.c
> @@ -133,6 +133,8 @@ static struct flow_list flow_list;
>  static struct sysctl_params_ctx sysctl = { -1, -1 };
>  
>  static unsigned int cols, rows;
> +static WINDOW *screen;
> +static int skip_lines;
>  
>  static unsigned int interval = 1;
>  static bool show_src = false;
> @@ -156,6 +158,10 @@ enum tbl_flow_col {
>  
>  static struct ui_table flows_tbl;
>  
> +enum tab_entry {
> + TAB_FLOWS,
> +};
> +
>  static const char *short_options = "vhTUsDIS46ut:nGb";
>  static const struct option long_options[] = {
>   {"ipv4",no_argument,NULL, '4'},
> @@ -1197,11 +1203,15 @@ static void flows_table_init(struct ui_table *tbl)
>   ui_table_header_color_set(_tbl, COLOR(BLACK, GREEN));
>  }
>  
> +static void tab_main_on_open(struct ui_tab *tab, enum ui_tab_event_t evt, 
> uint32_t id)
> +{
> + draw_flows(screen, _list, skip_lines);
> +}
> +
>  static void presenter(void)
>  {
>   bool show_help = false;
> - int skip_lines = 0;
> - WINDOW *screen;
> + struct ui_tab *tab_main;
>  
>   lookup_init(LT_PORTS_TCP);
>   lookup_init(LT_PORTS_UDP);
> @@ -1219,6 +1229,12 @@ static void presenter(void)
>  
>  flows_table_init(_tbl);
>  
> + tab_main = ui_tab_create();
> + ui_tab_event_cb_set(tab_main, tab_main_on_open);
> + ui_tab_pos_set(tab_main, 2, 0);
> + ui_tab_active_color_set(tab_main, COLOR(BLACK, GREEN));
> + ui_tab_entry_add(tab_main, TAB_FLOWS, "Flows");
> +
>   rcu_register_thread();
>   while (!sigint) {
>   int ch;
> @@ -1278,6 +1294,9 @@ static void presenter(void)
>   show_option_toggle(ch);
>   do_reload_flows = true;
>   break;
> + case '\t':
> + ui_tab_event_send(tab_main, UI_EVT_SELECT_NEXT);
> + break;
>   default:
>   fflush(stdin);
>   break;
> @@ -1288,13 +1307,14 @@ static void presenter(void)
>   if (show_help)
>   draw_help();
>   else
> - draw_flows(screen, _list, skip_lines);
> + ui_tab_show(tab_main);
>  
>   draw_footer();
>   }
>   rcu_unregister_thread();
>  
>   ui_table_uninit(_tbl);
> + ui_tab_destroy(tab_main);
>  
>   screen_end();
>   lookup_cleanup(LT_PORTS_UDP);
> diff --git a/list.h b/list.h
> new file mode 100644
> index 000..44dc8a2
> --- /dev/null
> +++ b/list.h

How about naming this something like list-compat.h or
urcu-list-compat.h to make its propose a bit clearer?

> @@ -0,0 +1,19 @@
> +#ifndef LIST_H
> +#define LIST_H

Please #include  here.

> +
> +#ifndef cds_list_last_entry
> +#define cds_list_last_entry(ptr, type, member) \
> + cds_list_entry((ptr)->prev, type, member)
> +#endif
> +
> +#ifndef cds_list_next_entry
> +#define cds_list_next_entry(pos, member) \
> + cds_list_entry((pos)->member.next, typeof(*(pos)), member)
> +#endif
> +
> +#ifndef cds_list_prev_entry
> +#define cds_list_prev_entry(pos, member) \
> + cds_list_entry((pos)->member.prev, typeof(*(pos)), member)
> +#endif
> +
> +#endif /* LIST_H */
> diff --git a/ui.c b/ui.c
> index 78d1560..d8e0213 100644
> --- a/ui.c
> +++ b/ui.c
> @@ -7,6 +7,8 @@
>  
>  #include "ui.h"
>  #include "str.h"
> +#include "list.h"
> +#include "screen.h"
>  #include "xmalloc.h"
>  
>  static struct ui_text *ui_text_alloc(size_t len)
> @@ -62,6 +64,7 @@ void ui_table_init(struct ui_table *tbl)
>   tbl->height  = LINES - 2;
>   tbl->col_pad = 1;
>   tbl->row = ui_text_alloc(tbl->width);
> + tbl->delim   = " ";
>  
>   CDS_INIT_LIST_HEAD(>cols);
>  }
> @@ -134,6 +137,11 @@ void ui_table_col_align_set(struct ui_table *tbl, int 
> col_id, enum ui_align alig
>   col->align = align;
>  }
>  
> +void ui_table_col_delim_set(struct ui_table *tbl, char *delim)

const char *delim?

> +{
> + tbl->delim = delim;
> +}
> +
>  void ui_table_row_add(struct ui_table *tbl)
>  {
>   tbl->rows_y++;
> @@ -166,7 +174,7 @@ static void 

[netsniff-ng] Re: list: Remove cds_list_* wrappers

[Tobias Klauser]

On 2017-01-17 at 16:36:53 +0100, Vadim Kochan  wrote:
> On Tue, Jan 17, 2017 at 5:30 PM, Vadim Kochan  wrote:
> > Hi Tobias,
> >
> > Just some thoughts from me regarding this commit, if the below
> > can make a sense.
> >
> > In case if list_head (now cds_list_head) will be used by other modules
> > and lets imagine that we will get rid of liburcu dependency (for example
> > if we can make flowtop single threaded) then we will need to have copy-paste
> > cds_* API from urcu or have a wrappers for it, so may be it is better
> > to have generic name w/o cds_ prefix ?
> >
> > Regards,
> > Vadim Kochan
> 
> Sorry for the noise.
> 
> Well now I think that cds_ will be used for RCU like list usage, and
> if to use simple list_head then we will need to have
> local list.h to do not make dpendencies to liburcu.

Exactly. And if you read the reference from the commit, the reason for
the name choice is clearly stated. IMO it's not really nice to hide data
structures and APIs behind #defines just because we don't like the name
or whatever. This will also unnecessarily confuse readers of the code.

-- 
You received this message because you are subscribed to the Google Groups 
"netsniff-ng" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[netsniff-ng] Re: [PATCH v2] flowtop: Replace single linked list by list_head from list.h

[Tobias Klauser]

On 2017-01-13 at 12:06:07 +0100, Vadim Kochan  wrote:
> list.h provides generic Linux-like linked list API which also supports
> RCU list operations.
> 
> Also additionally was removed the spinlock which is not needed for
> RCU-list operations, for the list_del_rcu(...) case it is needed
> additionally call call_rcu(...) before free the flow entry.
> 
> Because of full RCU support now flows are freed after grace-period
> (after presenter leaves RCU lock) via calling call_rcu(), because
> of that for the new entries we return NFCT_CB_STOLEN to tell conntrack
> API do not automatically free received nfct_conntrack object, it will be
> freed by us via call_rcu(...) therefor no need to use nfct_clone(n).
> 
> Signed-off-by: Vadim Kochan 

Applied, thank you!

-- 
You received this message because you are subscribed to the Google Groups 
"netsniff-ng" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[netsniff-ng] Re: [PATCH] flowtop: Replace single linked list by list_head from list.h

[Tobias Klauser]

On 2017-01-12 at 15:54:31 +0100, Vadim Kochan <vadi...@gmail.com> wrote:
> On Thu, Jan 12, 2017 at 4:28 PM, Tobias Klauser <tklau...@distanz.ch> wrote:
[...]
> >>  enum flow_direction {
> >> @@ -355,15 +357,15 @@ static inline struct flow_entry 
> >> *flow_entry_xalloc(void)
> >>  static inline void flow_entry_xfree(struct flow_entry *n)
> >>  {
> >>   if (n->ct)
> >> - nfct_destroy(n->ct);
> >> + xfree(n->ct);
> >
> > This would leak memory allocated internally in struct nf_contrack, no?
> > What's the reason for this change?
> 
> nfct_destroy(ct) may fail if we free entry after nfct event processing
> (our free is defered now because of RCU),
> so using just free(x) is safer because we do just nfct_clone(ct) which
> just allocated another nfct_conntrack entry for us,
> but ofcourse it would be better to have something nfct_free(x), so I
> agree this is not nice but safer.

As long as it causes memory to be leaked IMO it is equally bad.

If flow_entry_xfree() and thus nfct_destroy() is called via the
call_rcu() wrapper I proposed, it should no longer be unsafe to call
nfct_destroy()

-- 
You received this message because you are subscribed to the Google Groups 
"netsniff-ng" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[netsniff-ng] Re: [PATCH] flowtop: Replace single linked list by list_head from list.h

[Tobias Klauser]

Thanks for the patch and sorry for the delay in reply.

I'm not too familiar with the RCU list interface, but from a quick
glance this looks OK in general. A few remarks below.

Daniel, if you have time could you maybe have a quick look?

On 2017-01-09 at 07:26:07 +0100, Vadim Kochan  wrote:
> list.h provides generic Linux-like linked list API which also supports
> RCU list operations.
> 
> Also additionally was removed the spinlock which is not needed for
> RCU-list operations, for the list_del_rcu(...) case it is needed
> additionally call synchronize_rcu(...) before free the flow entry.
> 
> Because of full RCU support now flows are freed after grace-period via
> calling synchronize_rcu() and these removed-and-ready-to-free entries
> are kept in separate struct list_head.

Do we really need the separate free list? Couldn't we add a struct
rcu_head to struct flow_entry instead and then use something along the
lines of:

static void flow_entry_xfree_rcu(struct rcu_head *rcu)
{
struct flow_entry *entry = caa_container_of(rcu, struct flow_entry, 
rcu_head);
flow_entry_free(entry);
}

call_rcu(>rcu_head, free_entry_rcu);

to free the entries after the RCU grace period?

> Signed-off-by: Vadim Kochan 

> ---
>  flowtop.c | 121 
> +++---
>  1 file changed, 44 insertions(+), 77 deletions(-)
> 
> diff --git a/flowtop.c b/flowtop.c
> index f48f5c8..a5ac2bb 100644
> --- a/flowtop.c
>+++ b/flowtop.c

The #include "locking.h" should be removed as non of the symbols defined
there are used anymore in flowtop.c

> @@ -51,6 +51,9 @@
>  #endif
>  
>  struct flow_entry {
> + struct list_head entry;
> + struct list_head free;
> +
>   uint32_t flow_id, use, status;
>   uint8_t  l3_proto, l4_proto;
>   uint32_t ip4_src_addr, ip4_dst_addr;
> @@ -65,7 +68,6 @@ struct flow_entry {
>   char city_src[128], city_dst[128];
>   char rev_dns_src[256], rev_dns_dst[256];
>   char procname[256];
> - struct flow_entry *next;
>   int inode;
>   unsigned int procnum;
>   bool is_visible;
> @@ -78,8 +80,8 @@ struct flow_entry {
>  };
>  
>  struct flow_list {
> - struct flow_entry *head;
> - struct spinlock lock;
> + struct list_head head;
> + struct list_head free;
>  };
>  
>  enum flow_direction {
> @@ -355,15 +357,15 @@ static inline struct flow_entry *flow_entry_xalloc(void)
>  static inline void flow_entry_xfree(struct flow_entry *n)
>  {
>   if (n->ct)
> - nfct_destroy(n->ct);
> + xfree(n->ct);

This would leak memory allocated internally in struct nf_contrack, no?
What's the reason for this change?

>   xfree(n);
>  }
>  
>  static inline void flow_list_init(struct flow_list *fl)
>  {
> - fl->head = NULL;
> - spinlock_init(>lock);
> + INIT_LIST_HEAD(>head);
> + INIT_LIST_HEAD(>free);
>  }
>  
>  static inline bool nfct_is_dns(const struct nf_conntrack *ct)
> @@ -392,84 +394,60 @@ static void flow_list_new_entry(struct flow_list *fl, 
> const struct nf_conntrack
>   flow_entry_from_ct(n, ct);
>   flow_entry_get_extended(n);
>  
> - rcu_assign_pointer(n->next, fl->head);
> - rcu_assign_pointer(fl->head, n);
> + list_add_rcu(>entry, >head);
>  
>   n->is_visible = true;
>  }
>  
> -static struct flow_entry *flow_list_find_id(struct flow_list *fl,
> - uint32_t id)
> +static struct flow_entry *flow_list_find_id(struct flow_list *fl, uint32_t 
> id)
>  {
> - struct flow_entry *n = rcu_dereference(fl->head);
> + struct flow_entry *n;
>  
> - while (n != NULL) {
> + list_for_each_entry_rcu(n, >head, entry) {
q
>   if (n->flow_id == id)
>   return n;
> -
> - n = rcu_dereference(n->next);
>   }
>  
>   return NULL;
>  }
>  
> -static struct flow_entry *flow_list_find_prev_id(const struct flow_list *fl,
> -  uint32_t id)
> +static void flow_list_del_entry(struct flow_list *fl, const struct 
> nf_conntrack *ct)
>  {
> - struct flow_entry *prev = rcu_dereference(fl->head), *next;
> -
> - if (prev->flow_id == id)
> - return NULL;
> -
> - while ((next = rcu_dereference(prev->next)) != NULL) {
> - if (next->flow_id == id)
> - return prev;
> + struct flow_entry *n;
>  
> - prev = next;
> + n = flow_list_find_id(fl, nfct_get_attr_u32(ct, ATTR_ID));
> + if (n) {
> + list_del_rcu(>entry);
> + list_add_rcu(>free, >free);

Here would be the call_rcu() outlined above (instead of list_add_rcu)

>   }
> -
> - return NULL;
>  }
>  
> -static void flow_list_destroy_entry(struct flow_list *fl,
> - const struct nf_conntrack *ct)
> +static void flow_list_clear(struct flow_list *fl)
>  {
> - struct flow_entry *n1, *n2;
> - 

[netsniff-ng] Re: [PATCH 0/4] Introduce new pcap io API for pcap r/w accesses

[Tobias Klauser]

On 2017-01-12 at 13:59:05 +0100, Vadim Kochan <vadi...@gmail.com> wrote:
> On Wed, Dec 14, 2016 at 11:33 PM, Vadim Kochan <vadi...@gmail.com> wrote:
> > On Wed, Dec 14, 2016 at 11:26 PM, Tobias Klauser <tklau...@distanz.ch> 
> > wrote:
> >> On 2016-12-12 at 22:09:52 +0100, Vadim Kochan <vadi...@gmail.com> wrote:
> >>> Add new pcap io API to make pcap read/write accesses more
> >>> simpler and generic. Added pcap_io & pcap_packet struct's to
> >>> keep some internal pcap state like magic, link type & packet header
> >>> instead of to pass them like parameters and keep it all within 
> >>> netsniff-ng.c.
> >>>
> >>> Also such approach might be used to unify sniffing from ring buffer via
> >>> pcap io API similary as it is done with regular files.
> >>>
> >>> Some fast-path sensitive or setter/getter functions were inlined in 
> >>> pcap_io.h.
> >>
> >> This series touches very sensitive fast-path code. Before even reviewing
> >> it, I'd like to see performance measurments proofing that the your
> >> patches don't introduce any performance regressions and that inlining of
> >> these functions indeed does the job.
> >>
> >> Thanks!
> >
> > Hm, yeah, I need to think how to perform this on my laptop ...
> 
> I will try to measure time execution of critical code via clock()
> syscall for both versions if it will be OK for you, if yes -
> would be it OK to add some #IFDEF's (ahhh not too nice probably) to
> have such calculation persistent in the code ?

I think it should also be possible to test basic performance by just
invoking netsniff-ng under perf and then comparing profiles.

Of course you're also free to instrument the code for performance
measurments locally, but I'd rather not put any instrumentation code
into the upstream repo.

Thanks
Tobias

-- 
You received this message because you are subscribed to the Google Groups 
"netsniff-ng" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[netsniff-ng] Re: [PATCH v3] man: trafgen: Add short description about field offset using

[Tobias Klauser]

On 2016-12-21 at 19:33:33 +0100, Vadim Kochan  wrote:
> Add short note about field offset syntax with an example.
> 
> Signed-off-by: Vadim Kochan 

Applied, thanks!

-- 
You received this message because you are subscribed to the Google Groups 
"netsniff-ng" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[netsniff-ng] Re: [PATCH v2 6/7] trafgen: parser: Allow to set function at field offset

[Tobias Klauser]

On 2016-12-21 at 10:13:03 +0100, Vadim Kochan <vadi...@gmail.com> wrote:
> On Wed, Dec 21, 2016 at 10:17:17AM +0100, Tobias Klauser wrote:
> > On 2016-12-20 at 22:10:54 +0100, Vadim Kochan <vadi...@gmail.com> wrote:
> > > On Tue, Dec 20, 2016 at 12:33:47PM +0100, Tobias Klauser wrote:
[...]
> > > > And third, it's not immediately intuitive to which item the index
> > > > refers, i.e. is the MAC address ab:bb:cc:dd:ee:ff afterwards or
> > > > aa:bb:cc:dd:ee:00?
> > > 
> > > Indexing starts from 0 byte in the network order, please see a below 
> > > examples of
> > > trafgen + netsniff-ng outputs:
> > 
> > Ok. Could you please mention this fact (that indices start from byte 0
> > in network order) in the man page?
> > 
> 
> Hm, actually I did it in the man patch, but may be too shortly ...

Oops, sorry. I obviously didn't read the patch careful enough. Maybe you
could add one of the examples to the man page as well to make it
immediately obvious what the result will be?

[...]
> > > > Let me think about this a bit more...
> > > 
> > > I think we can have 1 or 3 different solutions:
> > > 
> > >   1) Current
> > > 
> > >   2) Current + function param default value
> > > 
> > >   3) Array-like
> > 
> > I'll go ahead and apply your series except for the man page patch. Could
> > you please send an update for this one incorporating the changes
> > mentioned above?
> > 
> > From there on we could then think about implementing option 3 in
> > addition...
> > 
> > Thanks a lot!
> 
> Sure I will collect your comments and use them for man page update.

Great, thanks.

Tobias

-- 
You received this message because you are subscribed to the Google Groups 
"netsniff-ng" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[netsniff-ng] Re: [PATCH v2 6/7] trafgen: parser: Allow to set function at field offset

[Tobias Klauser]

On 2016-12-18 at 10:52:49 +0100, Vadim Kochan  wrote:
> Extend proto field expression to:
> 
> proto_field[{index}:{len}] = {func}

I like the idea behind this very much, but I'm not particularly happy
with the syntax of this.

First, I find it a bit confusing that the length is specified and not
the end index (as at least I would assume it i.e. like array indexing in
Python works).

Second, the need to mention a field twice with
something like:

  eth(saddr=aa:bb:cc:dd:ee:ff, saddr[0]=dinc())

isn't very intuitive to understand IMO. And moreover, from the existing
syntax I'd assume some kind of function call semantics (like in C) with
the fields being parameters to the function. But indexing a function
parameter isn't really what I'm used to (but then again this is just
personal taste).

And third, it's not immediately intuitive to which item the index
refers, i.e. is the MAC address ab:bb:cc:dd:ee:ff afterwards or
aa:bb:cc:dd:ee:00?

But then I currently don't immediately see a better way to implement
this behavior and I also see why you'd chosen the indexing to work with
length rather than end index.

One option might be to let the user specify multi-byte/-word fields as a
C-array-like initializer list and support function calls therein, i.e.
the example above could be written as:

  eth(saddr={ 0xaa, 0xbb, 0xcc, 0xdd, 0xee, drnd() })

which would be a bit more in line with the traditional trafgen syntax.

Let me think about this a bit more...

-- 
You received this message because you are subscribed to the Google Groups 
"netsniff-ng" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[netsniff-ng] Re: [PATCH v2 5/7] trafgen: parser: Parse IPv6 address by strict match pattern

[Tobias Klauser]

On 2016-12-18 at 10:52:48 +0100, Vadim Kochan  wrote:
> Used IPv6 pattern from nftables project to match valid only IPv6
> address to do not mess with MAC or other syntax patterns with ':' symbol.

Please mention the nftables source where you took this from in a comment
in the lexer file.

-- 
You received this message because you are subscribed to the Google Groups 
"netsniff-ng" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[netsniff-ng] Re: [PATCH v4] flowtop: Move & refactor walk_processes(...) to proc.c

[Tobias Klauser]

On 2016-12-17 at 16:57:22 +0100, Vadim Kochan  wrote:
> Add proc_find_by_inode(...) in proc.c which finds pid by inode & gets
> processe's command line and use it in the flowtop.c instead of
> walk_processes(...).
> 
> Signed-off-by: Vadim Kochan 

Applied, thanks.

-- 
You received this message because you are subscribed to the Google Groups 
"netsniff-ng" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[netsniff-ng] Re: [PATCH 2/2] flowtop: Replace walk_processes(...) by proc_find_by_inode(...)

[Tobias Klauser]

On 2016-12-13 at 02:36:02 +0100, Vadim Kochan  wrote:
> Use proc_find_by_inode(...) from proc.c to find pid by inode
> and it's command line instead of having own specific process walker.
> 
> Signed-off-by: Vadim Kochan 
> ---
>  flowtop.c | 76 
> ---
>  1 file changed, 9 insertions(+), 67 deletions(-)
> 
> diff --git a/flowtop.c b/flowtop.c
> index 4f8cbcf..0691001 100644
> --- a/flowtop.c
> +++ b/flowtop.c

The #includes for sys/stat.h and dirent.h could be dropped from flowtop
now, no?

-- 
You received this message because you are subscribed to the Google Groups 
"netsniff-ng" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[netsniff-ng] Re: [PATCH 0/4] Introduce new pcap io API for pcap r/w accesses

[Tobias Klauser]

On 2016-12-12 at 22:09:52 +0100, Vadim Kochan  wrote:
> Add new pcap io API to make pcap read/write accesses more
> simpler and generic. Added pcap_io & pcap_packet struct's to
> keep some internal pcap state like magic, link type & packet header
> instead of to pass them like parameters and keep it all within netsniff-ng.c.
> 
> Also such approach might be used to unify sniffing from ring buffer via
> pcap io API similary as it is done with regular files.
> 
> Some fast-path sensitive or setter/getter functions were inlined in pcap_io.h.

This series touches very sensitive fast-path code. Before even reviewing
it, I'd like to see performance measurments proofing that the your
patches don't introduce any performance regressions and that inlining of
these functions indeed does the job.

Thanks!

-- 
You received this message because you are subscribed to the Google Groups 
"netsniff-ng" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[netsniff-ng] Re: [PATCH 1/2] proc: Add function for find process by inode

[Tobias Klauser]

On 2016-12-13 at 02:36:01 +0100, Vadim Kochan  wrote:
> Add proc_find_by_inode(...) which finds pid by inode & gets processe's
> command line.
> 
> Actually this function was taken from flowtop.c walk_process(...)
> and refactored to look more generic.
> 
> Signed-off-by: Vadim Kochan 
> ---
>  proc.c | 74 
> ++
>  proc.h |  1 +
>  2 files changed, 75 insertions(+)
> 
> diff --git a/proc.c b/proc.c
> index 76e3c93..8119226 100644
> --- a/proc.c
> +++ b/proc.c
> @@ -3,11 +3,13 @@
>  #endif
>  #include 
>  #include 
> +#include 
>  #include 
>  #include 
>  #include 
>  #include 
>  #include 
> +#include 
>  
>  #include "proc.h"
>  #include "die.h"
> @@ -83,6 +85,78 @@ ssize_t proc_get_cmdline(unsigned int pid, char *cmdline, 
> size_t len)
>   return ret;
>  }
>  
> +int __match_pid_by_inode(pid_t pid, ino_t ino)

Should be static as it is not used outside proc.c

-- 
You received this message because you are subscribed to the Google Groups 
"netsniff-ng" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[netsniff-ng] Re: [PATCH] ring_rx: Fix warning message if PACKET_FANOUT is not defined

[Tobias Klauser]

On 2016-11-22 at 00:09:32 +0100, Vadim Kochan  wrote:
> Move #ifdef PACKET_FANOUT at begining of join_fanout_group(...) function
> to fix warning message that 'fanout_opt' & 'ret' variables are not used
> if PACKET_FANOUT is not defined.
> 
> It may happen on 2.6.x Linux version.

Nack. This way we would always panic in ring_rx_setup() if PACKET_FANOUT
is not defined. We should allow to pass fanout_group = 0 (the default
setting, i.e. no fanout group) even for !PACKET_FANOUT.

-- 
You received this message because you are subscribed to the Google Groups 
"netsniff-ng" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [netsniff-ng] [ANNOUNCE] netsniff-ng 0.6.2

[Tobias Klauser]

On 2016-11-07 at 14:58:24 +0100, Tobias Klauser <tklau...@distanz.ch> wrote:
> It is our pleasure to announce the release of netsniff-ng 0.6.2.
> 
> The summary of changes and the short log of all changes since v0.6.1 can
> be found below and also on github at
> https://github.com/netsniff-ng/netsniff-ng/releases/tag/v0.6.2

I just realized I fat-fingered the version in the release commit
32804e5e and pushed it out too early (it says v0.5.2 instead of v0.6.2).
The tag, the GH release and all the tarballs show the correct version
though. Sorry about this!

-- 
You received this message because you are subscribed to the Google Groups 
"netsniff-ng" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[netsniff-ng] [ANNOUNCE] netsniff-ng 0.6.2

[Tobias Klauser]

It is our pleasure to announce the release of netsniff-ng 0.6.2.

The summary of changes and the short log of all changes since v0.6.1 can
be found below and also on github at
https://github.com/netsniff-ng/netsniff-ng/releases/tag/v0.6.2

Thanks to everybody who contributed to this release:

  Vadim Kochan, Paolo Abeni, Yousong Zhou, arch3y, Tommy Beadle, and
  Hisao Tanabe.

This release also means that the tree is again open for new features.

Happy packet sniffing!

---

netsniff-ng 0.6.2 (Circlon) has been released to the public.

It can be fetched via Git:

   git clone git://github.com/netsniff-ng/netsniff-ng.git
   git checkout v0.6.2

Or via HTTP from one of our mirrors:

   http://pub.netsniff-ng.org/netsniff-ng/netsniff-ng-0.6.2.tar.gz
   http://mirror.distanz.ch/netsniff-ng/netsniff-ng-0.6.2.tar.gz
   http://github.com/netsniff-ng/netsniff-ng/archive/v0.6.2.tar.gz

The release can be verified via Git (see README):

   git tag -v v0.6.2

Major high-level changes since the last release (v0.6.1) are:

1) Many improvements to the trafgen protocol generation functions, e.g. the
   ability to increment/randomize protocol fields at runtime using dinc()/rnd(),
   new functions to generate IPv6, ICMPv4/v6, PFC, and IEEE 802.3X PAUSH
   headers. From Vadim Kochan and Tobias Klauser.

2) flowtop UI improvements. From Vadim Kochan.

3) Packet counting fixes, e.g. to not account for duplicate packets received
   over the loopüback interface. From Paolo Abeni.

4) Allow IPv6 ranges to be specified for source and dest addresses in mausezahn.
   From Tommy Beadle.

5) Build system improvements, allowing to specify more options to the configure
   script. See './configure --help' for details. From Vadim Kochan and Tobias
   Klauser.

6) Various compilation fixes, e.g. for warnings or build failure certain
   systems. From Yousong Zhou and arch3y.

7) Various documentation improvements and fixes. From Vadim Kochan, Hisao
   Tanabe, and Tobias Klauser.

Contributions since last release:

 58  Vadim Kochan
 44  Tobias Klauser
  3  Paolo Abeni
  2  Yousong Zhou
  2  arch3y
  1  Tommy Beadle
  1  Hisao Tanabe

Git changelog since last release:

Vadim Kochan (58):
  geoip: Allow to get country 3-code
  flowtop: Change flows layout to 1-row view
  flowtop: Add display option to show src info
  screen: Add helpers to easy use color by name
  flowtop: Use new colors naming & helpers
  flowtop: Fix compilation error when build without geoip
  list: Add re-defined double-linked list API from liburcu
  ui: Implement UI table for flows printing
  flowtop: Use new UI table API for draw flows list
  ui: Print empty rows when clearing table
  flowtop: Get rid of clear() & refresh() calls
  flowtop: Simplify flows refresh delay
  flowtop: man: Add how-to activate conntrack by modprobe
  flowtop: Remove unused parameters from draw_flow_entry()
  netsniff-ng: pcap_io: Print unsupported magic number
  trafgen: parser: Split [e]type to separate keywords
  trafgen: parser: Replace 'mtype' by 'type'
  trafgen: proto: Add ICMPv4 header generation
  trafgen: ipv4: Set default proto as ipv6-in-ipv4 for ipv6()
  trafgen: ipv4: Do not use user-provided 'ihl' field to calculate csum
  trafgen: udp: Do not use user-provided 'len' field to calculate csum
  trafgen: Move applying of dynamic elements to own function
  trafgen: proto: Reference to packet from struct proto_hdr
  trafgen: proto: Move proto headers into packet
  trafgen: Allow to compile without libnl
  netsniff-ng: Allow to compile without libnl
  trafgen: proto: Use field id as array index
  colorize: Squash colorize macros into colors.h
  build: configure: Add option to enable debug symbols
  flowtop: Render table row via raw ncurses buffer
  flowtop: Add horizontal scrolling over flows table
  ui: Rename ui_table_row_print -> ui_table_row_col_set
  build: configure: Add option to specify install path for binary files
  build: configure: Add option to specify install path for /etc files
  mz: Check device argument by ifindex instead of name prefix
  trafgen: proto: Update field value at runtime
  trafgen: proto: Increment proto field at runtime
  trafgen: proto: Randomize proto field at runtime
  trafgen: ipv4: Update csum at runtime if needed
  trafgen: icmpv4: Update csum at runtime if needed
  trafgen: icmpv6: Update csum at runtime if needed
  trafgen: proto: Improve to find lower header by index
  trafgen: proto: Introduce proto_upper_header() function
  trafgen: udp: Update csum at runtime if needed
  trafgen: tcp: Update csum at runtime if it needed
  trafgen: parser: Unify proto field value parsing
  trafgen: parser: Add support of 'dinc' function for proto fields
  trafgen: parser: Add 'drnd()' function for proto fields
  trafge

[netsniff-ng] [ANNOUNCE] Tree closed for new features until v0.6.2 release

[Tobias Klauser]

It's already been more than 6 months since the last release and I think
it'd be time for v0.6.2 since there was quite some development since
v0.6.1.

Thus, we'd like to close the tree for new features as of today. This
means we'll only take bug fixes and small, non-intrusive fixes/cleanups
we deem safe for inclusion in v0.6.2. The final release is planned in
two weeks time. Afterwards the tree is open again for new features and
more experimental changes.

We'd like to ask everybody who's interested in the netsniff-ng toolkit
to give the current git HEAD some good testing.

Please see the shortlog below for an overview of the fixes and features
that were added since v0.6.1.

Thanks to everybody how contributed to the upcoming release so far!

-- Tobias


Hisao Tanabe (1):
  man: netsniff-ng: Fix usage example description

Paolo Abeni (3):
  netsniff-ng: Skip duplicated packets on loopback device
  netsniff-ng: Increment pkts_seen after packet type check
  netsniff-ng: Account skipped packets as 'seen' and 'dropped'

Tobias Klauser (41):
  AUTHORS: Add Tommy for his contribution
  AUTHORS: Add Arch3y for his contribution
  trafgen: proto: Inline proto_current_header() into its only caller
  build: configure: Write a comment to config.h for disabled features
  build: configure: Allow to disable support for libgeoip and zlib
  build: travis: Build with and without libGeoIP/zlib
  build: Always silence the no-op check command
  trafgen: proto: Replace panic() with bug() where applicable
  man: trafgen: Add example program, rewritten using the trafgen functions
  trafgen: proto: Add IPv6 header generation
  trafgen: proto: Add ICMPv6 header generation
  ioops: Add mkostemp_or_die() wrapper for mkostemp(3)
  trafgen: Use mkostemp_or_die() to create unique temporary file
  cpp: Use mkstemps() to create unique temporary file
  flowtop: Remove unused parameters from draw_help() and draw_footer()
  flowtop: Use fl argument instead of hard coded global flow_list
  ifpps: Mark arg parameter of on_panic_handler() as unused
  flowtop: Mark parameter in callback function as unused
  ring: Remove unused parameter sock from setup_ring_layout_generic()
  ring: Mark potentially unused parameters as such
  build: travis: Set up Coverity scan
  build: travis: Move coverity_scan to existing addons configuration
  doc: Document Travis CI Coverity integration
  man: mausezahn: Fix man warnings
  AUTHORS: Add Hisao Tanabe for his contribution
  AUTHORS: Add Paolo Abeni for his contribution
  dissectors: ethernet: Don't resolve OUI for locally administered addresses
  build: configure: Use command line options to disable optional libraries
  build: configure: Allow to compile tools without libnl
  trafgen: proto: Don't store context in each proto header
  trafgen: proto: Store registered protocols in an array
  trafgen: parser: Check read access to file before invoking cpp
  trafgen: proto: Make bytes pointer const in proto field set functions
  build: Disable all optimization options for debug builds
  trafgen: proto: Add space between switch statement and opening parenthesis
  trafgen: proto: Split static protocol definition out of struct proto_hdr
  trafgen: proto: Initialize lower proto header pointer on declaration
  ifpps: Prevent division by zero
  build: Simplify checking for PREFIX origin
  build: Unify default values for PREFIX and ETCDIR in configure
  AUTHORS: Add Yousong Zhou for his contribution

Tommy Beadle (1):
  mausezahn: Allow IPv6 ranges to be specified for source and dest addresses

Vadim Kochan (58):
  geoip: Allow to get country 3-code
  flowtop: Change flows layout to 1-row view
  flowtop: Add display option to show src info
  screen: Add helpers to easy use color by name
  flowtop: Use new colors naming & helpers
  flowtop: Fix compilation error when build without geoip
  list: Add re-defined double-linked list API from liburcu
  ui: Implement UI table for flows printing
  flowtop: Use new UI table API for draw flows list
  ui: Print empty rows when clearing table
  flowtop: Get rid of clear() & refresh() calls
  flowtop: Simplify flows refresh delay
  flowtop: man: Add how-to activate conntrack by modprobe
  flowtop: Remove unused parameters from draw_flow_entry()
  netsniff-ng: pcap_io: Print unsupported magic number
  trafgen: parser: Split [e]type to separate keywords
  trafgen: parser: Replace 'mtype' by 'type'
  trafgen: proto: Add ICMPv4 header generation
  trafgen: ipv4: Set default proto as ipv6-in-ipv4 for ipv6()
  trafgen: ipv4: Do not use user-provided 'ihl' field to calculate csum
  trafgen: udp: Do not use user-provided 'len' field to calculate csum
  trafgen: Move applying of dynamic elements to own 

[netsniff-ng] Re: [PATCH] trafgen: man: Fix Jesper's example link

[Tobias Klauser]

On 2016-09-23 at 14:20:19 +0200, Vadim Kochan <vadi...@gmail.com> wrote:
> The link:
> 
>   http://thread.gmane.org/gmane.linux.network/257155
> 
> does not work, so replaced to another one at:
> 
>   https://marc.info/?l=linux-netdev=135903630614184
> 
> Signed-off-by: Vadim Kochan <vadi...@gmail.com>
> Suggested-by: Tobias Klauser <tklau...@distanz.ch>

Applied, thanks.

-- 
You received this message because you are subscribed to the Google Groups 
"netsniff-ng" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[netsniff-ng] Re: [PATCH v2 0/2] build: configure: Add support for '--prefix' & '--sysconfdir' options

[Tobias Klauser]

On 2016-08-22 at 23:17:10 +0200, Vadim Kochan  wrote:
> Add new 'configure' options:
> 
> --prefix - to control of installation of binaries & shared files
> --sysconfdir - to control of installation of config files
> 
> The old 'make VAR=VALUE' should still work.
> 
> v2:
> 1) Reword commit subjects to do not contain option names.
> 
> Vadim Kochan (2):
>   build: configure: Add option to specify install path for binary files
>   build: configure: Add option to specify install path for /etc files

Applied, thanks. I simplified the help text a bit - since we only have
two directory related options, I combined them as I think it's easier to
read (even though we don't follow autotools help message format that
way).

-- 
You received this message because you are subscribed to the Google Groups 
"netsniff-ng" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[netsniff-ng] Re: [PATCH 0/3] flowtop: Add horizontal scrolling over flows

[Tobias Klauser]

On 2016-09-12 at 16:47:12 +0200, Vadim Kochan  wrote:
> On Tue, Aug 23, 2016 at 12:06 AM, Vadim Kochan  wrote:
> > In case of smaller than 130 chars display, some of the fields might
> > be hidden, therefore add ability to scroll to left/right over the end
> > of screen to see hidden columns.
> >
> > Also it will allow to extend flow line with additional fields (pps).
> >
> > Used approach from 'htop' tool where scrolling is made over custom
> > char-attr buffer element.
> >
> > Vadim Kochan (3):
> >   flowtop: Render table row via raw ncurses buffer
> >   flowtop: Add horizontal scrolling over flows table
> >   ui: Rename ui_table_row_print -> ui_table_row_col_set
> >
> >  flowtop.c | 75 +--
> >  ui.c  | 99 
> > ---
> >  ui.h  | 20 +++--
> >  3 files changed, 147 insertions(+), 47 deletions(-)
> >
> > --
> > 2.9.2
> >
> 
> Softly ping for review ...

Series now applied, thanks for the patience and sorry for the delay.

-- 
You received this message because you are subscribed to the Google Groups 
"netsniff-ng" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[netsniff-ng] Re: [PATCH] build: configure: Add option to enable debug symbols

[Tobias Klauser]

On 2016-08-13 at 01:39:33 +0200, Vadim Kochan  wrote:
> Add '--enable-debug' option to enable debugging symbols,
> the reason is to simplify it by ./configure it once and
> do not execute 'make DEBUG=1 ...' each time for a long
> debug session.
> 
> Signed-off-by: Vadim Kochan 

Appled, thanks.

-- 
You received this message because you are subscribed to the Google Groups 
"netsniff-ng" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[netsniff-ng] Re: [PATCH] tstamping: Move code to the sock.c

[Tobias Klauser]

On 2016-08-12 at 23:09:28 +0200, Vadim Kochan  wrote:
> Move hw timestamp enable/disable code to sock.c as
> it is a socket related functionality, and it makes
> have less files in source tree.
> 
> Move tstamping related includes under HAVE_TSTAMPING config
> to compile them only if hw timestamping is suported.
> 
> Signed-off-by: Vadim Kochan 
> ---
>  netsniff-ng.c|  1 -
>  netsniff-ng/Makefile |  3 ---
>  sock.c   | 39 +++
>  sock.h   |  1 +
>  tstamping.c  | 38 --
>  tstamping.h  | 15 ---
>  6 files changed, 40 insertions(+), 57 deletions(-)
>  delete mode 100644 tstamping.c
>  delete mode 100644 tstamping.h
> 
> diff --git a/netsniff-ng.c b/netsniff-ng.c
> index ce37e10..ec060f8 100644
> --- a/netsniff-ng.c
> +++ b/netsniff-ng.c
> @@ -44,7 +44,6 @@
>  #include "lockme.h"
>  #include "tprintf.h"
>  #include "timer.h"
> -#include "tstamping.h"
>  #include "dissector.h"
>  #include "xmalloc.h"
>  
> diff --git a/netsniff-ng/Makefile b/netsniff-ng/Makefile
> index d1d8a85..c6a531f 100644
> --- a/netsniff-ng/Makefile
> +++ b/netsniff-ng/Makefile
> @@ -74,9 +74,6 @@ endif
>  ifeq ($(CONFIG_GEOIP), 1)
>  netsniff-ng-objs +=  geoip.o
>  endif
> -ifeq ($(CONFIG_HWTSTAMP), 1)
> -netsniff-ng-objs +=  tstamping.o
> -endif
>  ifeq ($(CONFIG_LIBNL), 1)
>  netsniff-ng-objs +=  mac80211.o \
>   proto_nlmsg.o
> diff --git a/sock.c b/sock.c
> index a84796c..a4ec434 100644
> --- a/sock.c
> +++ b/sock.c
> @@ -1,7 +1,15 @@
> +#include "config.h"
> +
>  #include 
>  #include 
>  #include 
>  #include 
> +#ifdef HAVE_HARDWARE_TIMESTAMPING
> +#include 
> +#include 
> +#include 
> +#include 
> +#endif
>  #include 
>  #include 
>  
> @@ -196,3 +204,34 @@ void reset_system_socket_memory(int *vals, size_t len)
>   set_system_socket_mem(sock_wmem_max, vals[2]);
>   set_system_socket_mem(sock_wmem_def, vals[3]);
>  }
> +
> +int set_sockopt_hwtimestamp(int sock, const char *dev)
> +{
> +#ifdef HAVE_HARDWARE_TIMESTAMPING
> + int timesource, ret;
> + struct hwtstamp_config hwconfig;
> + struct ifreq ifr;
> +
> + if (!strncmp("any", dev, strlen("any")))
> + return -1;
> +
> + memset(, 0, sizeof(hwconfig));
> + hwconfig.tx_type = HWTSTAMP_TX_OFF;
> + hwconfig.rx_filter = HWTSTAMP_FILTER_ALL;
> +
> + memset(, 0, sizeof(ifr));
> + strlcpy(ifr.ifr_name, dev, sizeof(ifr.ifr_name));
> + ifr.ifr_data = 
> +
> + ret = ioctl(sock, SIOCSHWTSTAMP, );
> + if (ret < 0)
> + return -1;
> +
> + timesource = SOF_TIMESTAMPING_RAW_HARDWARE;
> +
> + return setsockopt(sock, SOL_PACKET, PACKET_TIMESTAMP, ,
> +   sizeof(timesource));
> +#else
> + return -1;
> +#endif
> +}

Please add the #ifdef around the entire function and keep the static
inline function returning -1 for !HAVE_HARDWARE_TIMESTAMPING in sock.h

-- 
You received this message because you are subscribed to the Google Groups 
"netsniff-ng" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[netsniff-ng] Re: [RFC PATCH] trafgen: proto: Split static protocol definition out of struct proto_hdr

[Tobias Klauser]

On 2016-08-11 at 20:11:02 +0200, Vadim Kochan <vadi...@gmail.com> wrote:
> On Thu, Aug 11, 2016 at 06:16:27PM +0200, Tobias Klauser wrote:
> > Currently struct proto_hdr is used twofold:
> > 
> >   1) Statically define protocol behavior, i.e. all the *_hdr definitions in
> >  trafgen_l{2,3,4}.c which map a protocol id/layer to a set of callback
> >  functions.
> > 
> >   2) For each packet created at parse time the struct is memcpy()'ed
> >  (including all the static information from 1) and then used to store
> >  dynamic information at parse/run time.
> > 
> > Thus, struct proto_hdr members such as the proto id, layer and the
> > pointers callback functions get copied for each created packet (in
> > addition to the other fields which get changed during parsing). Also,
> > static/dynamic information get mixed and we e.g. can't make the protocol
> > definitions const to ensure they'll not get changed by mistake.
> > 
> > Rather than copying the struct proto_hdr for every packet, clearly
> > separate the two purposes defined above by splitting struct proto_hdr
> > into two structs:
> > 
> >   1) struct proto_ops for the static (const) protocol behavior definition
> > 
> >   2) struct proto_hdr (reduced) for dynamic information
> > 
> > struct proto_hdr keeps a pointer to the corresponding proto_ops instance
> > and uses it to execute the corresponding callbacks.
> 
> You can add my "Acked-by" if it is needed.

Thanks, will add it to the final patch.

> > 
> > Reference: 
> > https://groups.google.com/forum/#!msg/netsniff-ng/20RvwJdh50Y/eMkbmKSaBgAJ
> > Signed-off-by: Tobias Klauser <tklau...@distanz.ch>
> > ---
> >  trafgen_l2.c | 24 ++---
> >  trafgen_l3.c |  8 +++
> >  trafgen_l4.c | 22 +--
> >  trafgen_parser.y |  2 +-
> >  trafgen_proto.c  | 65 
> > +---
> >  trafgen_proto.h  | 40 +-
> >  6 files changed, 83 insertions(+), 78 deletions(-)
> > 
> > diff --git a/trafgen_l2.c b/trafgen_l2.c
> > index 1863332b3543..f09b2a61cacc 100644
> > --- a/trafgen_l2.c
> > +++ b/trafgen_l2.c
> > @@ -47,7 +47,7 @@ static void eth_header_init(struct proto_hdr *hdr)
> > proto_field_set_default_dev_mac(hdr, ETH_SRC_ADDR);
> >  }
> >  
> >  
> > -struct proto_hdr *proto_header_init(enum proto_id pid)
> > +struct proto_hdr *proto_header_new(enum proto_id pid)
> 
> The old name was really bad by may be something like:
> 
> {proto,packet}_header_{add,push}
> 
> might be better ?

I agree, renamed it to proto_header_push.

-- 
You received this message because you are subscribed to the Google Groups 
"netsniff-ng" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[netsniff-ng] Re: [PATCH] colorize: Squash into colors.h

[Tobias Klauser]

On 2016-08-12 at 00:42:24 +0200, Vadim Kochan  wrote:
> Move colorize_xxx macroses to colors.h, the reason
> is to have one file for coloring stuff and have less
> files as possible in the source tree.
> 
> Signed-off-by: Vadim Kochan 

Applied, thanks!

-- 
You received this message because you are subscribed to the Google Groups 
"netsniff-ng" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[netsniff-ng] [RFC PATCH] trafgen: proto: Split static protocol definition out of struct proto_hdr

[Tobias Klauser]

Currently struct proto_hdr is used twofold:

  1) Statically define protocol behavior, i.e. all the *_hdr definitions in
 trafgen_l{2,3,4}.c which map a protocol id/layer to a set of callback
 functions.

  2) For each packet created at parse time the struct is memcpy()'ed
 (including all the static information from 1) and then used to store
 dynamic information at parse/run time.

Thus, struct proto_hdr members such as the proto id, layer and the
pointers callback functions get copied for each created packet (in
addition to the other fields which get changed during parsing). Also,
static/dynamic information get mixed and we e.g. can't make the protocol
definitions const to ensure they'll not get changed by mistake.

Rather than copying the struct proto_hdr for every packet, clearly
separate the two purposes defined above by splitting struct proto_hdr
into two structs:

  1) struct proto_ops for the static (const) protocol behavior definition

  2) struct proto_hdr (reduced) for dynamic information

struct proto_hdr keeps a pointer to the corresponding proto_ops instance
and uses it to execute the corresponding callbacks.

Reference: 
https://groups.google.com/forum/#!msg/netsniff-ng/20RvwJdh50Y/eMkbmKSaBgAJ
Signed-off-by: Tobias Klauser <tklau...@distanz.ch>
---
 trafgen_l2.c | 24 ++---
 trafgen_l3.c |  8 +++
 trafgen_l4.c | 22 +--
 trafgen_parser.y |  2 +-
 trafgen_proto.c  | 65 +---
 trafgen_proto.h  | 40 +-
 6 files changed, 83 insertions(+), 78 deletions(-)

diff --git a/trafgen_l2.c b/trafgen_l2.c
index 1863332b3543..f09b2a61cacc 100644
--- a/trafgen_l2.c
+++ b/trafgen_l2.c
@@ -47,7 +47,7 @@ static void eth_header_init(struct proto_hdr *hdr)
proto_field_set_default_dev_mac(hdr, ETH_SRC_ADDR);
 }
 
-static struct proto_hdr eth_hdr = {
+static const struct proto_ops eth_proto_ops = {
.id = PROTO_ETH,
.layer  = PROTO_L2,
.header_init= eth_header_init,
@@ -74,13 +74,13 @@ static void vlan_header_init(struct proto_hdr *hdr)
 
proto_header_fields_add(hdr, vlan_fields, array_size(vlan_fields));
 
-   if (lower->id == PROTO_ETH)
+   if (lower->ops->id == PROTO_ETH)
lower_etype = proto_field_get_u16(lower, ETH_TYPE);
-   else if (lower->id == PROTO_VLAN)
+   else if (lower->ops->id == PROTO_VLAN)
lower_etype = proto_field_get_u16(lower, VLAN_ETYPE);
 
proto_field_set_be16(hdr, VLAN_ETYPE, lower_etype);
-   proto_field_set_default_be16(hdr, VLAN_TPID, pid_to_eth(hdr->id));
+   proto_field_set_default_be16(hdr, VLAN_TPID, pid_to_eth(hdr->ops->id));
 }
 
 static void vlan_set_next_proto(struct proto_hdr *hdr, enum proto_id pid)
@@ -89,7 +89,7 @@ static void vlan_set_next_proto(struct proto_hdr *hdr, enum 
proto_id pid)
proto_field_set_be16(hdr, VLAN_ETYPE, pid_to_eth(pid));
 }
 
-static struct proto_hdr vlan_hdr = {
+static const struct proto_ops vlan_proto_ops = {
.id = PROTO_VLAN,
.layer  = PROTO_L2,
.header_init= vlan_header_init,
@@ -114,7 +114,7 @@ static void arp_header_init(struct proto_hdr *hdr)
 
lower = proto_lower_default_add(hdr, PROTO_ETH);
 
-   if (lower->id == PROTO_ETH) {
+   if (lower->ops->id == PROTO_ETH) {
const uint8_t bcast[6] = { 0xff, 0xff, 0xff, 0xff, 0xff, 0xff };
 
proto_field_set_default_bytes(lower, ETH_DST_ADDR, bcast);
@@ -133,7 +133,7 @@ static void arp_header_init(struct proto_hdr *hdr)
proto_field_set_default_dev_ipv4(hdr, ARP_TPA);
 }
 
-static struct proto_hdr arp_hdr = {
+static const struct proto_ops arp_proto_ops = {
.id = PROTO_ARP,
.layer  = PROTO_L2,
.header_init= arp_header_init,
@@ -161,7 +161,7 @@ static void mpls_set_next_proto(struct proto_hdr *hdr, enum 
proto_id pid)
proto_field_set_default_be32(hdr, MPLS_LAST, 0);
 }
 
-static struct proto_hdr mpls_hdr = {
+static const struct proto_ops mpls_proto_ops = {
.id = PROTO_MPLS,
.layer  = PROTO_L2,
.header_init= mpls_header_init,
@@ -170,8 +170,8 @@ static struct proto_hdr mpls_hdr = {
 
 void protos_l2_init(void)
 {
-   proto_header_register(_hdr);
-   proto_header_register(_hdr);
-   proto_header_register(_hdr);
-   proto_header_register(_hdr);
+   proto_ops_register(_proto_ops);
+   proto_ops_register(_proto_ops);
+   proto_ops_register(_proto_ops);
+   proto_ops_register(_proto_ops);
 }
diff --git a/trafgen_l3.c b/trafgen_l3.c
index 1cdd041ed057..2eabef12667d 100644
--- a/trafgen_l3.c
+++ b/trafgen_l3.c
@@ -86,7 +86,7 @@ static void ipv4_set_next_proto(struct proto_hdr *hdr, enum 
proto_id pid)
proto_field_set_default_u8(hdr,

[netsniff-ng] Re: [PATCH v2 05/14] trafgen: icmpv6: Update csum at runtime if needed

[Tobias Klauser]

On 2016-08-11 at 00:57:07 +0200, Vadim Kochan  wrote:
> Use same function to calculate csum for packet_update
> and for packet_finish events.
> 
> Allow update csum if one of the ICMPv6 fields was changed.
> 
> Signed-off-by: Vadim Kochan 
> ---
>  trafgen_l4.c | 26 +++---
>  1 file changed, 15 insertions(+), 11 deletions(-)
> 
> diff --git a/trafgen_l4.c b/trafgen_l4.c
> index 19ebb37..908641c 100644
> --- a/trafgen_l4.c
> +++ b/trafgen_l4.c
> @@ -202,39 +202,43 @@ static void icmpv6_header_init(struct proto_hdr *hdr)
>   proto_header_fields_add(hdr, icmpv6_fields, array_size(icmpv6_fields));
>  }
>  
> -static void icmpv6_packet_finish(struct proto_hdr *hdr)
> +static void icmpv6_csum_update(struct proto_hdr *hdr)
>  {
>   struct proto_hdr *lower = proto_lower_header(hdr);
> - struct packet *pkt = current_packet();
> + struct packet *pkt = packet_get(hdr->pkt_id);
>   uint16_t total_len;
>   uint16_t csum;
>  

Why no check for hdr->is_csum_valid here?

>   if (proto_field_is_set(hdr, ICMPV6_CSUM))
>   return;
> -
>   if (!lower)
>   return;
>  
>   total_len = pkt->len - hdr->pkt_offset;
>  
> - switch (lower->id) {
> - case PROTO_IP6:
> + proto_field_set_be16(hdr, ICMPV6_CSUM, 0);
> +
> + if (lower->id == PROTO_IP6) {
>   csum = p6_csum((void *) proto_header_ptr(lower), 
> proto_header_ptr(hdr),
>   total_len, IPPROTO_ICMPV6);
> - break;
> - default:
> - csum = 0;
> - break;
> +
> + proto_field_set_be16(hdr, ICMPV6_CSUM, bswap_16(csum));
> + hdr->is_csum_valid = true;
>   }
> +}
>  
> - proto_field_set_be16(hdr, ICMPV6_CSUM, bswap_16(csum));
> +static void icmpv6_field_changed(struct proto_hdr *hdr, struct proto_field 
> *field)
> +{
> + hdr->is_csum_valid = false;
>  }
>  
>  static struct proto_hdr icmpv6_hdr = {
>   .id = PROTO_ICMP6,
>   .layer  = PROTO_L4,
>   .header_init= icmpv6_header_init,
> - .packet_finish  = icmpv6_packet_finish,
> + .packet_finish  = icmpv6_csum_update,
> + .packet_update  = icmpv6_csum_update,
> + .field_changed  = icmpv6_field_changed,
>  };
>  
>  void protos_l4_init(void)
> -- 
> 2.9.2
> 

-- 
You received this message because you are subscribed to the Google Groups 
"netsniff-ng" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[netsniff-ng] Re: [PATCH v2 03/14] trafgen: ipv4: Update csum at runtime if needed

[Tobias Klauser]

On 2016-08-11 at 00:57:05 +0200, Vadim Kochan  wrote:
> Handle 'field_changed' callback to check if IPv4 csum is needed
> to be recalculated, if so - update it on 'packet_update' event.
> 
> Added 'is_csum_valid' to proto_hdr struct to check if csum needs to
> be updated.
> 
> Signed-off-by: Vadim Kochan 
> ---
>  trafgen_l3.c| 33 +++--
>  trafgen_proto.h |  2 ++
>  2 files changed, 29 insertions(+), 6 deletions(-)
> 
> diff --git a/trafgen_l3.c b/trafgen_l3.c
> index 1cdd041..8db7e5d 100644
> --- a/trafgen_l3.c
> +++ b/trafgen_l3.c
> @@ -43,6 +43,30 @@ static void ipv4_header_init(struct proto_hdr *hdr)
>   proto_field_set_default_dev_ipv4(hdr, IP4_SADDR);
>  }
>  
> +static void ipv4_field_changed(struct proto_hdr *hdr, struct proto_field 
> *field)
> +{
> + hdr->is_csum_valid = false;
> +}
> +
> +static void ipv4_csum_update(struct proto_hdr *hdr)
> +{
> + struct packet *pkt;
> + uint16_t csum;
> +
> + if (proto_field_is_set(hdr, IP4_CSUM))
> + return;
> + if (hdr->is_csum_valid)
> + return;

Minor nit: The check for hdr->is_csum_valid is less expensive and should
come first (in the patch for icmpv4 it's already done like this).

Otherwise the patch looks good.

-- 
You received this message because you are subscribed to the Google Groups 
"netsniff-ng" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[netsniff-ng] Re: [PATCH v2 02/14] trafgen: proto: Randomize proto field at runtime

[Tobias Klauser]

On 2016-08-11 at 00:57:04 +0200, Vadim Kochan  wrote:
> Add dynamic proto field function which can generate
> random value in specified range (default 0 - MAX_UINT32).
> 
> Signed-off-by: Vadim Kochan 
> ---
>  trafgen_proto.c | 27 +++
>  trafgen_proto.h |  1 +
>  2 files changed, 28 insertions(+)
> 
> diff --git a/trafgen_proto.c b/trafgen_proto.c
> index 72dbad3..072cbcd 100644
> --- a/trafgen_proto.c
> +++ b/trafgen_proto.c
> @@ -466,6 +466,31 @@ static void field_inc_func(struct proto_field *field)
>   }
>  }
>  
> +static inline unsigned int field_rand(struct proto_field *field)

Return type should be uint32_t.

Otherwise the patch looks good to apply.

-- 
You received this message because you are subscribed to the Google Groups 
"netsniff-ng" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[netsniff-ng] Re: [PATCH 00/15] trafgen: Support dinc & drnd for proto fields

[Tobias Klauser]

Hi Vadim

On 2016-07-26 at 21:35:05 +0200, Vadim Kochan  wrote:
> Implemented 'dinc' and 'drnd' functions to be used for proto fields,
> and generate values at runtime. Parsing of proto field values
> for unified to make extending of field functions more easier w/o
> copy/paste similar rules for each proto field. Instead of that
> the field_expr struct is used to keep specified field and value which
> might be a func in the following form:
> 
> (=())
> 
> Fields which has dynamic functions are stored in packet_dyn structure and
> generated after each packet run. After fields are updated the L3/L4 headers
> updates its csum if it is needed.
> 
> Changed field lookup logic to make field set/get operations a little bit 
> faster by
> using field id as index in the array, as usually fields are defined regarding
> its index.
> 
> Example:
> 
> { eth(sa=11:22:33:44:55:66, sa=dinc()), udp(sp=drnd(), dp=drnd()) }
> 
> Vadim Kochan (15):
>   trafgen: Move applying dynamic elements to function
>   trafgen: proto: Reference to packet from proto_hdr struct
>   trafgen: proto: Move proto headers into packet
>   trafgen: proto: Force field id to be index in array
>   trafgen: proto: Increment proto field at runtime
>   trafgen: proto: Randomize proto field at runtime
>   trafgen: ipv4: Update csum at runtime if needed
>   trafgen: icmpv4: Update csum at runtime if needed
>   trafgen: icmpv6: Update csum at runtime if needed
>   trafgen: udp: Update csum at runtime if needed
>   trafgen: tcp: Update csum at runtime if it needed
>   trafgen: parser: Unify proto field value parsing
>   trafgen: parser: Add support of 'dinc' function for proto fields
>   trafgen: parser: Add 'drnd()' function for proto fields
>   trafgen: man: Add description for 'dinc' and 'drnd' field functions

While reviewing (and partially applying) this series I noticed something
about the current implementation of trafgen protos which I find quite
odd:

The struct proto_hdr is on one hand used to statically define protocol
behavior (i.e. all the *_hdr definitions in trafgen_l{2,3,4}.c) and
these are then memcpy()'ed for each created packed at parse time and
then used to store dynamically changing information at parse/run time.
This way, members such as the proto id, layer and the pointers callback
functions get copied for each created packet (in addition to the other
fields which get changed during parsing) and static/dynamic information
get mixed and we e.g. can't make the protocol definitions const to
ensure they'll not get changed.

Rather than copying the struct proto_hdr for every packet, I think it
would be cleaner to separate it into two structs, one for the statically
defined protocol (let's call it struct proto_ops) and one for the
dynamic information (this would be the 'new', reduced struct proto_hdr).
The struct proto_hdr would then have a pointer to the corresponding
proto_ops instance and could use it to execute callbacks. The new
structs would look something like this:

struct proto_ops {
enum proto_id id;
enum proto_layer layer;

void (*header_init)(struct proto_hdr *hdr);
void (*header_finish)(struct proto_hdr *hdr);
void (*packet_finish)(struct proto_hdr *hdr);
void (*set_next_proto)(struct proto_hdr *hdr, enum proto_id pid);
};

struct proto_hdr {
struct proto_ops *ops;
uint16_t pkt_offset;
uint32_t pkt_id;
struct proto_field *fields;
size_t fields_count;
size_t len;
};

Maybe I missed something important, why this wouldn't work or isn't a
good idea? But if not, and if you agree I'd like to make this change
before the reroll of your series.

If that's OK, I'll prepare the series and send it as an RFC to you,
Daniel and the list for review.

-- 
You received this message because you are subscribed to the Google Groups 
"netsniff-ng" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[netsniff-ng] Re: [PATCH 04/15] trafgen: proto: Force field id to be index in array

[Tobias Klauser]

On 2016-07-26 at 21:35:09 +0200, Vadim Kochan  wrote:
> Usually proto fields array is sorted in the same order as related enum,
> so id may be used as index for faster lookup, it will make
> csum field calculation little faster at runtime.
> 
> Signed-off-by: Vadim Kochan 

I now applied the patch (with added comment and bug_on() check) now and
as a follow-up also did the same thing for the 'registered' array, see
current HEAD.

-- 
You received this message because you are subscribed to the Google Groups 
"netsniff-ng" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[netsniff-ng] Re: [PATCH 05/15] trafgen: proto: Increment proto field at runtime

[Tobias Klauser]

On 2016-08-08 at 21:04:20 +0200, Vadim Kochan <vadi...@gmail.com> wrote:
> On Sat, Aug 06, 2016 at 01:36:26AM +0300, Vadim Kochan wrote:
> > On Wed, Aug 03, 2016 at 04:27:14PM +0200, Tobias Klauser wrote:
> > > On 2016-07-26 at 21:35:10 +0200, Vadim Kochan <vadi...@gmail.com> wrote:
> > > > Extended 'struct packet_dyn' with proto fields which has
> > > > dynamically changing values at runtime.
> > > > 
> > > > Implement incrementing of proto field at runtime with min & max
> > > > parameters, by default if the 'min' parameter is not specified
> > > > then original value is used. For fields which len is greater
> > > > than 4 - last 4 bytes are incremented as unsigned int value.
> > > > 
> > > > Added 'field_changed' callback for proto header which
> > > > may be used for check if csum updating is needed. This callback
> > > > is called after field was changed at runtime.
> > > > 
> > > > Added 'packet_update' callback to let proto header know
> > > > when to apply final proto header changes at runtime (csum update).
> > > 
> > > The documentation of these callbacks would also make sense where they're
> > > defined.
> > > 
> > > > Signed-off-by: Vadim Kochan <vadi...@gmail.com>
> > > > ---
> > > >  trafgen.c   |  9 ++
> > > >  trafgen_conf.h  |  7 
> > > >  trafgen_proto.c | 99 
> > > > +
> > > >  trafgen_proto.h | 26 +++
> > > >  4 files changed, 141 insertions(+)
> > > > 
> > > > diff --git a/trafgen.c b/trafgen.c
> > > > index b76b5d7..553dfa5 100644
> > > > --- a/trafgen.c
> > > > +++ b/trafgen.c
> > > >  void proto_packet_finish(void)
> > > >  {
> > > > struct proto_hdr **headers = _packet()->headers[0];
> > > > @@ -433,3 +446,89 @@ void proto_packet_finish(void)
> > > > p->packet_finish(p);
> > > > }
> > > >  }
> > > > +
> > > > +static inline unsigned int field_inc(struct proto_field *field)
> > > > +{
> > > > +   uint32_t val;
> > > > +
> > > > +   val = field->func.val - field->func.min;
> > > > +   val = (val + field->func.inc) % field->func.max;
> > > 
> > > Shouldn't this be
> > > 
> > >   val = (val + field->func.inc) % (field->func.max - field->func.min + 1)
> > > 
> > > to be consistent with apply_counter()?
> 
> I simplified it and now it really works well:
> 
>   #define max_int32(a, b) 
> \
>   ({  
> \
>   int32_t _a = (int32_t) (a); 
> \
>   int32_t _b = (int32_t) (b); 
> \
>   _a - ((_a - _b) & ((_a - _b) >> (sizeof(int32_t) * 8 - 
> 1)));\

The line above definitely needs an explanatory comment. How and why does
this work? Why is it implemented like this instead of the "obvious"
max() solution?

>   })
> 
>   static inline unsigned int field_inc(struct proto_field *field)
>   {
>  uint32_t min = field->func.min;
>  uint32_t max = field->func.max;
>  uint32_t val = field->func.val;
>  uint32_t inc = field->func.inc;
>  uint32_t next;
> 
>  next = (val + inc) % (max + 1);
>  field->func.val = max_int32(next, min);
> 
>  return val;
>   }
> 
> so max_int32(a,b) should be fast enough w/o branching.

Have you profiled this against other possible implementations,
especially the max_int32() implementation above? If your profiling doesn't
show significant improvements of this max_int32() variant I'd rather
stay with the "obvious" one, i.e. a > b ? a : b

> > Sure, I tried this approach while implementing 1st version but when I
> > used the following case:
> > 
> > trafgen/trafgen -o lo -n 10 --cpu 1 '{ eth(type=0x800), fill(0xff, 10), 
> > dinc(5, 20, 5) }'
> > 
> > then interval between 5 & 20 changes very differently. But in my version
> > it repeats from 5 till 20 (yes here is a little difference that initial
> > value is incremented i

[netsniff-ng] Re: [PATCH v3 3/3] configure: Add option to compile tools without libnl dependency

[Tobias Klauser]

On 2016-08-08 at 16:31:11 +0200, Vadim Kochan <vadi...@gmail.com> wrote:
> On Mon, Aug 8, 2016 at 5:24 PM, Tobias Klauser <tklau...@distanz.ch> wrote:
> > On 2016-08-06 at 11:59:05 +0200, Vadim Kochan <vadi...@gmail.com> wrote:
> >> On Sat, Aug 6, 2016 at 12:41 PM, Tobias Klauser <tklau...@distanz.ch> 
> >> wrote:
> >> > On 2016-08-05 at 22:51:54 +0200, Vadim Kochan <vadi...@gmail.com> wrote:
> >> Yes, I understand, I said so because I really remember we talked about
> >> disabling of libnl support earier ago and you or Daniel pointed that
> >> it is better to have this dependency by default, this is how I
> >> remember this, I might be wrong, but anyway
> >> it simplifies build netsniff-ng/trafgen manually w/o searching libnl-
> >> packages on specific dist.
> >
> > I can't remember we talked about it, but that doesn't mean we didn't :)
> > In any case if we discussed a particular feature/patch/RFC/... before
> > and you patch is based on the previous discussion, please include a link
> > to the respective mailing list thread into the description. This may
> > help to remind us of past conversations ;)
> >
> > Thanks!
> 
> This is a link with our discussion (within libnl-route fix titled with
> "[PATCH] build: Check for libnl-route"), but I looked for this
> just now so I did not consider it while writing previous email, so I
> might be wrong for some conclusions):
> 
> https://groups.google.com/forum/?hl=en#!searchin/netsniff-ng/Check$20for$20libnl-route%7Csort:relevance/netsniff-ng/0Ws7ecy1H_Y/DIzprE4KBAAJ

Thanks for the link. I don't see detailled discussion of how this should
be implemented, just the general idea. And there, I think, we all agreed
that it would be fine as long as it doesn't introduce too much
additional complexity.

-- 
You received this message because you are subscribed to the Google Groups 
"netsniff-ng" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[netsniff-ng] Re: [PATCH v3 3/3] configure: Add option to compile tools without libnl dependency

[Tobias Klauser]

On 2016-08-06 at 11:59:05 +0200, Vadim Kochan <vadi...@gmail.com> wrote:
> On Sat, Aug 6, 2016 at 12:41 PM, Tobias Klauser <tklau...@distanz.ch> wrote:
> > On 2016-08-05 at 22:51:54 +0200, Vadim Kochan <vadi...@gmail.com> wrote:
> >> Hm,
> >>
> >> My version was to keep current strict dependency on libnl by default,
> >> so if there is no libnl then
> >> netsniff-ng & trafgen will be removed from build list, but make
> >> possible to skip this
> >> rule by --disable-libnl option. Now I am not sure there is a reason
> >> for --disable-libnl as if there is no libnl
> >> then netsniff-ng & trafgen will be added to build list, and will be
> >> compiled w/o libnl dependency..
> >
> > Ok, but that intention of your patch wasn't really clear from the
> > description, sorry.
> 
> Yes, usually I put no so much description into patches.

Please try to be as verbose as possible in patch descriptions (at least
for non-trivial patches). The description is there to help others
understand the rationale behind a patch (before applying it and
especially afterwards e.g. when chasing bugs).

> >> So does --disable-libnl makes sense ?
> >
> > If you have libnl-dev installed but want to compile netsniff-ng/trafgen
> > without libnl support it still makes sense.
> >
> > IMO the current version is more in line with the behavior we already
> > have for geoip and libz.
> >
> > Thanks
> > Tobias
> 
> Yes, I understand, I said so because I really remember we talked about
> disabling of libnl support earier ago and you or Daniel pointed that
> it is better to have this dependency by default, this is how I
> remember this, I might be wrong, but anyway
> it simplifies build netsniff-ng/trafgen manually w/o searching libnl-
> packages on specific dist.

I can't remember we talked about it, but that doesn't mean we didn't :)
In any case if we discussed a particular feature/patch/RFC/... before
and you patch is based on the previous discussion, please include a link
to the respective mailing list thread into the description. This may
help to remind us of past conversations ;)

Thanks!

-- 
You received this message because you are subscribed to the Google Groups 
"netsniff-ng" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[netsniff-ng] Re: [PATCH 06/15] trafgen: proto: Randomize proto field at runtime

[Tobias Klauser]

On 2016-08-06 at 01:44:42 +0200, Vadim Kochan <vadi...@gmail.com> wrote:
> On Fri, Aug 05, 2016 at 09:26:24AM +0200, Tobias Klauser wrote:
> > On 2016-07-26 at 21:35:11 +0200, Vadim Kochan <vadi...@gmail.com> wrote:
> > > +static inline unsigned int field_rand(struct proto_field *field)
> > > +{
> > > + return field->func.min + (rand() % ((field->func.max - field->func.min) 
> > > + 1));
> > > +}
> > > +
> > > +static void field_rnd_func(struct proto_field *field)
> > > +{
> > > + unsigned int val = field_rand(field);
> > > +
> > > + if (field->len == 1) {
> > > + proto_field_set_u8(field->hdr, field->id, (uint8_t) val);
> > > + } else if (field->len == 2) {
> > > + proto_field_set_be16(field->hdr, field->id, (uint16_t) val);
> > > + } else if (field->len == 4) {
> > > + proto_field_set_be32(field->hdr, field->id, (uint32_t) val);
> > > + } else if (field->len > 4) {
> > > + uint8_t *bytes = __proto_field_get_bytes(field);
> > > + uint32_t i;
> > > +
> > > + for (i = 0; i < field->len; i++, val = field_rand(field))
> > > + bytes[i] = (uint8_t) val;
> > 
> > No need for the assigment of `val' in the loop, should rather be like
> > this:
> > 
> > for (i = 0; i < field->len; i++)
> > bytes[i] = (uint8_t) field_rand(field);
> > > + }
> 
> I did so because field_rand(field) was calculated at the beginning so in
> the loop it will be calculated only on the next round. Do you think it
> is not needed ?

IMO it's easy to overread the val assignment in the for statement as it
is not such a common pattern. We'd not even save the call to
field_rand(), so I think it's worth to have this explicitely stated in
the loop body rather than the for(...) statement.

-- 
You received this message because you are subscribed to the Google Groups 
"netsniff-ng" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[netsniff-ng] Re: [PATCH 13/15] trafgen: parser: Add support of 'dinc' function for proto fields

[Tobias Klauser]

On 2016-08-06 at 00:43:56 +0200, Vadim Kochan <vadi...@gmail.com> wrote:
> On Wed, Aug 03, 2016 at 04:32:06PM +0200, Tobias Klauser wrote:
> > On 2016-07-26 at 21:35:18 +0200, Vadim Kochan <vadi...@gmail.com> wrote:
> > > +static void proto_field_func_setup(struct proto_field *field, struct 
> > > proto_field_func *func)
> > > +{
> > > + struct packet_dyn *pkt_dyn;
> > > +
> > > + proto_field_func_add(field->hdr, field->id, func);
> > > +
> > > + pkt_dyn = _dyn[packetd_last];
> > > + pkt_dyn->flen++;
> > > + pkt_dyn->fields = (struct proto_field **)xrealloc(pkt_dyn->fields, 
> > > pkt_dyn->flen *
> > > + sizeof(struct proto_field *));
> > 
> > No need to case the return value of xrealloc()
> 
> Sorry, I did not get it, may be you meant "No need to cast ..." ?

Yes, sorry. That's a typo and should read "...cast the return value...".

-- 
You received this message because you are subscribed to the Google Groups 
"netsniff-ng" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[netsniff-ng] Re: [PATCH v3 2/3] netsniff-ng: Allow compile without libnl

[Tobias Klauser]

On 2016-08-04 at 18:30:19 +0200, Vadim Kochan  wrote:
> There is reason to do not install libnl-xxx packages just
> for sniffing packets, for example if netsniff-ng will be compiled
> on embedded or switch system.
> 
> Hide libnl depended code if CONFIG_LIBNL=0.
> 
> In case if user will use --rfraw option the panic message will be printed,
> in case if netlink messages will be sniffed the noop dissector will be used.
> 
> Signed-off-by: Vadim Kochan 

Applied with minor modifications (due to my changes in PATCH 3/3).

-- 
You received this message because you are subscribed to the Google Groups 
"netsniff-ng" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[netsniff-ng] Re: [PATCH v3 1/3] trafgen: Allow to compile without libnl

[Tobias Klauser]

On 2016-08-04 at 18:30:18 +0200, Vadim Kochan  wrote:
> trafgen uses libnl only to inject mac80211 frames but
> it might be not needed in some embedded or switch environments,
> so lets make possible to disable this feature.
> 
> In case if --rfraw option will be used - user will get the panic message.
> 
> Signed-off-by: Vadim Kochan 

Applied with minor modifications (due to my changes in PATCH 3/3).

-- 
You received this message because you are subscribed to the Google Groups 
"netsniff-ng" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[netsniff-ng] Re: [PATCH v3 3/3] configure: Add option to compile tools without libnl dependency

[Tobias Klauser]

On 2016-08-04 at 18:30:20 +0200, Vadim Kochan  wrote:
> Add command line parsing function which allows to compile tools
> (trafgen, netsniff-ng) without libnl-xxx libraries.

But it does in its current state not allow to compile
netsniff-ng/trafgen without libnl if libnl is not installed (i.e. not
found by check_libnl).

I rewrote this patch to support this case and make it more in line with
the existing library checking/disabling.

Thanks!

-- 
You received this message because you are subscribed to the Google Groups 
"netsniff-ng" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[netsniff-ng] Re: [PATCH 06/15] trafgen: proto: Randomize proto field at runtime

[Tobias Klauser]

On 2016-07-26 at 21:35:11 +0200, Vadim Kochan  wrote:
> Add dynamic proto field function which can generate
> random value in specified range (default 0 - MAX_UINT32).
> 
> Signed-off-by: Vadim Kochan 
> ---
>  trafgen_proto.c | 29 +
>  trafgen_proto.h |  1 +
>  2 files changed, 30 insertions(+)
> 
> diff --git a/trafgen_proto.c b/trafgen_proto.c
> index 069aa00..f57d390 100644
> --- a/trafgen_proto.c
> +++ b/trafgen_proto.c
> @@ -486,6 +486,30 @@ static void field_inc_func(struct proto_field *field)
>   }
>  }
>  
> +static inline unsigned int field_rand(struct proto_field *field)
> +{
> + return field->func.min + (rand() % ((field->func.max - field->func.min) 
> + 1));
> +}
> +
> +static void field_rnd_func(struct proto_field *field)
> +{
> + unsigned int val = field_rand(field);
> +
> + if (field->len == 1) {
> + proto_field_set_u8(field->hdr, field->id, (uint8_t) val);
> + } else if (field->len == 2) {
> + proto_field_set_be16(field->hdr, field->id, (uint16_t) val);
> + } else if (field->len == 4) {
> + proto_field_set_be32(field->hdr, field->id, (uint32_t) val);
> + } else if (field->len > 4) {
> + uint8_t *bytes = __proto_field_get_bytes(field);
> + uint32_t i;
> +
> + for (i = 0; i < field->len; i++, val = field_rand(field))
> + bytes[i] = (uint8_t) val;

No need for the assigment of `val' in the loop, should rather be like
this:

for (i = 0; i < field->len; i++)
bytes[i] = (uint8_t) field_rand(field);
> + }
> +}
> +
>  void proto_field_func_add(struct proto_hdr *hdr, uint32_t fid,
> struct proto_field_func *func)
>  {
> @@ -522,6 +546,11 @@ void proto_field_func_add(struct proto_hdr *hdr, 
> uint32_t fid,
>  
>   field->func.update_field = field_inc_func;
>   }
> +
> + if (func->type & PROTO_FIELD_FUNC_RND) {
> + field->func.max = field->func.max ?: (uint32_t)~0 - 1;

UINT32_MAX, please. Also in all other places in this series.

> + field->func.update_field = field_rnd_func;
> + }
>  }
>  
>  void proto_field_dyn_apply(struct proto_field *field)
> diff --git a/trafgen_proto.h b/trafgen_proto.h
> index 64f4366..40817a8 100644
> --- a/trafgen_proto.h
> +++ b/trafgen_proto.h
> @@ -38,6 +38,7 @@ struct proto_hdr;
>  enum proto_field_func_t {
>   PROTO_FIELD_FUNC_INC = 1 << 0,
>   PROTO_FIELD_FUNC_MIN = 1 << 1,
> + PROTO_FIELD_FUNC_RND = 1 << 2,
>  };
>  
>  struct proto_field_func {
> -- 
> 2.6.3
> 

-- 
You received this message because you are subscribed to the Google Groups 
"netsniff-ng" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[netsniff-ng] Re: [PATCH v2 0/3] build: Allow compile trafgen & netsniff-ng without libnl

[Tobias Klauser]

On 2016-08-03 at 10:58:00 +0200, Vadim Kochan  wrote:
> It might be useful only if to compile trafgen or netsniff-ng in environment
> (or manually w/o needs for rfraw) where is no needed libnl (embedded/switch 
> server),
> and it would be good to have ability to disable libnl dependency at all even 
> w/o
> such features like rfraw dissect/inject & nlmsg dissecting. For example
> netsniff-ng might be used just for sniffing, and trafgen - send packets to 
> wired
> network only.
> 
> Therefor added --no-libnl option in ./confiure script which sets make option &
> CPP definition which are used to ignore libnl- related code/modules/libraries.
> 
> v2:
> 1) Reorder commits to be "configure" related changes  1st

I just realized I was wrong by requesting this. In the current
order we might break disectability if --disable-libnl is used. Your
original order of patches was correct. Care to change it again?

Sorry about that!

-- 
You received this message because you are subscribed to the Google Groups 
"netsniff-ng" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[netsniff-ng] Re: [PATCH 13/15] trafgen: parser: Add support of 'dinc' function for proto fields

[Tobias Klauser]

On 2016-07-26 at 21:35:18 +0200, Vadim Kochan  wrote:
> Add 'dinc()' function in 'field_expr' rules to be used for dynamically
> incrementing of any specified field:
> 
> SYNTAX := dinc() | dinc(inc) | dinc(min, max) | dinc(inc, min, max)

This should definitely follow the same parameter order as the existing
dinc() function!!!

> EXAMPLES:
> { udp(sport=dinc() }
> { udp(sport=dinc(1) }
> { udp(sport=dinc(5, 100, 125) }

> Signed-off-by: Vadim Kochan 
> ---
>  trafgen_parser.y | 51 +++
>  1 file changed, 51 insertions(+)
> 
> diff --git a/trafgen_parser.y b/trafgen_parser.y
> index 58ac999..7872b7c 100644
> --- a/trafgen_parser.y
> +++ b/trafgen_parser.y
> @@ -74,6 +74,7 @@ enum field_expr_type_t {
>   FIELD_EXPR_MAC,
>   FIELD_EXPR_IP4_ADDR,
>   FIELD_EXPR_IP6_ADDR,
> + FIELD_EXPR_INC,
>  };
>  
>  struct proto_field_expr {
> @@ -85,6 +86,7 @@ struct proto_field_expr {
>   struct in6_addr ip6_addr;
>   long long int number;
>   uint8_t bytes[256];
> + struct proto_field_func func;
>   } val;
>  };
>  
> @@ -125,6 +127,12 @@ static inline void __init_new_csum_slot(struct 
> packet_dyn *slot)
>   slot->slen = 0;
>  }
>  
> +static inline void __init_new_fields_slot(struct packet_dyn *slot)
> +{
> + slot->fields = NULL;
> + slot->flen = 0;
> +}
> +
>  static inline void __setup_new_counter(struct counter *c, uint8_t start,
>  uint8_t stop, uint8_t stepping,
>  int type)
> @@ -167,6 +175,7 @@ static void realloc_packet(void)
>   __init_new_counter_slot(_dyn[packetd_last]);
>   __init_new_randomizer_slot(_dyn[packetd_last]);
>   __init_new_csum_slot(_dyn[packetd_last]);
> + __init_new_fields_slot(_dyn[packetd_last]);
>  }
>  
>  struct packet *current_packet(void)
> @@ -376,6 +385,19 @@ static void proto_field_set(uint32_t fid)
>   field_expr.field = proto_field_by_id(hdr, fid);
>  }
>  
> +static void proto_field_func_setup(struct proto_field *field, struct 
> proto_field_func *func)
> +{
> + struct packet_dyn *pkt_dyn;
> +
> + proto_field_func_add(field->hdr, field->id, func);
> +
> + pkt_dyn = _dyn[packetd_last];
> + pkt_dyn->flen++;
> + pkt_dyn->fields = (struct proto_field **)xrealloc(pkt_dyn->fields, 
> pkt_dyn->flen *
> + sizeof(struct proto_field *));

No need to case the return value of xrealloc()

> + pkt_dyn->fields[pkt_dyn->flen - 1] = field;
> +}
> +
>  static void proto_field_expr_eval(void)
>  {
>   struct proto_field *field = field_expr.field;
> @@ -405,6 +427,14 @@ static void proto_field_expr_eval(void)
>   (uint8_t *)_expr.val.ip6_addr.s6_addr);
>   break;
>  
> + case FIELD_EXPR_INC:
> + if (field_expr.val.func.min > field_expr.val.func.max)
> + panic("dinc(): min(%u) can't be greater than max(%u)\n",
> + field_expr.val.func.min, 
> field_expr.val.func.max);
> +
> + proto_field_func_setup(field, _expr.val.func);
> + break;
> +
>   case FIELD_EXPR_UNKNOWN:
>   default:
>   bug();
> @@ -685,6 +715,26 @@ field_expr
>field_expr.val.ip4_addr = $1; }
>   | ip6_addr { field_expr.type = FIELD_EXPR_IP6_ADDR;
>field_expr.val.ip6_addr = $1; }
> + | K_DINC '(' ')' { field_expr.type = FIELD_EXPR_INC;
> +field_expr.val.func.type = PROTO_FIELD_FUNC_INC;
> +field_expr.val.func.inc = 1; }
> + | K_DINC '(' number ')'
> + { field_expr.type = FIELD_EXPR_INC;
> +   field_expr.val.func.inc = $3; }
> + | K_DINC '(' number delimiter number ')'
> + { field_expr.type = FIELD_EXPR_INC;
> +   field_expr.val.func.type  = PROTO_FIELD_FUNC_INC;
> +   field_expr.val.func.type |= PROTO_FIELD_FUNC_MIN;
> +   field_expr.val.func.inc = 1;
> +   field_expr.val.func.min = $3;
> +   field_expr.val.func.max = $5; }
> + | K_DINC '(' number delimiter number delimiter number ')'
> + { field_expr.type = FIELD_EXPR_INC;
> +   field_expr.val.func.type  = PROTO_FIELD_FUNC_INC;
> +   field_expr.val.func.type |= PROTO_FIELD_FUNC_MIN;
> +   field_expr.val.func.inc = $3;
> +   field_expr.val.func.min = $5;
> +   field_expr.val.func.max = $7; }
>   ;
>  
>  eth_proto
> @@ -1097,6 +1147,7 @@ void cleanup_packets(void)
>   for (i = 0; i < dlen; ++i) {
>   free(packet_dyn[i].cnt);
>   free(packet_dyn[i].rnd);
> + free(packet_dyn[i].fields);
>   }
>  
>   

[netsniff-ng] Re: [PATCH 05/15] trafgen: proto: Increment proto field at runtime

[Tobias Klauser]

On 2016-07-26 at 21:35:10 +0200, Vadim Kochan  wrote:
> Extended 'struct packet_dyn' with proto fields which has
> dynamically changing values at runtime.
> 
> Implement incrementing of proto field at runtime with min & max
> parameters, by default if the 'min' parameter is not specified
> then original value is used. For fields which len is greater
> than 4 - last 4 bytes are incremented as unsigned int value.
> 
> Added 'field_changed' callback for proto header which
> may be used for check if csum updating is needed. This callback
> is called after field was changed at runtime.
> 
> Added 'packet_update' callback to let proto header know
> when to apply final proto header changes at runtime (csum update).

The documentation of these callbacks would also make sense where they're
defined.

> Signed-off-by: Vadim Kochan 
> ---
>  trafgen.c   |  9 ++
>  trafgen_conf.h  |  7 
>  trafgen_proto.c | 99 
> +
>  trafgen_proto.h | 26 +++
>  4 files changed, 141 insertions(+)
> 
> diff --git a/trafgen.c b/trafgen.c
> index b76b5d7..553dfa5 100644
> --- a/trafgen.c
> +++ b/trafgen.c
> @@ -619,6 +619,15 @@ static inline void packet_apply_dyn_elements(int idx)
>   apply_randomizer(idx);
>   apply_csum16(idx);
>   }
> +
> + if (packet_dyn_has_fields(_dyn[idx])) {
> + uint32_t i;
> +
> + for (i = 0; i < packet_dyn[idx].flen; i++)
> + proto_field_dyn_apply(packet_dyn[idx].fields[i]);
> +
> + proto_packet_update(idx);
> + }
>  }
>  
>  static void xmit_slowpath_or_die(struct ctx *ctx, unsigned int cpu, unsigned 
> long orig_num)
> diff --git a/trafgen_conf.h b/trafgen_conf.h
> index 934f8fe..7f3616c 100644
> --- a/trafgen_conf.h
> +++ b/trafgen_conf.h
> @@ -49,6 +49,8 @@ struct packet_dyn {
>   size_t rlen;
>   struct csum16 *csum;
>   size_t slen;
> + struct proto_field **fields;
> + uint32_t flen;
>  };
>  
>  static inline bool packet_dyn_has_elems(struct packet_dyn *p)
> @@ -61,6 +63,11 @@ static inline bool packet_dyn_has_only_csums(struct 
> packet_dyn *p)
>   return (p->clen == 0 && p->rlen == 0 && p->slen);
>  }
>  
> +static inline bool packet_dyn_has_fields(struct packet_dyn *p)
> +{
> + return !!p->flen;
> +}
> +
>  extern void compile_packets_str(char *str, bool verbose, unsigned int cpu);
>  extern void compile_packets(char *file, bool verbose, unsigned int cpu,
>   bool invoke_cpp, char *const cpp_argv[]);
> diff --git a/trafgen_proto.c b/trafgen_proto.c
> index ce389ce..069aa00 100644
> --- a/trafgen_proto.c
> +++ b/trafgen_proto.c
> @@ -419,6 +419,19 @@ void protos_init(const char *dev)
>   p->ctx = 
>  }
>  
> +void proto_packet_update(uint32_t idx)
> +{
> + struct packet *pkt = packet_get(idx);
> + ssize_t i;
> +
> + for (i = pkt->headers_count - 1; i >= 0; i--) {
> + struct proto_hdr *hdr = pkt->headers[i];
> +
> + if (hdr->packet_update)
> + hdr->packet_update(hdr);
> + }
> +}
> +
>  void proto_packet_finish(void)
>  {
>   struct proto_hdr **headers = _packet()->headers[0];
> @@ -433,3 +446,89 @@ void proto_packet_finish(void)
>   p->packet_finish(p);
>   }
>  }
> +
> +static inline unsigned int field_inc(struct proto_field *field)
> +{
> + uint32_t val;
> +
> + val = field->func.val - field->func.min;
> + val = (val + field->func.inc) % field->func.max;

Shouldn't this be

val = (val + field->func.inc) % (field->func.max - field->func.min + 1)

to be consistent with apply_counter()?

Also, I think you should probably get rid of as many pointer
dereferences as possible in these runtime functions, i.e. store max and
min in temporary variables.

> + field->func.val = val + field->func.min;
> +
> + return field->func.val;
> +}
> +
> +static void field_inc_func(struct proto_field *field)
> +{
> + if (field->len == 1) {
> + uint8_t val;
> +
> + val = field_inc(field);
> + proto_field_set_u8(field->hdr, field->id, val);

Assignment on declaration please. Or even better:

proto_field_set_u8(field->hdr, field->id, field_inc(field));

> + } else if (field->len == 2) {
> + uint16_t val;
> +
> + val = field_inc(field);
> + proto_field_set_be16(field->hdr, field->id, val);

Same.

> + } else if (field->len == 4) {
> + uint32_t val;
> +
> + val = field_inc(field);
> + proto_field_set_be32(field->hdr, field->id, val);

Same.

> + } else if (field->len > 4) {
> + uint8_t *bytes = __proto_field_get_bytes(field);
> + uint32_t val;
> +
> + bytes += field->len - 4;
> + val = field_inc(field);
> +
> + *(uint32_t *)bytes = bswap_32(val);

This 

[netsniff-ng] Re: [PATCH 04/15] trafgen: proto: Force field id to be index in array

[Tobias Klauser]

On 2016-08-02 at 22:26:28 +0200, Vadim Kochan  wrote:
> Yes, bug_on(x) is good point, so I am expecting you will continue
> review the rest patches ?

Yes

-- 
You received this message because you are subscribed to the Google Groups 
"netsniff-ng" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[netsniff-ng] Re: [PATCH 1/3] trafgen: Allow to compile without libnl

[Tobias Klauser]

On 2016-08-02 at 22:17:41 +0200, Vadim Kochan <vadi...@gmail.com> wrote:
> On Tue, Aug 2, 2016 at 11:23 AM, Tobias Klauser <tklau...@distanz.ch> wrote:
> > On 2016-07-31 at 23:13:16 +0200, Vadim Kochan <vadi...@gmail.com> wrote:
> >> trafgen uses libnl only to inject mac80211 frames but
> >> it might be not needed in some embedded or switch environments,
> >> so lets make possible to disable this feature.
> >>
> >> In case if --rfraw option will be used - user will get the panic message.
> >>
> >> Signed-off-by: Vadim Kochan <vadi...@gmail.com>
> >> ---
> >>  mac80211.c   |  1 -
> >>  mac80211.h   | 15 +++
> >>  trafgen/Makefile | 21 +++--
> >>  3 files changed, 30 insertions(+), 7 deletions(-)
> >>
> >> diff --git a/mac80211.c b/mac80211.c
> >> index f22b600..9aea5a0 100644
> >> --- a/mac80211.c
> >> +++ b/mac80211.c
> >> @@ -24,7 +24,6 @@
> >>  #include 
> >>  #include 
> >>
> >> -#include "die.h"
> >>  #include "str.h"
> >>  #include "dev.h"
> >>  #include "mac80211.h"
> >> diff --git a/mac80211.h b/mac80211.h
> >> index dea4ae0..2780c03 100644
> >> --- a/mac80211.h
> >> +++ b/mac80211.h
> >> @@ -1,7 +1,22 @@
> >>  #ifndef MAC80211_H
> >>  #define MAC80211_H
> >>
> >> +#include "die.h"
> >> +#include "config.h"
> >> +
> >> +#ifdef CONFIG_NO_LIBNL
> >> +static inline void enter_rfmon_mac80211(const char *device, char **mondev)
> >> +{
> >> +panic("enter_rfmon_mac80211: CONFIG_NO_LIBNL option needs to be 
> >> disabled\n");
> >> +}
> >> +
> >> +static inline void leave_rfmon_mac80211(const char *mondev)
> >> +{
> >> +panic("leave_rfmon_mac80211: CONFIG_NO_LIBNL option needs to be 
> >> disabled\n");
> >> +}
> >
> > These messages both have a double negative which isn't very easy to
> > understand. I suggest to call the CONFIG directive CONFIG_LIBNL instead
> > which is also more in line with the other library CONFIG_* directives we
> > have.
> >
> > Also, you use the CONFIG_NO_LIBNL without setting it anywhere, this is
> > only done in patch 3/3. Please change the series order to make the
> > configure change come first.
> 
> I really think that CONFIG_{DISABLE or ENABLE}_LIBNL might be better
> because we really enable/disable it,
> but CONFIG_LIBNL is rather about existence of package, if to use this
> way then it will be needed (probably) to add C defines for
> CONFIG_LIBNL_ROUTE, CONFIG_LIBNL_GENL.

Form the point of view of compiling nesniff-ng/trafgen there is no
difference of whether an external library "exists" or was explicitely
disabled by the user. I'd really prefer CONFIG_LIBNL for this and I
don't think more fine-grained control (CONFIG_LIBNL_*) will be needed,
as we gain relatively little compared to the amount of #ifdef complexity
we introduce.
> 

-- 
You received this message because you are subscribed to the Google Groups 
"netsniff-ng" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[netsniff-ng] Re: [PATCH 04/15] trafgen: proto: Force field id to be index in array

[Tobias Klauser]

On 2016-07-26 at 21:35:09 +0200, Vadim Kochan  wrote:
> Usually proto fields array is sorted in the same order as related enum,
> so id may be used as index for faster lookup, it will make
> csum field calculation little faster at runtime.
> 
> Signed-off-by: Vadim Kochan 
> ---
>  trafgen_l3.h| 4 ++--
>  trafgen_proto.c | 8 +---
>  2 files changed, 3 insertions(+), 9 deletions(-)
> 
> diff --git a/trafgen_l3.h b/trafgen_l3.h
> index a1b1523..e0c9a1c 100644
> --- a/trafgen_l3.h
> +++ b/trafgen_l3.h
> @@ -10,14 +10,14 @@ enum ip4_field {
>   IP4_LEN,
>   IP4_ID,
>   IP4_FLAGS,
> + IP4_MF,
> + IP4_DF,
>   IP4_FRAG_OFFS,
>   IP4_TTL,
>   IP4_PROTO,
>   IP4_CSUM,
>   IP4_SADDR,
>   IP4_DADDR,
> - IP4_DF,
> - IP4_MF,
>  };
>  
>  enum ip6_field {
> diff --git a/trafgen_proto.c b/trafgen_proto.c
> index a1d56cf..ce389ce 100644
> --- a/trafgen_proto.c
> +++ b/trafgen_proto.c
> @@ -118,13 +118,7 @@ void proto_header_fields_add(struct proto_hdr *hdr,
>  
>  static struct proto_field *proto_field_by_id(struct proto_hdr *hdr, uint32_t 
> fid)
>  {
> - int i;
> -
> - for (i = 0; i < hdr->fields_count; i++)
> - if (hdr->fields[i].id == fid)
> - return >fields[i];
> -
> - panic("Failed lookup field id %u for proto id %u\n", fid, hdr->id);
> + return >fields[fid];

Please add an inline comment here explaining this behaviour. Otherwise
someone will stumble upon it in the future for sure.

Also, something along the lines of

  bug_on(hdr->fields[fid].id != fid);

should be added to catch future erroneous fields definitions.

-- 
You received this message because you are subscribed to the Google Groups 
"netsniff-ng" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[netsniff-ng] Re: [PATCH 03/15] trafgen: proto: Move proto headers into packet

[Tobias Klauser]

On 2016-07-26 at 21:35:08 +0200, Vadim Kochan  wrote:
> Till now headers were used only for packet creation at compile time
> only, which does not allow to handle dynamic fields update at runtime.
> It needs that proto_hdr entries will be not freed after compile is done.
> 
> Signed-off-by: Vadim Kochan 
> ---
>  trafgen_conf.h   |  6 ++
>  trafgen_parser.y | 17 ++---
>  trafgen_proto.c  | 40 +++-
>  trafgen_proto.h  |  2 ++
>  4 files changed, 41 insertions(+), 24 deletions(-)
> 
> diff --git a/trafgen_conf.h b/trafgen_conf.h
> index efce29c..934f8fe 100644
> --- a/trafgen_conf.h
> +++ b/trafgen_conf.h
> @@ -5,6 +5,10 @@
>  #include 
>  #include 
>  
> +#include "trafgen_proto.h"

This produces a circular header dependency together with

  #include "trafgen_conf.h"

in trafgen_proto.h. I guess the latter is not needed, so I dropped it
and applied the patch.

-- 
You received this message because you are subscribed to the Google Groups 
"netsniff-ng" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[netsniff-ng] Re: [PATCH 02/15] trafgen: proto: Reference to packet from proto_hdr struct

[Tobias Klauser]

On 2016-07-26 at 21:35:07 +0200, Vadim Kochan  wrote:
> Using of current_packet() is not relevant for dynamically updated
> fields so lets keep packet index in proto_hdr struct.
> 
> Signed-off-by: Vadim Kochan 

Applied, thanks!

-- 
You received this message because you are subscribed to the Google Groups 
"netsniff-ng" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[netsniff-ng] Re: [PATCH 3/3] configure: Add option to compile tools without libnl dependency

[Tobias Klauser]

On 2016-07-31 at 23:13:18 +0200, Vadim Kochan  wrote:
> Add command line parsing function which allows to compile tools (trafgen, 
> netsniff-ng)
> without libnl-xxx libraries.
> 
> Option --no-libnl sets CONFIG_NO_LIBNL=1.
> 
> Signed-off-by: Vadim Kochan 
> ---
>  configure | 52 +---
>  1 file changed, 49 insertions(+), 3 deletions(-)
> 
> diff --git a/configure b/configure
> index 105b1ec..85f2178 100755
> --- a/configure
> +++ b/configure
> @@ -18,8 +18,36 @@ HAVE_LIBGEOIP=0
>  HAVE_LIBZ=0
>  HAVE_TPACKET3=0
>  
> +CONFIG_NO_LIBNL=0
> +
>  # use "CROSS_COMPILE= SYSROOT= ./configure && make" for cross 
> compilation
>  
> +usage()
> +{
> + echo "Usage: ./configure [OPTIONS]"
> + echo -e  "\t-h, --help - print usage"
> +echo -ne "\t--no-libnl - compile without libnl"

Indenting is inconsistent.

> + echo "Some features (rfraw, nlmsg dissect) will not work in trafgen, 
> netsniff-ng."
> +
> + exit 0
> +}
> +
> +while [ $# -gt 0 ]
> +do
> + case "$1" in
> + -h|--help)
> + usage
> + ;;
> + --no-libnl)
> + CONFIG_NO_LIBNL=1
> + ;;

Most autoconf configure scripts use --disable-xxx. I suggest we stick to
that scheme (even though we don't use autotools).

-- 
You received this message because you are subscribed to the Google Groups 
"netsniff-ng" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[netsniff-ng] Re: [PATCH 1/3] trafgen: Allow to compile without libnl

[Tobias Klauser]

On 2016-07-31 at 23:13:16 +0200, Vadim Kochan  wrote:
> trafgen uses libnl only to inject mac80211 frames but
> it might be not needed in some embedded or switch environments,
> so lets make possible to disable this feature.
> 
> In case if --rfraw option will be used - user will get the panic message.
> 
> Signed-off-by: Vadim Kochan 
> ---
>  mac80211.c   |  1 -
>  mac80211.h   | 15 +++
>  trafgen/Makefile | 21 +++--
>  3 files changed, 30 insertions(+), 7 deletions(-)
> 
> diff --git a/mac80211.c b/mac80211.c
> index f22b600..9aea5a0 100644
> --- a/mac80211.c
> +++ b/mac80211.c
> @@ -24,7 +24,6 @@
>  #include 
>  #include 
>  
> -#include "die.h"
>  #include "str.h"
>  #include "dev.h"
>  #include "mac80211.h"
> diff --git a/mac80211.h b/mac80211.h
> index dea4ae0..2780c03 100644
> --- a/mac80211.h
> +++ b/mac80211.h
> @@ -1,7 +1,22 @@
>  #ifndef MAC80211_H
>  #define MAC80211_H
>  
> +#include "die.h"
> +#include "config.h"
> +
> +#ifdef CONFIG_NO_LIBNL
> +static inline void enter_rfmon_mac80211(const char *device, char **mondev)
> +{
> +panic("enter_rfmon_mac80211: CONFIG_NO_LIBNL option needs to be 
> disabled\n");
> +}
> +
> +static inline void leave_rfmon_mac80211(const char *mondev)
> +{
> +panic("leave_rfmon_mac80211: CONFIG_NO_LIBNL option needs to be 
> disabled\n");
> +}

These messages both have a double negative which isn't very easy to
understand. I suggest to call the CONFIG directive CONFIG_LIBNL instead
which is also more in line with the other library CONFIG_* directives we
have.

Also, you use the CONFIG_NO_LIBNL without setting it anywhere, this is
only done in patch 3/3. Please change the series order to make the
configure change come first.

-- 
You received this message because you are subscribed to the Google Groups 
"netsniff-ng" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [netsniff-ng] [PATCH] netsniff-ng: account skipped packets as 'seen' and 'dropped'

[Tobias Klauser]

On 2016-07-27 at 15:59:08 +0200, Paolo Abeni  wrote:
> The packets filtered out due to pkt_type are incoming packets
> effectively dropped and should be accounted as such.
> This patch explicitly accounts the skipped packets number in
> skip_packet() and add such number to the 'drop' and 'seen'
> counters in update_rx_stats().
> 
> Signed-off-by: Paolo Abeni 

Applied, thanks Paolo!

-- 
You received this message because you are subscribed to the Google Groups 
"netsniff-ng" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [netsniff-ng] [PATCH 01/15] trafgen: Move applying dynamic elements to function

[Tobias Klauser]

On 2016-07-26 at 21:35:06 +0200, Vadim Kochan  wrote:
> Same code for applying dynamic elements is used in both places
> for slow & fast path modes, lets move them into one inlined function.
> 
> Signed-off-by: Vadim Kochan 

Applied, thanks.

-- 
You received this message because you are subscribed to the Google Groups 
"netsniff-ng" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [netsniff-ng] Re: [PATCH 00/15] trafgen: Support dinc & drnd for proto fields

[Tobias Klauser]

Nice work Vadim! This looks good from a bird's-eye view, but due to the
size of the changes it will probably take me some time to review it
completely and give you feedback (with the exception of patch 1/15 which
is a no-brainer and I already applied it).

On 2016-07-27 at 13:04:45 +0200, Vadim Kochan  wrote:
> I really missed one thing related to csum - so if I have proto field
> with dynamic function then csum will be calculated
> minimum twice - after packet is build and after all dynamic fields are
> applied, well I have no particular solution for this,
> but let me know if you want to have it fixed in this series or after
> it will be applied.

I think this can go into a separate series.

Thanks
Tobias

-- 
You received this message because you are subscribed to the Google Groups 
"netsniff-ng" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.