Hi,
thanks for the new version of nfdump. It looks really good and many
of the features are what we were waiting for. Good work was done.
I've tried to use it on own systems and I come across a problem which
occurs when the aggregation feature (-A) is used.
A simple command:
nfdump -r ../CESNET.anon/2010-01-10/nfcapd.201001101300 -A proto -a
leads to segmentation fault error:
$ nfdump -r ../CESNET.anon/2010-01-10/nfcapd.201001101300 -A proto -a
Date flow start Duration Proto Packets Bytes bps
Bpp Flows
2010-01-10 12:59:03.575 3542.511 IGMP 59 1652 3
28 59
2010-01-10 12:58:02.778 3598.948 ICMP6 3004 222391 494
74 1193
Segmentation fault
This behaviour not happens everytime. For exmaple when the "inif"
attribute is used to aggregation then nfdump works fine.
[r...@coyote 2010-01-10]# nfdump -r nfcapd.201001101300 -a -A inif
Date flow start Duration Input Packets Bytes bps
Bpp Flows
2010-01-10 12:53:25.961 3936.489 2 185.9 M 138.9 G 282.3 M
747 8218831
2010-01-10 12:53:21.996 3940.237 1 180.9 M 129.6 G 263.1 M
716 8214775
Summary: total flows: 16433606, total bytes: 268.5 G, total packets:
366.8 M, avg bps: 545.2 M, avg pps: 93079, avg bpp: 732
Time window: 2010-01-10 12:53:21 - 2010-01-10 13:59:02
Total flows processed: 16433606, Blocks skipped: 0, Bytes read: 855079096
Sys: 8.226s flows/second: 1997582.3 Wall: 8.867s flows/second: 1853241.3
The similar situation comes when either srcip or dstip item is used. The
srcip works fine, but the dstip fails.
I tried both 32 and 64 platform with the same result.
If you are interested in the issue I put several anonymized files where
the problem appears on the http://hawk.cis.vutbr.cz/~tpoder/tmp/nfdump/
site.
Best regards,
Tomas
Peter Haag wrote:
> Dear all,
> I'm happy to announce, that nfdump-1.6 is available for downloading
> @ Sourceforge. Several new features have been added ( see list below )
> nfdump-1.6 is mostly compatible with nfdump-1.5.x.
> nfdump-1.6 works with current NfSen 1.3.2, however, the new features
> are not
> accessible using the interface.
> *** Please note: *** PortTracker from NfSen 1.3.2 does *NOT* work with
> nfdump-1.6.
> An updated version for NfSen/PortTracker will be released later.
>
>
> NEW in 1.6 since 1.5.8 ( latest on top )
> ----------------------
> o Add router IP extension.
> o Add router ID extension (engine type/ID)
> o Add srcmask and dstmask aggregation
> o Aggregated ( -a, -A, -b, -B ) or sorted flows ( -m ) can be written back
> to binary files ( -w )
> Note: This results in a behaviour change for -w in combination
> with aggregation
> o Extend -N ( do not scale numbers ) to all text output not just summary
> o Remove header lines of -s stat, when using -q ( quiet )
> Note: This results in a behaviour change for -N
> o Remove legacy v1.4 file compatibility
> o Remove -S option from nfdump ( legacy 1.4 compatibility )
> o Make use of log (syslog) functions for nfprofile.
> o Move log functions to util.c
> o Update sflow collector.
> o Add parse_csv.pl script as an example to parse csv output
> o Add csv output format ( -o cvs ) as replacement for -o pipe - keep
> -o pipe for now.
> o Flow-tools converter updated - supports all common elements.
> o Sflow collector updated. Supports more common elements.
> o Add sampling to nfdump. Sampling is automatically recognised
> in undocumented v5 header fields and in v9 option templates.
> see nfcapd(1)
> o Add @include option for filter to include more filter files.
> o Add bidirectional aggregation ( -b, -B ) - experimental feature
> o Add flexible aggregation comparable to Flexible Netflow (FNF)
> over all available v9 tags
> o All new tags can be selected in -o fmt:... see nfdump(1)
> o topN stat for all new tags is implemented
> o Integrate developer code to read from pcap files into stable branch
> o Update filter syntax for new tags
> o Add flexible storage option for nfcapd. To save disk space, the
> data extensions to be stored in the data file are user selectable.
> o Added more v9 tags for netflow v9.
> The detailed tags are listed in nfcapd(1) Beside of MAC addresses
> and VLAN labels, also MPLS labels and many more v9 tags are now
> supported. AS numbers and interface numbers are now 32bit clean.
> Adding new tags also extended the binary file format with
> data block type 2, which is extension based. File format
> for version <= 1.5.* ( Data block format type 1 ) is read
> transparently. ( --enable-compat15 ) Data block type 2 are skipped
> by nfdump 1.5.8.
> o Added option for multiple netflow stream to same port.
> -n <Ident,IP,base_directory>
> Example: -n router1,192.168.100.1,/var/nfdump/router1
> So multiple -n options may be given at the command line
> Old style syntax still works for compatibility, ( -I .. -l ... )
> but then only one source is supported.
> o Move to automake for building nfdump
> o Make nfdump fully 64bit compliant. ( 32/64bit data alignments and
> access )
> Compiles and runs cleanly on 32/64bit systems
> o Switch scaling factor ( k, M, G ) from 1024 to 1000.
>
>
------------------------------------------------------------------------------
This SF.Net email is sponsored by the Verizon Developer Community
Take advantage of Verizon's best-in-class app development support
A streamlined, 14 day to market process makes app distribution fast and easy
Join now and get one step closer to millions of Verizon customers
http://p.sf.net/sfu/verizon-dev2dev
_______________________________________________
Nfdump-discuss mailing list
Nfdump-discuss@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfdump-discuss
------------------------------------------------------------------------------
This SF.Net email is sponsored by the Verizon Developer Community
Take advantage of Verizon's best-in-class app development support
A streamlined, 14 day to market process makes app distribution fast and easy
Join now and get one step closer to millions of Verizon customers
http://p.sf.net/sfu/verizon-dev2dev
_______________________________________________
Nfdump-discuss mailing list
Nfdump-discuss@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfdump-discuss
