-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Hi Joerg
- --On February 19, 2008 13:00:17 +0100 [EMAIL PROTECTED] wrote:
| Hi Peter,
|
| it seems that you are _the_ expert in this project - are you? Thank you for
your patience with us/me. I am very new to
| netflowing and need some advice and knowledge input.
|
| If you can point me to a good FAQ or something else I would be pleased but
maybe I am out of "frequently".
There are lots of papers on the net describing netflow. So dig the net for any
plain netflow papers, which show how netflow
works. In the end netflow can become more than complex.
|
|
| I don't know if it is relevant for my question but I use standard flow
expire/export of five minutes at the probes and
| standard five minute rotation interval at nfsen. So defaults everywhere. If
my data file contains flows over several days,
| this was done by nfcapd.
The files contains the flows, which were exported during the 5min periode. The
start/end times of a flow are specified in the
flow record, and are filled in by the exporter. These time stamps in msec are
relative to the boot time of the exporter device.
nfdump only display, what netflow records contains. So I can't tell you why you
exporter sends these flows.
|
|
| You mentioned the risk of 32bit overflow at the exporter. What is the
"exporter"? Is nfdump the exporter? How can I take care
| of counter overflows? If a flow lasts some days - well the tools (nfsen) must
handle this long lasting flows - if this is the
| problem with counter overflow at all.
No - an exporter exports flows, as the name says. nfdump comes with a collector
(nfcapd ) and a flow processing tool (nufump)
- Peter
|
| --->
|
| I would like to understand the whole netflow process. From the probe to the
sensor to the exporter.
|
| May I draft my understanding of flow processing?
|
| 1. netflow probe handles flow counting
|
| If a flow expires (either by timer/max traffic/max flow/FIN/RST or what
ever), netflow information is send to the sensor and
| the probe forgets all about this flow. The probe accumulates only a five
minute stateful flow information. After expiration
| every packet newly arriving creates a new entry in the flow table of the
probe.
|
| 2. nfsen processes the netflow information and tries to provide something
like a stateful flow information from the real
| beginning to the end of a flow. To accomplish this, nfcapd seems to look into
the historical files in the profile-data
| directory to see if any of the "new" flows were active in the past. Nfcapd
retrieves the flow starttime from historical data
| and puts it into the profile-data file of the current timeslot so that flow
starttime is carried over into the timeslot data
| file.
|
| Am I right?
|
| What other historical data is carried over into the new data file of the
current timeslot? The flags, as you mentioned. I
| hope that the number of bytes and packets is not accumulated with the past
but only flow information gathered by the probe
| and arriving during the 5 minute interval is stored into the data file of
current timeslot.
|
| I understand that because of two different 5 minute intervals at the probe
and at the sensor and because the flow expiring
| can take place at any time during this intervals it is possible that traffic
information is sometimes stored a little bit to
| late.
|
|
| So beside this inaccuracy is it true that the "Packets" and "Bytes" printed
be nfdump with "-s record/packets" option are
| only the packets and bytes that had been seen by the probe during this
interval? When I do a "-o extended" dump, are the
| ratio values of pps and bps calculated at a timeslot base or at the
flow-duration base? I would expect that if I select a
| single timeslot to dump that calculation is done for that single timeslot
only but as you explained that all time values are
| taken from the data file maybe pps und bps are calculated since flow
starttime.
|
|
|
| Joerg
|
| -----Original Message-----
| From: Peter Haag [mailto:[EMAIL PROTECTED]
| Sent: Monday, February 18, 2008 3:29 PM
| To: Pichel, Jörg; [email protected]
| Subject: Re: [Nfsen-discuss] How to interpret nfdump output
|
| -----BEGIN PGP SIGNED MESSAGE-----
| Hash: SHA1
|
| Hi Joerg,
|
| A flow has a dedicated start a and end time. When aggregating flows ( as it's
done in -s ) then more records could be summed
| up. In any case the resulted flow has a start and end time resulting in a
duration, regardless, whether the flow is not yet
| finished. All rate values are calculated for each flow, which its specific
duration, to get an average value for bps, pps etc.
| for each flow.
|
| The summary lines at the bottom give an overall average value, in the time
window seen total packets/bytes divided by
| duration. The fact if a flow is completed or not is not relevant.
|
| So time values are taken from the flows found in the file. The file name, or
other issues are not relevant. So your file
| contains flows over several days. Be also aware, that the exporter must be
able to deal with the 32bit overflow from sysup
| time otherwise bizare flows could result - maybe yours below.
|
| As for the flags - yes they are all cumulated OR
|
|
| - Peter
|
| - --On February 18, 2008 14:02:01 +0100 [EMAIL PROTECTED] wrote:
|
| |
| |
| | I tried to analyze a very high pks/s peak. For this I selected one
| | single timeslot "Feb 18 2008 - 10:45", called nfdump and got this output:
| |
| |
| | ** nfdump -M /sw/pkg/nfsen/profiles-data/live/mucwan -T -r
| | 2008/02/18/nfcapd.200802181045 -n 20 -s record/packets -o long nfdump
filter:
| | any
| | Aggregated flows 22705
| | Top 20 flows ordered by packets:
| | Date flow start Duration Proto Src IP Addr:Port Dst
IP Addr:Port Flags Tos Packets Bytes Flows
| | 2008-02-12 12:56:04.110 510524.741 TCP 10.40.232.5:445 ->
10.40.200.5:1814 .APR.. 0 5.1 M 1.5 G 1
| | 2008-02-12 12:56:04.110 510524.741 TCP 10.40.200.5:1814 ->
10.40.232.5:445 .AP... 0 4.9 M 659.2 M 1
| | 2008-02-12 12:56:30.231 510498.626 TCP 10.40.232.5:445 ->
10.40.208.5:1388 .APR.. 0 4.6 M 1.4 G 1
| | 2008-02-12 12:56:30.231 510498.626 TCP 10.40.208.5:1388 ->
10.40.232.5:445 .AP... 0 4.4 M 603.7 M 1
| | 2008-02-12 12:55:58.184 510530.659 TCP 10.40.232.5:445 ->
10.40.216.5:3228 .APR.. 0 1.9 M 648.2 M 1
| | 2008-02-12 12:55:58.184 510530.659 TCP 10.40.216.5:3228 ->
10.40.232.5:445 .AP... 0 1.8 M 238.9 M 1
| | 2008-02-12 12:56:32.902 510495.941 TCP 10.40.232.5:445 ->
10.40.224.5:1779 .APR.. 0 1.7 M 624.4 M 1
| | 2008-02-12 12:55:58.184 510530.674 TCP 10.40.232.5:445 ->
10.40.204.5:2848 .APR.. 0 1.7 M 620.6 M 1
| | 2008-02-12 12:56:32.902 510495.941 TCP 10.40.224.5:1779 ->
10.40.232.5:445 .AP... 0 1.5 M 209.8 M 1
| | 2008-02-12 12:55:58.184 510530.674 TCP 10.40.204.5:2848 ->
10.40.232.5:445 .AP... 0 1.5 M 208.3 M 1
| | 2008-02-16 07:31:50.615 184378.288 TCP 10.40.232.5:445 ->
10.40.240.32:1084 .APRS. 0 871746 262.1 M 1
| | 2008-02-16 07:31:50.615 184378.288 TCP 10.40.240.32:1084 ->
10.40.232.5:445 .AP.S. 0 827661 110.1 M 1
| | 2008-02-17 18:10:15.823 59673.029 TCP 10.40.232.5:445 ->
10.40.220.5:3577 .APRS. 0 452797 159.3 M 1
| | 2008-02-17 18:10:15.823 59673.029 TCP 10.40.220.5:3577 ->
10.40.232.5:445 .AP.S. 0 422606 55.0 M 1
| | 2008-02-18 10:35:23.499 565.379 TCP 10.40.212.5:1083 ->
10.40.232.5:445 .AP.S. 0 26698 4.0 M 1
| | 2008-02-18 10:35:23.499 565.379 TCP 10.40.232.5:445 ->
10.40.212.5:1083 .APRS. 0 26084 4.0 M 1
| | 2008-02-18 10:22:39.148 1108.766 TCP 80.90.99.41:3505 ->
10.40.224.6:1152 .AP.SF 0 15114 16.6 M 1
| | 2008-02-18 10:45:40.760 45.540 TCP 10.40.232.7:8080 ->
10.40.208.7:33746 .AP.S. 0 13181 18.4 M 1
| |
| | Summary: total flows: 22707, total bytes: 7.4 G, total packets: 32.0
| | M, avg bps: 124932, avg pps: 65, avg bpp: 237 Time window: 2008-02-12
| | 12:55:58 - 2008-02-18 10:46:44 Total flows processed: 22707, Records
skipped: 0, Bytes read: 1180788
| | Sys: 0.032s flows/second: 709571.6 Wall: 0.030s flows/second: 744174.6
| |
| |
| | Are the numbers of pkts/s, bytes/s that are printed in the table the
| | number of pakets and bytes that flew during the 5-minute interval from
| | 10:40 - 10:45 or are these the packets and bytes that flew since
| | flowstart, which are mostly some days in the past. The "Time window" given
in the "Summary" information could lead to this
| | opinion - but this is not the intention if I select a single timeslot to
inspect.
| |
| | How can nfdump know when the flow started when I give it only one
| | single timeslot to inspect? Does nfdump read all the old data as well to
find the flow start or is this information
| | collected by nfcapd and stored in the five minutes dump files?
| |
| | And what about the "Flags" - Is this a cummulation of TCP-Flags during
| | this inspection interval (given as single timeslot
| | 10:45) or since flow start? And can I conclude that if there have been
| | "R"eset Flags seen in the flow that this flow has ended during the
inspection interval and the others are potentially still
| | active?
| |
|
|
| -------------------------------------------------------------------------
| This SF.net email is sponsored by: Microsoft
| Defy all challenges. Microsoft(R) Visual Studio 2008.
| http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
| _______________________________________________
| Nfsen-discuss mailing list
| [email protected]
| https://lists.sourceforge.net/lists/listinfo/nfsen-discuss
- --
_______ SWITCH - The Swiss Education and Research Network ______
Peter Haag, Security Engineer, Member of SWITCH CERT
PGP fingerprint: D9 31 D5 83 03 95 68 BA FB 84 CA 94 AB FC 5D D7
SWITCH, Werdstrasse 2, P.O. Box, CH-8021 Zurich, Switzerland
E-mail: [EMAIL PROTECTED] Web: http://www.switch.ch/
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.3 (Darwin)
iQCVAwUBR70uNP5AbZRALNr/AQKuOAP9H/p1yuzaqEkh9iGMMAtF3nhXm2KVvNVc
LlHLilN2kSIzcI5wj+MdVb5mxwjQc2BM3jaux9IV3EnFxPzGOKwATKupmTxWkTHe
Ry2J8GEgp9uzYx2yqaE8orTj0xBoerF2ip0uuQabwR0QCknQiZCZT539GSbkAPAH
4FopBxoN4sA=
=hO4s
-----END PGP SIGNATURE-----
-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
_______________________________________________
Nfsen-discuss mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/nfsen-discuss
