On 05/06/2016 15:06, Peter Haag wrote:
This time-wrap is a kind of hassle since ever. It actually has been tested on
many exporters - ok mainly on Cisco's, as it seemed to be used widely.
<sorry was busy...>
I ran a tcpdump, I will send you a pcap file off-list shortly.
I haven't looked into softflowd overruns, as I expected them too behave the
same.
I am open to help debugging the stuff, you have me a pcap with an overflow,
which could be rather difficult to produce.
Alternatively, you could add some LogInfo() messages in case of a wrap around
to log the original values to see how compensation needs to be done correctly
or even better to fix softflowd.
How do I generate LogInfo() messages?
Anyway, looking at my netflow, I picked a host which had a relatively
small number of flows/packets in the 5 minute window, but also a silly
duration, and then restricted my query just to that host:
** nfdump -M /var/nfsen/profiles-data/live/lch-fw1 -T -r
2016/06/09/nfcapd.201606091700 -n 500 -s ip/bytes
nfdump filter:
host 74.125.206.156
Top 500 IP Addr ordered by bytes:
Date first seen Duration Proto IP Addr Flows(%)
Packets(%) Bytes(%) pps bps bpp
2016-04-20 23:52:34.099 4294727.209 any10.26.1.189
<http://localhost:8888/nfsen/nfsen.php#null> 2(100.0) 44(100.0)
7967(100.0) 0 0 181
2016-04-20 23:52:34.099 4294727.209 any74.125.206.156
<http://localhost:8888/nfsen/nfsen.php#null> 2(100.0) 44(100.0)
7967(100.0) 0 0 181
Summary: total flows: 2, total bytes: 7967, total packets: 44, avg bps: 0, avg
pps: 0, avg bpp: 181
Time window: 2016-04-20 23:39:14 - 2016-06-09 17:02:35
Total flows processed: 18130, Blocks skipped: 0, Bytes read: 1169124
Sys: 0.008s flows/second: 2266250.0 Wall: 0.008s flows/second: 2044429.4
Looking at the individual flows I an see only "date first seen":
** nfdump -M /var/nfsen/profiles-data/live/lch-fw1 -T -r
2016/06/09/nfcapd.201606091700 -c 20
nfdump filter:
host 74.125.206.156
Date first seen Event XEvent Proto Src IP Addr:Port Dst
IP Addr:Port X-Src IP Addr:Port X-Dst IP Addr:Port In Byte Out Byte
2016-06-09 15:45:27.091 INVALID Ignore TCP10.26.1.189:62498
<http://localhost:8888/nfsen/nfsen.php#null> ->74.125.206.156:443
<http://localhost:8888/nfsen/nfsen.php#null> 0.0.0.0:0
<http://localhost:8888/nfsen/nfsen.php#null> ->0.0.0.0:0
<http://localhost:8888/nfsen/nfsen.php#null> 1984 0
2016-04-20 23:52:34.099 INVALID Ignore TCP74.125.206.156:443
<http://localhost:8888/nfsen/nfsen.php#null> ->10.26.1.189:62498
<http://localhost:8888/nfsen/nfsen.php#null> 0.0.0.0:0
<http://localhost:8888/nfsen/nfsen.php#null> ->0.0.0.0:0
<http://localhost:8888/nfsen/nfsen.php#null> 5983 0
Summary: total flows: 2, total bytes: 7967, total packets: 44, avg bps: 0, avg
pps: 0, avg bpp: 181
Time window: 2016-04-20 23:39:14 - 2016-06-09 17:02:35
Total flows processed: 18130, Blocks skipped: 0, Bytes read: 1169124
Sys: 0.008s flows/second: 2266250.0 Wall: 0.009s flows/second: 1945696.5
Doing this again at the CLI with -o raw, I can see "first" and "last": $
nfdump -M /var/nfsen/profiles-data/live/lch-fw1 -T -r
2016/06/09/nfcapd.201606091700 -c 20 -o raw 'host 74.125.206.156' Flow
Record: Flags = 0x06 FLOW, Unsampled export
sysid = 1 size = 56
first = 1465483527 [2016-06-09 15:45:27] last
= 1465487481 [2016-06-09 16:51:21] msec_first
= 91 msec_last = 308 src addr
= 10.26.1.189 dst addr = 74.125.206.156 src port
= 62498 dst port = 443 fwd status
= 0 tcp flags = 0x1b .AP.SF
proto = 6 TCP (src)tos = 0
(in)packets = 21 (in)bytes = 1984
Flow Record: Flags = 0x06 FLOW, Unsampled export
sysid = 1 size = 56
first = 1461192754 [2016-04-20 23:52:34] last
= 1465487481 [2016-06-09 16:51:21] msec_first
= 99 msec_last = 308 src addr
= 74.125.206.156 dst addr = 10.26.1.189 src port
= 443 dst port = 62498 fwd status
= 0 tcp flags = 0x1b .AP.SF
proto = 6 TCP (src)tos = 0
(in)packets = 23 (in)bytes = 5983
Summary: total flows: 2, total bytes: 7967, total packets: 44, avg bps:
0, avg pps: 0, avg bpp: 181 Time window: 2016-04-20 23:39:14 -
2016-06-09 17:02:35 Total flows processed: 18130, Blocks skipped: 0,
Bytes read: 1169124 Sys: 0.012s flows/second: 1510833.3 Wall: 0.010s
flows/second: 1730457.2 Now, looking at my pcap file: First packet after
Template has Arrival Time: Jun 9, 2016 16:42:51.064505000 BST
Epoch Time: 1465486971.064505000 seconds If I now decode this with
tshark: $ sudo tshark -r fw1.pcap -nnV -d udp.port==9995,cflow | less
... search for 74.125.206.156 Frame 21073: 534 bytes on wire (4272
bits), 534 bytes captured (4272 bits) Arrival Time: Jun 9, 2016
17:37:09.499219000 BST Epoch Time: 1465490229.499219000 seconds ...
Cisco NetFlow/IPFIX Version: 9 Count: 15 SysUptime: 273579393
Timestamp: Jun 9, 2016 17:37:09.000000000 BST CurrentSecs:
1465490229 FlowSequence: 4885983 SourceId: 0 FlowSet 1
FlowSet Id: (Data) (1024) FlowSet Length: 472
Flow 1 SrcAddr: 10.26.1.189 (10.26.1.189)
DstAddr: 74.125.206.156 (74.125.206.156) [Duration:
-240.057000000 seconds] StartTime: 273335.501000000
seconds EndTime: 273095.444000000 seconds
Octets: 2223 Packets: 20 SrcPort: 63141
DstPort: 443 Protocol: 6 TCP Flags:
0x1b IPVersion: 04 Flow 2 SrcAddr:
74.125.206.156 (74.125.206.156) DstAddr: 10.26.1.189
(10.26.1.189) [Duration: -240.057000000 seconds]
StartTime: 273335.501000000 seconds
EndTime: 273095.444000000 seconds Octets: 2031
Packets: 20 SrcPort: 443 DstPort: 63141
Protocol: 6 TCP Flags: 0x1b
IPVersion: 04 This is not exactly the same flow. However you can see
that according to tshark, StartTime is after EndTime; this seems to be
true for other flows too. This might just be a bug in softflowd
(although strange if no one has noticed it before). I'll send you the
pcap file. Cheers, Brian.
------------------------------------------------------------------------------
What NetFlow Analyzer can do for you? Monitors network bandwidth and traffic
patterns at an interface-level. Reveals which users, apps, and protocols are
consuming the most bandwidth. Provides multi-vendor support for NetFlow,
J-Flow, sFlow and other flows. Make informed decisions using capacity
planning reports. https://ad.doubleclick.net/ddm/clk/305295220;132659582;e
_______________________________________________
Nfdump-discuss mailing list
Nfdump-discuss@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfdump-discuss
