Hi,
Thanks Adrian. That worked well (What a newbie am I !!!) .
From what I can see, The problem comes from the exporter (pflow) which
sends flows with an end date 1 second oldest than the start date
example :
Cisco NetFlow/IPFIX
Version: 5
Count: 30
SysUptime: 251457000
Timestamp: Jul 26, 2012 11:05:48.025111758
CurrentSecs: 1343293548
CurrentNSecs: 25111758
FlowSequence: 104587950
EngineType: 42
EngineId: 42
00.. .... .... .... = SamplingMode: No sampling mode configured (0)
..00 0000 0000 0000 = SampleRate: 0
pdu 1/30
....
pdu 20/30
SrcAddr: X.X.X.X (X.X.X.X)
DstAddr: 194.57.169.116 (194.57.169.116)
NextHop: 0.0.0.0 (0.0.0.0)
InputInt: 0
OutputInt: 0
Packets: 1
Octets: 60
[Duration: 4294966.296000000 seconds]
StartTime: 251367.000000000 seconds
EndTime: 251366.000000000 seconds
SrcPort: 53
DstPort: 55680
padding
TCP Flags: 0x00
Protocol: 6
IP ToS: 0x00
SrcAS: 0
DstAS: 0
SrcMask: 0 (prefix: X.X.X.X/32)
DstMask: 0 (prefix: 194.57.169.116/32)
padding
.....
The exporter seems to be on time and the problem appears more often than
time correction frequency.
We are looking with firewall administrators what solution we can find.
Do somebody uses pf as an netflow exporter on the list ?
Regards,
Cédric
Le 25/07/2012 09:39, Adrian Popa a écrit :
To capture flow data instruct tcpdump to write to a file (-s 1500 -w
/tmp/mycapture.cap), leave it on for a while and then open it in
wireshark. Select "Decode as" cflow in wireshark. It may still not
display the full payload if it doesn't capture a template file, which
is sent periodically (as quick as every second, or as slow as once in
30 minutes - it depends).
On Tue, Jul 24, 2012 at 7:33 PM, cedric.delaunay
<cedric.delau...@gmail.com <mailto:cedric.delau...@gmail.com>> wrote:
Hi,
Thanks for your answers.
I'll try to upgrade nfsen and nfdump to lastest versions
(currently 1.3.5 and 1.6.1p1)
I can't see how to capture the flows as a tcpdump on collector's
machine won't give me packets content. Am I wrong ? Any Idea howto
? Maybe with "-E" option on nfcapd but data are already interpreted...
I'm unable to read data like
00:05:00:1e:06:32:4d:c0:50:0e:c8:44:00:9e:61:4e:03:22:64:3a:2a:2a:00:00:26:71:a5:50:81:14:5e:1b:00:00:00:00:00:00:00:00:00:00:00:04:00:00:00:ae:06:30:e2:78:06:30
;)
I will see when I'll be back thursday.
Bye
Le 24/07/2012 17:22, Jan Pazdera a écrit :
Hi,
this can sometimes happen, if the exporter shifts its system
time, for example during NTP time correction. Try to check,
whether the exporter does this or not and if so, turn it off to
see, if this is the reason.
The problem is connected with nfcapd > v1.6. When it detects a
time shift, it performs a time correction, which sometimes leads
to overflow.
Regards,
Jan
Jan Pazdera <pazdera atinvea.cz <http://invea.cz>>
INVEA-TECH a.s.
U Vodarny 2965/2, 61600 Brno, Czech Republic
Tel:+420 511 205 251 <tel:%2B420%20511%20205%20251>
www.invea-tech.com <http://www.invea-tech.com>
Key 0x89F62F78: 41A7 28C2 C624 FBD1 E236 6827 42EB 3694 89F6 2F78
On 24.7.2012 17:05, Adrian Popa wrote:
I guess a packet capture of those flows might help determine if
they are sent with this duration timestamp incorrectly set, or
if nfcapd interprets them incorrectly.
Peter will be able to tell you more.
On Tue, Jul 24, 2012 at 6:00 PM, Adrian Popa
<adrian.popa...@gmail.com <mailto:adrian.popa...@gmail.com>> wrote:
Usually the network device which sends flows (the router)
will expire flows based on flow termination or on a specific
timeout value. For Cisco you can set timeouts for active
flows to force them to expire even if there is still traffic
(useful in case of a DOS attack).
I don't remember the commands but I can look them up if you
need them.
I don't think nfcapd can do anything about this - the
timestamps you are getting look like a bug - it looks
awfully close to 4294967296 - which is 2^32. So I would
suspect a variable overflow somewhere. Negative time
perhaps, with a positive int?
On Tue, Jul 24, 2012 at 5:02 PM, cedric.delaunay
<cedric.delau...@gmail.com
<mailto:cedric.delau...@gmail.com>> wrote:
Hi all,
One small question before holidays.
As we know, flow expiration on the exporter runs if no
packet comes in a
flow or if a "end of session" tcp flag is detected.
Then the exporter will inform nfcapd in an udp packet.
What's happens if this packet is lost on the network ?
Will nfsen never
see that this flow has expired ?
I found flows with duration up to 4 000 000 000 ms and
only 1 flow.
here a example anonymized :
Date flow start Duration Proto Src IP
Addr:Port Dst IP Addr:Port Packets Bytes Flows
2012-07-24 10:59:54.007 2.000 UDP
113.107.214.100:61918 <http://113.107.214.100:61918> ->
216.67.102.45:2122 <http://216.67.102.45:2122> 1
131 1
2012-07-24 10:59:54.007 2.000 UDP 216.67.102.45:2122
<http://216.67.102.45:2122> -> 113.107.214.100:61918
<http://113.107.214.100:61918> 1 305 1
2012-07-24 10:59:59.007 3.000 UDP 62.252.190.196:123
<http://62.252.190.196:123> -> 61.192.94.167:123
<http://61.192.94.167:123> 4 304 1
2012-07-24 10:59:59.007 3.000 UDP 61.192.94.167:123
<http://61.192.94.167:123> -> 62.252.190.196:123
<http://62.252.190.196:123> 4 304 1
2012-06-04 17:59:57.711 4294966.296 UDP
113.107.184.123:27057 <http://113.107.184.123:27057> ->
216.67.102.45:2122 <http://216.67.102.45:2122> 1
126 1
2012-06-04 17:59:57.711 4294966.296 UDP
216.67.102.45:2122 <http://216.67.102.45:2122> ->
113.107.184.123:27057 <http://113.107.184.123:27057>
1 309 1
2012-07-24 11:32:08.008 116.000 TCP
113.107.219.116:36157 <http://113.107.219.116:36157> ->
218.185.100.221:80 <http://218.185.100.221:80>
7 730 1
2012-07-24 11:32:08.008 116.000 TCP 218.185.100.221:80
<http://218.185.100.221:80> -> 113.107.219.116:36157
<http://113.107.219.116:36157> 5 1764 1
2012-07-24 11:54:54.008 9.000 TCP
113.107.79.246:38264 <http://113.107.79.246:38264> ->
242.194.34.210:25 <http://242.194.34.210:25> 3
156 1
2012-07-24 11:59:59.008 1.000 UDP 62.252.190.196:123
<http://62.252.190.196:123> -> 61.192.94.167:123
<http://61.192.94.167:123> 4 304 1
2012-07-24 11:59:59.008 1.000 UDP 61.192.94.167:123
<http://61.192.94.167:123> -> 62.252.190.196:123
<http://62.252.190.196:123> 4 304 1
IP addresses anonymized
Summary: total flows: 11, total bytes: 4737, total
packets: 35, avg bps: 0, avg pps: 0, avg bpp: 135
Time window: 2012-06-04 17:59:57 - 2012-07-24 12:00:00
Total flows processed: 3783450, Blocks skipped: 0, Bytes
read: 196757126
Sys: 1.618s flows/second: 2337262.1 Wall: 1.612s
flows/second: 2345863.0
All flows with duration > 4000000000 started the same
day : 2012-04-06
Am I wrong if I think this should not happen ?
Could a packet loss be the reason of my problem ? What
else if not ?
Is there a way to force nfcapd to expire flows for which
he recieves no
more information ?
The exporter is a Packetfilter firewall running on
OpenBSD with pflow
enabled.
Thanks
------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's
security and
threat landscape has changed and how IT managers can
respond. Discussions
will include endpoint security, mobile security and the
latest in malware
threats.
http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
Nfsen-discuss mailing list
Nfsen-discuss@lists.sourceforge.net
<mailto:Nfsen-discuss@lists.sourceforge.net>
https://lists.sourceforge.net/lists/listinfo/nfsen-discuss
------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats.http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
Nfsen-discuss mailing list
Nfsen-discuss@lists.sourceforge.net
<mailto:Nfsen-discuss@lists.sourceforge.net>
https://lists.sourceforge.net/lists/listinfo/nfsen-discuss
------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats.http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
Nfsen-discuss mailing list
Nfsen-discuss@lists.sourceforge.net
<mailto:Nfsen-discuss@lists.sourceforge.net>
https://lists.sourceforge.net/lists/listinfo/nfsen-discuss
------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond.
Discussions
will include endpoint security, mobile security and the latest in
malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
Nfsen-discuss mailing list
Nfsen-discuss@lists.sourceforge.net
<mailto:Nfsen-discuss@lists.sourceforge.net>
https://lists.sourceforge.net/lists/listinfo/nfsen-discuss
------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
Nfsen-discuss mailing list
Nfsen-discuss@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfsen-discuss
