Hi all,
Same problem here after upgrade yesterday from nfsen 1.3.5/nfdump 1.6.1
to lastest releases :1.3.6p1 and 1.6.6.
Nfcapd capture flows.
Live profile looks good (I guess rrd works) but alerts and other
profiles (continuous or continus/shadow on my box) don't create graphs.
plugins installed : ddd, SSHCURE, SURFmap, Porttracker(doesn't work,
but long), Events and a home made one.
=> Profiles :
- nfcapd files are no longer copied in continuous profiles directory's
but they were before update.
- tried to create a new/fresh one withous success
You have to know that some profile filters are edited nightly by cronjob
and are long. Example :
wc /data/nfsen/profiles-stat/protocoles/dns/sortant-filter.txt
1 1032 6818 /data/nfsen/profiles-stat/protocoles/dns/sortant-filter.txt
here are my logs for profiles :
Jul 31 10:35:15 dionysos user.info nfsen[28507]: Run periodic at Tue Jul
31 10:35:00 2012
Jul 31 10:35:15 dionysos user.info nfsen[28507]: Prepare profiling
'./Traffic_interne'
Jul 31 10:35:15 dionysos user.info nfsen[28507]: Prepare profiling './live'
Jul 31 10:35:15 dionysos user.info nfsen[28507]: Prepare profiling
'./testucop1'
Jul 31 10:35:15 dionysos user.info nfsen[28507]: Prepare profiling
'./traffic_ucop'
Jul 31 10:35:15 dionysos user.info nfsen[28507]: Prepare profiling
'protocoles/dns'
Jul 31 10:35:15 dionysos user.info nfsen[28507]: Prepare profiling
'protocoles/dns_vers_serveurs_inconnu'
Jul 31 10:35:15 dionysos user.info nfsen[28507]: Prepare profiling
'protocoles/ntpAncien'
Jul 31 10:35:15 dionysos user.info nfsen[28507]: Prepare profiling
'protocoles/partage_windows'
Jul 31 10:35:15 dionysos user.info nfsen[28507]: Prepare profiling
'protocoles/smtp'
Jul 31 10:35:15 dionysos user.info nfsen[28507]: Prepare profiling
'protocoles/web'
Jul 31 10:35:15 dionysos user.notice nfsen[28507]: 17 channels/alerts to
profile
Jul 31 10:35:15 dionysos user.info nfsen[28508]: comm child[28739]
terminated with no exit value
Jul 31 10:35:15 dionysos user.info nfsen[28740]: nfsend: exit
child[28741] Exit: 255, Signal: 0, Core: 0
Jul 31 10:35:15 dionysos user.info nfsen[28744]: nfsend: exit
child[28746] Exit: 255, Signal: 0, Core: 0
Jul 31 10:35:15 dionysos user.info nfsen[28745]: nfsend: exit
child[28747] Exit: 255, Signal: 0, Core: 0
Jul 31 10:35:15 dionysos user.info nfsen[28748]: nfsend: exit
child[28749] Exit: 255, Signal: 0, Core: 0
Jul 31 10:35:15 dionysos user.info nfsen[28742]: nfsend: exit
child[28743] Exit: 255, Signal: 0, Core: 0
Jul 31 10:35:15 dionysos user.info nfsen[28742]: profile opts:
.#Traffic_interne#2#cs6513#cs6513 for profiler 1
Jul 31 10:35:15 dionysos user.info nfsen[28742]: profile opts:
protocoles#dns#6#sortant#ucop1-V4|ucop2-V4 for profiler 1
Jul 31 10:35:15 dionysos user.info nfsen[28740]: profile opts:
protocoles#dns#6#entrant#ucop1-V4|ucop2-V4 for profiler 0
Jul 31 10:35:15 dionysos user.info nfsen[28740]: profile opts:
protocoles#smtp#6#cs7204#ucop1-V4|ucop2-V4 for profiler 0
Jul 31 10:35:15 dionysos user.info nfsen[28742]: profile opts:
protocoles#web#6#sortant#ucop1-V4|ucop2-V4 for profiler 1
Jul 31 10:35:15 dionysos user.info nfsen[28745]: profile opts:
.#traffic_ucop#2#ucop1-V4#ucop1-V4 for profiler 3
Jul 31 10:35:15 dionysos user.info nfsen[28745]: profile opts:
protocoles#ntpAncien#6#VersCoeur#cs6513 for profiler 3
Jul 31 10:35:15 dionysos user.info nfsen[28745]: profile opts:
.#~CherchePics#8#CherchePics#ucop1-V4|ucop2-V4 for profiler 3
Jul 31 10:35:15 dionysos user.info nfsen[28740]: profiler 0 started
Jul 31 10:35:15 dionysos user.info nfsen[28742]: profiler 1 started
Jul 31 10:35:15 dionysos user.info nfsen[28507]: nfsend: exit
child[28742] Exit: 0, Signal: 13, Core: 0
Jul 31 10:35:15 dionysos user.info nfsen[28507]: nfsend: exit
child[28740] Exit: 0, Signal: 13, Core: 0
Jul 31 10:35:15 dionysos user.info nfsen[28744]: profile opts:
.#testucop1#6#ucop1-V4#ucop1-V4 for profiler 2
Jul 31 10:35:15 dionysos user.info nfsen[28744]: profile opts:
protocoles#dns_vers_serveurs_inconnu#6#cs6513#cs6513 for profiler 2
Jul 31 10:35:15 dionysos user.info nfsen[28744]: profile opts:
.#~botdetect#8#botdetect#ucop1-V4|ucop2-V4 for profiler 2
Jul 31 10:35:15 dionysos user.info nfsen[28744]: profiler 2 started
Jul 31 10:35:15 dionysos user.info nfsen[28745]: profiler 3 started
Jul 31 10:35:15 dionysos user.info nfsen[28748]: profile opts:
.#traffic_ucop#2#ucop2-V4#ucop2-V4 for profiler 4
Jul 31 10:35:15 dionysos user.info nfsen[28748]: profile opts:
protocoles#ntpAncien#6#versVntp#cs6513 for profiler 4
Jul 31 10:35:15 dionysos user.info nfsen[28748]: profile opts:
.#~DNSCchangerMalwareVictims#8#DNSCchangerMalwareVictims#ucop1-V4|ucop2-V4
for profiler 4
Jul 31 10:35:15 dionysos user.info nfsen[28748]: profiler 4 started
Jul 31 10:35:15 dionysos user.info nfsen[28750]: nfsend: exit
child[28751] Exit: 255, Signal: 0, Core: 0
Jul 31 10:35:15 dionysos user.info nfsen[28750]: profile opts:
protocoles#dns#6#interne#cs6513 for profiler 5
Jul 31 10:35:15 dionysos user.info nfsen[28750]: profile opts:
protocoles#partage_windows#6#cs6513#cs6513 for profiler 5
Jul 31 10:35:15 dionysos user.info nfsen[28750]: profile opts:
.#~test#8#test#cs6513|ucop1-V4|ucop2-V4 for profiler 5
Jul 31 10:35:15 dionysos user.info nfsen[28750]: profiler 5 started
Jul 31 10:35:15 dionysos user.notice nfsen[28507]: Update profile
Traffic_interne in group .
Jul 31 10:35:15 dionysos user.info nfsen[28507]: Add channel size
30285926400
Jul 31 10:35:15 dionysos user.info nfsen[28507]: Set new profile size:
30285926400
Jul 31 10:35:15 dionysos user.info nfsen[28507]: Add
.:Traffic_interne:201207311030 for plugin processing
Jul 31 10:35:16 dionysos user.notice nfsen[28507]: Update profile live
in group .
Jul 31 10:35:16 dionysos user.info nfsen[28507]: Add channel size
544696852480
Jul 31 10:35:16 dionysos user.info nfsen[28507]: Add channel size
232831922176
Jul 31 10:35:16 dionysos user.info nfsen[28507]: Add channel size 1008914432
Jul 31 10:35:16 dionysos user.info nfsen[28507]: Set new profile size:
778537689088
Jul 31 10:35:16 dionysos user.info nfsen[28507]: Add .:live:201207311030
for plugin processing
Jul 31 10:35:17 dionysos user.notice nfsen[28507]: Update profile
testucop1 in group .
Jul 31 10:35:18 dionysos user.notice nfsen[28507]: Update profile
traffic_ucop in group .
Jul 31 10:35:18 dionysos user.info nfsen[28507]: Add channel size
30558482432
Jul 31 10:35:18 dionysos user.info nfsen[28507]: Add channel size 87322624
Jul 31 10:35:18 dionysos user.info nfsen[28507]: Set new profile size:
30645805056
Jul 31 10:35:18 dionysos user.info nfsen[28507]: Add
.:traffic_ucop:201207311030 for plugin processing
Jul 31 10:35:19 dionysos user.notice nfsen[28507]: Update profile dns in
group protocoles
Jul 31 10:35:19 dionysos user.notice nfsen[28507]: Update profile
dns_vers_serveurs_inconnu in group protocoles
Jul 31 10:35:20 dionysos user.notice nfsen[28507]: Update profile
ntpAncien in group protocoles
Jul 31 10:35:21 dionysos user.notice nfsen[28507]: Update profile
partage_windows in group protocoles
Jul 31 10:35:22 dionysos user.notice nfsen[28507]: Update profile smtp
in group protocoles
Jul 31 10:35:23 dionysos user.notice nfsen[28507]: Update profile web in
group protocoles
Jul 31 10:35:24 dionysos user.info nfsen[28507]: Run plugins for
201207311030
=> Alerts :
Trying to understand, I patched file /.../.../nfsen/libexec/NfAlert.pm
like this :
$err = "No flow file for requested time slot : $file";
here are my logs for alerts :
Jul 31 10:35:33 dionysos user.info nfsen[28507]: Check alerts for Tue
Jul 31 10:30:00 2012
Jul 31 10:35:33 dionysos user.notice nfsen[28507]: Process alert 'botdetect'
Jul 31 10:35:33 dionysos user.info nfsen[28507]: alert 'botdetect':
conditions based on plugin
Jul 31 10:35:33 dionysos user.crit nfsen[28507]: Error reading statinfo
of 'botdetect': No flow file for requested time slot :
/data/nfsen/profiles-data/~botdetect/botdetect/nfcapd.201207311030
Jul 31 10:35:33 dionysos user.info nfsen[28507]: Alert 'botdetect' done.
Jul 31 10:35:33 dionysos user.notice nfsen[28507]: Process alert
'CherchePics'
Jul 31 10:35:33 dionysos user.info nfsen[28507]: alert 'CherchePics':
conditions based on total flow summary
Jul 31 10:35:33 dionysos user.crit nfsen[28507]: Error reading statinfo
of 'CherchePics': No flow file for requested time slot :
/data/nfsen/profiles-data/~CherchePics/CherchePics/nfcapd.201207311030
Jul 31 10:35:33 dionysos user.info nfsen[28507]: Alert 'CherchePics' done.
Jul 31 10:35:33 dionysos user.notice nfsen[28507]: Process alert
'DNSCchangerMalwareVictims'
Jul 31 10:35:33 dionysos user.info nfsen[28507]: alert
'DNSCchangerMalwareVictims': conditions based on total flow summary
Jul 31 10:35:33 dionysos user.crit nfsen[28507]: Error reading statinfo
of 'DNSCchangerMalwareVictims': No flow file for requested time slot :
/data/nfsen/profiles-data/~DNSCchangerMalwareVictims/DNSCchangerMalwareVictims/nfcapd.201207311030
Jul 31 10:35:33 dionysos user.info nfsen[28507]: Alert
'DNSCchangerMalwareVictims' done.
Jul 31 10:35:33 dionysos user.notice nfsen[28507]: Process alert 'test'
Jul 31 10:35:33 dionysos user.info nfsen[28507]: alert 'test':
conditions based on total flow summary
Jul 31 10:35:33 dionysos user.crit nfsen[28507]: Error reading statinfo
of 'test': No flow file for requested time slot :
/data/nfsen/profiles-data/~test/test/nfcapd.201207311030
Jul 31 10:35:33 dionysos user.info nfsen[28507]: Alert 'test' done.
Jul 31 10:35:33 dionysos user.info nfsen[28507]: Check alerts done.
Any Idea ?
Thanks a lot
Cédric
Le 30/07/2012 10:39, teslenko.and...@gmail.com a écrit :
> Hello Peter,
>> Make sure your rrd library and Perl module for RRD are linked correctly,
>> as well as the loader will find them.
> The live profile works fine. The graphs looks good.
> I am not use empy filters for subprofile. I use like 'proto udp' or 'as
> ASnum'
> Now i set filter 'any' and type 1:1, but results the same.
>
> I am not use nfcad. I use flow-capture, and
> pile up data in folder for profile 'live' after converting data by the
> utility ft2nfdump .
> The data processing occurs before the nfsen process is run.
> I repeat again -- the profile 'live' works fine.
> Maybe I am not understand correctly how must works a sub-profile.
> I think the nfprofile process apply filter for file which was stored in
> folder for profile 'live'
> and after that it is copying result file in to sub-profile directory.
>
> Is it right?
>
>
>
>
> 29.07.2012 13:27, Peter Haag пишет:
>> Make sure your rrd library and Perl module for RRD are linked correctly,
>> as well as the loader will find them.
>> for a 1:1 profile use filer 'any' an empty filter means 'not any'
>>
>> - Peter
>>
>>
>> On 23/7/12 1:19 PM, Anrey Teslenko wrote:
>>> Hello all,
>>>
>>> Let me to raise this issue once again. I've seen a lot of
>>> correspondences on this topic but has not found the answer to resolve
>>> it
>>>
>>> My nfdump: Version: 1.6.6 $Date: 2012-03-11 11:57:45 +0100 (Sun, 11 Mar
>>> 2012)
>>> My nfsen: Version: 1.3.6p1 $Id: nfsen 53 2012-01-23 16:36:02Z peter
>>>
>>> I have a problem when nftsen trying to collect the data from "live" profile
>>> into
>>> other new profile. All nfcapd files for all profiles are empty with a
>>> size of 312 bytes.
>>>
>>>
>>> This is a output of nfdump for this files.
>>>
>>> Date flow start Duration Proto Src IP Addr:Port
>>> Dst IP Addr:Port Packets Bytes Flows
>>> Summary: total flows: 0, total bytes: 0, total packets: 0, avg bps: 0,
>>> avg pps: 0, avg bpp: 0
>>> Time window: <unknown>
>>> Total flows processed: 0, Blocks skipped: 0, Bytes read: 36
>>> Sys: 0.000s flows/second: 0.0 Wall: 0.000s flows/second: 0.0
>>>
>>> I see zeros in rrd file.
>>> For appropriate file in profile live all looks good .
>>> I see next in log file
>>>
>>> Run periodic at Mon Jul 23 13:40:00 2012
>>> Prepare profiling './FlowStatistics'
>>> Prepare profiling './live'
>>> 1 channels/alerts to profile
>>> nfprofile run: /usr/local/bin/nfprofile -I -t 1343039700 -p
>>> /home/netflow/flows -P /var/www/nfsen/profiles-stat -L local3 -M
>>> /home/netflow/flows/live/upstreams -r nfcapd.201207231335
>>> Limit profilers: 1
>>> profile opts: .#FlowStatistics#2#AS6703#upstreams for profiler 0
>>> profiler 0 started
>>> comm child[16690] terminated with no exit value
>>> Process line '.#FlowStatistics#2#AS6703#upstreams#012'
>>> Setup channel 'AS6703' in profile 'FlowStatistics' group '.',
>>> channellist 'upstreams'
>>> profiler 0 finished
>>> Update profile FlowStatistics in group .
>>> Add channel size 20480
>>> Set new profile size: 20480
>>> Add .:FlowStatistics:201207231335 for plugin processing
>>>
>>>
>>> When i trying to start from command line
>>> /usr/local/bin/nfprofile -I -t 1343041440 -p /home/netflow/flows -P
>>> /var/www/php-cisco/profiles-stat -L local3 -M
>>> /home/netflow/flows/live/upstreams -r nfcapd.201207231402
>>>
>>> Process just sleep and do nothing (may be hung)
>>>
>>> www-data 22884 0.0 0.1 9312 1648 pts/1 S+ 14:06 0:00
>>> /usr/local/bin/nfprofile -I -t 1343039700 -p /home/netflow/flows -P
>>> /var/www/php-cisco/profiles-stat -L local3 -M
>>> /home/netflow/flows/live/upstreams -r nfcapd.201207231335
>>>
>>> I see next in syslog when i try push ENTER
>>>
>>> Jul 23 14:10:16 nettools nfprofile[22884]: Process line '#012'
>>> Jul 23 14:10:16 nettools nfprofile[22884]: Incomplete line - channel
>>> skipped.
>>> Jul 23 14:10:16 nettools nfprofile[22884]: Process line '#012'
>>> Jul 23 14:10:16 nettools nfprofile[22884]: Incomplete line - channel
>>> skipped.
>>> Jul 23 14:10:16 nettools nfprofile[22884]: Process line '#012'
>>> Jul 23 14:10:16 nettools nfprofile[22884]: Incomplete line - channel
>>> skipped.
>>>
>>> ------------------------------------------------------------------------------
>>> Live Security Virtual Conference
>>> Exclusive live event will cover all the ways today's security and
>>> threat landscape has changed and how IT managers can respond. Discussions
>>> will include endpoint security, mobile security and the latest in malware
>>> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
>>> _______________________________________________
>>> Nfsen-discuss mailing list
>>> Nfsen-discuss@lists.sourceforge.net
>>> https://lists.sourceforge.net/lists/listinfo/nfsen-discuss
>>>
>
>
> ------------------------------------------------------------------------------
> Live Security Virtual Conference
> Exclusive live event will cover all the ways today's security and
> threat landscape has changed and how IT managers can respond. Discussions
> will include endpoint security, mobile security and the latest in malware
> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
> _______________________________________________
> Nfsen-discuss mailing list
> Nfsen-discuss@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/nfsen-discuss
------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
Nfsen-discuss mailing list
Nfsen-discuss@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfsen-discuss
