Hi Adrian - et al, All those of you using snapshot-20070110 and plugins, should consider to update to the snapshot-20070126 appended. The issues described be Adrian, concering the multiple END block calls are fixed. It turned out, that the current design using the standard Perl END blocks for terminating the plugins do no longer work. Therefore I have changed that part into a Cleanup function which actually does the same from the plugin perspective, but is ways more flexible also for upcoming releases.
Things to note: For the plugins to work you must replace the sub END by
sub Cleanup. Simply replace the two words END -> Cleanup.
Further more this snapshot contains bug fixes for bugs found in
snapshot-20070110.
Sorry for the trouble, but that's the life of snapshots. But thanks to all
for testing. I'll upload the snapshot to sourceforge, if feedback is
positive.
- Peter
> Peter, I have good news.
>
> First of all, you were right... I had to wait about 15-20 minutes before
> I got the second 'Process_v9: [0] Add template 256' message; anyway,
> both sources are collected now.
>
> Next, I installed again the newest snapshot and applied the patch, as
> you instructed. Unfortunatelly, the patch couldn't be applied:
> [EMAIL PROTECTED] libexec]# patch Nfcomm.pm nfcomm.patch
> patching file Nfcomm.pm
> patch: **** malformed patch at line 4: syslog('err', "Register plugin
> '$module' for profile '$profile' in profile group '$profilegroup' does
> not exists!");
>
> ...but I opened Nfcomm.pm and added the changes by hand (changed
> '$profile' to '$profilegroup/$profile') and now, it loads both plugins
> without problems. (Thanks!!!)
>
> As far as calling the END subroutine, this is what I found: Each time
> the comm server restarts the END subroutines from all the plugins are
> called... The good thing is that the plugins are not interrupted, just
> the subroutine is called (and I get logging messages). A quick
> workaround would be to disable the syslog messages from the END
> subroutine... :)
>
> Jan 25 14:36:36 hail prefixStats: comm server started: 7953
> Jan 25 14:36:36 hail prefixStats: prefixStats END
> Jan 25 14:36:36 hail prefixStats: floodsearch END
> Jan 25 14:36:37 hail prefixStats: comm server started: 7954
> Jan 25 14:36:37 hail prefixStats: prefixStats END
> Jan 25 14:36:37 hail prefixStats: floodsearch END
>
> Anyway, everything seems to be working now - thank you very much for
> your help. I'll put the box into production the following days and see
> if I get any more problems.
>
> Thank you for your support,
> Adrian Popa
>
> Peter Haag wrote:
>> Hi Adrian ,
>>
>>
>>> Thank you for your reply, Peter.
>>>
>>> It seems that the plugins were the least of my problem...
>>>
>>> Today I inserted the new collector in the place of the old one, for
>>> about an hour, to see how it responds. Unfortunatelly, it seemed to be
>>> collecting data only from one source (out of two who are exporting).
>>> Left without ideas, I uninstalled the snapshot versions of nfsen and
>>> nfdump and proceeded with installing the older versions which are
>>> running beautifully on my old collector. These versions are:
>>> nfdump-1.5.2
>>> nfsen-snapshot-20060810
>>>
>>> Unfortunatelly, the same problem appeared with the older versions
>>> too...
>>> Here is how some debugging information look like:
>>>
>>> * I have packets coming in from source1 (7304bb2) on udp port 9900:
>>>
>>> 11:52:20.785322 IP 193.231.103.96.54515 > 80.97.223.7.9900: UDP, length
>>> 1436
>>> 11:52:20.789370 IP 193.231.103.96.54515 > 80.97.223.7.9900: UDP, length
>>> 1436
>>> 11:52:20.790053 IP 193.231.103.96.54515 > 80.97.223.7.9900: UDP, length
>>> 1436
>>> 11:52:20.794553 IP 193.231.103.96.54515 > 80.97.223.7.9900: UDP, length
>>> 1436
>>> 11:52:20.799603 IP 193.231.103.96.54515 > 80.97.223.7.9900: UDP, length
>>> 1436
>>>
>>> * I have packets coming in from source2 (7304bcnt2) on udp port
>>> 9901:
>>>
>>> 11:52:30.949127 IP 193.231.103.95.52750 > 80.97.223.7.9901: UDP, length
>>> 1436
>>> 11:52:30.952642 IP 193.231.103.95.52750 > 80.97.223.7.9901: UDP, length
>>> 1436
>>> 11:52:30.955807 IP 193.231.103.95.52750 > 80.97.223.7.9901: UDP, length
>>> 1436
>>> 11:52:30.960206 IP 193.231.103.95.52750 > 80.97.223.7.9901: UDP, length
>>> 1436
>>> 11:52:30.963720 IP 193.231.103.95.52750 > 80.97.223.7.9901: UDP, length
>>> 1436
>>>
>>> * My firewall is disabled (allows all packets to enter):
>>>
>>> [EMAIL PROTECTED] bin]# iptables -L
>>> Chain INPUT (policy ACCEPT)
>>> target prot opt source destination
>>> Chain FORWARD (policy ACCEPT)
>>> target prot opt source destination
>>> Chain OUTPUT (policy ACCEPT)
>>> target prot opt source destination
>>> Chain RH-Firewall-1-INPUT (0 references)
>>> target prot opt source destination
>>>
>>> * I start nfcapd -p 9900 -E
>>> Flow Record:
>>> Flags = 0x00000000
>>> size = 52
>>> mark = 0
>>> srcaddr = 86.20.248.67
>>> dstaddr = 86.35.20.209
>>> first = 1169718601 [2007-01-25 11:50:01]
>>> last = 1169718605 [2007-01-25 11:50:05]
>>> msec_first = 888
>>> msec_last = 372
>>> dir = 0
>>> tcp_flags = 0x18 .AP...
>>> prot = 6
>>> tos = 0
>>> input = 7
>>> output = 0
>>> srcas = 6453
>>> dstas = 0
>>> srcport = 17827
>>> dstport = 3991
>>> dPkts = 2
>>> dOctets = 84
>>> * I start nfcapd -p 9901 -E
>>> o (I get no output, no matter how long I wait)
>>> * Logging looks like this:
>>>
>>> Jan 25 11:50:41 hail nfcapd[591]: Startup.
>>> Jan 25 11:50:41 hail nfcapd[591]: Process_v9: New exporter domain 0
>>> Jan 25 11:50:42 hail nfcapd[591]: Process_v9: [0] Add template 256
>>> Jan 25 11:50:43 hail nfcapd[591]: Ident: 'none' Flows: 7792, Packets:
>>> 163878, Bytes: 120587576, Sequence Errors: 286, Bad Packets: 0
>>> Jan 25 11:50:43 hail nfcapd[591]: Terminating nfcapd.
>>> Jan 25 11:51:03 hail nfcapd[602]: Startup.
>>> Jan 25 11:51:03 hail nfcapd[602]: Process_v9: New exporter domain 0
>>> Jan 25 11:51:09 hail nfcapd[602]: Ident: 'none' Flows: 0, Packets: 0,
>>> Bytes: 0, Sequence Errors: 1603, Bad Packets: 0
>>> Jan 25 11:51:09 hail nfcapd[602]: Terminating nfcapd.
>>>
>>> The difference I see is that the collector that works displays this
>>> line: Process_v9: [0] Add template 256.
>>>
>>> I don't know what that means, but I would surely like to make it
>>> work...
>>> I don't know if it's a linux problem (aparently the packets reach the
>>> application) or if it's a nfcapd problem (worked like a charm on the
>>> older collector box).
>>>
>>> Any ideas for testing are *highly* appreciated!
>>>
>>
>> I assume, that the second exporter feeding at 9901 is a netflow v9
>> exporter as well. This is an issue of v9:
>> Each netflow v9 exporter sends different type of records: template
>> records
>> as well as data records. To correctly decode the netflow v9 data, a
>> collector needs to receive first the required template records, which
>> describe, how to interpret the data records. Without the template
>> records
>> the collector has no clue, how to compile netflow data from the data
>> record. When you restart the collector at any given time, it has to wait
>> for the template records to arrive, in order to successfully decode the
>> data records. According the RFC each exporter has to resend the template
>> records at regular intervalls ( interval not specified ). Up to that
>> point
>> the collector discards all data records, and therefore you don not get
>> any
>> output in nfcpad -E . This is so for all nfcapd version and has nothing
>> to
>> do with the snapshot.
>>
>> So the solution: Just wait until the template arrives .. the exporter
>> must
>> resend them. You'll see the 'add template' message therafter.
>>
>> - Peter
>>
>>
>>
>>
>>> If I can collect both sources, I will upgrade to the new version and
>>> try
>>> the plugin patch.
>>>
>>> I greatly appreciate your help,
>>> Thanks,
>>> Adrian Popa
>>>
>>>
>>>
>>
>>
>>
>
> -------------------------------------------------------------------------
> Take Surveys. Earn Cash. Influence the Future of IT
> Join SourceForge.net's Techsay panel and you'll get the chance to share
> your
> opinions on IT & business topics through brief surveys - and earn cash
> http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV_______________________________________________
> Nfsen-discuss mailing list
> [email protected]
> https://lists.sourceforge.net/lists/listinfo/nfsen-discuss
>
--
_______ SWITCH - The Swiss Education and Research Network ______
Peter Haag, Security Engineer, Member of SWITCH CERT
PGP fingerprint: D9 31 D5 83 03 95 68 BA FB 84 CA 94 AB FC 5D D7
SWITCH, Limmatquai 138, CH-8001 Zurich, Switzerland
E-mail: [EMAIL PROTECTED] Web: http://www.switch.ch/
nfsen-snapshot-20070126.tar.gz
Description: Binary data
------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys - and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
_______________________________________________ Nfsen-discuss mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/nfsen-discuss
