The new nftrack.c does fix the issue. I have statistics that make sense, and rrdtool graphs to go with it.
I did have to update the do_compile script to add $NFDUMP/minilzo.o to the end of the NFDUMP_OBJ line. # nfdump objects NFDUMP_OBJ="$NFDUMP/util.o $NFDUMP/nftree.o $NFDUMP/grammar.o $NFDUMP/scanner.o $NFDUMP/nffile.o $NFDUMP/flist.o $NFDUMP/nf_common.o $NFDUMP/panonymizer.o $NFDUMP/rijndael.o $NFDUMP/ipconv.o $NFDUMP/minilzo.o" Thanks for the help and quick fix. Danny On Mon, 2008-05-26 at 10:57 +0200, Peter Haag wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > Hi Tor, > The additional notice of the failure of PortTracker with compressed files > made me suspicious. So I checked the files, > which are shipped in the tar ball of 1.3, and there I found the reason: Due > to a mistake, nftrack.c got not replaced > correctly when compression was introduced. So find appended the correct > nftrack.c for NfSen 1.3. Please replace it and > recompile nftrack. > This fixes the error with compressed flow files. > > Sorry for that! > > - Peter > > Tor Inge Skaar wrote: > | There seems to be some kind of correlation here anyways. First I ran > | PortTracker on compressed flows, but the graphs and table data were > | completely off scale (count and bytes in the 10^15 scale!) and the port > | distribution is off as well (top tcp; 63142, 3719, ..) > | > | Then I simply turn off the compression flag for the nfcapd processes, > | disables PortTracker, reload nfsen, delete all portracker data, > | initialize the db by running nftrack -I -d /path/to/porttracker/data, > | enables PortTracker plugin in nfsen.conf again and finally reloads nfsen > | again. > | > | The graph and table data now are exactly as expected, with port 80/tcp > | and 53/udp on top. > | > | I'm not sure why this is, but apparently compression makes a difference. > | > | I compiled nftrack (through do_compile) as instructed, linking to the > | nfdump 1.5.7 source, and rrdtool 1.2.27. > | > | Tor I. Skaar > | > | Peter Haag wrote: > |> Porttracker reads transparently nfcapd files, whether they are compressed > or not. All files linked to nfdump code, have > |> the same reader routines. > | > |> - Peter > | > | > |> Danny Rappleyea wrote: > |> | Can anyone confirm whether the PortTracker plugin is able to read > |> | compressed files? I have a new NfSen installation that I'm testing. I > |> | had the PortTracker plugin half working, where the summary at the bottom > |> | had correct-looking data but the graphs were blank. Now the summary is > |> | garbage, with 66635 as the only port with more bytes than is possible. > |> | Between then and now, I did enable compression in the nfsen.conf file. > |> | > |> | I did a couple of tests using nftrack on one of the original > |> | uncompressed files and a newer compressed file. It looks like from the > |> | results that nftrack can't deal with a compressed file. > |> | > |> | --- > |> | [EMAIL PROTECTED] 19]# /usr/local/bin/nftrack -r nfcapd.200805191720 > |> | -d /local/nfsen/plugins-data/PortTracker -s -t 200805191720 -p > |> | 1211232000 > |> | 10 0 0 > |> | 515 80 25 443 524 135 21 3396 113 139 > |> | 4667 2893 926 389 203 187 183 160 136 105 > |> | 10 1 0 > |> | 80 524 1976 3389 49409 35182 443 1979 25 515 > |> | 47574 31628 23494 16627 16343 10244 7422 7276 6281 5823 > |> | 10 2 0 > |> | 1976 35182 9100 80 2495 2510 2522 2811 2540 524 > |> | 18952960 15213336 8561650 6610134 6503022 5609872 5444351 4953478 > |> | 3593795 2817093 > |> | 10 0 1 > |> | 7000 161 7001 0 53 1347 1346 2967 137 123 > |> | 12545 7260 6403 4036 3714 2587 1818 1188 1173 661 > |> | 10 1 1 > |> | 7000 7001 161 0 2967 53 1347 1346 137 1851 > |> | 25411 12216 12152 8239 6791 5656 4757 3336 1561 1489 > |> | 10 2 1 > |> | 7001 0 2967 7000 1346 1347 1851 161 53 694 > |> | 15540246 11160809 3034373 2385728 1546072 1378910 1226395 985230 414838 > |> | 383264 > |> | [EMAIL PROTECTED] 21]# /usr/local/bin/nftrack -r nfcapd.200805211550 > |> | -d /local/nfsen/plugins-data/PortTracker -s -t 200805211550 -p > |> | 1211399400 > |> | 10 0 0 > |> | 0 65535 65534 65533 65532 65531 65530 65529 65528 65527 > |> | 0 0 0 0 0 0 0 0 0 0 > |> | 10 1 0 > |> | 0 65535 65534 65533 65532 65531 65530 65529 65528 65527 > |> | 0 0 0 0 0 0 0 0 0 0 > |> | 10 2 0 > |> | 0 65535 65534 65533 65532 65531 65530 65529 65528 65527 > |> | 0 0 0 0 0 0 0 0 0 0 > |> | 10 0 1 > |> | 0 65535 65534 65533 65532 65531 65530 65529 65528 65527 > |> | 0 0 0 0 0 0 0 0 0 0 > |> | 10 1 1 > |> | 0 65535 65534 65533 65532 65531 65530 65529 65528 65527 > |> | 0 0 0 0 0 0 0 0 0 0 > |> | 10 2 1 > |> | 0 65535 65534 65533 65532 65531 65530 65529 65528 65527 > |> | 0 0 0 0 0 0 0 0 0 0 > |> | --- > |> | > |> | Are there any tricks with nfdump or other tools that could uncompress > |> | the file and feed it to stdin on nftrack? Any other workarounds? > |> | > |> | Best regards, > |> | > |> | Danny > |> | > |> | > |> | > |> | ------------------------------------------------------------------------- > |> | This SF.net email is sponsored by: Microsoft > |> | Defy all challenges. Microsoft(R) Visual Studio 2008. > |> | http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ > |> | _______________________________________________ > |> | Nfsen-discuss mailing list > |> | [email protected] > |> | https://lists.sourceforge.net/lists/listinfo/nfsen-discuss > | > | > | ------------------------------------------------------------------------- > | This SF.net email is sponsored by: Microsoft > | Defy all challenges. Microsoft(R) Visual Studio 2008. > | http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ > | _______________________________________________ > | Nfsen-discuss mailing list > | [email protected] > | https://lists.sourceforge.net/lists/listinfo/nfsen-discuss > | > | > > - -- > _______ SWITCH - The Swiss Education and Research Network ______ > Peter Haag, Security Engineer, Member of SWITCH CERT > PGP fingerprint: D9 31 D5 83 03 95 68 BA FB 84 CA 94 AB FC 5D D7 > SWITCH, Werdstrasse 2, P.O. Box, CH-8021 Zurich, Switzerland > E-mail: [EMAIL PROTECTED] Web: http://www.switch.ch/ > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1.4.5 (Darwin) > > iQCVAwUBSDp7Wv5AbZRALNr/AQJfPwP/ZbDQi3G/4CkFCh16ykAERYl2ykYdSupB > urjgHwPy4SUFA+QWRXd2avzDFoq7OekIYUUiwyWMbotEpfqEx1qZAaY59mLrD+CK > j9CVnV4QHEu/wQCfTCAGVImbt2ya/HKLsBnKbiNUi5mt50CNzWgTdvJeRryKJ8RQ > RpLxcdiGhic= > =pyu6 > -----END PGP SIGNATURE----- ------------------------------------------------------------------------- This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ _______________________________________________ Nfsen-discuss mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/nfsen-discuss
