-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
- --On March 11, 2008 3:38:25 -0700 Chris Waters <[EMAIL PROTECTED]> wrote:
| Thanks Peter. I guess I did not read through the nfsen-dist.conf file
| as I used my config from a much older version durring my upgrade. The
No problem - upgrade do keep all old setting, but do not add by default new
ones.
| only thing it complianed about during the install (after the "-" issue
| from the other thread) was the high water mark format. Anyway moving
| along.
|
| Another question along these lines. I have about 1.2 million files on
| average, so if my compression tests are satisfactory I will want to
| convert them. By running the manual compression on them, do you know
| what happens if I run a big recursive script to compress them and it
| encounters a file that is already compressed or do I need to validate
| each file first by parsing the output of nfdump -v? I belive the
| documentation suggests that the "-j" option is bi-directional, meaning
| it will compress uncompressed files and decompress compressed ones.
| Just looking for clarification.
Yes - that's correct.
|
| Also, if I change (or in my case add) the compression parameters do I
| need to do a reconfig?
No - just add it in nfsen.conf and stop/start NfSen, so that the collector gets
restarted and everything is done.
- Peter
|
| Thanks!
|
| Chris Waters
| Technology Services - Network Group
| JELD-WEN, Inc.
| Information Systems
| [EMAIL PROTECTED]
|
| -----Original Message-----
| From: Peter Haag [mailto:[EMAIL PROTECTED]
| Sent: Tuesday, March 11, 2008 12:53 AM
| To: Chris Waters; [email protected]
| Subject: Re: [Nfsen-discuss] Compression in 1.5.7
|
| -----BEGIN PGP SIGNED MESSAGE-----
| Hash: SHA1
|
|
| Hi Chris,
| Compression is configured in nfsen.conf:
|
| # Compress flows while collecting 0 or 1
| $ZIPcollected = 1;
|
| # Compress flows in profiles 0 or 1
| $ZIPprofiles = 1;
|
| Which means the collected flows and/or the created profiles are
| compressed.
|
| These tags should be in nfsen-dist.conf with comes with NfSen.
|
| As of nfcapd itself - it compresses the flows be adding -z as described
| in nfcapd(1). Whether you want to compress your flows or not, depends on
| your system I/O speed , CPU or disk capacity. You can do a rather small
| test by running.
|
| ./nftest <path/to/a/nfcapd.file>
|
| ( nftest is only in the src directory and built, when doing a 'make
| test' )
|
| It does not alter the file but takes your flow data to compare speed and
| compression.
| The compression algorithm selected is a bit less effective than zip, but
| *very* fast, what matters most, when processing flows. As the size of
| the files shrinks down to 50% in average, I think this is a good
| trade-off.
|
| Already existing nfcapd files can be compressed/uncompressed by using
| ./nfdump -j nfcapd.file
|
| Hope this helps.
|
| - Peter
|
|
| - --On March 10, 2008 14:27:35 -0700 Chris Waters <[EMAIL PROTECTED]>
| wrote:
|
| | I would like to know how to use the compression feature of nfdump
| 1.5.7.
| | I'm not sure where it's documented but I don't seem to be able to find
|
| | in the mail archives or in the man pages for it.
| |
| | 1. How do you enable it? It seems that it should be an config option
|
| | in nfsen.conf but I don't see it. I am using nfdump 1.5.7 with a
| | current svn snapshot of nfsen. It doesn't look like compression is on
|
| | by default.
| |
| | Here is a current file:
| | [root]# /usr/local/svn/nfdump/bin/nfdump -v nfcapd.200803101335
| | File : nfcapd.200803101335
| | Version : 1 - not compressed
| | Blocks : 1
| | Records : 15900
| |
| | Here is a current nfcapd process:
| | netflow 22959 1 0 Mar09 ? 00:00:00
| | /usr/local/svn/nfdump/bin/nfcapd -w -D -I rca-cal-md-1 -p 10060 -u
| | netflow -g apache -B 200000 -l /data/nfsen/profiles/live/rca-cal-md-1
| | -P /data/nfsen/var/run/rca-cal-md-1.pid
| |
| | 2. Once enabled, what performance impact will it have when running
| | nfsen? Longer query times etc?
| |
| | 3. Anything else to watch out for?
| |
| | If I have missed the documentation, please just point me to it.
| |
| | Thanks.
| |
| |
| | Chris Waters
| |
| | Technology Services - Networks Group
| |
| |
| |
| | JELD-WEN, inc.
| |
| | Information Systems
| |
| | [EMAIL PROTECTED]
| |
| | RELIABILITY for real life(r)
| |
| | This correspondence is for the named person's use only. It may contain
|
| | confidential or legally privileged information and is intended solely
| | for the named addressee. If you receive this correspondence in error,
| | please notify the sender and delete it from your system. You must not
| | disclose, copy or rely on any part of this correspondence if you are
| | not the intended recipient.
| |
| |
|
|
|
| - --
| _______ SWITCH - The Swiss Education and Research Network ______ Peter
| Haag, Security Engineer, Member of SWITCH CERT PGP fingerprint: D9 31
| D5 83 03 95 68 BA FB 84 CA 94 AB FC 5D D7
| SWITCH, Werdstrasse 2, P.O. Box, CH-8021 Zurich, Switzerland
| E-mail: [EMAIL PROTECTED] Web: http://www.switch.ch/ -----BEGIN PGP
| SIGNATURE-----
| Version: GnuPG v1.4.3 (Darwin)
|
| iQCVAwUBR9Y6ef5AbZRALNr/AQKgywQAipn/TD5YAhVPfrDa4yQXBcvpE8H3qKxq
| Q+cfdQB4FxCQ5EgZBZcmaIgw3FQDxQX2ElZpxImIbnStwPtzCCrxJk72uh4b1vng
| H3wBTaytJOhNcwocrE3CJ1olc2rsff7WYBQggPQR5O/NAcjC8Y0N/oYMN88UBk8r
| Lfjq1pwU85I=
| =8TT8
| -----END PGP SIGNATURE-----
|
|
| -------------------------------------------------------------------------
| This SF.net email is sponsored by: Microsoft
| Defy all challenges. Microsoft(R) Visual Studio 2008.
| http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
| _______________________________________________
| Nfsen-discuss mailing list
| [email protected]
| https://lists.sourceforge.net/lists/listinfo/nfsen-discuss
- --
_______ SWITCH - The Swiss Education and Research Network ______
Peter Haag, Security Engineer, Member of SWITCH CERT
PGP fingerprint: D9 31 D5 83 03 95 68 BA FB 84 CA 94 AB FC 5D D7
SWITCH, Werdstrasse 2, P.O. Box, CH-8021 Zurich, Switzerland
E-mail: [EMAIL PROTECTED] Web: http://www.switch.ch/
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.3 (Darwin)
iQCVAwUBR9aEEP5AbZRALNr/AQI0RgP/ViUo44oKSstYnE1h+5YvkT6ZZzk1MMjW
BWNed7vYoI3xCFMBOAEGsIphgVPFdOKPfIG03/4fYv0miXwq4eBJvbGuD5Cd8JrO
UGQ+VWQ52//AW/uuAMvATjU2iK+gPT1s1e+Q5DhNk98zvKpvwdmSILeXJ7F2+NSk
skgdCroIwo8=
=c17t
-----END PGP SIGNATURE-----
-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
_______________________________________________
Nfsen-discuss mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/nfsen-discuss
