Re: [pmacct-discussion] Fwd: minb - version 1.5.3

2016-08-02 Thread Stephen Clark

Hi Paolo,

I had to look at the code but I figured it out.

needed minb= not minb>=

On 08/02/2016 09:36 AM, Steve Clark wrote:


Hi Paolo,

I am trying to limit netflow aggregates to greater than 100 bytes before 
insertion into my PG database, but

I can't seem to get it to work. All aggregates are being inserted.

my config:
daemonize: true
debug: false
pidfile: /var/run/nfacctd.pid
syslog: daemon
pre_tag_map: ./my.pretag.map
nfacctd_disable_checks: true
nfacctd_time_new: false
aggregate: tag, src_host, dst_host, src_port, dst_port, proto, tos
plugin_pipe_size: 4096000
plugin_buffer_size: 4096
plugins: pgsql
sql_table: netflow
sql_data: typed
sql_dont_try_update: true
sql_use_copy: true
sql_db: pmacct
sql_host: 127.0.0.1
sql_passwd: arealsmartpwd
sql_user: pmacct
sql_refresh_time: 60
sql_optimize_clauses: true
sql_history: 1m
sql_history_roundoff: m
sql_recovery_logfile: /var/lib/pmacct/recovery_log
sql_preprocess: minb>=100
sql_locking_style: row
sql_cache_entries: 19
imt_buckets: 65537
imt_mem_pools_size: 1024000
nfacctd_port: 2055

Here is what I get in my table - notice 1173 under 100 bytes.
pmacct=# truncate netflow ;
TRUNCATE TABLE
pmacct=# select count(*),sum(bytes)as bytes,sum(packets)as packets from 
netflow where agent_id = '246' and bytes <100;

 count | bytes | packets
---+---+-
  1173 | 89321 |1205
(1 row)

pmacct=# select count(*),sum(bytes)as bytes,sum(packets)as packets from 
netflow where agent_id = '246';

count |  bytes   | packets
---+--+-
3690 | 63424928 |  105921
(1 row)

Also this is what shows from /var/log/messages
Aug  2 08:06:01 netflow nfacctd[4073]: INFO ( default/pgsql ): *** Purging 
cache - START (PID: 4073) ***
Aug  2 08:06:01 netflow nfacctd[4073]: INFO ( default/pgsql ): *** Purging 
cache - END (PID: 4073, QN: 3690/3690, ET: 0) ***


   KEY: minb
DESC: check. Aggregates on the queue are evaluated one-by-one; 
each object is marked valid
only if the bytes counter is '>=' minb value. An interesting idea is to set 
its value
  to a fraction of the link capacity. Remember that you 
have also a timeframe reference:

  the 'sql_refresh_time' seconds. All plugins.

  For example, given the following parameters:
  Link Capacity = 8Mbit/s, THreshold = 0.1%, TImeframe = 60s
  minb = ((LC / 8) * TI) * TH -> ((8Mbit/s / 8) * 60s) * 
0.1% = 6 bytes.


  Given a 8Mbit link, all aggregates which have accounted 
for at least 60Kb of traffic

  in the last 60 seconds, will be written to the DB.



Any suggestions?

Thanks,
Steve



___
pmacct-discussion mailing list
http://www.pmacct.net/#mailinglists



--

"They that give up essential liberty to obtain temporary safety,
deserve neither liberty nor safety."  (Ben Franklin)

"The course of history shows that as a government grows, liberty
decreases."  (Thomas Jefferson)


___
pmacct-discussion mailing list
http://www.pmacct.net/#mailinglists

[pmacct-discussion] Fwd: minb - version 1.5.3

2016-08-02 Thread Steve Clark


Hi Paolo,

I am trying to limit netflow aggregates to greater than 100 bytes before 
insertion into my PG database, but

I can't seem to get it to work. All aggregates are being inserted.

my config:
daemonize: true
debug: false
pidfile: /var/run/nfacctd.pid
syslog: daemon
pre_tag_map: ./my.pretag.map
nfacctd_disable_checks: true
nfacctd_time_new: false
aggregate: tag, src_host, dst_host, src_port, dst_port, proto, tos
plugin_pipe_size: 4096000
plugin_buffer_size: 4096
plugins: pgsql
sql_table: netflow
sql_data: typed
sql_dont_try_update: true
sql_use_copy: true
sql_db: pmacct
sql_host: 127.0.0.1
sql_passwd: arealsmartpwd
sql_user: pmacct
sql_refresh_time: 60
sql_optimize_clauses: true
sql_history: 1m
sql_history_roundoff: m
sql_recovery_logfile: /var/lib/pmacct/recovery_log
sql_preprocess: minb>=100
sql_locking_style: row
sql_cache_entries: 19
imt_buckets: 65537
imt_mem_pools_size: 1024000
nfacctd_port: 2055

Here is what I get in my table - notice 1173 under 100 bytes.
pmacct=# truncate netflow ;
TRUNCATE TABLE
pmacct=# select count(*),sum(bytes)as bytes,sum(packets)as packets from netflow 
where agent_id = '246' and bytes <100;

 count | bytes | packets
---+---+-
  1173 | 89321 |1205
(1 row)

pmacct=# select count(*),sum(bytes)as bytes,sum(packets)as packets from netflow 
where agent_id = '246';

count |  bytes   | packets
---+--+-
3690 | 63424928 | 105921
(1 row)

Also this is what shows from /var/log/messages
Aug  2 08:06:01 netflow nfacctd[4073]: INFO ( default/pgsql ): *** Purging cache 
- START (PID: 4073) ***
Aug  2 08:06:01 netflow nfacctd[4073]: INFO ( default/pgsql ): *** Purging cache 
- END (PID: 4073, QN: 3690/3690, ET: 0) ***


   KEY: minb
DESC: check. Aggregates on the queue are evaluated one-by-one; 
each object is marked valid
only if the bytes counter is '>=' minb value. An interesting idea is to set its 
value
  to a fraction of the link capacity. Remember that you 
have also a timeframe reference:

  the 'sql_refresh_time' seconds. All plugins.

  For example, given the following parameters:
  Link Capacity = 8Mbit/s, THreshold = 0.1%, TImeframe = 60s
  minb = ((LC / 8) * TI) * TH -> ((8Mbit/s / 8) * 60s) * 
0.1% = 6 bytes.


  Given a 8Mbit link, all aggregates which have accounted 
for at least 60Kb of traffic

  in the last 60 seconds, will be written to the DB.



Any suggestions?

Thanks,
Steve

___
pmacct-discussion mailing list
http://www.pmacct.net/#mailinglists