Re: [pmacct-discussion] Capturing NetFlow in PostgresSQL Database

[Paolo Lucente]

Hi Anthony,

ET is the Estimated Time it took to complete the purging event. So, in
your case, 8978 seconds. QN is the Query Number: how many were sent to
the database / how many entries are in the cache. So, in your case,
12482 are sent out of 32819 that are cached. This for sure means that
nfacctd_time_new is set to false (default is false; read more in [*])
and you may want to revisit such choice (at least for a test). 

Cheers,
Paolo

[*] https://github.com/pmacct/pmacct/blob/master/CONFIG-KEYS

On Thu, Apr 28, 2016 at 09:04:33AM -0400, Anthony Rodriguez wrote:
> Yeah we have considered this, and are attempting it. What does the
> QN and ET parts of the statement mean when purging?
> Ex: QN: 12482/32819, ET: 8978
> 
> On 04/28/2016 08:01 AM, Matthew Walster wrote:
> >On 27 April 2016 at 20:42, Anthony Rodriguez
> > >>wrote:
> >
> >Hi,
> >
> >I have set up a test capturing NF into a postgresql 8.4 DB. It
> >seems to work OK but the writing to the DB seems
> >to get far behind. I had one writer take 1.5 hours to sync. Any
> >hints would be appreciated.
> >
> >Apr 27 10:30:01  nfacctd[31907]: INFO ( default/pgsql ): ***
> >Purging cache - START (PID: 31907) ***
> >Apr 27 12:59:39  nfacctd[31907]: INFO ( default/pgsql ): ***
> >Purging cache - END (PID: 31907,QN: 12482/32819, ET: 8978) ***
> >
> >
> >​ Have you considered moving to INSERTs rather than UPDATEs?​
> >​ Config key is "
> >sql_dont_try_update
> >​ "
> >
> >M ​
> >​
> >
> >
> >
> >___
> >pmacct-discussion mailing list
> >http://www.pmacct.net/#mailinglists
> 
> -- 
> Anthony Rodriguez
> Software Engineer I
> NetWolves
> Email: anthony.rodrig...@netwolves.com
> www.netwolves.com
> 

> ___
> pmacct-discussion mailing list
> http://www.pmacct.net/#mailinglists


___
pmacct-discussion mailing list
http://www.pmacct.net/#mailinglists

Re: [pmacct-discussion] Capturing NetFlow in PostgresSQL Database

[Matthew Walster]

On 27 April 2016 at 20:42, Anthony Rodriguez <
anthony.rodrig...@netwolves.com> wrote:

> Hi,
>
> I have set up a test capturing NF into a postgresql 8.4 DB. It seems to
> work OK but the writing to the DB seems
> to get far behind. I had one writer take 1.5 hours to sync. Any hints
> would be appreciated.
>
> Apr 27 10:30:01  nfacctd[31907]: INFO ( default/pgsql ): *** Purging cache
> - START (PID: 31907) ***
> Apr 27 12:59:39  nfacctd[31907]: INFO ( default/pgsql ): *** Purging cache
> - END (PID: 31907, QN: 12482/32819, ET: 8978) ***
>
>
​Have you considered moving to INSERTs rather than UPDATEs?​

​Config key is "
sql_dont_try_update
​"

M​
​
___
pmacct-discussion mailing list
http://www.pmacct.net/#mailinglists

[pmacct-discussion] Capturing NetFlow in PostgresSQL Database

[Anthony Rodriguez]

Hi,

I have set up a test capturing NF into a postgresql 8.4 DB. It seems to 
work OK but the writing to the DB seems
to get far behind. I had one writer take 1.5 hours to sync. Any hints 
would be appreciated.


Apr 27 10:30:01  nfacctd[31907]: INFO ( default/pgsql ): *** Purging 
cache - START (PID: 31907) ***
Apr 27 12:59:39  nfacctd[31907]: INFO ( default/pgsql ): *** Purging 
cache - END (PID: 31907,QN: 12482/32819, ET: 8978) ***


What does the ending information signify.

nfacctd config

daemonize: true
debug: false
pidfile: /var/run/nfacctd.pid
syslog: daemon
pre_tag_map: /root/my.pretag.map
nfacctd_disable_checks: true
nfacctd_time_new: false
aggregate: tag, src_host, dst_host, src_port, dst_port, proto, tos
plugin_pipe_size: 4096000
plugin_buffer_size: 4096
plugins: pgsql
sql_table: acct_v9
sql_data: typed
sql_db: pmacct
sql_passwd: arealsmartpwd
sql_user: pmacct
sql_refresh_time: 300
sql_optimize_clauses: true
sql_history: 1h
sql_history_roundoff: m
sql_recovery_logfile: /var/lib/pmacct/recovery_log
sql_table_version: 9
!sql_startup_delay: 900
sql_preprocess: qnum=1000, minp=5
imt_buckets: 65537
imt_mem_pools_size: 1024000
nfacctd_port: 2055

Thanks,
Anthony

--
Anthony Rodriguez
Software Engineer I
NetWolves
Email: anthony.rodrig...@netwolves.com
www.netwolves.com

___
pmacct-discussion mailing list
http://www.pmacct.net/#mailinglists