Re: [pmacct-discussion] 1.7.5 with static ndpi

2020-06-25 Thread Stephen Clark

Hi Paolo,

We have pmacct installed on a number of remote systems and
it just more moving parts to keep updated with having to also install/update 
nDPI.

Also I have used the following configure line

./configure '--enable-ndpi' --with-ndpi-static-lib=/usr/local/lib/ '--enable-l2' 
'--enable-traffic-bins' '--enable-bgp-bins' '--enable-bmp-bins' '--enable-st-bins'


and still get a dynamically linked pmacctd. Also the dynamic lib and static lib 
are both in /usr/local/lib


I just removed the dynamic libs and got pmacctd built - I am testing it now.

Thanks for your help,
Steve


On 6/24/20 4:30 PM, Paolo Lucente wrote:

Hi Steve,

Apart from asking the obvious - personal curiosity! - why do you want to
link against a static nDPI library. There are a couple main avenues i
can point you to depending on your goal:

1) You can supply configure with a --with-ndpi-static-lib knob; guess
the static lib and the dynamic lib are in different places, you should
be game. Even simplifying further: should you make the 'shared object'
library disappear then things will be forced onto the static library;

2) did you see the "pmacct & Docker" email that did just circulate on
the list? In the seek for a static library? Perhaps time to look into a
container instead? :-D

Paolo

On Tue, Jun 23, 2020 at 01:44:32PM -0400, Stephen Clark wrote:

Hello,

Can anyone give the magic configuration items I need to build using a static
libndpi.a

I have spend all day trying to do this without any success. It seem like I
tried every combination
that ./configure --help displays.

Any help would be appreciated.

Thanks,
Steve


___
pmacct-discussion mailing list
http://www.pmacct.net/#mailinglists



--

"They that give up essential liberty to obtain temporary safety,
deserve neither liberty nor safety."  (Ben Franklin)

"The course of history shows that as a government grows, liberty
decreases."  (Thomas Jefferson)

"Beer is proof God loves us and wants us to be happy!" (Ben Franklin)


___
pmacct-discussion mailing list
http://www.pmacct.net/#mailinglists


[pmacct-discussion] 1.7.5 with static ndpi - updated -again

2020-06-24 Thread Stephen Clark

Spoke to soon,

Still created pmacctd using shared lib for ndpi

 ldd pmacct/src/pmacctd
    linux-vdso.so.1 =>  (0x7ffeb1be6000)
    libndpi.so.3 => /usr/local/lib/libndpi.so.3 (0x7f4258388000)

Updating - I logged out - logged back in

used
./configure  '--enable-ndpi' --with-ndpi-static-lib=/usr/local/lib '--enable-l2' 
'--enable-traffic-bins' '--enable-bgp-bins' '--enable-bmp-bins' '--enable-st-bins'


and it built using ndpi static lib just fine.


Hello,

Can anyone give the magic configuration items I need to build using a static 
libndpi.a


I have spend all day trying to do this without any success. It seem like I tried 
every combination

that ./configure --help displays.

Any help would be appreciated.

Thanks,
Steve




___
pmacct-discussion mailing list
http://www.pmacct.net/#mailinglists


[pmacct-discussion] 1.7.5 with static ndpi - updated

2020-06-24 Thread Stephen Clark

Updating - I logged out - logged back in

used
./configure  '--enable-ndpi' --with-ndpi-static-lib=/usr/local/lib '--enable-l2' 
'--enable-traffic-bins' '--enable-bgp-bins' '--enable-bmp-bins' '--enable-st-bins'


and it built using ndpi static lib just fine.


Hello,

Can anyone give the magic configuration items I need to build using a static 
libndpi.a


I have spend all day trying to do this without any success. It seem like I tried 
every combination

that ./configure --help displays.

Any help would be appreciated.

Thanks,
Steve



___
pmacct-discussion mailing list
http://www.pmacct.net/#mailinglists


[pmacct-discussion] 1.7.5 with static ndpi

2020-06-23 Thread Stephen Clark

Hello,

Can anyone give the magic configuration items I need to build using a static 
libndpi.a


I have spend all day trying to do this without any success. It seem like I tried 
every combination

that ./configure --help displays.

Any help would be appreciated.

Thanks,
Steve


___
pmacct-discussion mailing list
http://www.pmacct.net/#mailinglists


Re: [pmacct-discussion] nbar/nbar2

2020-01-22 Thread Stephen Clark

On 1/22/20 2:40 PM, Stephen Clark wrote:

On 1/22/20 11:20 AM, Stephen Clark wrote:

Hi Paolo,

can nfprobe export nbar data like cisco's?

Thanks,
Steve

Answering my own question it appears that is can. Is anybody using it sending 
NetFlows to

SolarWinds.


Answering my question again - it appears SW accepts the data as nbar. Yippee and
pmacct really rocks!

Thanks Paolo.

Regards,
Steve

"Beer is proof God loves us and wants us to be happy!" (Ben Franklin)


___
pmacct-discussion mailing list
http://www.pmacct.net/#mailinglists


Re: [pmacct-discussion] nbar/nbar2

2020-01-22 Thread Stephen Clark

On 1/22/20 11:20 AM, Stephen Clark wrote:

Hi Paolo,

can nfprobe export nbar data like cisco's?

Thanks,
Steve

Answering my own question it appears that is can. Is anybody using it sending 
NetFlows to

SolarWinds.

--

"They that give up essential liberty to obtain temporary safety,
deserve neither liberty nor safety."  (Ben Franklin)

"The course of history shows that as a government grows, liberty
decreases."  (Thomas Jefferson)

"Beer is proof God loves us and wants us to be happy!" (Ben Franklin)


___
pmacct-discussion mailing list
http://www.pmacct.net/#mailinglists


[pmacct-discussion] nbar/nbar2

2020-01-22 Thread Stephen Clark

Hi Paolo,

can nfprobe export nbar data like cisco's?

Thanks,
Steve

--


___
pmacct-discussion mailing list
http://www.pmacct.net/#mailinglists


Re: [pmacct-discussion] effort to relicense pmacct from GPL to a BSD-style license

2020-01-12 Thread Stephen Clark

I am too.

On 1/11/20 3:38 PM, Mike Jager wrote:

On 9 Jan 2020, at 2:52, Job Snijders wrote:


We need explicit approval from all contributors, and carefully keep
track of those agreements. If a contributor doesn't agree or answer,
we'll have to re-implement the contributed functionality or remove the
contribution from the code base.

REQUEST TO THE PMACCT CONTRIBUTOR COMMUNITY
---

If you have contributed to the pmacct project (your name may be listed
below), please consider a reply-all to this email expressing your
explicit consent (or disapproval) to change the license governing your
contributions to the pmacct project, to the following license:

 """
 Permission to use, copy, modify, and distribute this software for
 any purpose with or without fee is hereby granted, provided that the
 above copyright notice and this permission notice appear in all
 copies.

 THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL
 WARRANTIES WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED
 WARRANTIES OF MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE
 AUTHOR BE LIABLE FOR ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL
 DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA
 OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER
 TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR
 PERFORMANCE OF THIS SOFTWARE.
 """

---

I'm happy to change the license governing my contributions to the above.

Cheers
Mike

___
pmacct-discussion mailing list
http://www.pmacct.net/#mailinglists




--

"They that give up essential liberty to obtain temporary safety,
deserve neither liberty nor safety."  (Ben Franklin)

"The course of history shows that as a government grows, liberty
decreases."  (Thomas Jefferson)

"Beer is proof God loves us and wants us to be happy!" (Ben Franklin)


___
pmacct-discussion mailing list
http://www.pmacct.net/#mailinglists


Re: [pmacct-discussion] specify source address for netflow packets from pmacctd

2019-10-17 Thread Stephen Clark

Hi Felix,

You are correct - I'll try out the nprobe_source_ip - that sounds like what I am 
looking for.


Thanks,
Steve

On 10/17/19 11:51 AM, Felix Stolba wrote:

Hey Steve

I'm assuming you're generating Netflow from locally received/generated traffic, 
e.g. using pmacct as a Netflow probe. Therefore I think the option called 
nfprobe_source_ip might be the one you're looking for.
Let me know if that fits your use-case.

Regards
Felix


Am 17.10.19, 16:56 schrieb "pmacct-discussion im Auftrag von Stephen Clark" 
:

 Hi Paolo,
 
 We have multiple nic ports and ips on systems we are using pmacctd on. Is there

 a way to specify the ip address pmacctd is using for the source address in 
the
 netflow packets it is sending? I didn't see anything in the config-keys 
file but
 I could
 have missed it.
 
 As an example suppose we have:
 
 eth0: 172.24.10.1

 eth1: 10.9.0.1
 
 We would like to be able to have the netflow packets generated by pmacctd to be

 10.9.0.1.
 
 Thanks for the great software.
 
 Regards,

 Steve
 
 
 ___

 pmacct-discussion mailing list
 http://www.pmacct.net/#mailinglists
 


___
pmacct-discussion mailing list
http://www.pmacct.net/#mailinglists



--

"They that give up essential liberty to obtain temporary safety,
deserve neither liberty nor safety."  (Ben Franklin)

"The course of history shows that as a government grows, liberty
decreases."  (Thomas Jefferson)

"Beer is proof God loves us and wants us to be happy!" (Ben Franklin)


___
pmacct-discussion mailing list
http://www.pmacct.net/#mailinglists

[pmacct-discussion] specify source address for netflow packets from pmacctd

2019-10-17 Thread Stephen Clark

Hi Paolo,

We have multiple nic ports and ips on systems we are using pmacctd on. Is there
a way to specify the ip address pmacctd is using for the source address in the
netflow packets it is sending? I didn't see anything in the config-keys file but 
I could

have missed it.

As an example suppose we have:

eth0: 172.24.10.1
eth1: 10.9.0.1

We would like to be able to have the netflow packets generated by pmacctd to be 
10.9.0.1.


Thanks for the great software.

Regards,
Steve


___
pmacct-discussion mailing list
http://www.pmacct.net/#mailinglists


Re: [pmacct-discussion] Segfault 1.6.1 and current master

2017-08-23 Thread Stephen Clark
Hi Paolo,

After doing some more investigation this looks like this could be the problem.
From cfg.h
/* global vars */
EXT char *cfg[SRVBUFLEN],

From cfg.c
  while (!feof(file)) {
if (rows == LARGEBUFLEN) {
  Log(LOG_ERR, "ERROR: [%s] maximum number of %d lines reached.\n",
filename, LARGEBUFLEN);

Shouldn't *cfg[SRVBUFLEN] be *cfg[LARGEBUFLEN] ?

It looks like there are not enough array elements to hold all the config item we
have.

Regards,
Steve

On 08/22/2017 03:06 PM, Stephen Clark wrote:
> Hi Paolo,
>
> We have a large nfacctd.conf file around 530 lines. When we try to
> start nfacctd with the -F flag we get a segfault at line below.
>
>   /* splitting key, value and name */
>   delim = strchr(cfg[index], ':');
>   *delim = '\0';   <<<<<<<<<
>   key = cfg[index];
>   value = delim+1;
>
>
> I changed the code like below to avoid the segfault - note this happens on the
> last line of the file.
> Also we only get the segfault if we use the -F flag.
>
>   delim = strchr(cfg[index], ':');
>   if (delim) {
>   *delim = '\0';
>   key = cfg[index];
>   value = delim+1;
>   } else {
>   index++;
>   continue;
>   }
>
> Regards,
> Steve
>
>
>
>
> ___
> pmacct-discussion mailing list
> http://www.pmacct.net/#mailinglists


-- 

"They that give up essential liberty to obtain temporary safety, 
deserve neither liberty nor safety."  (Ben Franklin)

"The course of history shows that as a government grows, liberty 
decreases."  (Thomas Jefferson)

"Beer is proof God loves us and wants us to be happy!" (Ben Franklin)



signature.asc
Description: OpenPGP digital signature
___
pmacct-discussion mailing list
http://www.pmacct.net/#mailinglists

[pmacct-discussion] nDPI

2017-07-26 Thread Stephen Clark
Hi Paolo,

Noticed an error in the example you gave in the documentation.

5) Configure pmacct. The following sample configuration is based on pmacctd and
   the print plugin with formatted output to stdout:

   daemonize: true
   interface: eth0
   snaplen: 700
   !
   plugins: print
   !
   aggregate: src_host, dst_host, src_port dst_port, proto, tos, class<<
missing comma between src_port and dst_port

   What enables packet classification is the use of the 'class' primitive as 
part
   of the supplied aggregation method. Further classification-related options,
   such as timers, attempts, etc., are documented in the CONFIG-KEYS document
(classifier_* directives).


Regards,
Steve




signature.asc
Description: OpenPGP digital signature
___
pmacct-discussion mailing list
http://www.pmacct.net/#mailinglists

[pmacct-discussion] nDPI

2017-07-25 Thread Stephen Clark
Hi Paolo,

I did a minimal test of the new nDPI integration. It looks promising.

What is the first Unknown suppose to represent?
Unknown/Kerberos 
Unknown/Kerberos 
Unknown/Kerberos 

This is a little confusing - this was traffic between the same host - very close
together but
only one is identified as SSL traffic.
Unknown/Unknown   443   57137 tcp 0  8 
8618
Unknown/SSL   443   57137 tcp 0  2848 
3076024

Some feedback,

Thanks,
Steve




signature.asc
Description: OpenPGP digital signature
___
pmacct-discussion mailing list
http://www.pmacct.net/#mailinglists

[pmacct-discussion] packet classification - nDPI

2017-05-09 Thread Stephen Clark
Hi,

has anyone hooked nDPI into pmacctd for packet classification?

Thanks,
Steve

-- 

"They that give up essential liberty to obtain temporary safety, 
deserve neither liberty nor safety."  (Ben Franklin)

"The course of history shows that as a government grows, liberty 
decreases."  (Thomas Jefferson)





signature.asc
Description: OpenPGP digital signature
___
pmacct-discussion mailing list
http://www.pmacct.net/#mailinglists

[pmacct-discussion] nfacctd and pfring

2017-03-22 Thread Stephen Clark
Hi Paolo,

Does nfacctd make use of pfring or is it only used by pmacctd?

Thanks,
Steve




signature.asc
Description: OpenPGP digital signature
___
pmacct-discussion mailing list
http://www.pmacct.net/#mailinglists

Re: [pmacct-discussion] forwarding netflow

2016-11-17 Thread Stephen Clark

Hmm...

Doesn't samplicate do this?

and each  should be specified as
[/[/]], where

  IP address of the receiver
  port UDP number of the receiver (default 2000)
  number of received datagrams between successive
copied datagrams for this receiver.

On 11/17/2016 10:10 AM, Paul Lockaby wrote:

Ah, yeah, hmm. Doesn't seem to support any sampling of the data it forwards so 
I guess I'll have to find something else.

Thanks,
-Paul


On Nov 17, 2016, at 12:36 AM, Tristan Bendall  wrote:

Hi Paul

Pretty sure "tee" does this? Have a look below:

http://wiki.pmacct.net/OfficialConfigKeys

Tristan

-Original Message-
From: pmacct-discussion [mailto:pmacct-discussion-boun...@pmacct.net] On Behalf 
Of Paul Lockaby
Sent: 16 November 2016 21:57
To: pmacct-discussion@pmacct.net
Subject: [pmacct-discussion] forwarding netflow

Does pmacct/nfacctd support forwarding netflow/ipfix, like samplicator does? If 
it does, does it support forwarding sampled netflow/ipfix? E.g. I have netflow 
coming in to one collector host and I need to send it on to one person who 
wants it 1:1 and another person who wants it 10:1.

-Paul
___
pmacct-discussion mailing list
http://www.pmacct.net/#mailinglists

___
pmacct-discussion mailing list
http://www.pmacct.net/#mailinglists


___
pmacct-discussion mailing list
http://www.pmacct.net/#mailinglists




--

"They that give up essential liberty to obtain temporary safety,
deserve neither liberty nor safety."  (Ben Franklin)

"The course of history shows that as a government grows, liberty
decreases."  (Thomas Jefferson)



___
pmacct-discussion mailing list
http://www.pmacct.net/#mailinglists


Re: [pmacct-discussion] performance issue

2016-11-14 Thread Stephen Clark

Hi Paolo,

This is our config.
daemonize: true
debug: false
pidfile: /var/run/nfacctd.pid
syslog: daemon
pre_tag_map: /etc/pmacct/my.pretag.map
!nfacctd_disable_checks: false
nfacctd_disable_checks: true

nfacctd_time_new: false

aggregate: tag, src_host, dst_host, src_port, dst_port, proto, tos


!plugin_pipe_size: 8192000
!plugin_buffer_size: 8192
plugin_pipe_size: 4096000
plugin_buffer_size: 4096

plugins: pgsql

!sql_table: acct_uni_custom
sql_table: netflow
sql_data: typed

!sql_multi_values: 512000
sql_dont_try_update: true
sql_use_copy: true
sql_db: pmacct
sql_host: 127.0.0.1
sql_passwd: arealsmartpwd
sql_user: pmacct
!sql_refresh_time: 300
sql_refresh_time: 60
sql_optimize_clauses: true
!sql_history: 5m
sql_history: 1m
sql_history_roundoff: m
sql_preprocess:  fsrc=2
sql_locking_style: row
sql_cache_entries: 800011

imt_buckets: 65537
imt_mem_pools_size: 1024000

nfacctd_port: 2055

Thanks for your support.

Steve

On 11/12/2016 09:31 AM, Paolo Lucente wrote:

Hi Steve,

Canyou please post your integral config to try to reproduce the issue?
It smells like something is wrong (bug).

Cheers,
Paolo

On Wed, Nov 09, 2016 at 10:38:02AM -0500, Stephen Clark wrote:

Hi Paolo,

it seems that using the sql_preprocess:  fsrc=20
causes the problem when start getting more than 20 netflows per
minute.

If we comment this line out we don't see any issue, minimal cpu usage only one
sql writer process.

We are trying to limit the size of our database, that is why we used it.

Any ideas why using this causes a problem?

Thanks,
Steve

On 11/09/2016 09:47 AM, Stephen Clark wrote:

Oops - we just hit 10 writers.

On 11/09/2016 08:54 AM, Stephen Clark wrote:

Hi Paolo,

Thanks for the response. Do you see anything in our confguration
that I could adjust to mitigate the situation.

We never reach 10 sql writers.

Would increasing the any of these help?
sql_refresh_time: 60
sql_optimize_clauses: true
!sql_history: 5m
sql_history: 1m
sql_history_roundoff: m

Thanks,
Steve

On 11/09/2016 07:58 AM, Paolo Lucente wrote:

Hi Steve,

You are experiencing a few connected problems, i guess. The root issue
should be that the PostgreSQL database is not coping with the insert or
update rate and/or with the size of the dataset.

The list of plugins you see there are, in fact, all DB writers. They are
queued up, waiting for the table they want to write to will unlock. What
typically happens, wrt the seg fault, is that each of these processes
does take some memory; you stack many of them, memory goes away; you try
to stack more, they go seg fault due to lack of system resources. You
can prevent that lowering the amoung of writers allowed to stack up, by
default 10, via sql_max_writers.

Finally, it is normal that the active DB writer can take 100% CPU in order
to dump all its data to the backend. On shortage of system resources the
situation can get unstabe and you may see multiple processes in such
state as they start competing with kernel, ie. swap, etc.

Cheers,
Paolo

On Tue, Nov 08, 2016 at 03:34:57PM -0500, Stephen Clark wrote:

Hi,

I am having a problem with nfacctd getting way behind with ver 1.5.3
Everything is ok until I add a server that is sending a lot of netflows
then things start bogging down. I see the nfacctd plugins
using 100% cpu using top.

Then I start getting seg faults:
Nov  8 15:28:01 netflow2 kernel: nfacctd[14296]: segfault at 0 ip
0046d588 sp 7fffe95e72c0 error 4 in
nfacctd[40+14f000]

11389 ?R  8:15 nfacctd: pgsql Plugin -- DB Writer [default]
11993 ?R  7:11 nfacctd: pgsql Plugin -- DB Writer [default]
12229 ?R  6:14 nfacctd: pgsql Plugin -- DB Writer [default]
12372 ?R  5:15 nfacctd: pgsql Plugin -- DB Writer [default]
12435 ?R  4:14 nfacctd: pgsql Plugin -- DB Writer [default]
12678 ?R  3:17 nfacctd: pgsql Plugin -- DB Writer [default]
13187 ?R  2:17 nfacctd: pgsql Plugin -- DB Writer [default]
13499 ?R  1:13 nfacctd: pgsql Plugin -- DB Writer [default]
13711 ?R  0:24 nfacctd: pgsql Plugin -- DB Writer [default]


daemonize: true
debug: false
pidfile: /var/run/nfacctd.pid
syslog: daemon

pre_tag_map: /etc/pmacct/my.pretag.map

nfacctd_disable_checks: true

nfacctd_time_new: false

aggregate: tag, src_host, dst_host, src_port, dst_port, proto, tos


!plugin_pipe_size: 8192000
!plugin_buffer_size: 8192
plugin_pipe_size: 4096000
plugin_buffer_size: 4096

plugins: pgsql

sql_table: netflow
sql_data: typed

sql_dont_try_update: true
sql_use_copy: true
sql_db: pmacct
sql_host: 127.0.0.1
sql_passwd: **
sql_user: pmacct

sql_refresh_time: 60
sql_optimize_clauses: true

sql_history: 1m
sql_history_roundoff: m
sql_recovery_logfile: /var/lib/pmacct/recovery_log

sql_preprocess:  fsrc=20

sql_locking_style: row
sql_cache_entries: 29

imt_buckets: 65537
imt_mem_pools_size: 1024000

nfacctd_port: 2055


Suggestions?

Thanks,
Steve

Re: [pmacct-discussion] building version 1.6.1

2016-11-10 Thread Stephen Clark

On 11/09/2016 09:21 PM, Vincent Bernat wrote:

  ❦  9 novembre 2016 11:56 -0500, Stephen Clark <sclar...@earthlink.net> :


LIBS . : -L/usr/pgsql-9.4/lib -ldl -L/usr/local/lib -lpfring
-lpcap -lrt -lnuma -lz -lpthread

If libpfring is linked to one version of libpcap and your local libpcap
is another one, there will a version conflict unless libpcap symbols are
versioned (they are not). Could you check with ldd what is in libpfring,
dependency-wise?

Yes, that is probably what is happening. The pf_ring from ntopng
replace libpcap with its own
version that uses pf_ring.





Yes, that is probably what is happening. The pf_ring from ntopng
replace libpcap with its own
version that uses pf_ring.

So, if libpfring is exporting directly the bpf_validate symbols (could
you check with objdump -T?), you can try to not link with -lpcap at all.
I just took the easy way out and compiled on a system that didn't have pf_ring 
installed.

Thanks for the info though.

Steve




___
pmacct-discussion mailing list
http://www.pmacct.net/#mailinglists

Re: [pmacct-discussion] performance issue

2016-11-09 Thread Stephen Clark

Oops - we just hit 10 writers.

On 11/09/2016 08:54 AM, Stephen Clark wrote:

Hi Paolo,

Thanks for the response. Do you see anything in our confguration
that I could adjust to mitigate the situation.

We never reach 10 sql writers.

Would increasing the any of these help?
 sql_refresh_time: 60
sql_optimize_clauses: true
!sql_history: 5m
sql_history: 1m
sql_history_roundoff: m

Thanks,
Steve

On 11/09/2016 07:58 AM, Paolo Lucente wrote:

Hi Steve,

You are experiencing a few connected problems, i guess. The root issue
should be that the PostgreSQL database is not coping with the insert or
update rate and/or with the size of the dataset.

The list of plugins you see there are, in fact, all DB writers. They are
queued up, waiting for the table they want to write to will unlock. What
typically happens, wrt the seg fault, is that each of these processes
does take some memory; you stack many of them, memory goes away; you try
to stack more, they go seg fault due to lack of system resources. You
can prevent that lowering the amoung of writers allowed to stack up, by
default 10, via sql_max_writers.

Finally, it is normal that the active DB writer can take 100% CPU in order
to dump all its data to the backend. On shortage of system resources the
situation can get unstabe and you may see multiple processes in such
state as they start competing with kernel, ie. swap, etc.

Cheers,
Paolo

On Tue, Nov 08, 2016 at 03:34:57PM -0500, Stephen Clark wrote:

Hi,

I am having a problem with nfacctd getting way behind with ver 1.5.3
Everything is ok until I add a server that is sending a lot of netflows
then things start bogging down. I see the nfacctd plugins using 100% cpu 
using top.


Then I start getting seg faults:
Nov  8 15:28:01 netflow2 kernel: nfacctd[14296]: segfault at 0 ip
0046d588 sp 7fffe95e72c0 error 4 in
nfacctd[40+14f000]

11389 ?R  8:15 nfacctd: pgsql Plugin -- DB Writer [default]
11993 ?R  7:11 nfacctd: pgsql Plugin -- DB Writer [default]
12229 ?R  6:14 nfacctd: pgsql Plugin -- DB Writer [default]
12372 ?R  5:15 nfacctd: pgsql Plugin -- DB Writer [default]
12435 ?R  4:14 nfacctd: pgsql Plugin -- DB Writer [default]
12678 ?R  3:17 nfacctd: pgsql Plugin -- DB Writer [default]
13187 ?R  2:17 nfacctd: pgsql Plugin -- DB Writer [default]
13499 ?R  1:13 nfacctd: pgsql Plugin -- DB Writer [default]
13711 ?R  0:24 nfacctd: pgsql Plugin -- DB Writer [default]


daemonize: true
debug: false
pidfile: /var/run/nfacctd.pid
syslog: daemon

pre_tag_map: /etc/pmacct/my.pretag.map

nfacctd_disable_checks: true

nfacctd_time_new: false

aggregate: tag, src_host, dst_host, src_port, dst_port, proto, tos


!plugin_pipe_size: 8192000
!plugin_buffer_size: 8192
plugin_pipe_size: 4096000
plugin_buffer_size: 4096

plugins: pgsql

sql_table: netflow
sql_data: typed

sql_dont_try_update: true
sql_use_copy: true
sql_db: pmacct
sql_host: 127.0.0.1
sql_passwd: **
sql_user: pmacct

sql_refresh_time: 60
sql_optimize_clauses: true

sql_history: 1m
sql_history_roundoff: m
sql_recovery_logfile: /var/lib/pmacct/recovery_log

sql_preprocess:  fsrc=20

sql_locking_style: row
sql_cache_entries: 29

imt_buckets: 65537
imt_mem_pools_size: 1024000

nfacctd_port: 2055


Suggestions?

Thanks,
Steve

___
pmacct-discussion mailing list
http://www.pmacct.net/#mailinglists






--

"They that give up essential liberty to obtain temporary safety,
deserve neither liberty nor safety."  (Ben Franklin)

"The course of history shows that as a government grows, liberty
decreases."  (Thomas Jefferson)



___
pmacct-discussion mailing list
http://www.pmacct.net/#mailinglists


Re: [pmacct-discussion] performance issue

2016-11-09 Thread Stephen Clark

Hi Paolo,

Thanks for the response. Do you see anything in our confguration
that I could adjust to mitigate the situation.

We never reach 10 sql writers.

Would increasing the any of these help?
 sql_refresh_time: 60
sql_optimize_clauses: true
!sql_history: 5m
sql_history: 1m
sql_history_roundoff: m

Thanks,
Steve

On 11/09/2016 07:58 AM, Paolo Lucente wrote:

Hi Steve,

You are experiencing a few connected problems, i guess. The root issue
should be that the PostgreSQL database is not coping with the insert or
update rate and/or with the size of the dataset.

The list of plugins you see there are, in fact, all DB writers. They are
queued up, waiting for the table they want to write to will unlock. What
typically happens, wrt the seg fault, is that each of these processes
does take some memory; you stack many of them, memory goes away; you try
to stack more, they go seg fault due to lack of system resources. You
can prevent that lowering the amoung of writers allowed to stack up, by
default 10, via sql_max_writers.

Finally, it is normal that the active DB writer can take 100% CPU in order
to dump all its data to the backend. On shortage of system resources the
situation can get unstabe and you may see multiple processes in such
state as they start competing with kernel, ie. swap, etc.

Cheers,
Paolo

On Tue, Nov 08, 2016 at 03:34:57PM -0500, Stephen Clark wrote:

Hi,

I am having a problem with nfacctd getting way behind with ver 1.5.3
Everything is ok until I add a server that is sending a lot of netflows
then things start bogging down. I see the nfacctd plugins using 100% cpu using 
top.

Then I start getting seg faults:
Nov  8 15:28:01 netflow2 kernel: nfacctd[14296]: segfault at 0 ip
0046d588 sp 7fffe95e72c0 error 4 in
nfacctd[40+14f000]

11389 ?R  8:15 nfacctd: pgsql Plugin -- DB Writer [default]
11993 ?R  7:11 nfacctd: pgsql Plugin -- DB Writer [default]
12229 ?R  6:14 nfacctd: pgsql Plugin -- DB Writer [default]
12372 ?R  5:15 nfacctd: pgsql Plugin -- DB Writer [default]
12435 ?R  4:14 nfacctd: pgsql Plugin -- DB Writer [default]
12678 ?R  3:17 nfacctd: pgsql Plugin -- DB Writer [default]
13187 ?R  2:17 nfacctd: pgsql Plugin -- DB Writer [default]
13499 ?R  1:13 nfacctd: pgsql Plugin -- DB Writer [default]
13711 ?R  0:24 nfacctd: pgsql Plugin -- DB Writer [default]


daemonize: true
debug: false
pidfile: /var/run/nfacctd.pid
syslog: daemon

pre_tag_map: /etc/pmacct/my.pretag.map

nfacctd_disable_checks: true

nfacctd_time_new: false

aggregate: tag, src_host, dst_host, src_port, dst_port, proto, tos


!plugin_pipe_size: 8192000
!plugin_buffer_size: 8192
plugin_pipe_size: 4096000
plugin_buffer_size: 4096

plugins: pgsql

sql_table: netflow
sql_data: typed

sql_dont_try_update: true
sql_use_copy: true
sql_db: pmacct
sql_host: 127.0.0.1
sql_passwd: **
sql_user: pmacct

sql_refresh_time: 60
sql_optimize_clauses: true

sql_history: 1m
sql_history_roundoff: m
sql_recovery_logfile: /var/lib/pmacct/recovery_log

sql_preprocess:  fsrc=20

sql_locking_style: row
sql_cache_entries: 29

imt_buckets: 65537
imt_mem_pools_size: 1024000

nfacctd_port: 2055


Suggestions?

Thanks,
Steve

___
pmacct-discussion mailing list
http://www.pmacct.net/#mailinglists



--

"They that give up essential liberty to obtain temporary safety,
deserve neither liberty nor safety."  (Ben Franklin)

"The course of history shows that as a government grows, liberty
decreases."  (Thomas Jefferson)



___
pmacct-discussion mailing list
http://www.pmacct.net/#mailinglists


Re: [pmacct-discussion] building version 1.6.1

2016-11-09 Thread Stephen Clark

Hi Paolo,

I compiled it on a system without pf_ring installed and it compiled OK.

Regards,
Steve

On 11/09/2016 07:52 AM, Paolo Lucente wrote:

Hi Steve,

This is the same issue as described here:

https://github.com/pmacct/pmacct/issues/40

See the wontfix flag, i'm unable to reproduce the issue. If you solve it
yourself, please contribute a patch. I'd be fully to look into it for
you but it smells i need (unprivileged) access to the box.

Cheers,
Paolo


On Wed, Nov 09, 2016 at 07:22:36AM -0500, Stephen Clark wrote:

Hi,

I get the following error when trying to build 1.6.1 on CentOS 6.0

   CCLD   pmacctd
/usr/local/lib/libpcap.a(bpf_filter.o): In function `bpf_validate':
(.text+0x0): multiple definition of `bpf_validate'
./.libs/libdaemons.a(libdaemons_la-bpf_filter.o):/var/lib/pgsql/sclark/pmacct-1.6.1/src/bpf_filter.c:528:
first defined here
/usr/local/lib/libpcap.a(bpf_filter.o): In function `bpf_filter':
(.text+0x590): multiple definition of `bpf_filter'
./.libs/libdaemons.a(libdaemons_la-bpf_filter.o):/var/lib/pgsql/sclark/pmacct-1.6.1/src/bpf_filter.c:201:
first defined here

PLATFORM . : x86_64
OS ... : Linux 2.6.32-504.16.2.el6.x86_64
COMPILER . : gcc
CFLAGS ... : -O2 -g -O2
LIBS . : -L/usr/pgsql-9.4/lib -ldl -L/usr/local/lib -lpfring
-lpcap -lrt -lnuma -lz -lpthread
LDFLAGS .. : -Wl,--export-dynamic
PLUGINS .. :

Is there a way to exclude pfring?

Thanks,
Steve


___
pmacct-discussion mailing list
http://www.pmacct.net/#mailinglists



--

"They that give up essential liberty to obtain temporary safety,
deserve neither liberty nor safety."  (Ben Franklin)

"The course of history shows that as a government grows, liberty
decreases."  (Thomas Jefferson)



___
pmacct-discussion mailing list
http://www.pmacct.net/#mailinglists


[pmacct-discussion] building version 1.6.1

2016-11-09 Thread Stephen Clark

Hi,

I get the following error when trying to build 1.6.1 on CentOS 6.0

  CCLD   pmacctd
/usr/local/lib/libpcap.a(bpf_filter.o): In function `bpf_validate':
(.text+0x0): multiple definition of `bpf_validate'
./.libs/libdaemons.a(libdaemons_la-bpf_filter.o):/var/lib/pgsql/sclark/pmacct-1.6.1/src/bpf_filter.c:528: 
first defined here

/usr/local/lib/libpcap.a(bpf_filter.o): In function `bpf_filter':
(.text+0x590): multiple definition of `bpf_filter'
./.libs/libdaemons.a(libdaemons_la-bpf_filter.o):/var/lib/pgsql/sclark/pmacct-1.6.1/src/bpf_filter.c:201: 
first defined here


PLATFORM . : x86_64
OS ... : Linux 2.6.32-504.16.2.el6.x86_64
COMPILER . : gcc
CFLAGS ... : -O2 -g -O2
LIBS . : -L/usr/pgsql-9.4/lib -ldl -L/usr/local/lib -lpfring -lpcap -lrt 
-lnuma -lz -lpthread

LDFLAGS .. : -Wl,--export-dynamic
PLUGINS .. :

Is there a way to exclude pfring?

Thanks,
Steve


___
pmacct-discussion mailing list
http://www.pmacct.net/#mailinglists


[pmacct-discussion] performance issue

2016-11-08 Thread Stephen Clark

Hi,

I am having a problem with nfacctd getting way behind with ver 1.5.3
Everything is ok until I add a server that is sending a lot of netflows
then things start bogging down. I see the nfacctd plugins using 100% cpu using 
top.

Then I start getting seg faults:
Nov  8 15:28:01 netflow2 kernel: nfacctd[14296]: segfault at 0 ip 
0046d588 sp 7fffe95e72c0 error 4 in nfacctd[40+14f000]


11389 ?R  8:15 nfacctd: pgsql Plugin -- DB Writer [default]
11993 ?R  7:11 nfacctd: pgsql Plugin -- DB Writer [default]
12229 ?R  6:14 nfacctd: pgsql Plugin -- DB Writer [default]
12372 ?R  5:15 nfacctd: pgsql Plugin -- DB Writer [default]
12435 ?R  4:14 nfacctd: pgsql Plugin -- DB Writer [default]
12678 ?R  3:17 nfacctd: pgsql Plugin -- DB Writer [default]
13187 ?R  2:17 nfacctd: pgsql Plugin -- DB Writer [default]
13499 ?R  1:13 nfacctd: pgsql Plugin -- DB Writer [default]
13711 ?R  0:24 nfacctd: pgsql Plugin -- DB Writer [default]


daemonize: true
debug: false
pidfile: /var/run/nfacctd.pid
syslog: daemon

pre_tag_map: /etc/pmacct/my.pretag.map

nfacctd_disable_checks: true

nfacctd_time_new: false

aggregate: tag, src_host, dst_host, src_port, dst_port, proto, tos


!plugin_pipe_size: 8192000
!plugin_buffer_size: 8192
plugin_pipe_size: 4096000
plugin_buffer_size: 4096

plugins: pgsql

sql_table: netflow
sql_data: typed

sql_dont_try_update: true
sql_use_copy: true
sql_db: pmacct
sql_host: 127.0.0.1
sql_passwd: **
sql_user: pmacct

sql_refresh_time: 60
sql_optimize_clauses: true

sql_history: 1m
sql_history_roundoff: m
sql_recovery_logfile: /var/lib/pmacct/recovery_log

sql_preprocess:  fsrc=20

sql_locking_style: row
sql_cache_entries: 29

imt_buckets: 65537
imt_mem_pools_size: 1024000

nfacctd_port: 2055


Suggestions?

Thanks,
Steve

___
pmacct-discussion mailing list
http://www.pmacct.net/#mailinglists


Re: [pmacct-discussion] [SPAM] Re: sampling

2016-08-24 Thread Stephen Clark

Hi Tim,

My question is can nfprobe from pmacct package do the sampling at the 
origination of the flow.


Thanks,
Steve

On 08/24/2016 10:55 AM, Tim Jackson wrote:
If the probe is doing sampling, you can have pmacct re-normalize sampling via 
either a static sampling value, or via the flow/ipfix information that tells 
it how it was sampled..


sample_rate: 
sample_map: 
(s|n|p|u)facctd_renormalize:true

If you're using a tap that samples traffic, you can tell the daemon that 
upstream traffic is sampled by X:


pmacctd_ext_sampling_rate | uacctd_ext_sampling_rate

http://wiki.pmacct.net/OfficialConfigKeys

--
Tim


On Wed, Aug 24, 2016 at 9:37 AM, Stephen Clark <sclar...@earthlink.net 
<mailto:sclar...@earthlink.net>> wrote:


Hi Paolo,

I looked thru the CONFIG_KEYS and didn't find the ability to do sampling
except
in the SQL_preprocess keys. Is it possible to do the sampling at the point
the neflow records are first created - in other words by nfprobe?

Thanks,
Steve

-- 


"They that give up essential liberty to obtain temporary safety,
deserve neither liberty nor safety."  (Ben Franklin)

"The course of history shows that as a government grows, liberty
decreases."  (Thomas Jefferson)



___
pmacct-discussion mailing list
http://www.pmacct.net/#mailinglists <http://www.pmacct.net/#mailinglists>





--

"They that give up essential liberty to obtain temporary safety,
deserve neither liberty nor safety."  (Ben Franklin)

"The course of history shows that as a government grows, liberty
decreases."  (Thomas Jefferson)


___
pmacct-discussion mailing list
http://www.pmacct.net/#mailinglists

[pmacct-discussion] sampling

2016-08-24 Thread Stephen Clark

Hi Paolo,

I looked thru the CONFIG_KEYS and didn't find the ability to do sampling except
in the SQL_preprocess keys. Is it possible to do the sampling at the point
the neflow records are first created - in other words by nfprobe?

Thanks,
Steve

--

"They that give up essential liberty to obtain temporary safety,
deserve neither liberty nor safety."  (Ben Franklin)

"The course of history shows that as a government grows, liberty
decreases."  (Thomas Jefferson)



___
pmacct-discussion mailing list
http://www.pmacct.net/#mailinglists


Re: [pmacct-discussion] collecting large number of netflows

2016-08-18 Thread Stephen Clark

On 08/17/2016 08:38 AM, Jentsch, Mario wrote:

Hey Steve,

that question can't be answered without a lot of assumptions about the details of your project and 
we made the experience that even with project details it is a hard thing to predict due to the 
nature of network traffic patterns. Pmacct (namely nfacctd) can handle that number of flows - even 
with only one instance - and is most probably not the bottleneck. If it is possible what you plan 
to do, depends on questions like "how many records per timebin do you have after aggregation 
in nfacctd" - this is what your backend DB has to handle and "how is this data processed 
later on?" - this has more or less impact on DB performance and the time it takes to create 
reports or feed any user interfaces.

Regards,
Mario

Hi Mario,

Thanks for the response. We will be collecting data from about 200 probes. This 
is a new endeavor so I guess we be learning on the fly. We are planning on using
fsrc sampling feature set at 20 flows per minute with inserts only into a 
postgresql 9.4 DB running on CentOS 6.8 in VMware on a hefty Cisco UCS system.


Regards,
Steve

-Original Message-
From: pmacct-discussion [mailto:pmacct-discussion-boun...@pmacct.net]
On Behalf Of Stephen Clark
Sent: Thursday, August 04, 2016 5:01 PM
To: pmacct-discussion@pmacct.net
Subject: [pmacct-discussion] collecting large number of netflows

Hi List,

I am looking to collect a large number of netflow records, on the order of a
100
million a day,
and store them in a postgres DB. Has anyone done this or something similar
using
pmacct?

Thanks,
Steve





___
pmacct-discussion mailing list
http://www.pmacct.net/#mailinglists


Re: [pmacct-discussion] collecting large number of netflows

2016-08-04 Thread Stephen Clark

Hmm...

I don't think pmacct directly support cassandra, but is does support  MongoDB.
Also I would like to be able to filter/sample the data at the point of origin, 
but
I don't think that is possible with pmacctd and the nfprobe module. It only 
seems
to be able to do it at the collector - nfacctd.

Steve

On 08/04/2016 11:48 AM, David McKen wrote:
For that type of scale maybe a SQL like NoSQL db like cassandra may work 
better for you.


On Thu, Aug 4, 2016 at 11:01 AM, Stephen Clark <sclar...@earthlink.net 
<mailto:sclar...@earthlink.net>> wrote:


Hi List,

I am looking to collect a large number of netflow records, on the order of
a 100 million a day,
and store them in a postgres DB. Has anyone done this or something similar
using pmacct?

Thanks,
Steve


___
pmacct-discussion mailing list
http://www.pmacct.net/#mailinglists





--

"They that give up essential liberty to obtain temporary safety,
deserve neither liberty nor safety."  (Ben Franklin)

"The course of history shows that as a government grows, liberty
decreases."  (Thomas Jefferson)


___
pmacct-discussion mailing list
http://www.pmacct.net/#mailinglists

[pmacct-discussion] collecting large number of netflows

2016-08-04 Thread Stephen Clark

Hi List,

I am looking to collect a large number of netflow records, on the order of a 100 
million a day,
and store them in a postgres DB. Has anyone done this or something similar using 
pmacct?


Thanks,
Steve


___
pmacct-discussion mailing list
http://www.pmacct.net/#mailinglists


Re: [pmacct-discussion] Fwd: minb - version 1.5.3

2016-08-02 Thread Stephen Clark

Hi Paolo,

I had to look at the code but I figured it out.

needed minb= not minb>=

On 08/02/2016 09:36 AM, Steve Clark wrote:


Hi Paolo,

I am trying to limit netflow aggregates to greater than 100 bytes before 
insertion into my PG database, but

I can't seem to get it to work. All aggregates are being inserted.

my config:
daemonize: true
debug: false
pidfile: /var/run/nfacctd.pid
syslog: daemon
pre_tag_map: ./my.pretag.map
nfacctd_disable_checks: true
nfacctd_time_new: false
aggregate: tag, src_host, dst_host, src_port, dst_port, proto, tos
plugin_pipe_size: 4096000
plugin_buffer_size: 4096
plugins: pgsql
sql_table: netflow
sql_data: typed
sql_dont_try_update: true
sql_use_copy: true
sql_db: pmacct
sql_host: 127.0.0.1
sql_passwd: arealsmartpwd
sql_user: pmacct
sql_refresh_time: 60
sql_optimize_clauses: true
sql_history: 1m
sql_history_roundoff: m
sql_recovery_logfile: /var/lib/pmacct/recovery_log
sql_preprocess: minb>=100
sql_locking_style: row
sql_cache_entries: 19
imt_buckets: 65537
imt_mem_pools_size: 1024000
nfacctd_port: 2055

Here is what I get in my table - notice 1173 under 100 bytes.
pmacct=# truncate netflow ;
TRUNCATE TABLE
pmacct=# select count(*),sum(bytes)as bytes,sum(packets)as packets from 
netflow where agent_id = '246' and bytes <100;

 count | bytes | packets
---+---+-
  1173 | 89321 |1205
(1 row)

pmacct=# select count(*),sum(bytes)as bytes,sum(packets)as packets from 
netflow where agent_id = '246';

count |  bytes   | packets
---+--+-
3690 | 63424928 |  105921
(1 row)

Also this is what shows from /var/log/messages
Aug  2 08:06:01 netflow nfacctd[4073]: INFO ( default/pgsql ): *** Purging 
cache - START (PID: 4073) ***
Aug  2 08:06:01 netflow nfacctd[4073]: INFO ( default/pgsql ): *** Purging 
cache - END (PID: 4073, QN: 3690/3690, ET: 0) ***


   KEY: minb
DESC: check. Aggregates on the queue are evaluated one-by-one; 
each object is marked valid
only if the bytes counter is '>=' minb value. An interesting idea is to set 
its value
  to a fraction of the link capacity. Remember that you 
have also a timeframe reference:

  the 'sql_refresh_time' seconds. All plugins.

  For example, given the following parameters:
  Link Capacity = 8Mbit/s, THreshold = 0.1%, TImeframe = 60s
  minb = ((LC / 8) * TI) * TH -> ((8Mbit/s / 8) * 60s) * 
0.1% = 6 bytes.


  Given a 8Mbit link, all aggregates which have accounted 
for at least 60Kb of traffic

  in the last 60 seconds, will be written to the DB.



Any suggestions?

Thanks,
Steve



___
pmacct-discussion mailing list
http://www.pmacct.net/#mailinglists



--

"They that give up essential liberty to obtain temporary safety,
deserve neither liberty nor safety."  (Ben Franklin)

"The course of history shows that as a government grows, liberty
decreases."  (Thomas Jefferson)


___
pmacct-discussion mailing list
http://www.pmacct.net/#mailinglists

Re: [pmacct-discussion] pgsql insert only on version 1.5.3

2016-07-28 Thread Stephen Clark

Thanks Paolo,

I'll give that a try.

On 07/28/2016 02:39 PM, Paolo Lucente wrote:

Hi Steve,

Try setting 'nfacctd_time_new: true' which would take as reference
time of arrival of the flow to the collector; you should get your
desired behaviour. Another solution is to keep nfacctd_time_new to
false and decrease to the minimum the active timeout on your NetFlow
exporter (what is happening now is that some long-lived flows is
being trapped at the exporter long time before being exported to
the collector).

Cheers,
Paolo

On Wed, Jul 27, 2016 at 11:30:47AM -0400, Stephen Clark wrote:

Hi List,

Maybe someone can point out what I am doing wrong. I am trying to
get nfacctd to only do inserts and not do updates
but my data looks like it is still doing updates, see row from pgsql below:
tag | ip_src  | ip_dst  | port_src | port_dst |
ip_proto | tos | packets |   bytes   |   stamp_inserted|
stamp_updated|   id| agent_id
-+-+-+--+--+--+-+-+---+-+-+-+--
   0 | 172.24.110.112  | 19x.xx.xxx.xx   |60391 | 443 |6
|   0 |   8 |   328 | 2016-07-27 10:55:00 | 2016-07-27
11:10:01 | 1313720 |  246

Notice stamp_inserted and stamp_updated - I would expect them to be
the same if the pgsql plugin was only doing inserts.

Here is my config.

daemonize: true
debug: false
pidfile: /var/run/nfacctd.pid
syslog: daemon
!logfile: /home/arodriguez/pmacct/pmacct-1.5.3/logfile
pre_tag_map: ./my.pretag.map
nfacctd_disable_checks: false

nfacctd_time_new: false

aggregate: tag, src_host, dst_host, src_port, dst_port, proto, tos


plugin_pipe_size: 4096000
plugin_buffer_size: 4096

plugins: pgsql

sql_table: acct_uni_custom
sql_data: typed

!sql_multi_values: 512000
sql_dont_try_update: true
sql_use_copy: true
sql_db: pmacct
sql_host: 127.0.0.1
sql_passwd: arealsmartpwd
sql_user: pmacct
sql_refresh_time: 300
sql_optimize_clauses: true
sql_history: 5m
sql_history_roundoff: m
sql_recovery_logfile: /var/lib/pmacct/recovery_log
!sql_table_version: 9
sql_preprocess: qnum=1000, minp=5
sql_locking_style: row
sql_cache_entries: 19

imt_buckets: 65537
imt_mem_pools_size: 1024000

nfacctd_port: 2055
!nfacctd_ip: 127.0.0.1
!nfacctd_time_new: true
!nfacctd_allow_file: /etc/pmacct/allow

Any clarification would be appreciated.

Thanks,
Steve

___
pmacct-discussion mailing list
http://www.pmacct.net/#mailinglists



--

"They that give up essential liberty to obtain temporary safety,
deserve neither liberty nor safety."  (Ben Franklin)

"The course of history shows that as a government grows, liberty
decreases."  (Thomas Jefferson)



___
pmacct-discussion mailing list
http://www.pmacct.net/#mailinglists


[pmacct-discussion] pgsql insert only on version 1.5.3

2016-07-27 Thread Stephen Clark

Hi List,

Maybe someone can point out what I am doing wrong. I am trying to get nfacctd to 
only do inserts and not do updates

but my data looks like it is still doing updates, see row from pgsql below:
tag | ip_src  | ip_dst  | port_src | port_dst | ip_proto | tos | 
packets |   bytes   |   stamp_inserted| stamp_updated|   id| agent_id

-+-+-+--+--+--+-+-+---+-+-+-+--
  0 | 172.24.110.112  | 19x.xx.xxx.xx   |60391 | 443 |6 |   0 
|   8 |   328 | 2016-07-27 10:55:00 | 2016-07-27 11:10:01 | 1313720 
|  246


Notice stamp_inserted and stamp_updated - I would expect them to be the same if 
the pgsql plugin was only doing inserts.


Here is my config.

daemonize: true
debug: false
pidfile: /var/run/nfacctd.pid
syslog: daemon
!logfile: /home/arodriguez/pmacct/pmacct-1.5.3/logfile
pre_tag_map: ./my.pretag.map
nfacctd_disable_checks: false

nfacctd_time_new: false

aggregate: tag, src_host, dst_host, src_port, dst_port, proto, tos


plugin_pipe_size: 4096000
plugin_buffer_size: 4096

plugins: pgsql

sql_table: acct_uni_custom
sql_data: typed

!sql_multi_values: 512000
sql_dont_try_update: true
sql_use_copy: true
sql_db: pmacct
sql_host: 127.0.0.1
sql_passwd: arealsmartpwd
sql_user: pmacct
sql_refresh_time: 300
sql_optimize_clauses: true
sql_history: 5m
sql_history_roundoff: m
sql_recovery_logfile: /var/lib/pmacct/recovery_log
!sql_table_version: 9
sql_preprocess: qnum=1000, minp=5
sql_locking_style: row
sql_cache_entries: 19

imt_buckets: 65537
imt_mem_pools_size: 1024000

nfacctd_port: 2055
!nfacctd_ip: 127.0.0.1
!nfacctd_time_new: true
!nfacctd_allow_file: /etc/pmacct/allow

Any clarification would be appreciated.

Thanks,
Steve

___
pmacct-discussion mailing list
http://www.pmacct.net/#mailinglists