[pmacct-discussion] nDPI warning

2020-07-23 Thread Steve Clark

Hi Paolo,

just sending this because the message said to.

pmacctd[26649]: WARN ( default/core ): nDPI support for fragmented traffic not 
implemented. If you see this message, please get in touch: Paolo Lucente 


Regards,
Steve
Email Confidentiality Notice: The information contained in this transmission 
may contain privileged and confidential and/or protected health information 
(PHI) and may be subject to protection under the law, including the Health 
Insurance Portability and Accountability Act of 1996, as amended (HIPAA). This 
transmission is intended for the sole use of the individual or entity to whom 
it is addressed. If you are not the intended recipient, you are notified that 
any use, dissemination, distribution, printing or copying of this transmission 
is strictly prohibited and may subject you to criminal or civil penalties. If 
you have received this transmission in error, please contact the sender 
immediately and delete this email and any attachments from any computer. Vaso 
Corporation and its subsidiary companies are not responsible for data leaks 
that result from email messages received that contain privileged and 
confidential and/or protected health information (PHI).

___
pmacct-discussion mailing list
http://www.pmacct.net/#mailinglists


Re: [pmacct-discussion] master - ndpi on 32bit CentOS 6

2020-07-10 Thread Steve Clark

Hi Paolo,

could you send me the output of ldd pmacctd so I can see the versions of the 
libraries being used?
also the ouput of pmacctd -V


Thanks,
Steve

On 07/09/2020 03:06 PM, Paolo Lucente wrote:

I did test on a Debian 10:

4.19.0-8-686-pae #1 SMP Debian 4.19.98-1 (2020-01-26) i686 GNU/Linux

As i was suspecting, passing the pcap you sent me through a daemon
compiled on this box went fine (that is, i can't reproduce the issue).
From what i see, by the way, this is not something related to nDPI.

Paolo

On 09/07/2020 18:19, Steve Clark wrote:


Thanks for checking, could you tell what distro and version you tested on?

Also when I compile on 32 bit I get a lot of warning of redefines
between ndpi.h and pmacct.h
do you get those also?




On 07/09/2020 11:55 AM, Paolo Lucente wrote:


Hi Steve,

I do have avail of a i686-based VM. I can't say everything is tested on
i686 but i tend to check every now and then that nothing fundamental is
broken. I took the example config you used, compiled master code with
the same config switches as you did (essentially --enable-ndpi) and had
no joy reproducing the issue.

You could send me privately your capture and i may try with that one
(although i am not highly positive it will be a successful test); or you
could arrange me access to your box to read the pcap. Let me know.

Paolo

On 09/07/2020 14:54, Steve Clark wrote:


Hi Paolo,

I have compiled master with nDPI on both 32bit and 64bit CentOS 6
systems. The 64 bit pmacctd seems
to work fine. But I get bogus byte counts when I run the 32bit version
against the same pcap file.

Just wondered if you have done any testing on 32bit intel system with
the above combination.

below is the output when using 32bit pmacctd - first the pmacctd
invocation then the nfacctd output
pmacct/src/pmacctd -f ./mypaolo.conf -I v1.7.5_v9_ndpi_class_paolo.pcap
INFO ( default/core ): Promiscuous Mode Accounting Daemon, pmacctd
1.7.6-git (20200707-01)
INFO ( default/core ):  '--enable-ndpi'
'--with-ndpi-static-lib=/usr/local/lib/' '--enable-l2'
'--enable-traffic-bins' '--enable-bgp-bins' '--enable-bmp-bins'
'--enable-st-bins'
INFO ( default/core ): Reading configuration file
'/var/lib/pgsql/sclark/mypaolo.conf'.
INFO ( p4p1/nfprobe ): NetFlow probe plugin is originally based on
softflowd 0.9.7 software, Copyright 2002 Damien Miller 
<mailto:d...@mindrot.org>
All rights reserved.
INFO ( p4p1/nfprobe ):   TCP timeout: 3600s
INFO ( p4p1/nfprobe ):  TCP post-RST timeout: 120s
INFO ( p4p1/nfprobe ):  TCP post-FIN timeout: 300s
INFO ( p4p1/nfprobe ):   UDP timeout: 300s
INFO ( p4p1/nfprobe ):  ICMP timeout: 300s
INFO ( p4p1/nfprobe ):   General timeout: 3600s
INFO ( p4p1/nfprobe ):  Maximum lifetime: 604800s
INFO ( p4p1/nfprobe ):   Expiry interval: 60s
INFO ( default/core ): PCAP capture file, sleeping for 2 seconds
INFO ( p4p1/nfprobe ): Exporting flows to [172.24.109.157]:rrac
WARN ( p4p1/nfprobe ): Shutting down on user request.
INFO ( default/core ): OK, Exiting ...

src/nfacctd -f examples/nfacctd-print.conf.example
INFO ( default/core ): NetFlow Accounting Daemon, nfacctd 1.7.6-git
(20200623-00)
INFO ( default/core ):  '--enable-ndpi'
'--with-ndpi-static-lib=/usr/local/lib/' '--enable-l2'
'--enable-traffic-bins' '--enable-bgp-bins' '--enable-bmp-bins'
'--enable-st-bins'
INFO ( default/core ): Reading configuration file
'/var/lib/pgsql/sclark/pmacct/examples/nfacctd-print.conf.example'.
INFO ( default/core ): waiting for NetFlow/IPFIX data on :::5678
INFO ( foo/print ): cache entries=16411 base cache memory=56322552 bytes
WARN ( foo/print ): no print_output_file and no print_output_lock_file
defined.
INFO ( foo/print ): *** Purging cache - START (PID: 21926) ***
CLASS SRC_IP
DST_IP SRC_PORT  DST_PORT
PROTOCOLPACKETS   BYTES
NetFlow   172.24.110.104
172.24.109.247 41900 2055
udp 26 1576253010996
NetFlow   172.24.110.104
172.24.109.247 58131 2055
udp 211576253008620
INFO ( foo/print ): *** Purging cache - END (PID: 21926, QN: 2/2, ET:
0) ***
^CINFO ( foo/print ): *** Purging cache - START (PID: 21559) ***
INFO ( foo/print ): *** Purging cache - END (PID: 21559, QN: 0/0, ET:
X) ***
INFO ( default/core ): OK, Exiting ...

Now the output when using and the same .pcap file 64bit version of
pmacctd

sudo /root/pmacctd-176 -f ./mypaolo.conf -I
v1.7.5_v9_ndpi_class_paolo.pcap
INFO ( default/core ): Promiscuous Mode Accounting Daemon, pmacctd
1.7.6-git (20200623-00)
INFO ( default/core ):  '--enable-ndpi'
'--with-ndpi-static-lib=/usr/local/lib/' '--enable-l2'
'--enable-traffic-bins' '--enable-bgp-bins' '--enable-bmp-bins'
'--enable-st-bins'
INFO ( default/core ): Reading configuration file
'/var/lib/pgsql/sclark/mypaolo.conf'.
INFO ( p4p1/nfprobe ): NetFlow probe plugin is originally based on
softflowd 0.9.7 so

Re: [pmacct-discussion] master - ndpi on 32bit CentOS 6

2020-07-09 Thread Steve Clark

Thanks for checking, could you tell what distro and version you tested on?

Also when I compile on 32 bit I get a lot of warning of redefines between 
ndpi.h and pmacct.h
do you get those also?




On 07/09/2020 11:55 AM, Paolo Lucente wrote:

Hi Steve,

I do have avail of a i686-based VM. I can't say everything is tested on
i686 but i tend to check every now and then that nothing fundamental is
broken. I took the example config you used, compiled master code with
the same config switches as you did (essentially --enable-ndpi) and had
no joy reproducing the issue.

You could send me privately your capture and i may try with that one
(although i am not highly positive it will be a successful test); or you
could arrange me access to your box to read the pcap. Let me know.

Paolo

On 09/07/2020 14:54, Steve Clark wrote:

Hi Paolo,

I have compiled master with nDPI on both 32bit and 64bit CentOS 6
systems. The 64 bit pmacctd seems
to work fine. But I get bogus byte counts when I run the 32bit version
against the same pcap file.

Just wondered if you have done any testing on 32bit intel system with
the above combination.

below is the output when using 32bit pmacctd - first the pmacctd
invocation then the nfacctd output
pmacct/src/pmacctd -f ./mypaolo.conf -I v1.7.5_v9_ndpi_class_paolo.pcap
INFO ( default/core ): Promiscuous Mode Accounting Daemon, pmacctd
1.7.6-git (20200707-01)
INFO ( default/core ):  '--enable-ndpi'
'--with-ndpi-static-lib=/usr/local/lib/' '--enable-l2'
'--enable-traffic-bins' '--enable-bgp-bins' '--enable-bmp-bins'
'--enable-st-bins'
INFO ( default/core ): Reading configuration file
'/var/lib/pgsql/sclark/mypaolo.conf'.
INFO ( p4p1/nfprobe ): NetFlow probe plugin is originally based on
softflowd 0.9.7 software, Copyright 2002 Damien Miller 
All rights reserved.
INFO ( p4p1/nfprobe ):   TCP timeout: 3600s
INFO ( p4p1/nfprobe ):  TCP post-RST timeout: 120s
INFO ( p4p1/nfprobe ):  TCP post-FIN timeout: 300s
INFO ( p4p1/nfprobe ):   UDP timeout: 300s
INFO ( p4p1/nfprobe ):  ICMP timeout: 300s
INFO ( p4p1/nfprobe ):   General timeout: 3600s
INFO ( p4p1/nfprobe ):  Maximum lifetime: 604800s
INFO ( p4p1/nfprobe ):   Expiry interval: 60s
INFO ( default/core ): PCAP capture file, sleeping for 2 seconds
INFO ( p4p1/nfprobe ): Exporting flows to [172.24.109.157]:rrac
WARN ( p4p1/nfprobe ): Shutting down on user request.
INFO ( default/core ): OK, Exiting ...

src/nfacctd -f examples/nfacctd-print.conf.example
INFO ( default/core ): NetFlow Accounting Daemon, nfacctd 1.7.6-git
(20200623-00)
INFO ( default/core ):  '--enable-ndpi'
'--with-ndpi-static-lib=/usr/local/lib/' '--enable-l2'
'--enable-traffic-bins' '--enable-bgp-bins' '--enable-bmp-bins'
'--enable-st-bins'
INFO ( default/core ): Reading configuration file
'/var/lib/pgsql/sclark/pmacct/examples/nfacctd-print.conf.example'.
INFO ( default/core ): waiting for NetFlow/IPFIX data on :::5678
INFO ( foo/print ): cache entries=16411 base cache memory=56322552 bytes
WARN ( foo/print ): no print_output_file and no print_output_lock_file
defined.
INFO ( foo/print ): *** Purging cache - START (PID: 21926) ***
CLASS SRC_IP
DST_IP SRC_PORT  DST_PORT
PROTOCOLPACKETS   BYTES
NetFlow   172.24.110.104
172.24.109.247 41900 2055
udp 26 1576253010996
NetFlow   172.24.110.104
172.24.109.247 58131 2055
udp 211576253008620
INFO ( foo/print ): *** Purging cache - END (PID: 21926, QN: 2/2, ET: 0) ***
^CINFO ( foo/print ): *** Purging cache - START (PID: 21559) ***
INFO ( foo/print ): *** Purging cache - END (PID: 21559, QN: 0/0, ET: X) ***
INFO ( default/core ): OK, Exiting ...

Now the output when using and the same .pcap file 64bit version of pmacctd

sudo /root/pmacctd-176 -f ./mypaolo.conf -I v1.7.5_v9_ndpi_class_paolo.pcap
INFO ( default/core ): Promiscuous Mode Accounting Daemon, pmacctd
1.7.6-git (20200623-00)
INFO ( default/core ):  '--enable-ndpi'
'--with-ndpi-static-lib=/usr/local/lib/' '--enable-l2'
'--enable-traffic-bins' '--enable-bgp-bins' '--enable-bmp-bins'
'--enable-st-bins'
INFO ( default/core ): Reading configuration file
'/var/lib/pgsql/sclark/mypaolo.conf'.
INFO ( p4p1/nfprobe ): NetFlow probe plugin is originally based on
softflowd 0.9.7 software, Copyright 2002 Damien Miller 
All rights reserved.
INFO ( default/core ): PCAP capture file, sleeping for 2 seconds
INFO ( p4p1/nfprobe ):   TCP timeout: 3600s
INFO ( p4p1/nfprobe ):  TCP post-RST timeout: 120s
INFO ( p4p1/nfprobe ):  TCP post-FIN timeout: 300s
INFO ( p4p1/nfprobe ):   UDP timeout: 300s
INFO ( p4p1/nfprobe ):  ICMP timeout: 300s
INFO ( p4p1/nfprobe ):   General timeout: 3600s
INFO ( p4p1/nfprobe ):  Maximum lifetime: 604800s
INFO ( p4p1/nfprobe ):   Expiry interval: 60s
INFO ( p4p1/nfprobe ): Exporting flows to [172.24.109.157

[pmacct-discussion] master - ndpi on 32bit CentOS 6

2020-07-09 Thread Steve Clark

Hi Paolo,

I have compiled master with nDPI on both 32bit and 64bit CentOS 6 systems. The 
64 bit pmacctd seems
to work fine. But I get bogus byte counts when I run the 32bit version against 
the same pcap file.

Just wondered if you have done any testing on 32bit intel system with the above 
combination.

below is the output when using 32bit pmacctd - first the pmacctd invocation 
then the nfacctd output
pmacct/src/pmacctd -f ./mypaolo.conf -I v1.7.5_v9_ndpi_class_paolo.pcap
INFO ( default/core ): Promiscuous Mode Accounting Daemon, pmacctd 1.7.6-git 
(20200707-01)
INFO ( default/core ):  '--enable-ndpi' 
'--with-ndpi-static-lib=/usr/local/lib/' '--enable-l2' '--enable-traffic-bins' 
'--enable-bgp-bins' '--enable-bmp-bins' '--enable-st-bins'
INFO ( default/core ): Reading configuration file 
'/var/lib/pgsql/sclark/mypaolo.conf'.
INFO ( p4p1/nfprobe ): NetFlow probe plugin is originally based on softflowd 0.9.7 
software, Copyright 2002 Damien Miller 
 All rights reserved.
INFO ( p4p1/nfprobe ):   TCP timeout: 3600s
INFO ( p4p1/nfprobe ):  TCP post-RST timeout: 120s
INFO ( p4p1/nfprobe ):  TCP post-FIN timeout: 300s
INFO ( p4p1/nfprobe ):   UDP timeout: 300s
INFO ( p4p1/nfprobe ):  ICMP timeout: 300s
INFO ( p4p1/nfprobe ):   General timeout: 3600s
INFO ( p4p1/nfprobe ):  Maximum lifetime: 604800s
INFO ( p4p1/nfprobe ):   Expiry interval: 60s
INFO ( default/core ): PCAP capture file, sleeping for 2 seconds
INFO ( p4p1/nfprobe ): Exporting flows to [172.24.109.157]:rrac
WARN ( p4p1/nfprobe ): Shutting down on user request.
INFO ( default/core ): OK, Exiting ...

src/nfacctd -f examples/nfacctd-print.conf.example
INFO ( default/core ): NetFlow Accounting Daemon, nfacctd 1.7.6-git 
(20200623-00)
INFO ( default/core ):  '--enable-ndpi' 
'--with-ndpi-static-lib=/usr/local/lib/' '--enable-l2' '--enable-traffic-bins' 
'--enable-bgp-bins' '--enable-bmp-bins' '--enable-st-bins'
INFO ( default/core ): Reading configuration file 
'/var/lib/pgsql/sclark/pmacct/examples/nfacctd-print.conf.example'.
INFO ( default/core ): waiting for NetFlow/IPFIX data on :::5678
INFO ( foo/print ): cache entries=16411 base cache memory=56322552 bytes
WARN ( foo/print ): no print_output_file and no print_output_lock_file defined.
INFO ( foo/print ): *** Purging cache - START (PID: 21926) ***
CLASS SRC_IP DST_IP 
SRC_PORT  DST_PORT  PROTOCOLPACKETS 
  BYTES
NetFlow   172.24.110.104 172.24.109.247 
41900 2055  udp 26  
  1576253010996
NetFlow   172.24.110.104 172.24.109.247 
58131 2055  udp 21  
  1576253008620
INFO ( foo/print ): *** Purging cache - END (PID: 21926, QN: 2/2, ET: 0) ***
^CINFO ( foo/print ): *** Purging cache - START (PID: 21559) ***
INFO ( foo/print ): *** Purging cache - END (PID: 21559, QN: 0/0, ET: X) ***
INFO ( default/core ): OK, Exiting ...

Now the output when using and the same .pcap file 64bit version of pmacctd

sudo /root/pmacctd-176 -f ./mypaolo.conf -I v1.7.5_v9_ndpi_class_paolo.pcap
INFO ( default/core ): Promiscuous Mode Accounting Daemon, pmacctd 1.7.6-git 
(20200623-00)
INFO ( default/core ):  '--enable-ndpi' 
'--with-ndpi-static-lib=/usr/local/lib/' '--enable-l2' '--enable-traffic-bins' 
'--enable-bgp-bins' '--enable-bmp-bins' '--enable-st-bins'
INFO ( default/core ): Reading configuration file 
'/var/lib/pgsql/sclark/mypaolo.conf'.
INFO ( p4p1/nfprobe ): NetFlow probe plugin is originally based on softflowd 0.9.7 
software, Copyright 2002 Damien Miller 
 All rights reserved.
INFO ( default/core ): PCAP capture file, sleeping for 2 seconds
INFO ( p4p1/nfprobe ):   TCP timeout: 3600s
INFO ( p4p1/nfprobe ):  TCP post-RST timeout: 120s
INFO ( p4p1/nfprobe ):  TCP post-FIN timeout: 300s
INFO ( p4p1/nfprobe ):   UDP timeout: 300s
INFO ( p4p1/nfprobe ):  ICMP timeout: 300s
INFO ( p4p1/nfprobe ):   General timeout: 3600s
INFO ( p4p1/nfprobe ):  Maximum lifetime: 604800s
INFO ( p4p1/nfprobe ):   Expiry interval: 60s
INFO ( p4p1/nfprobe ): Exporting flows to [172.24.109.157]:rrac
WARN ( p4p1/nfprobe ): Shutting down on user request.
INFO ( default/core ): OK, Exiting ...

src/nfacctd -f examples/nfacctd-print.conf.example
INFO ( default/core ): NetFlow Accounting Daemon, nfacctd 1.7.6-git 
(20200623-00)
INFO ( default/core ):  '--enable-ndpi' 
'--with-ndpi-static-lib=/usr/local/lib/' '--enable-l2' '--enable-traffic-bins' 
'--enable-bgp-bins' '--enable-bmp-bins' '--enable-st-bins'
INFO ( default/core ): Reading configuration file 
'/var/lib/pgsql/sclark/pmacct/examples/nfacctd-print.conf.example'.
INFO ( default/core ): waiting for NetFlow/IPFIX data on :::5678
INFO ( foo/print ): 

Re: [pmacct-discussion] fabric path header patch not in master

2020-06-25 Thread Steve Clark

Sorry for the noise - my git foo it not that good.

Recloned everything and my patches are in master.

Oh it turns out I was on the wrong system - DUH.


On 06/25/2020 02:03 PM, Steve Clark wrote:

Hi Paolo,

Is there a reason my CFP patch is in 1.7.5 but not in master?

Thanks,
Steve


___
pmacct-discussion mailing list
http://www.pmacct.net/#mailinglists



Email Confidentiality Notice: The information contained in this transmission 
may contain privileged and confidential and/or protected health information 
(PHI) and may be subject to protection under the law, including the Health 
Insurance Portability and Accountability Act of 1996, as amended (HIPAA). This 
transmission is intended for the sole use of the individual or entity to whom 
it is addressed. If you are not the intended recipient, you are notified that 
any use, dissemination, distribution, printing or copying of this transmission 
is strictly prohibited and may subject you to criminal or civil penalties. If 
you have received this transmission in error, please contact the sender 
immediately and delete this email and any attachments from any computer. Vaso 
Corporation and its subsidiary companies are not responsible for data leaks 
that result from email messages received that contain privileged and 
confidential and/or protected health information (PHI).

___
pmacct-discussion mailing list
http://www.pmacct.net/#mailinglists


[pmacct-discussion] fabric path header patch not in master

2020-06-25 Thread Steve Clark

Hi Paolo,

Is there a reason my CFP patch is in 1.7.5 but not in master?

Thanks,
Steve
Email Confidentiality Notice: The information contained in this transmission 
may contain privileged and confidential and/or protected health information 
(PHI) and may be subject to protection under the law, including the Health 
Insurance Portability and Accountability Act of 1996, as amended (HIPAA). This 
transmission is intended for the sole use of the individual or entity to whom 
it is addressed. If you are not the intended recipient, you are notified that 
any use, dissemination, distribution, printing or copying of this transmission 
is strictly prohibited and may subject you to criminal or civil penalties. If 
you have received this transmission in error, please contact the sender 
immediately and delete this email and any attachments from any computer. Vaso 
Corporation and its subsidiary companies are not responsible for data leaks 
that result from email messages received that contain privileged and 
confidential and/or protected health information (PHI).

___
pmacct-discussion mailing list
http://www.pmacct.net/#mailinglists


Re: [pmacct-discussion] 1.7.5 with static ndpi

2020-06-25 Thread Steve Clark

Thanks for the info Marc.

On 06/25/2020 08:50 AM, Marc Sune wrote:
Steve,

Missatge de Stephen Clark 
mailto:sclar...@earthlink.net>> del dia dj., 25 de juny 
2020 a les 13:56:
Hi Paolo,

We have pmacct installed on a number of remote systems and
it just more moving parts to keep updated with having to also install/update 
nDPI.

Not sure what your requirements are, but if the concern is remote system's 
connectivity, docker images can be downloaded in a tar.gz and locally imported, 
without the need to have external connectivity.

https://docs.docker.com/engine/reference/commandline/save/


Also I have used the following configure line

./configure '--enable-ndpi' --with-ndpi-static-lib=/usr/local/lib/ '--enable-l2'
'--enable-traffic-bins' '--enable-bgp-bins' '--enable-bmp-bins' 
'--enable-st-bins'

and still get a dynamically linked pmacctd. Also the dynamic lib and static lib
are both in /usr/local/lib

Can you the resulting config.log?

marc


I just removed the dynamic libs and got pmacctd built - I am testing it now.

Thanks for your help,
Steve


On 6/24/20 4:30 PM, Paolo Lucente wrote:

Hi Steve,

Apart from asking the obvious - personal curiosity! - why do you want to
link against a static nDPI library. There are a couple main avenues i
can point you to depending on your goal:

1) You can supply configure with a --with-ndpi-static-lib knob; guess
the static lib and the dynamic lib are in different places, you should
be game. Even simplifying further: should you make the 'shared object'
library disappear then things will be forced onto the static library;

2) did you see the "pmacct & Docker" email that did just circulate on
the list? In the seek for a static library? Perhaps time to look into a
container instead? :-D

Paolo

On Tue, Jun 23, 2020 at 01:44:32PM -0400, Stephen Clark wrote:

Hello,

Can anyone give the magic configuration items I need to build using a static
libndpi.a

I have spend all day trying to do this without any success. It seem like I
tried every combination
that ./configure --help displays.

Any help would be appreciated.

Thanks,
Steve


___
pmacct-discussion mailing list
http://www.pmacct.net/#mailinglists



--

"They that give up essential liberty to obtain temporary safety,
deserve neither liberty nor safety."  (Ben Franklin)

"The course of history shows that as a government grows, liberty
decreases."  (Thomas Jefferson)

"Beer is proof God loves us and wants us to be happy!" (Ben Franklin)


___
pmacct-discussion mailing list
http://www.pmacct.net/#mailinglists



___
pmacct-discussion mailing list
http://www.pmacct.net/#mailinglists


Email Confidentiality Notice: The information contained in this transmission 
may contain privileged and confidential and/or protected health information 
(PHI) and may be subject to protection under the law, including the Health 
Insurance Portability and Accountability Act of 1996, as amended (HIPAA). This 
transmission is intended for the sole use of the individual or entity to whom 
it is addressed. If you are not the intended recipient, you are notified that 
any use, dissemination, distribution, printing or copying of this transmission 
is strictly prohibited and may subject you to criminal or civil penalties. If 
you have received this transmission in error, please contact the sender 
immediately and delete this email and any attachments from any computer. Vaso 
Corporation and its subsidiary companies are not responsible for data leaks 
that result from email messages received that contain privileged and 
confidential and/or protected health information (PHI).
___
pmacct-discussion mailing list
http://www.pmacct.net/#mailinglists


[pmacct-discussion] Fwd: minb - version 1.5.3

2016-08-02 Thread Steve Clark


Hi Paolo,

I am trying to limit netflow aggregates to greater than 100 bytes before 
insertion into my PG database, but

I can't seem to get it to work. All aggregates are being inserted.

my config:
daemonize: true
debug: false
pidfile: /var/run/nfacctd.pid
syslog: daemon
pre_tag_map: ./my.pretag.map
nfacctd_disable_checks: true
nfacctd_time_new: false
aggregate: tag, src_host, dst_host, src_port, dst_port, proto, tos
plugin_pipe_size: 4096000
plugin_buffer_size: 4096
plugins: pgsql
sql_table: netflow
sql_data: typed
sql_dont_try_update: true
sql_use_copy: true
sql_db: pmacct
sql_host: 127.0.0.1
sql_passwd: arealsmartpwd
sql_user: pmacct
sql_refresh_time: 60
sql_optimize_clauses: true
sql_history: 1m
sql_history_roundoff: m
sql_recovery_logfile: /var/lib/pmacct/recovery_log
sql_preprocess: minb>=100
sql_locking_style: row
sql_cache_entries: 19
imt_buckets: 65537
imt_mem_pools_size: 1024000
nfacctd_port: 2055

Here is what I get in my table - notice 1173 under 100 bytes.
pmacct=# truncate netflow ;
TRUNCATE TABLE
pmacct=# select count(*),sum(bytes)as bytes,sum(packets)as packets from netflow 
where agent_id = '246' and bytes <100;

 count | bytes | packets
---+---+-
  1173 | 89321 |1205
(1 row)

pmacct=# select count(*),sum(bytes)as bytes,sum(packets)as packets from netflow 
where agent_id = '246';

count |  bytes   | packets
---+--+-
3690 | 63424928 | 105921
(1 row)

Also this is what shows from /var/log/messages
Aug  2 08:06:01 netflow nfacctd[4073]: INFO ( default/pgsql ): *** Purging cache 
- START (PID: 4073) ***
Aug  2 08:06:01 netflow nfacctd[4073]: INFO ( default/pgsql ): *** Purging cache 
- END (PID: 4073, QN: 3690/3690, ET: 0) ***


   KEY: minb
DESC: check. Aggregates on the queue are evaluated one-by-one; 
each object is marked valid
only if the bytes counter is '>=' minb value. An interesting idea is to set its 
value
  to a fraction of the link capacity. Remember that you 
have also a timeframe reference:

  the 'sql_refresh_time' seconds. All plugins.

  For example, given the following parameters:
  Link Capacity = 8Mbit/s, THreshold = 0.1%, TImeframe = 60s
  minb = ((LC / 8) * TI) * TH -> ((8Mbit/s / 8) * 60s) * 
0.1% = 6 bytes.


  Given a 8Mbit link, all aggregates which have accounted 
for at least 60Kb of traffic

  in the last 60 seconds, will be written to the DB.



Any suggestions?

Thanks,
Steve

___
pmacct-discussion mailing list
http://www.pmacct.net/#mailinglists

[pmacct-discussion] Out of Office. (was: Out of Office. (was: Out of Office. (was: Out of Office. (was: Nfacct - Missing src_port, and dst_port))))

2016-04-13 Thread Steve Clark

  
  
Sorry, I will out of the office til 4/21/16.

-- 
  Stephen=C2=A0Clark
  NetWolves Managed Services, LLC.
  Director=C2=A0of=C2=A0Technology
  Phone:=C2=A0813-579-3200
  Fax:=C2=A0813-882-0209
  Email:=C2=A0steve.clark@netwolves.c=
om
  http://www.netwolves.com

  



___
pmacct-discussion mailing list
http://www.pmacct.net/#mailinglists

[pmacct-discussion] Out of Office. (was: Out of Office. (was: Out of Office. (was: Nfacct - Missing src_port, and dst_port)))

2016-04-13 Thread Steve Clark

  
  
Sorry, I will out of the office til 4/21/16.

-- 
  Stephen=C2=A0Clark
  NetWolves Managed Services, LLC.
  Director=C2=A0of=C2=A0Technology
  Phone:=C2=A0813-579-3200
  Fax:=C2=A0813-882-0209
  Email:=C2=A0steve.clark@netwolves.c=
om
  http://www.netwolves.com

  



___
pmacct-discussion mailing list
http://www.pmacct.net/#mailinglists

[pmacct-discussion] Out of Office. (was: Out of Office. (was: Nfacct - Missing src_port, and dst_port))

2016-04-13 Thread Steve Clark

  
  
Sorry, I will out of the office til 4/21/16.

-- 
  Stephen=C2=A0Clark
  NetWolves Managed Services, LLC.
  Director=C2=A0of=C2=A0Technology
  Phone:=C2=A0813-579-3200
  Fax:=C2=A0813-882-0209
  Email:=C2=A0steve.clark@netwolves.c=
om
  http://www.netwolves.com

  



___
pmacct-discussion mailing list
http://www.pmacct.net/#mailinglists

[pmacct-discussion] Out of Office. (was: Nfacct - Missing src_port, and dst_port)

2016-04-13 Thread Steve Clark

  
  
Sorry, I will out of the office til 4/21/16.

-- 
  Stephen=C2=A0Clark
  NetWolves Managed Services, LLC.
  Director=C2=A0of=C2=A0Technology
  Phone:=C2=A0813-579-3200
  Fax:=C2=A0813-882-0209
  Email:=C2=A0steve.clark@netwolves.c=
om
  http://www.netwolves.com

  



___
pmacct-discussion mailing list
http://www.pmacct.net/#mailinglists

Re: [pmacct-discussion] netflow v9 ifindex always 0 - pmacct version 1.5.1

2015-07-17 Thread Steve Clark

Hi Paolo,

On 07/17/2015 01:58 AM, Paolo Lucente wrote:

Hi Steve,

libpcap does not report such info due to no integration with the
underlying OS. This is an advantage of using ULOG due to its tight
coupling to the OS. Plus, in the QUICKSTART document Quickstart
guide to setup a NetFlow agent/probe chapter it is described how
pmacct can help setting direction and interface indexes basing on
MAC or IP addresses.

In my case I just need to be able to have one value in the InputInt: and 
OutputInt:
fields, it doesn't need to be set based on any criteria. I have read both the 
CONFIG-KEYS
and the QUICKSTART guide, though I am not sure I understand them completely.

Am I not able to simply put something like:

interface: p4p1
aggregate: src_host, dst_host, src_port, dst_port, proto, tos, in_iface, 
out_iface
plugins: nfprobe[p4p1]
nfprobe_receiver: 10.0.129.71:2055
nfprobe_version: 9
nfprobe_ifindex[p4p1]: 4

in my config file?

I tried to use a pre-tag filter like
nfprobe_ifindex[p4p1]: tag
pre_tag_map: ./my.pretag.map


then edited my.pretag.map as follows:
set_tag=4 filter='net 0.0.0.0/0'

and still only saw the value 0 in the InputInt: and OutputInt: fields.



Thanks for taking the time to respond and making pmacct available.




Cheers,
Paolo

On Thu, Jul 16, 2015 at 12:27:01PM -0400, Steve Clark wrote:

Hello,

I have read the discussing in this email thread:
https://www.mail-archive.com/pmacct-discussion@pmacct.net/msg02187.html
But still can't see anything but zero in the InputInt: and OutputInt: when 
looking at the exported packets with
wireshark:


Here is my simple config - could someone explain what I am doing wrong?

!
! pmacctd configuration example
!
! Did you know CONFIG-KEYS contains the detailed list of all configuration keys
! supported by 'nfacctd' and 'pmacctd' ?
!
! debug: true
daemonize: false
interface: p4p1
aggregate: src_host, dst_host, src_port, dst_port, proto, tos, in_iface, 
out_iface
plugins: nfprobe[p4p1]
nfprobe_receiver: 10.0.129.71:2055
nfprobe_version: 9
nfprobe_ifindex[p4p1]: 4
! nfprobe_engine: 1:1
! nfprobe_timeouts: tcp=120:maxlife=3600
!
! networks_file: /path/to/networks.lst
! classifiers: /path/to/classifiers/
! snaplen: 700

Startup command:

sudo ../src/pmacctd  -f ./probe_netflow.conf
INFO ( default/core ): Reading configuration file 
'/var/lib/pgsql/pmacct-1.5.1/examples/probe_netflow.conf'.
INFO ( p4p1/nfprobe ): NetFlow probe plugin is originally based on softflowd 0.9.7 
software, Copyright 2002 Damien Miller d...@mindrot.org All rights reserved.
INFO ( p4p1/nfprobe ):   TCP timeout: 3600s
INFO ( p4p1/nfprobe ):  TCP post-RST timeout: 120s
INFO ( p4p1/nfprobe ):  TCP post-FIN timeout: 300s
INFO ( p4p1/nfprobe ):   UDP timeout: 300s
INFO ( p4p1/nfprobe ):  ICMP timeout: 300s
INFO ( p4p1/nfprobe ):   General timeout: 3600s
INFO ( p4p1/nfprobe ):  Maximum lifetime: 604800s
INFO ( p4p1/nfprobe ):   Expiry interval: 60s
INFO ( p4p1/nfprobe ): Exporting flows to [10.0.129.71]:iop
OK ( default/core ): link type is: 1
WARN ( default/core ): p4p1: no IPv4 address assigned
^CWARN ( p4p1/nfprobe ): Shutting down on user request.
OK: Exiting ...

Thanks,

--
Stephen Clark

___
pmacct-discussion mailing list
http://www.pmacct.net/#mailinglists


___
pmacct-discussion mailing list
http://www.pmacct.net/#mailinglists




--
Stephen Clark
*NetWolves Managed Services, LLC.*
Director of Technology
Phone: 813-579-3200
Fax: 813-882-0209
Email: steve.cl...@netwolves.com
http://www.netwolves.com
___
pmacct-discussion mailing list
http://www.pmacct.net/#mailinglists

[pmacct-discussion] netflow v9 ifindex always 0 - pmacct version 1.5.1

2015-07-16 Thread Steve Clark

Hello,

I have read the discussing in this email thread:
https://www.mail-archive.com/pmacct-discussion@pmacct.net/msg02187.html
But still can't see anything but zero in the InputInt: and OutputInt: when 
looking at the exported packets with
wireshark:


Here is my simple config - could someone explain what I am doing wrong?

!
! pmacctd configuration example
!
! Did you know CONFIG-KEYS contains the detailed list of all configuration keys
! supported by 'nfacctd' and 'pmacctd' ?
!
! debug: true
daemonize: false
interface: p4p1
aggregate: src_host, dst_host, src_port, dst_port, proto, tos, in_iface, 
out_iface
plugins: nfprobe[p4p1]
nfprobe_receiver: 10.0.129.71:2055
nfprobe_version: 9
nfprobe_ifindex[p4p1]: 4
! nfprobe_engine: 1:1
! nfprobe_timeouts: tcp=120:maxlife=3600
!
! networks_file: /path/to/networks.lst
! classifiers: /path/to/classifiers/
! snaplen: 700

Startup command:

sudo ../src/pmacctd  -f ./probe_netflow.conf
INFO ( default/core ): Reading configuration file 
'/var/lib/pgsql/pmacct-1.5.1/examples/probe_netflow.conf'.
INFO ( p4p1/nfprobe ): NetFlow probe plugin is originally based on softflowd 0.9.7 
software, Copyright 2002 Damien Miller d...@mindrot.org All rights reserved.
INFO ( p4p1/nfprobe ):   TCP timeout: 3600s
INFO ( p4p1/nfprobe ):  TCP post-RST timeout: 120s
INFO ( p4p1/nfprobe ):  TCP post-FIN timeout: 300s
INFO ( p4p1/nfprobe ):   UDP timeout: 300s
INFO ( p4p1/nfprobe ):  ICMP timeout: 300s
INFO ( p4p1/nfprobe ):   General timeout: 3600s
INFO ( p4p1/nfprobe ):  Maximum lifetime: 604800s
INFO ( p4p1/nfprobe ):   Expiry interval: 60s
INFO ( p4p1/nfprobe ): Exporting flows to [10.0.129.71]:iop
OK ( default/core ): link type is: 1
WARN ( default/core ): p4p1: no IPv4 address assigned
^CWARN ( p4p1/nfprobe ): Shutting down on user request.
OK: Exiting ...

Thanks,

--
Stephen Clark

___
pmacct-discussion mailing list
http://www.pmacct.net/#mailinglists