Re: [pmacct-discussion] pretag.map not working when running nfacctd in a container
Hi Rich,
I was wondering if you had any log availble from nfacctd; for example,
is it possible that the file 'pretag.map' with no paths supplied is not
found, causing the issue?
Paolo
On Thu, Dec 07, 2023 at 11:34:56PM +, Compton, Rich A wrote:
> Hi, hoping that someone can help me with this issue. I am trying to run
> nfacctd in a container and I’m using a pretag.map file to filter only certain
> netflow records. When I remove the “pre_tag_map:” line and
> “pre_tag_label_filter” from the config file, I am able to export the netflow
> records to the mysql database. When I add the same config back in, I get no
> netflow records in my database.
> The same config with the pre_tag_map config seems to work when running
> nfacctd natively on the host OS.
> Anybody have any ideas what the issue is?
> Here’s a sample of my template config file:
>
> daemonize: false
> nfacctd_port: 2055
> nfacctd_time_new: true
> pre_tag_map: pretag.map
> maps_index: true
> maps_entries: 1
> plugins: mysql[dns], mysql[ntp], mysql[ssdp], mysql[snmp], mysql[chargen],
> mysql[ldap], mysql[portmap]
> aggregate: src_host, src_port, dst_host, dst_port, proto, src_as, dst_as,
> in_iface, out_iface, peer_src_ip
> pre_tag_label_filter[dns]: dns
> aggregate_filter[dns]: dst port 53
> pre_tag_label_filter[ntp]: ntp
> aggregate_filter[ntp]: dst port 123
> pre_tag_label_filter[ssdp]: ssdp
> aggregate_filter[ssdp]: dst port 1900
> pre_tag_label_filter[snmp]: snmp
> aggregate_filter[snmp]: dst port 161
> pre_tag_label_filter[chargen]: chargen
> aggregate_filter[chargen]: dst port 19
> pre_tag_label_filter[ldap]: ldap
> aggregate_filter[ldap]: dst port 389
> pre_tag_label_filter[portmap]: portmap
> aggregate_filter[portmap]: dst port 111
>
> sql_db[dns]: honeypot_feed
> sql_optimize_clauses[dns]: true
> sql_table[dns]: netflow
> sql_host[dns]: ${SQL_HOST}
> sql_passwd[dns]: ${SQL_PASSWORD}
> sql_user[dns]: ${SQL_USER}
> sql_refresh_time[dns]: 10
> sql_history[dns]: 1m
> sql_history_roundoff[dns]: mh
>
> sql_db[ntp]: honeypot_feed
> sql_optimize_clauses[ntp]: true
> sql_table[ntp]: netflow
> sql_host[ntp]: ${SQL_HOST}
> sql_passwd[ntp]: ${SQL_PASSWORD}
> sql_user[ntp]: ${SQL_USER}
> sql_refresh_time[ntp]: 10
> sql_history[ntp]: 1m
> sql_history_roundoff[ntp]: mh
>
> sql_db[snmp]: ${SQL_DATABASE}
> sql_optimize_clauses[snmp]: true
> sql_table[snmp]: netflow
> sql_host[snmp]: ${SQL_HOST}
> sql_passwd[snmp]: ${SQL_PASSWORD}
> sql_user[snmp]: ${SQL_USER}
> sql_refresh_time[snmp]: 10
> sql_history[snmp]: 1m
> sql_history_roundoff[snmp]: mh
>
> sql_db[ssdp]: ${SQL_DATABASE}
> sql_optimize_clauses[ssdp]: true
> sql_table[ssdp]: netflow
> sql_host[ssdp]: ${SQL_HOST}
> sql_passwd[ssdp]: ${SQL_PASSWORD}
> sql_user[ssdp]: ${SQL_USER}
> sql_refresh_time[ssdp]: 10
> sql_history[ssdp]: 1m
> sql_history_roundoff[ssdp]: mh
>
> sql_db[ldap]: ${SQL_DATABASE}
> sql_optimize_clauses[ldap]: true
> sql_table[ldap]: netflow
> sql_host[ldap]: ${SQL_HOST}
> sql_passwd[ldap]: ${SQL_PASSWORD}
> sql_user[ldap]: ${SQL_USER}
> sql_refresh_time[ldap]: 10
> sql_history[ldap]: 1m
> sql_history_roundoff[ldap]: mh
>
> sql_db[chargen]: ${SQL_DATABASE}
> sql_optimize_clauses[chargen]: true
> sql_table[chargen]: netflow
> sql_host[chargen]: ${SQL_HOST}
> sql_passwd[chargen]: ${SQL_PASSWORD}
> sql_user[chargen]: ${SQL_USER}
> sql_refresh_time[chargen]: 10
> sql_history[chargen]: 1m
> sql_history_roundoff[chargen]: mh
>
> sql_db[portmap]: ${SQL_DATABASE}
> sql_optimize_clauses[portmap]: true
> sql_table[portmap]: netflow
> sql_host[portmap]: ${SQL_HOST}
> sql_passwd[portmap]: ${SQL_PASSWORD}
> sql_user[portmap]: ${SQL_USER}
> sql_refresh_time[portmap]: 10
> sql_history[portmap]: 1m
> sql_history_roundoff[portmap]: mh
>
>
> ---cut-
> Example of pretag.map file:
> set_label=dns src_net=1.2.3.0/24
> set_label=ntp src_net=1.2.3.0/24
> set_label=snmp src_net=1.2.3.0/24
> set_label=ssdp src_net=1.2.3.0/24
> set_label=chargen src_net=1.2.3.0/24
> set_label=portmap src_net=1.2.3.0/24
> set_label=ldap src_net=1.2.3.0/24
>
>
>
>
> [signature_1767717039]
>
> Rich Compton| Principal Eng |314.596.2828
> 8560 Upland Drive, Suite B | Englewood, CO 80112
> ___
> pmacct-discussion mailing list
> http://www.pmacct.net/#mailinglists
___
pmacct-discussion mailing list
http://www.pmacct.net/#mailinglists
[pmacct-discussion] pretag.map not working when running nfacctd in a container
Hi, hoping that someone can help me with this issue. I am trying to run
nfacctd in a container and I’m using a pretag.map file to filter only certain
netflow records. When I remove the “pre_tag_map:” line and
“pre_tag_label_filter” from the config file, I am able to export the netflow
records to the mysql database. When I add the same config back in, I get no
netflow records in my database.
The same config with the pre_tag_map config seems to work when running nfacctd
natively on the host OS.
Anybody have any ideas what the issue is?
Here’s a sample of my template config file:
daemonize: false
nfacctd_port: 2055
nfacctd_time_new: true
pre_tag_map: pretag.map
maps_index: true
maps_entries: 1
plugins: mysql[dns], mysql[ntp], mysql[ssdp], mysql[snmp], mysql[chargen],
mysql[ldap], mysql[portmap]
aggregate: src_host, src_port, dst_host, dst_port, proto, src_as, dst_as,
in_iface, out_iface, peer_src_ip
pre_tag_label_filter[dns]: dns
aggregate_filter[dns]: dst port 53
pre_tag_label_filter[ntp]: ntp
aggregate_filter[ntp]: dst port 123
pre_tag_label_filter[ssdp]: ssdp
aggregate_filter[ssdp]: dst port 1900
pre_tag_label_filter[snmp]: snmp
aggregate_filter[snmp]: dst port 161
pre_tag_label_filter[chargen]: chargen
aggregate_filter[chargen]: dst port 19
pre_tag_label_filter[ldap]: ldap
aggregate_filter[ldap]: dst port 389
pre_tag_label_filter[portmap]: portmap
aggregate_filter[portmap]: dst port 111
sql_db[dns]: honeypot_feed
sql_optimize_clauses[dns]: true
sql_table[dns]: netflow
sql_host[dns]: ${SQL_HOST}
sql_passwd[dns]: ${SQL_PASSWORD}
sql_user[dns]: ${SQL_USER}
sql_refresh_time[dns]: 10
sql_history[dns]: 1m
sql_history_roundoff[dns]: mh
sql_db[ntp]: honeypot_feed
sql_optimize_clauses[ntp]: true
sql_table[ntp]: netflow
sql_host[ntp]: ${SQL_HOST}
sql_passwd[ntp]: ${SQL_PASSWORD}
sql_user[ntp]: ${SQL_USER}
sql_refresh_time[ntp]: 10
sql_history[ntp]: 1m
sql_history_roundoff[ntp]: mh
sql_db[snmp]: ${SQL_DATABASE}
sql_optimize_clauses[snmp]: true
sql_table[snmp]: netflow
sql_host[snmp]: ${SQL_HOST}
sql_passwd[snmp]: ${SQL_PASSWORD}
sql_user[snmp]: ${SQL_USER}
sql_refresh_time[snmp]: 10
sql_history[snmp]: 1m
sql_history_roundoff[snmp]: mh
sql_db[ssdp]: ${SQL_DATABASE}
sql_optimize_clauses[ssdp]: true
sql_table[ssdp]: netflow
sql_host[ssdp]: ${SQL_HOST}
sql_passwd[ssdp]: ${SQL_PASSWORD}
sql_user[ssdp]: ${SQL_USER}
sql_refresh_time[ssdp]: 10
sql_history[ssdp]: 1m
sql_history_roundoff[ssdp]: mh
sql_db[ldap]: ${SQL_DATABASE}
sql_optimize_clauses[ldap]: true
sql_table[ldap]: netflow
sql_host[ldap]: ${SQL_HOST}
sql_passwd[ldap]: ${SQL_PASSWORD}
sql_user[ldap]: ${SQL_USER}
sql_refresh_time[ldap]: 10
sql_history[ldap]: 1m
sql_history_roundoff[ldap]: mh
sql_db[chargen]: ${SQL_DATABASE}
sql_optimize_clauses[chargen]: true
sql_table[chargen]: netflow
sql_host[chargen]: ${SQL_HOST}
sql_passwd[chargen]: ${SQL_PASSWORD}
sql_user[chargen]: ${SQL_USER}
sql_refresh_time[chargen]: 10
sql_history[chargen]: 1m
sql_history_roundoff[chargen]: mh
sql_db[portmap]: ${SQL_DATABASE}
sql_optimize_clauses[portmap]: true
sql_table[portmap]: netflow
sql_host[portmap]: ${SQL_HOST}
sql_passwd[portmap]: ${SQL_PASSWORD}
sql_user[portmap]: ${SQL_USER}
sql_refresh_time[portmap]: 10
sql_history[portmap]: 1m
sql_history_roundoff[portmap]: mh
---cut-
Example of pretag.map file:
set_label=dns src_net=1.2.3.0/24
set_label=ntp src_net=1.2.3.0/24
set_label=snmp src_net=1.2.3.0/24
set_label=ssdp src_net=1.2.3.0/24
set_label=chargen src_net=1.2.3.0/24
set_label=portmap src_net=1.2.3.0/24
set_label=ldap src_net=1.2.3.0/24
[signature_1767717039]
Rich Compton| Principal Eng |314.596.2828
8560 Upland Drive, Suite B | Englewood, CO 80112
___
pmacct-discussion mailing list
http://www.pmacct.net/#mailinglists
Re: [pmacct-discussion] pretag.map
Hi Stephen, If you do not filter over tags, ie. pre_tag_filter is not part of your config, then all will make to the database and those packets coming from a unit not in pretag.map will have a tag of zero. In other words both behaviours are possible (all make to the DB or filter things you are not interested out) and you are in command of which one you like. Cheers, Paolo On Thu, Nov 03, 2016 at 07:56:45AM -0400, Stephen Clark wrote: > Hi, > > I am using nfacctd to write netflows to postgres DB. I have my > pretag.map setup to map the > collectors ip to a tag. My question is what happens if I get a > netflow packet from a unit where > I have added it to my pretag.map file yet. Is an error generated? Is > the tag value 0? Is the netflow > packet ignored? > > Thanks in advance, > Steve > > -- > > "They that give up essential liberty to obtain temporary safety, > deserve neither liberty nor safety." (Ben Franklin) > > "The course of history shows that as a government grows, liberty > decreases." (Thomas Jefferson) > > > > ___ > pmacct-discussion mailing list > http://www.pmacct.net/#mailinglists ___ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists
[pmacct-discussion] pretag.map
Hi, I am using nfacctd to write netflows to postgres DB. I have my pretag.map setup to map the collectors ip to a tag. My question is what happens if I get a netflow packet from a unit where I have added it to my pretag.map file yet. Is an error generated? Is the tag value 0? Is the netflow packet ignored? Thanks in advance, Steve -- "They that give up essential liberty to obtain temporary safety, deserve neither liberty nor safety." (Ben Franklin) "The course of history shows that as a government grows, liberty decreases." (Thomas Jefferson) ___ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists
Re: [pmacct-discussion] pretag.map
Hello Paolo, With 'tag' keyword in aggregate field all works fine. Thanx. Paolo Lucente wrote: Hello Dmitriy, your pretag.map looks fine to my eyes. You should instead append the 'tag' keyword to your 'aggregate' directive in order to give values to the 'agent_id' field: aggregate: src_host,dst_host,src_mac,dst_mac,src_port,dst_port,tag Let me know whether the issue get solved. Cheers, Paolo ___ pmacct-discussion mailing list pmacct-discussion@pmacct.net http://muffin.area.ba.cnr.it/mailman/listinfo/pmacct-discussion
Re: [pmacct-discussion] pretag.map
Hello Dmitriy, your pretag.map looks fine to my eyes. You should instead append the 'tag' keyword to your 'aggregate' directive in order to give values to the 'agent_id' field: aggregate: src_host,dst_host,src_mac,dst_mac,src_port,dst_port,tag Let me know whether the issue get solved. Cheers, Paolo
[pmacct-discussion] pretag.map
Is working filter with "and" or "or" rules ? I have such config: /etc/pmacct.conf ! pmacctd configuration ! ! ! daemonize: true pidfile: /var/run/pmacctd.pid syslog: daemon ! ! interested in in and outbound traffic aggregate: src_host,dst_host,src_mac,dst_mac,src_port,dst_port ! on this network pcap_filter: net 195.235.115.0/22 or net 192.168.0.0/16 ! on this interface interface: eth1 ! ports_file: /etc/pmacct/ports ! storage methods plugins: mysql sql_host: mysql.xxx.net.ua sql_db: statistic sql_table: acct_v2 sql_table_version: 2 sql_user: xxx sql_passwd: xxx pre_tag_map: /etc/pmacct/pretag.map sql_refresh_time: 300 sql_history: 5m !sql_trigger_exec: /path/to/your/trigger.script sql_dont_try_update: true ! refresh the db every minute !sql_refresh_time: 60 ! reduce the size of the insert/update clause sql_optimize_clauses: true ! accumulate values in each row for up to an hour !sql_history: 1h ! create new rows on the minute, hour, day boundaries sql_history_roundoff: mhd ! in case of emergency, log to this file !sql_recovery_logfile: /var/lib/pmacct/recovery_log /etc/pmacct/pretag.map id=3filter='src net 192.168.0.0/16 and dst net 192.168.0.0/16' id=3filter='src net 192.168.0.0/16 and dst net 195.235.115.0/22' id=3filter='src net 195.235.115.0/22 and dst net 195.235.115.0/22' id=3filter='src net 195.235.115.0/22 and dst net 192.168.0.0/16' and all work, but i haven't agent_id = 3 in mysql table. But if i use some of pretag filter in tcpdump - i have traffic with that filter.
