Hi Federico,
Sure, please let's switch to unicast email as i'd need more info and/or
some example(s).
First and foremost, in summary, what is appearing not to be working
well: labelling flow or filtering of labels? From your last email i seem
to understand that it's the latter case, are we in sync?
Paolo
On 17/1/23 11:47, Federico Urtizberea wrote:
Hi Paolo, thanks for your answer.
I followed your suggestions and the results are:
# Disable pre_tag_label_encode_as_map
I had to change the different pretag.map files to tag the flows
correctly and none were captured. I then disabled the
pre_tag_label_filter and with a kafka consumer, I filtered the labeled
flows and I was able to see the properly labeled flows.
# enable pre_tag_label_encode_as_map
By disabling pre_tag_label_filter and using a kafka consumer, I filtered
the labeled flows and I was able to see the correctly labeled flows.
The pretag.map files were changing over the days, to really only mark
the searched traffic and have a clearer configuration.
If you need more accurate data, and a flow sample, I can send you by
unicast email.
Regards,
Federico
On 16/1/23 16:27, Paolo Lucente wrote:
Hi Federico,
I see the combo pre_tag_label_filter / pre_tag_label_encode_as_map,
can you please temporarily disable the latter
(pre_tag_label_encode_as_map) and see if the filtering does work as
expected? Should it not, can you also disable the filtering and check
what you see? Are labels applied correctly?
Paolo
On 12/1/23 11:21, Federico Urtizberea wrote:
Hi everyone, after looking the previous configuration, I changed it a
bit, but so far I still can't seeing the unknown traffic.
The actual configuration, is cleaner than previous one.
# /etc/pmacct/network.lst
192.168.0.0/24
192.168.1.0/24
172.16.0.0/23
172.16.2.0/24
172.16.250.0./24
# /etc/pmacct/pretag_in.map
set_label=client%wknwnnet1 dst_net=172.16.0.0/23 jeq=eval_type
set_label=client%wknwnnet1 dst_net=172.16.2.0/24 jeq=eval_type
set_label=client%wknwnnet2 dst_net=172.16.250.0/24 jeq=eval_type
set_label=type%mynet1 src_net=192.168.0.0/23 label=eval_type
set_label=type%mynet2 src_net=192.168.2.0/24 label=eval_type
set_label=type%tip src_net=0.0.0.0/0 label=eval_type
# /etc/pmacct/pretag_out.map
set_label=client%wknwnnet1 src_net=172.16.0.0/23 jeq=eval_type
set_label=client%wknwnnet1 src_net=172.16.2.0/24 jeq=eval_type
set_label=client%wknwnnet2 src_net=172.16.250.0/24 jeq=eval_type
set_label=type%mynet1 dst_net=192.168.0.0/23 label=eval_type
set_label=type%mynet2 dst_net=192.168.2.0/24 label=eval_type
set_label=type%tip dst_net=0.0.0.0/0 label=eval_type
# /etc/pmacct/pretag_unknown.map
dst_net=172.16.0.0/23
dst_net=172.16.2.0/24
dst_net=172.16.250.0/24
src_net=172.16.0.0/23
src_net=172.16.2.0/24
src_net=172.16.250.0/24
set_label=client%unknown src_net=0.0.0.0/0 jeq=eval_type
set_label=type%mynet1 dst_net=192.168.0.0/23 label=eval_type
set_label=type%mynet2 dst_net=192.168.2.0/24 label=eval_type
set_label=type%unknown dst_net=0.0.0.0/0 label=eval_type
#/etc/pmacct/sfacctd.conf
daemonize: false
debug: true
networks_file: /etc/pmacct/networks.lst
sfacctd_net: file
sfacctd_port: 8152
sfacctd_renormalize: true
sfacctd_time_new: true
plugin_buffer_size: 1024000
plugin_pipe_size: 10240000
propagate_signals: true
timestamps_secs: true
pre_tag_label_encode_as_map: true
plugins: kafka[in],kafka[out],kafka[unknown]
kafka_topic[in]: input_traffic
kafka_output[in]: json
kafka_broker_host[in]: 10.0.0.1
kafka_broker_port[in]: 5094
kafka_refresh_time[in]: 180
kafka_history[in]: 3m
kafka_history_roundoff[in]: m
pre_tag_map[in]: /etc/pmacct/pretag_in.map
aggregate_filter[in]: vlan and (dst net 172.16.0.0/23 or dst net
172.16.2.0/24 or dst net 172.16.250.0/24)
aggregate[in]: etype,label
kafka_topic[out]: output_traffic
kafka_output[out]: json
kafka_broker_host[out]: 10.0.0.1
kafka_broker_port[out]: 5094
kafka_refresh_time[out]: 180
kafka_history[out]: 3m
kafka_history_roundoff[out]: m
pre_tag_map[out]: /etc/pmacct/pretag_out.map
aggregate_filter[out]: vlan and (src net 172.16.0.0/23 or src net
172.16.2.0/24 or src net 172.16.250.0/24)
aggregate[out]: etype,label
kafka_topic[unknown]: unknown_traffic
kafka_output[unknown]: json
kafka_broker_host[unknown]: 10.0.0.1
kafka_broker_port[unknown]: 5094
kafka_refresh_time[unknown]: 180
kafka_history[unknown]: 3m
kafka_history_roundoff[unknown]: m
pre_tag_map[unknown]: /etc/pmacct/pretag_unknown.map
pre_tag_label_filter[unknown]: -null
aggregate[unknown]:
src_host,src_port,src_net,src_mask,dst_host,dst_port,dst_net,dst_mask,proto,etype,vlan,in_iface,out_iface,peer_src_ip,label
Any advice?
Regards,
Federico
On 10/1/23 21:46, Federico Urtizberea wrote:
An errata, in the copy and paste process I made a mistake. My
pretag.map file is:
/etc/pmacct/pretag.map
set_label=client%wknwnnet1 src_net=172.16.0.0/23 jeq=eval_out_type
set_label=client%wknwnnet1 src_net=172.16.2.0/24 jeq=eval_out_type
set_label=client%wknwnnet2 src_net=172.16.250.0/24 jeq=eval_out_type
set_label=client%wknwnnet1 dst_net=172.16.0.0/23 jeq=eval_in_type
set_label=client%wknwnnet1 dst_net=172.16.2.0/24 jeq=eval_in_type
set_label=client%wknwnnet2 dst_net=172.16.250.0/24 jeq=eval_in_type
set_label=direction%output,type%mynet1 dst_net=192.168.0.0/23
label=eval_out_type
set_label=direction%output,type%mynet2 dst_net=192.168.2.0/24
label=eval_out_type
set_label=direction%output,type%tip dst_net=0.0.0.0/0
label=eval_out_type
set_label=direction%input,type%mynet1 src_net=192.168.0.0/23
label=eval_in_type
set_label=direction%input,type%mynet2 src_net=192.168.2.0/24
label=eval_in_type
set_label=direction%input,type%tip src_net=0.0.0.0/0 label=eval_in_type
Regards,
Federico
On 10/1/23 20:55, Federico Urtizberea wrote:
Hi to all, i need some suggestions to resolve this.
I have severeral well known networks connected to my network, and i
provide transit to them. I need to measure the traffic between them
and my network, the ip transit traffic, and the unknown generate
traffic. To achieve this, I have configured several SFLOW exporters.
Let's say thay my networks are (mynet1) 192.168.0.0/24 and (mynet2)
192.168.1.0/24 and the well known networks 1 (wknwnnet1) are
172.16.0.0/23 and 172.16.2.0/24, and the well known network 2
(wknwnnet2) is 172.16.250.0/24.
So I wrote a network.lst file with all of these networks.
/etc/pmacct/network.lst
192.168.0.0/24
192.168.1.0/24
172.16.0.0/23
172.16.2.0/24
172.16.250.0./24
Then i wrote a pretag.map file, to set labels to the different kind
of traffic.
/etc/pmacct/pretag.map
set_label=client%wknwnnet1 src_net=172.16.0.0/23 jeq=eval_out_type
set_label=client%wknwnnet1 src_net=172.16.2.0/24 jeq=eval_out_type
set_label=client%wknwnnet2 src_net=172.16.250.0/24 jeq=eval_out_type
set_label=client%wknwnnet1 dst_net=172.16.0.0/23 jeq=eval_in_type
set_label=client%wknwnnet1 dst_net=172.16.2.0/24 jeq=eval_in_type
set_label=client%wknwnnet2 dst_net=172.16.250.0/24 jeq=eval_in_type
set_label=direction%output,type%mynet1 dst_net=192.168.0.0/23
label=eval_out_type
set_label=direction%output,type%mynet2 dst_net=192.168.2.0/24
label=eval_out_type
set_label=direction%output,type%tip dst_net=0.0.0.0/0
label=eval_out_type
set_label=direction%input,type%mynet1 src_net=192.168.0.0/23
label=eval_out_type
set_label=direction%input,type%mynet2 src_net=192.168.2.0/24
label=eval_out_type
set_label=direction%input,type%tip src_net=0.0.0.0/0
label=eval_out_type
and my sfacctd.conf is this:
daemonize: false
debug: true
networks_file: /etc/pmacct/networks.lst
sfacctd_net: file
sfacctd_port: 8152
sfacctd_renormalize: true
sfacctd_time_new: true
plugin_buffer_size: 1024000
plugin_pipe_size: 10240000
propagate_signals: true
timestamps_secs: true
pre_tag_map: /etc/pmacct/pretag.map
pre_tag_label_encode_as_map: true
plugins: kafka[known],kafka[unknown]
kafka_topic[known]: known
kafka_output[known]: json
kafka_broker_host[known]: 10.0.0.1
kafka_broker_port[known]: 5094
kafka_refresh_time[known]: 180
kafka_history[known]: 3m
kafka_history_roundoff[known]: m
aggregate_filter[known]: vlan and (net 172.16.0.0/23 or net
172.16.2.0/24 or net 172.16.250.0/24)
aggregate[known]: etype,label
kafka_topic[unknown]: unknown
kafka_output[unknown]: json
kafka_broker_host[unknown]: 10.0.0.1
kafka_broker_port[unknown]: 5094
kafka_refresh_time[unknown]: 180
kafka_history[unknown]: 3m
kafka_history_roundoff[unknown]: m
aggregate_filter[unknown]: vlan and not (net 172.16.0.0/23 or net
172.16.2.0/24 or net 172.16.250.0/24)
aggregate[unknown]:
src_host,src_port,src_net,src_mask,dst_host,dst_port,dst_net,dst_mask,proto,etype,vlan,in_iface,out_iface,peer_src_ip
The aggregated traffic for "known" plugin is working fine, but the
"unknown" plugin is not working, it doesn't aggregate any traffic.
If I remove the aggregate_filter for that plugin, and do the filter
in my backend "src_net=0.0.0.0/0 and dst_net=0.0.0.0/0" I obtain
the searched traffic.
Do you have any suggestions?
Thanks in advance,
Federico
_______________________________________________
pmacct-discussion mailing list
http://www.pmacct.net/#mailinglists
_______________________________________________
pmacct-discussion mailing list
http://www.pmacct.net/#mailinglists
