[pmacct-discussion] pmacct 1.7.8 released !
VERSION.
1.7.8
DESCRIPTION.
pmacct is a small set of multi-purpose passive network monitoring tools. It
can account, classify, aggregate, replicate and export forwarding-plane data,
ie. IPv4 and IPv6 traffic; collect and correlate control-plane data via BGP
and BMP; collect and correlate RPKI data; collect infrastructure data via
Streaming Telemetry. Each component works both as a standalone daemon and
as a thread of execution for correlation purposes (ie. enrich NetFlow with
BGP data).
A pluggable architecture allows to store collected forwarding-plane data into
memory tables, RDBMS (MySQL, PostgreSQL, SQLite), noSQL databases (MongoDB,
BerkeleyDB), AMQP (RabbitMQ) and Kafka message exchanges and flat-files.
pmacct offers customizable historical data breakdown, data enrichments like
BGP and IGP correlation and GeoIP lookups, filtering, tagging and triggers.
Libpcap, Linux Netlink/NFLOG, sFlow v2/v4/v5, NetFlow v5/v8/v9 and IPFIX are
all supported as inputs for forwarding-plane data. Replication of incoming
NetFlow, IPFIX and sFlow datagrams is also available. Collected data can
be easily exported (ie. via Kafka) to modern databases like ElasticSearch,
Apache Druid and ClickHouse and (ie. via flat-files) to classic tools
Cacti, RRDtool and MRTG, etc.
Control-plane and infrastructure data, collected via BGP, BMP and Streaming
Telemetry, can be all logged real-time or dumped at regular time intervals
to AMQP (RabbitMQ) and Kafka message exchanges and flat-files.
HOMEPAGE.
http://www.pmacct.net/
DOWNLOAD.
http://www.pmacct.net/pmacct-1.7.8.tar.gz
CHANGELOG.
+ Introduced support for eBPF for all daemons: if SO_REUSEPORT is
supported by the OS and eBPF support is compiled in, this allows
to load a custom load-balancer. To load-share, daemons have to
be part of the same cluster_name and each be configured with a
distinct cluster_id.
+ Introduced support for listening on VRF interfaces on Linux for
all daemons. The feature can be enabled via nfacctd_interface,
bgp_daemon_interface and equivalent knobs. Many thanks to
Marcel Menzel ( @WRMSRwasTaken ) for this contribution.
+ pre_tag_map: introduced limited tagging / labelling support for
BGP (pmbgpd), BMP (pmbmpd), Streaming Telemetry (pmtelemetryd)
daemons. ip, set_tag, set_label keys being currently supported.
+ pre_tag_map: defined a new pre_tag_label_encode_as_map config
knob to encode the output 'label' value as a map for JSON and
Apache Avro encodings, ie. in JSON "label": { "key1": "value1",
"key2": "value2" }. For keys and values to be correctly mapped,
the '%' delimiter is used when composing a pre_tag_map, ie.
"set_label=key1%value1,key2%value2 ip=0.0.0.0/0". Thanks to
Salvatore Cuzzilla ( @scuzzilla ) for this contribution.
+ pre_tag_map: introduced support for IP prefixes for src_net
and dst_net keys for indexed maps (maps_index set to true).
Indexing being an hash map, this feature currently tests data
against all defined IP prefix lenghts in the map for a match
(first defined matching prefix wins).
+ pre_tag_map: introduced two new 'is_nsel', 'is_nel' keys to
check for the presence of firewallEvent field (233) and
natEvent field (230) in NetFlow/IPFIX respectively in order
to infer whether data is NSEL / NEL. If set to 'true' this
does match NSEL / NEL data, if set to 'false' it does match
non NSEL / NEL data respectively.
+ Introduced a new mpls_label_stack primitive, encoded as a
string and includes a comma-separated list of integers (label
values). Thanks to Salvatore Cuzzilla ( @scuzzilla ) for this
contribution.
+ Introduced a new fw_event primitive, to support NetFlow v9/
IPFIX firewallEvent 233 Information Element.
+ Introduced a new tunnel_tcp_flags primitive for pmacctd and
sfacctd to record TCP flags for the inner layer of a tunneled
technology (ie. VXLAN). Also tunnel_dst_port decoding was
fixed for sfacctd.
+ Introduced support for in/out VLAN support for sfacctd. To be
savy, 'in_vlan' and 'vlan' were muxed onto the same primitive
depending on the daemon being used. Thanks to Jim Westfall
( @jwestfall69 ) for this contribution.
+ Introduced a new mpls_label_stack_encode_as_array config knob
to encode the MPLS label stack as an array for JSON and Apache
Avro encodings, ie. in JSON "mpls_label_stack": [ "0-label0",
"1-label1", "2-label2", "3-label3", "4-label4", "5-label5" ]
and in Avro "name": "mpls_label_stack", "type": { "type":
"array", "items": { "type": "string" } }. Thanks to Salvatore
Cuzzilla ( @scuzzilla ) for this contribution.
+ Introduced a new tcpflags_encode_as_array config knob to encode
TCP flags as an array for JSON and Apache Avro, ie. in JSON
"tcp_flags": [ "URG", "ACK", "PSH", "RST", "SYN", "FIN" ] and
in Avro "name": "tcp_flags", "type": { "type": "array",
"items": { "type": "string" } }. Thanks to Salvatore Cuzzilla
( @scuzzilla ) for this contribution.
+ Introduced a new fwd_status_encode_as_string config knob to
[pmacct-discussion] pmacct 1.7.8 released !
VERSION.
1.7.8
DESCRIPTION.
pmacct is a small set of multi-purpose passive network monitoring tools.
It can account, classify, aggregate, replicate and export forwarding
plane data, ie. IPv4 and IPv6 traffic; collect and correlate
control-plane data via BGP and BMP; collect and correlate RPKI data;
collect infrastructure data via Streaming Telemetry. Each component
works both as a standalone daemon and as a thread of execution for
correlation purposes (ie. enrich NetFlow with BGP data).
A pluggable architecture allows to store collected forwarding-plane data
into memory tables, RDBMS (MySQL, PostgreSQL, SQLite), noSQL databases
(MongoDB, BerkeleyDB), AMQP (RabbitMQ) and Kafka message exchanges and
flat-files. pmacct offers customizable historical data breakdown, data
enrichments like BGP and IGP correlation and GeoIP lookups, filtering,
tagging and triggers. Libpcap, Linux Netlink/NFLOG, sFlow v2/v4/v5,
NetFlow v5/v8/v9 and IPFIX are all supported as inputs for
forwarding-plane data. Replication of incoming NetFlow, IPFIX and sFlow
datagrams is also available. Collected data can be easily exported (ie.
via Kafka) to modern databases like ElasticSearch, Apache Druid and
ClickHouse and (ie. via flat-files) to classic tools Cacti, RRDtool and
MRTG, etc.
Control-plane and infrastructure data, collected via BGP, BMP and
Streaming Telemetry, can be all logged real-time or dumped at regular
time intervals to AMQP (RabbitMQ) and Kafka message exchanges and
flat-files.
HOMEPAGE.
http://www.pmacct.net/
DOWNLOAD.
http://www.pmacct.net/pmacct-1.7.8.tar.gz
CHANGELOG.
+ Introduced support for eBPF for all daemons: if SO_REUSEPORT is
supported by the OS and eBPF support is compiled in, this allows
to load a custom load-balancer. To load-share, daemons have to
be part of the same cluster_name and each be configured with a
distinct cluster_id.
+ Introduced support for listening on VRF interfaces on Linux for
all daemons. The feature can be enabled via nfacctd_interface,
bgp_daemon_interface and equivalent knobs. Many thanks to
Marcel Menzel ( @WRMSRwasTaken ) for this contribution.
+ pre_tag_map: introduced limited tagging / labelling support for
BGP (pmbgpd), BMP (pmbmpd), Streaming Telemetry (pmtelemetryd)
daemons. ip, set_tag, set_label keys being currently supported.
+ pre_tag_map: defined a new pre_tag_label_encode_as_map config
knob to encode the output 'label' value as a map for JSON and
Apache Avro encodings, ie. in JSON "label": { "key1": "value1",
"key2": "value2" }. For keys and values to be correctly mapped,
the '%' delimiter is used when composing a pre_tag_map, ie.
"set_label=key1%value1,key2%value2 ip=0.0.0.0/0". Thanks to
Salvatore Cuzzilla ( @scuzzilla ) for this contribution.
+ pre_tag_map: introduced support for IP prefixes for src_net
and dst_net keys for indexed maps (maps_index set to true).
Indexing being an hash map, this feature currently tests data
against all defined IP prefix lenghts in the map for a match
(first defined matching prefix wins).
+ pre_tag_map: introduced two new 'is_nsel', 'is_nel' keys to
check for the presence of firewallEvent field (233) and
natEvent field (230) in NetFlow/IPFIX respectively in order
to infer whether data is NSEL / NEL. If set to 'true' this
does match NSEL / NEL data, if set to 'false' it does match
non NSEL / NEL data respectively.
+ Introduced a new mpls_label_stack primitive, encoded as a
string and includes a comma-separated list of integers (label
values). Thanks to Salvatore Cuzzilla ( @scuzzilla ) for this
contribution.
+ Introduced a new fw_event primitive, to support NetFlow v9/
IPFIX firewallEvent 233 Information Element.
+ Introduced a new tunnel_tcp_flags primitive for pmacctd and
sfacctd to record TCP flags for the inner layer of a tunneled
technology (ie. VXLAN). Also tunnel_dst_port decoding was
fixed for sfacctd.
+ Introduced support for in/out VLAN support for sfacctd. To be
savy, 'in_vlan' and 'vlan' were muxed onto the same primitive
depending on the daemon being used. Thanks to Jim Westfall
( @jwestfall69 ) for this contribution.
+ Introduced a new mpls_label_stack_encode_as_array config knob
to encode the MPLS label stack as an array for JSON and Apache
Avro encodings, ie. in JSON "mpls_label_stack": [ "0-label0",
"1-label1", "2-label2", "3-label3", "4-label4", "5-label5" ]
and in Avro "name": "mpls_label_stack", "type": { "type":
"array", "items": { "type": "string" } }. Thanks to Salvatore
Cuzzilla ( @scuzzilla ) for this contribution.
+ Introduced a new tcpflags_encode_as_array config knob to encode
TCP flags as an array for JSON and Apache Avro, ie. in JSON
"tcp_flags": [ "URG", "ACK", "PSH", "RST", "SYN", "FIN" ] and
in Avro "name": "tcp_flags", "type": { "type": "array",
"items": { "type": "string" } }. Thanks to Salvatore Cuzzilla
( @scuzzilla ) for this contribution.
+ Introduced a new
