pmacct is a small set of multi-purpose passive network monitoring tools. It
can account, classify, aggregate, replicate and export forwarding-plane data,
ie. IPv4 and IPv6 traffic; collect and correlate control-plane data via BGP
and BMP; collect and correlate RPKI data; collect infrastructure data via
Streaming Telemetry. Each component works both as a standalone daemon and
as a thread of execution for correlation purposes (ie. enrich NetFlow with
BGP data).

A pluggable architecture allows to store collected forwarding-plane data into
memory tables, RDBMS (MySQL, PostgreSQL, SQLite), noSQL databases (MongoDB,
BerkeleyDB), AMQP (RabbitMQ) and Kafka message exchanges and flat-files.
pmacct offers customizable historical data breakdown, data enrichments like
BGP and IGP correlation and GeoIP lookups, filtering, tagging and triggers.
Libpcap, Linux Netlink/NFLOG, sFlow v2/v4/v5, NetFlow v5/v8/v9 and IPFIX are
all supported as inputs for forwarding-plane data. Replication of incoming
NetFlow, IPFIX and sFlow datagrams is also available. Statistics can be
easily exported to time-series databases like ElasticSearch and InfluxDB and
traditional tools Cacti RRDtool MRTG, Net-SNMP, GNUPlot, etc.

Control-plane and infrastructure data, collected via BGP, BMP and Streaming
Telemetry, can be all logged real-time or dumped at regular time intervals
to AMQP (RabbitMQ) and Kafka message exchanges and flat-files.



+ pmacct & Redis: pmacct daemons can now connect to a Redis cache.
  The main use-case currenly covered is: registering every stable
  daemon component in a table so to have, when running a cluster
  comprising several daemons / components, an olistic view of what
  is currently running and where; shall a component stop running
  or crash it will disappear from the inventory.
+ BMP daemon: as part of the IETF 107 vHackaton, preliminar support
  for draft-xu-grow-bmp-route-policy-attr-trace and draft-lucente-
  grow-bmp-tlv-ebit was introduced. Also added support for Peer
  Distinguisher field in the BMP Per-Peer Header.
+ BMP daemon: added support for reading from savefiles in libpcap
  format (pcap_savefile, pcap_savefile_delay, pcap_savefile_replay,
  pcap_filter) as an alternative to the use of
+ BMP daemon: re-worked, improved and generalized support for TLVs
  at the end of BMP messages. In this context, unknown Stats data
  is handled as a generic TLV. 
+ BMP daemon: added SO_KEEPALIVE TCP socket option (ie. to keep the
  sessions alive via a firewall / NAT kind of device). Thanks to
  Jared Mauch ( @jaredmauch ) for his patch. 
+ nfacctd, nfprobe plugin: added usec timestamp resolution to IPFIX
  collector and export via IEs #154, #155. For export, this can be
  configured via the new nfprobe_tstamp_usec knob.
+ nfacctd: new nfacctd_templates_receiver and nfacctd_templates_port
  config directives allow respectively to specify a destination
  where to copy NetFlow v9/IPFIX templates to and a port where to
  listen for templates from. If nfacctd_templates_receiver points to
  a replicator and the replicator exports to nfacctd_templates_port
  of a set of collectors then, for example, it gets possible to share
  templates among collectors in a cluster for the purpose of seamless
+ pmtelemetryd: in addition to existing TCP, UDP and ZeroMQ inputs,
  the daemon can now read Streaming Telemetry data in JSON format
  from a Kafka broker (telemetry_daemon_kafka_* config knobs).
+ Use of multiple processes for the Kafka Avro exporter
  to leverage the potential of multi-core/processors architectures.
  Code is from Raphael P. Barazzutti ( @rbarazzutti ).
+ added -F / --no-flatten command-line option to disable
  object flattening (default true for backward compatibility); also
  export to a Kafka broker for (flattened) JSON objects was added (in
  addition to existing export to ZeroMQ).
+ nDPI: introduced support for nDPI 3.2 and dropped support for all
  earlier versions of the library due to changes to the API.
+ Docker: embraced the technology for CI purposes; added a docker/
  directory in the file distribution where Dockerfile and scripts to
  build pmacct and dependencies are shared. Thanks to Claudio Ortega
  ( @claudio-ortega ) for contributing his excellent work in the area.
! fix, pmacctd: pcap_setdirection() enabled and moved to the right
  place in code. Libpcap tested for function presence. Thanks to
  Mikhail Sennikovsky for his patch.
! fix, pmacctd: SEGV has been detected if passing messages with an
  unsupported link layer. 
! fix, uacctd: handle non-ethernet packets correctly. Use mac_len = 0
  for non-ethernet packets in which case a zeroed ethernet header is
  used. Thanks to @aleksandrgilfanov for his patch.
! fix, BGP daemon: improved handling of withdrawals for label-unicast
  and mpls-vpn NLRIs.
! fix, BGP daemon: improved decoding of MPLS labels by passing it via
  a "0x%02x%02x%01x" formatter. In the past some labels may have been
  printed out incorrectly.
! fix, BGP daemon: decoding origin field correctly. Thanks to Peter
  Pothier ( @pothier-peter ) for his patch.
! BGP Looking Glass: LG code was moved from pmbgpd to pmacct libbgp
  so to be re-used in other components (ie. BGP thread of a flow
  daemon, BMP daemon, etc.).
! fix, BMP daemon: timestamps for Route Monitoring message were set
  to the current time rather than timestamp in the Per-Peer Header.
  Thanks to Peter Pothier ( @pothier-peter ) for reporting the issue.
! fix, BMP daemon: V flag test to allow IPv6 prefixes/peers was not
  correct (details in PR #362 on GitHub). Thanks @bcavns01 for his
! fix, BGP, BMP daemons: modified approach to message segmentation
  by ensuring reading exactly one message at a time (instead of the
  previous unaligned approach that was aiming to fill the available
  read buffer up) and waiting for all segments be available before
  processing a message (instead of the previous greedy approach that
  was trying to parse also partial messages). 
! fix, RPKI daemon: in rpki_prefix_lookup_node_match_cmp() prevent
  invalid results to over-write a previously computed valid one.
! fix, pmtelemetryd: recv() does include a MSG_WAITALL option to make
  sure all data is available before processing a given message; now
  n alarm() syscall is introduced so to prevent stalls due to bogus /
  incomplete data.
! fix, tee plugin: Tee_init_socks() now does not overwrite previously
  computed address length anymore: issues were repoted with IPv6 IPv4-
  mapped addresses. 
! fix, nfprobe plugin: take into account ingress/egress interfaces,
  if available, when comparing flows. Thanks to Mikhail Sennikovsky
  for his patch.
! fix, pretag_handlers.c: pretag_copy_label() instead of memcpy() in
  pretag_label_handler() to not borrow reference to label value and
  consequently lead to SEGV.
! fix, zmq_common.c: missing variable init in p_zmq_zap_handler() was
  giving troubles with gcc7 compiler optimizations. Thanks to Yuri
  Lachin ( @yuyutime ) for his support resolving this bug.
! fix, sql_common.c: print custom primitives with hex semantics among
  quotes. Hexs require a char type defined in the SQL table schema.
! fix, addr.c: passing right aligned argument to ip6_addr_cmp() in
  host_addr_mask_cmp(). This was found to impact selective replication
  of IPv6 flows basing on src_net and/or dst_net primitives in the
! fix, several code warnings catched by gcc7 and gcc8. Also several
  functions were renamed to avoid namespace conflicts with linked
- Obsoleted --enable-64bit knob which was already defaulting to true
  for some releases already.
- Obsoleted savefile_wait config knob (pcap_savefile_wait must be
  used instead).

See UPGRADE file.


pmacct-discussion mailing list

Reply via email to