Hi Thomas,
I think some confusion may be deriving from docs (to be improved) and
the fact 1.7.6 is old.
Nevertheless, from the interface indexes from your last output (ie.
1872541466, 3698069186, etc.) i can tell that you did configure
pcap_ifindex to 'hash' (being honored as you can see) in conjunction
with pcap_interfaces_map.
One issue in the code is for sure the fact to require the definition of
an ifindex always, even if pcap_ifindex is not set to 'map'. Another
issue was the silent discarding of pcap_interfaces_map without notifying
you with a warning. Both of these issues have been addressed in this
commit that i just passed:
https://github.com/pmacct/pmacct/commit/02080179aef3e87527e4d1158700eee729f1a5c3
Paolo
On 9/5/22 14:31, Thomas Eckert wrote:
Hi Paolo,
Thanks for the hint, I gave it a try. I'm observing the exact same
behavior between running pmacct in a container & directly on my host in
all cases. Tested with
* official docker image: 281904b7afd6
* official ubuntu 21.10 package: pmacct/impish,now 1.7.6-2 amd64
I *think* the problem is with the interfaces' ifindex parameter when
using the pcap_interfaces_map config key - everything works fine
(capture files are printed) when instead using the pcap_interface key.
Whenever I do not specify the 'ifindex' in the file specified as value
for the pcap_interfaces_map config key, I do not observe capture files
being printed. Vice versa, if I do specify the 'ifindex' parameter, then
capture files are printed.
In fact, if I do specify 'ifindex' for all interfaces listed when I run
"netstat -i", then pmacctd throws errors for my br-* & enx interfaces -
which it does not do when I omit 'ifindex' - almost as if it only then
realizes that it is supposed to access those interfaces at all. This
assumption is also based on the fact that I do see log lines such as these
INFO ( default/core ): Reading configuration file
'/etc/pmacct/pmacctd.conf'.
INFO ( default/core ): [/etc/pmacct/pcap-itf.conf] (re)loading map.
INFO ( default/core ): [/etc/pmacct/pcap-itf.conf] map successfully
(re)loaded.
INFO ( default/core ): [docker0,1872541466] link type is: 1 <=
INFO ( default/core ): [eno2,3698069186] link type is: 1 <=
INFO ( default/core ): [lo,2529615826] link type is: 1
<=
INFO ( default/core ): [tun0,3990258693] link type is: 12 <=
when specifying 'ifname' whereas the marked (<=) lines are missing
whenever I do not.
Reading through the config key documentation some more, I found the
config key pcap_ifindex. Interestingly enough, using it does not yield
any difference in results - neither for value "sys" nor for value "hash"
- irrespective of all other settings I played around with.
Assuming in pmacctd.conf the config key pcap_interfaces_map is used,
then this is what I speculate is effectively happening:
* pmacctd ignores config key pcap_ifindex
* instead, it expects 'ifindex' to be set in the interface mapping file
for each line
* each line where 'ifindex' is not set is ignored
* if 'ifindex' is missing on all lines, this results in a
"no-interface-being-listened-on" case without any warning/error
Summary: seems like 'ifname' is a mandatory parameter in the interface
mapping file whereas the documentation says "pmacctd: mandatory keys:
ifname."
My understanding of the documentation for above-mentioned config keys is
that the behavior I'm observing is not as intended (e.g. 'ifindex'
effectively being required, pcap_ifindex effectively being ignored) . So
I'm either making a mistake, e.g. in my config files, misunderstanding
the documentation or I'm encountering a bug - which I find difficult to
believe given how trivial my setup is.
Any Suggestions ?
Regards & Thanks,
Thomas
On Sun, May 8, 2022 at 1:43 PM Paolo Lucente <pa...@pmacct.net
<mailto:pa...@pmacct.net>> wrote:
Hi Thomas,
The simplest thing i may recommend is to check it all working outside a
container - this way you can easily isolate whether the issue is
somehow
related to the container (config or interaction of pmacctd with the
container) or with the pmacct config itself.
Paolo
On 6/5/22 06:05, Thomas Eckert wrote:
> Hi everyone,
>
> pmacct starter here, trying to get pmacctd working inside of a
container
> to listen to the (container's) host's traffic. I suppose this is
a, if
> not the, standard use case for pmacctd in a container. So I'm
sure it
> works in principle but I'm doing something wrong.
>
> Command for starting the container:
> docker run \
> --privileged --network=host \
> --name pmacctd \
> -v /tmp/pmacctd.conf:/etc/pmacct/pmacctd.conf:ro \
> -v /tmp/pcap-itf.conf:/etc/pmacct/pcap-itf.conf:ro \
> -v /tmp//captures:/var/pmacct/captures:rw pmacctd-debug \
> pmacct/pmacctd:latest
>
> Contents of pmacctd.conf:
> daemonize: false
> snaplen: 1000
> pcap_interfaces_map: /etc/pmacct/pcap-itf.conf
> aggregate: src_host, dst_host, src_port, dst_port, proto, class
> plugins: print
> print_output: json
> print_output_file: /var/pmacct/captures/capture-%Y%m%d_%H%M.txt
> print_output_file_append: true
> print_history: 1m
> print_history_roundoff: m
> print_refresh_time: 5
>
> pcap-itf.conf contains all interfaces of the host (as per netstat
-i) in
> the form
> ifname=eno2
> One line each, no other keys/values other than ifname.
> Possibly important note: There's a VPN (openconnect) constantly
running
> on the host. The VPN's interface is listed in netstat -i and, as
such,
> included in pcap-itf.conf.
>
> Starting the container yields this output:
> INFO ( default/core ): Promiscuous Mode Accounting Daemon,
pmacctd
> 1.7.7-git (20211107-0 (ef37a415))
> INFO ( default/core ): '--enable-mysql' '--enable-pgsql'
> '--enable-sqlite3' '--enable-kafka' '--enable-geoipv2'
> '--enable-jansson' '--enable-rabbitmq' '--enable-nflog'
'--enable-ndpi'
> '--enable-zmq' '--enable-avro' '--enable-serdes' '--enable-redis'
> '--enable-gnutls' 'AVRO_CFLAGS=-I/usr/local/avro/include'
> 'AVRO_LIBS=-L/usr/local/avro/lib -lavro' '--enable-l2'
> '--enable-traffic-bins' '--enable-bgp-bins' '--enable-bmp-bins'
> '--enable-st-bins'
> INFO ( default/core ): Reading configuration file
> '/etc/pmacct/pmacctd.conf'.
> INFO ( default/core ): [/etc/pmacct/pcap-itf.conf]
(re)loading map.
> INFO ( default/core ): [/etc/pmacct/pcap-itf.conf] map
successfully
> (re)loaded.
> INFO ( default_print/print ): cache entries=16411 base cache
> memory=67875896 bytes
> INFO ( default_print/print ): JSON: setting object handlers.
> INFO ( default_print/print ): *** Purging cache - START
(PID: 7) ***
> INFO ( default_print/print ): *** Purging cache - END (PID:
7, QN:
> 0/0, ET: X) ***
>
> Now, the problem is there are no files showing up in the 'captures'
> directory at all.
>
> I tried these things (as well as combinations thereof) to try to
> understand what's going on:
> * change the time related settings in pmacct.conf: to dump data
> more/less often - also waited (increasingly) long, at times up to
20 minutes
> * change 'snaplen' in pmacct.conf up & down - just to make sure
I'm not
> running into buffering problems (just guessing, haven't read
pmacct/d
> sources)
> * change pcap-itf.conf to contain all interfaces or only the
(host's)
> LAN + VPN interfaces (removing all others like docker's internal
'docker0')
> * check permission settings of the 'captures' directory - this
should be
> fine because a simple "touch /var/pmacct/captures/foobar" works
and the
> file does exist as observed in the directory on the host itself
> * run the container _not_ in host-sniffing mode, so just inside
its own
> network-bubble, then cause traffic against it and observe it writing
> data to the 'captures' directory - works!
>
> Because I started to doubt my own sanity I asked one of our
Docker/K8S
> experts to check my docker setup and he found no problem looking
over
> it, including via "docker inspect pmacct". So I'm fairly sure my
mistake
> is somewhere in the configuration of pmacctd but I cannot figure out
> what is. Would someone please point it out to me ?
>
> Regards & Thanks,
> Thomas
>
> PS: It's been almost 10 years since I've posted to a mailing list.
> Please forgive any conventions/best-practices missteps.
>
>
> _______________________________________________
> pmacct-discussion mailing list
> http://www.pmacct.net/#mailinglists
<http://www.pmacct.net/#mailinglists>
_______________________________________________
pmacct-discussion mailing list
http://www.pmacct.net/#mailinglists
_______________________________________________
pmacct-discussion mailing list
http://www.pmacct.net/#mailinglists
