Great, thanks! That was fast :-) Regards, Thomas
On Tue, May 10, 2022 at 4:29 PM Paolo Lucente <pa...@pmacct.net> wrote: > > Hi Thomas, > > I think some confusion may be deriving from docs (to be improved) and > the fact 1.7.6 is old. > > Nevertheless, from the interface indexes from your last output (ie. > 1872541466, 3698069186, etc.) i can tell that you did configure > pcap_ifindex to 'hash' (being honored as you can see) in conjunction > with pcap_interfaces_map. > > One issue in the code is for sure the fact to require the definition of > an ifindex always, even if pcap_ifindex is not set to 'map'. Another > issue was the silent discarding of pcap_interfaces_map without notifying > you with a warning. Both of these issues have been addressed in this > commit that i just passed: > > > https://github.com/pmacct/pmacct/commit/02080179aef3e87527e4d1158700eee729f1a5c3 > > Paolo > > > On 9/5/22 14:31, Thomas Eckert wrote: > > Hi Paolo, > > > > Thanks for the hint, I gave it a try. I'm observing the exact same > > behavior between running pmacct in a container & directly on my host in > > all cases. Tested with > > * official docker image: 281904b7afd6 > > * official ubuntu 21.10 package: pmacct/impish,now 1.7.6-2 amd64 > > > > I *think* the problem is with the interfaces' ifindex parameter when > > using the pcap_interfaces_map config key - everything works fine > > (capture files are printed) when instead using the pcap_interface key. > > Whenever I do not specify the 'ifindex' in the file specified as value > > for the pcap_interfaces_map config key, I do not observe capture files > > being printed. Vice versa, if I do specify the 'ifindex' parameter, then > > capture files are printed. > > > > In fact, if I do specify 'ifindex' for all interfaces listed when I run > > "netstat -i", then pmacctd throws errors for my br-* & enx interfaces - > > which it does not do when I omit 'ifindex' - almost as if it only then > > realizes that it is supposed to access those interfaces at all. This > > assumption is also based on the fact that I do see log lines such as > these > > INFO ( default/core ): Reading configuration file > > '/etc/pmacct/pmacctd.conf'. > > INFO ( default/core ): [/etc/pmacct/pcap-itf.conf] (re)loading map. > > INFO ( default/core ): [/etc/pmacct/pcap-itf.conf] map successfully > > (re)loaded. > > INFO ( default/core ): [docker0,1872541466] link type is: 1 <= > > INFO ( default/core ): [eno2,3698069186] link type is: 1 > <= > > INFO ( default/core ): [lo,2529615826] link type is: 1 > > <= > > INFO ( default/core ): [tun0,3990258693] link type is: 12 > <= > > when specifying 'ifname' whereas the marked (<=) lines are missing > > whenever I do not. > > > > Reading through the config key documentation some more, I found the > > config key pcap_ifindex. Interestingly enough, using it does not yield > > any difference in results - neither for value "sys" nor for value "hash" > > - irrespective of all other settings I played around with. > > > > Assuming in pmacctd.conf the config key pcap_interfaces_map is used, > > then this is what I speculate is effectively happening: > > * pmacctd ignores config key pcap_ifindex > > * instead, it expects 'ifindex' to be set in the interface mapping file > > for each line > > * each line where 'ifindex' is not set is ignored > > * if 'ifindex' is missing on all lines, this results in a > > "no-interface-being-listened-on" case without any warning/error > > Summary: seems like 'ifname' is a mandatory parameter in the interface > > mapping file whereas the documentation says "pmacctd: mandatory keys: > > ifname." > > > > My understanding of the documentation for above-mentioned config keys is > > that the behavior I'm observing is not as intended (e.g. 'ifindex' > > effectively being required, pcap_ifindex effectively being ignored) . So > > I'm either making a mistake, e.g. in my config files, misunderstanding > > the documentation or I'm encountering a bug - which I find difficult to > > believe given how trivial my setup is. > > > > Any Suggestions ? > > > > Regards & Thanks, > > Thomas > > > > On Sun, May 8, 2022 at 1:43 PM Paolo Lucente <pa...@pmacct.net > > <mailto:pa...@pmacct.net>> wrote: > > > > > > Hi Thomas, > > > > The simplest thing i may recommend is to check it all working > outside a > > container - this way you can easily isolate whether the issue is > > somehow > > related to the container (config or interaction of pmacctd with the > > container) or with the pmacct config itself. > > > > Paolo > > > > > > On 6/5/22 06:05, Thomas Eckert wrote: > > > Hi everyone, > > > > > > pmacct starter here, trying to get pmacctd working inside of a > > container > > > to listen to the (container's) host's traffic. I suppose this is > > a, if > > > not the, standard use case for pmacctd in a container. So I'm > > sure it > > > works in principle but I'm doing something wrong. > > > > > > Command for starting the container: > > > docker run \ > > > --privileged --network=host \ > > > --name pmacctd \ > > > -v /tmp/pmacctd.conf:/etc/pmacct/pmacctd.conf:ro \ > > > -v /tmp/pcap-itf.conf:/etc/pmacct/pcap-itf.conf:ro \ > > > -v /tmp//captures:/var/pmacct/captures:rw pmacctd-debug \ > > > pmacct/pmacctd:latest > > > > > > Contents of pmacctd.conf: > > > daemonize: false > > > snaplen: 1000 > > > pcap_interfaces_map: /etc/pmacct/pcap-itf.conf > > > aggregate: src_host, dst_host, src_port, dst_port, proto, > class > > > plugins: print > > > print_output: json > > > print_output_file: > /var/pmacct/captures/capture-%Y%m%d_%H%M.txt > > > print_output_file_append: true > > > print_history: 1m > > > print_history_roundoff: m > > > print_refresh_time: 5 > > > > > > pcap-itf.conf contains all interfaces of the host (as per netstat > > -i) in > > > the form > > > ifname=eno2 > > > One line each, no other keys/values other than ifname. > > > Possibly important note: There's a VPN (openconnect) constantly > > running > > > on the host. The VPN's interface is listed in netstat -i and, as > > such, > > > included in pcap-itf.conf. > > > > > > Starting the container yields this output: > > > INFO ( default/core ): Promiscuous Mode Accounting Daemon, > > pmacctd > > > 1.7.7-git (20211107-0 (ef37a415)) > > > INFO ( default/core ): '--enable-mysql' '--enable-pgsql' > > > '--enable-sqlite3' '--enable-kafka' '--enable-geoipv2' > > > '--enable-jansson' '--enable-rabbitmq' '--enable-nflog' > > '--enable-ndpi' > > > '--enable-zmq' '--enable-avro' '--enable-serdes' '--enable-redis' > > > '--enable-gnutls' 'AVRO_CFLAGS=-I/usr/local/avro/include' > > > 'AVRO_LIBS=-L/usr/local/avro/lib -lavro' '--enable-l2' > > > '--enable-traffic-bins' '--enable-bgp-bins' '--enable-bmp-bins' > > > '--enable-st-bins' > > > INFO ( default/core ): Reading configuration file > > > '/etc/pmacct/pmacctd.conf'. > > > INFO ( default/core ): [/etc/pmacct/pcap-itf.conf] > > (re)loading map. > > > INFO ( default/core ): [/etc/pmacct/pcap-itf.conf] map > > successfully > > > (re)loaded. > > > INFO ( default_print/print ): cache entries=16411 base cache > > > memory=67875896 bytes > > > INFO ( default_print/print ): JSON: setting object handlers. > > > INFO ( default_print/print ): *** Purging cache - START > > (PID: 7) *** > > > INFO ( default_print/print ): *** Purging cache - END (PID: > > 7, QN: > > > 0/0, ET: X) *** > > > > > > Now, the problem is there are no files showing up in the > 'captures' > > > directory at all. > > > > > > I tried these things (as well as combinations thereof) to try to > > > understand what's going on: > > > * change the time related settings in pmacct.conf: to dump data > > > more/less often - also waited (increasingly) long, at times up to > > 20 minutes > > > * change 'snaplen' in pmacct.conf up & down - just to make sure > > I'm not > > > running into buffering problems (just guessing, haven't read > > pmacct/d > > > sources) > > > * change pcap-itf.conf to contain all interfaces or only the > > (host's) > > > LAN + VPN interfaces (removing all others like docker's internal > > 'docker0') > > > * check permission settings of the 'captures' directory - this > > should be > > > fine because a simple "touch /var/pmacct/captures/foobar" works > > and the > > > file does exist as observed in the directory on the host itself > > > * run the container _not_ in host-sniffing mode, so just inside > > its own > > > network-bubble, then cause traffic against it and observe it > writing > > > data to the 'captures' directory - works! > > > > > > Because I started to doubt my own sanity I asked one of our > > Docker/K8S > > > experts to check my docker setup and he found no problem looking > > over > > > it, including via "docker inspect pmacct". So I'm fairly sure my > > mistake > > > is somewhere in the configuration of pmacctd but I cannot figure > out > > > what is. Would someone please point it out to me ? > > > > > > Regards & Thanks, > > > Thomas > > > > > > PS: It's been almost 10 years since I've posted to a mailing list. > > > Please forgive any conventions/best-practices missteps. > > > > > > > > > _______________________________________________ > > > pmacct-discussion mailing list > > > http://www.pmacct.net/#mailinglists > > <http://www.pmacct.net/#mailinglists> > > > > > > _______________________________________________ > > pmacct-discussion mailing list > > http://www.pmacct.net/#mailinglists >
_______________________________________________ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists
