Re: [pmacct-discussion] MySQL plugin processes terminating
Hi Klaas, Is it the main MySQL plugin failing on you or the writer processes (so the main MySQL plugin stays up and running)? Is it possible it is a simple memory issue, a-la you should throw more memory at it? You can collect more info on the crash (which may be useful for debug and troubleshooting) with the instruction here: https://github.com/pmacct/pmacct/blob/1.7.5/QUICKSTART#L2813-#L2828 As soon as you see malloc() appearing somewhere then it's a (lack of) memory issue. In case, instead, writers are thrown away because they pass the configured writer limit (sql_max_writers, 10 by default), then you would find a note in the logs. Paolo On 20/11/2020 13:02, Klaas Tammling wrote: Hi all, I've got the issue that regularly the threads for the MySQL plugin seem to silently crash. Is there any easy way to monitor these processes and restart them if needed? Thanks. Regards, Klaas ___ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists ___ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists
Re: [pmacct-discussion] Fragment/4 buffer full. Skipping fragments.
Hi Pierre, Maybe you need to increase the pmacctd_frag_buffer_size (by default 4MB and perhaps not sufficient for your traffic footprint): https://github.com/pmacct/pmacct/blob/1.7.5/CONFIG-KEYS Give that a try. Paolo On 20/11/2020 12:53, Pierre Grié wrote: Hello, We are using pmacct to generate Netflow v9 metrics. Yesterday, while we were under what we determined to be a heavy load of UDP fragmented packets, pmacct did not report any traffic peak. Our SNMP metrics reported 1Gbps+ of traffic at the same time. We noticed the following log message multiple times around then: "INFO ( default / core ): Fragment/4 buffer full. Skipping fragments.". Could this message be linked to the behavior we're seeing? Could it be caused by a misconfiguration our our side? Here's our current configuration: - daemonize: true pcap_ifindex: sys pcap_interfaces_map: /etc/pmacct/pcap_interfaces.map aggregate: src_host, dst_host, src_port, dst_port, proto, tcpflags, src_as, dst_as promisc: false syslog: local0 plugins: nfprobe[xxx], nfprobe[xxx] nfprobe_version: 9 nfprobe_source_ip: xxx ! Configuration for xxx nfprobe_receiver[xxx]: xxx nfprobe_direction[xxx]: tag pre_tag_map[xxx]: /etc/pmacct/pretag.map sampling_rate[xxx]: 200 plugin_pipe_size[xxx]: 12288000 plugin_buffer_size[xxx]: 122880 ! Configuration for nfprobe_receiver[xxx]: xxx nfprobe_direction[xxx]: tag pre_tag_map[xxx]: /etc/pmacct/pretag.map sampling_rate[xxx]: 1000 plugin_pipe_size[xxx]: 12288000 plugin_buffer_size[xxx]: 122880 pmacctd_as: file networks_file: /etc/pmacct/networks.list nfprobe_timeouts: tcp=1:maxlife=1:tcp.rst=1:tcp.fin=1:general=1:expint=5 - Thanks! ___ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists ___ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists
Re: [pmacct-discussion] pmacctd and OpenVPN
Hi Erik, Take a capture with tcpdump of some of these packets on the tun interface and send it via unicast email. Let's see what is possible or what is the issue. Paolo On 20/11/2020 11:34, Erik wrote: Hi, I am running a VPN server based on OpenVPN and recently there was a request to analyse some of the data flows. So I installed pmacct to do some experimenting. This is on Ubuntu 20.04 with pmacct 1.7.2 from the repository. The software installed fine and after configuration on the main NIC I was able to export flows in IPFIX format via nfprobe and look at the flows using nfdump. The next step was to configure it on the VPN server's TUN-adapter, but that caused pmacctd to fail to start, logging: ERROR ( default/core ): MAC aggregation not available for link type: 12 I have since looked through the archives and the Changelog for newer versions, but could not find anything relating to this error, other than some remark suggesting pmacct only supports or supported real NICs. I have not been able to test a newer version yet, to see if this has changed and it may take some time before I will be able to. Meanwhile, can someone confirm that pmacct does support TUN interfaces, or not? And maybe point me to an alternative if it doesn't? So far I have only found nprobe, which is a commercial alternative. Thanks, Erik ___ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists ___ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists
[pmacct-discussion] Fragment/4 buffer full. Skipping fragments.
Hello, We are using pmacct to generate Netflow v9 metrics. Yesterday, while we were under what we determined to be a heavy load of UDP fragmented packets, pmacct did not report any traffic peak. Our SNMP metrics reported 1Gbps+ of traffic at the same time. We noticed the following log message multiple times around then: "INFO ( default / core ): Fragment/4 buffer full. Skipping fragments.". Could this message be linked to the behavior we're seeing? Could it be caused by a misconfiguration our our side? Here's our current configuration: - daemonize: true pcap_ifindex: sys pcap_interfaces_map: /etc/pmacct/pcap_interfaces.map aggregate: src_host, dst_host, src_port, dst_port, proto, tcpflags, src_as, dst_as promisc: false syslog: local0 plugins: nfprobe[xxx], nfprobe[xxx] nfprobe_version: 9 nfprobe_source_ip: xxx ! Configuration for xxx nfprobe_receiver[xxx]: xxx nfprobe_direction[xxx]: tag pre_tag_map[xxx]: /etc/pmacct/pretag.map sampling_rate[xxx]: 200 plugin_pipe_size[xxx]: 12288000 plugin_buffer_size[xxx]: 122880 ! Configuration for nfprobe_receiver[xxx]: xxx nfprobe_direction[xxx]: tag pre_tag_map[xxx]: /etc/pmacct/pretag.map sampling_rate[xxx]: 1000 plugin_pipe_size[xxx]: 12288000 plugin_buffer_size[xxx]: 122880 pmacctd_as: file networks_file: /etc/pmacct/networks.list nfprobe_timeouts: tcp=1:maxlife=1:tcp.rst=1:tcp.fin=1:general=1:expint=5 - Thanks! ___ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists
[pmacct-discussion] pmacctd and OpenVPN
Hi, I am running a VPN server based on OpenVPN and recently there was a request to analyse some of the data flows. So I installed pmacct to do some experimenting. This is on Ubuntu 20.04 with pmacct 1.7.2 from the repository. The software installed fine and after configuration on the main NIC I was able to export flows in IPFIX format via nfprobe and look at the flows using nfdump. The next step was to configure it on the VPN server's TUN-adapter, but that caused pmacctd to fail to start, logging: ERROR ( default/core ): MAC aggregation not available for link type: 12 I have since looked through the archives and the Changelog for newer versions, but could not find anything relating to this error, other than some remark suggesting pmacct only supports or supported real NICs. I have not been able to test a newer version yet, to see if this has changed and it may take some time before I will be able to. Meanwhile, can someone confirm that pmacct does support TUN interfaces, or not? And maybe point me to an alternative if it doesn't? So far I have only found nprobe, which is a commercial alternative. Thanks, Erik ___ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists
[pmacct-discussion] MySQL plugin processes terminating
Hi all, I've got the issue that regularly the threads for the MySQL plugin seem to silently crash. Is there any easy way to monitor these processes and restart them if needed? Thanks. Regards, Klaas ___ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists
