Hi Adrien,
Thanks for your kind words :)
pmacct client can perform basic filtering but not sub-aggregation.
This said, the first thing it came to mind reading your question was
to query the memory table as "pmacct -s -O csv" then use a few touches
of awk for the sub-aggregation. Like:
For example, imagine src_host is field #1 src_port is field #3 and
proto is field #5 and bytes is field #8, you could select with awk
the fields of interest as:
pmacct -s -O csv | awk -F ',' '{ print $1","$3","$5","$8 }'
Then such output will contain duplicates that you can sum back as:
awk '{matrix[$1","$3","$5] += $8 } END {for (i in matrix) {print i, matrix[i]}}'
Please excuse typos - it was intended as an idea only and should not
be taken as working code. Let me know if it looks a viable solution
for you.
To your final question: no, unfortunately currently there is no way
to respawn died plugins. The condition should be checked at regular
intervals (ie. using 'pidfile' config directive) and the whole daemon
should be killed then restarted instead.
Cheers,
Paolo
On Wed, Aug 24, 2016 at 11:04:39PM +0200, Adrien Demarez wrote:
> Dear all,
>
> [first, thanks to pmacct authors for this very nice tool :) ]
>
> I am trying to use pmacct 1.6 daemon within a 8MB flash / 64MB RAM 3G
> OpenWRT router (MIPS/AR9341 platform). In order to aggregate conveniently
> the statistics (e.g. for ingress traffic I can group by destination IP, and
> for egress traffic I can group by src IP. Same for the src/dst port
> regarding client/server use), my configuration file was like that
> plugins:
> memory[clientin],memory[clientout],memory[serverin],memory[serverout]
> aggregate[clientin]: src_host,src_port,proto
> aggregate[clientout]: dst_host,dst_port,proto
> aggregate[serverin]: src_host,dst_port,proto
> aggregate[serverout]: dst_host,src_port,proto
> aggregate_filter[clientin]: inbound and not (dst portrange 1-1024)
> aggregate_filter[clientout]: outbound and not (dst portrange 1-1024)
> aggregate_filter[serverin]: inbound and (dst portrange 1-1024)
> aggregate_filter[serverout]: outbound and (dst portrange 1-1024)
>
> It works well and it does what I want on my sample router, but the main
> problem is... 4 instances of the daemon is just the limit of what my
> available RAM can support (in fact, the Linux OOM killer frequently kills
> one or several of the plugins so it doesn't work reliably, and any other
> simultaneous daemon or task would fail or be killed by the OOM killer).
> Same issues happen (even worse) with multiple "pgsql" or multiple "print"
> plugin instances. I can see that the memory occupation really depends a lot
> on the number of running plugins since when I run the following
> configuration it uses far less RAM :
> plugins: memory[in],memory[out]
> aggregate[in]: src_host,src_port,proto,dst_port
> aggregate[out]: dst_host,dst_port,proto,src_port
> aggregate_filter[in]: inbound
> aggregate_filter[out]: outbound
> (though of course, in that latter case it does not aggregate as efficiently
> as the first configuration and I have to do further postprocessing to
> manually aggregate what I want).
>
> => Since running multiple instance of pmacctd plugins is not good in my
> memory-constrained situation, is there any other/better way to achieve the
> same features as the first configuration with only one instance ? Ideally I
> would like to use only one instance of the memory plugin, yet be able to
> multiple aggregate_filters and aggregate ("group by") directives...
>
> Alternatively, is there any way to aggregate within pmacct _client_ ? (then
> I could run one instance of pmacctd that would not aggregate, and perform
> the aggregation later within the client)
> One more question : is there any easy way to "respawn" (automatically
> restart after crash or OOM killer) a killed plugin ?
>
> Best regards,
>
> --
> *Adrien Demarez*
> _______________________________________________
> pmacct-discussion mailing list
> http://www.pmacct.net/#mailinglists
_______________________________________________
pmacct-discussion mailing list
http://www.pmacct.net/#mailinglists
