Re: [pmacct-discussion] print plugin
Hi Fabien,
It smells like a crash. Please follow:
https://github.com/pmacct/pmacct/blob/master/QUICKSTART#L2031-L2060
We can then follow-up 1:1 with any output you may be able to gather.
Cheers,
Paolo
On Tue, Feb 13, 2018 at 10:21:54PM +0100, Fabien VINCENT wrote:
> Hi all,
>
> Trying to use the plugin print and seems I have an issue but can't find
> why
>
> aggregate[trafficin]: in_iface, tag2, peer_src_as, src_as, peer_src_ip,
> peer_dst_ip, std_comm, etype, tcpflags, dst_port, proto
> plugins: print[trafficin]
>
> print_refresh_time: 60
> print_history: 1m
> print_output: json
> print_output_file: /opt/data/nfacctd-in-%Y%m%d-%H%M.json
> !print_history_roundoff: m
> !print_output_file_append: true
>
> When starting, it's fine
>
> Feb 13 21:56:41 INFO ( default/core ): NetFlow Accounting Daemon,
> nfacctd 1.7.0-git (20170924-00)
> Feb 13 21:56:41 INFO ( default/core ): '--build=x86_64-linux-gnu'
> '--prefix=/usr/local' '--includedir=${prefix}/include'
> '--mandir=${prefix}/share/man' '--infodir=${prefix}/share/info'
> '--sysconfdir=/etc' '--localstatedir=/var'
> '--libdir=/usr/lib/x86_64-linux-gnu'
> '--libexecdir=${prefix}/lib/x86_64-linux-gnu' '--enable-pgsql'
> '--enable-mysql' '--enable-sqlite3' '--enable-ipv6' '--enable-64bit'
> '--enable-threads' '--enable-jansFeb 13 21:56:41 INFO ( default/core ):
> Reading configuration file '/etc/pmacct/nfacctd_in.conf'.
> Feb 13 21:56:41 INFO ( default/core/BGP ): maximum BGP peers allowed:
> 100
>
> But after few seconds :
>
> Feb 13 21:59:51 INFO ( trafficin/print ): JSON: setting object handlers.
> Feb 13 21:59:54 WARN ( default/core ): connection lost to
> 'trafficin-print'; closing connection.
> Feb 13 21:59:54 WARN ( default/core ): no more plugins active. Shutting
> down.
>
> Same with json and csv. Don't know why but sometimes it work ...
>
> > /usr/local/sbin/nfacctd -V
> NetFlow Accounting Daemon, nfacctd 1.7.0-git (20170924-00)
>
> Arguments:
> '--build=x86_64-linux-gnu' '--prefix=/usr/local'
> '--includedir=${prefix}/include' '--mandir=${prefix}/share/man'
> '--infodir=${prefix}/share/info' '--sysconfdir=/etc'
> '--localstatedir=/var' '--libdir=/usr/lib/x86_64-linux-gnu'
> '--libexecdir=${prefix}/lib/x86_64-linux-gnu' '--enable-pgsql'
> '--enable-mysql' '--enable-sqlite3' '--enable-ipv6' '--enable-64bit'
> '--enable-threads' '--enable-jansson' 'build_alias=x86_64-linux-gnu'
> '--enable-l2' '--enable-traffic-bins' '--enable-bgp-bins'
> '--enable-bmp-bins' '--enable-st-bins'
>
> Libs:
> libpcap version 1.6.2
> MySQL 5.5.58
> PostgreSQL 90415
> sqlite3 3.8.7.1
> jansson 2.7
>
> System:
> Linux 3.16.0-4-amd64 #1 SMP Debian 3.16.43-2+deb8u5 (2017-09-19) x86_64
>
> I was trying to do this, because the plugin pipe is missing data even
> with :
>
> plugin_pipe_size: 32768
> plugin_buffer_size: 327680
>
> --
>
> FABIEN VINCENT
> ---
> ___
> pmacct-discussion mailing list
> http://www.pmacct.net/#mailinglists
___
pmacct-discussion mailing list
http://www.pmacct.net/#mailinglists
[pmacct-discussion] print plugin
Hi all,
Trying to use the plugin print and seems I have an issue but can't find
why
aggregate[trafficin]: in_iface, tag2, peer_src_as, src_as, peer_src_ip,
peer_dst_ip, std_comm, etype, tcpflags, dst_port, proto
plugins: print[trafficin]
print_refresh_time: 60
print_history: 1m
print_output: json
print_output_file: /opt/data/nfacctd-in-%Y%m%d-%H%M.json
!print_history_roundoff: m
!print_output_file_append: true
When starting, it's fine
Feb 13 21:56:41 INFO ( default/core ): NetFlow Accounting Daemon,
nfacctd 1.7.0-git (20170924-00)
Feb 13 21:56:41 INFO ( default/core ): '--build=x86_64-linux-gnu'
'--prefix=/usr/local' '--includedir=${prefix}/include'
'--mandir=${prefix}/share/man' '--infodir=${prefix}/share/info'
'--sysconfdir=/etc' '--localstatedir=/var'
'--libdir=/usr/lib/x86_64-linux-gnu'
'--libexecdir=${prefix}/lib/x86_64-linux-gnu' '--enable-pgsql'
'--enable-mysql' '--enable-sqlite3' '--enable-ipv6' '--enable-64bit'
'--enable-threads' '--enable-jansFeb 13 21:56:41 INFO ( default/core ):
Reading configuration file '/etc/pmacct/nfacctd_in.conf'.
Feb 13 21:56:41 INFO ( default/core/BGP ): maximum BGP peers allowed:
100
But after few seconds :
Feb 13 21:59:51 INFO ( trafficin/print ): JSON: setting object handlers.
Feb 13 21:59:54 WARN ( default/core ): connection lost to
'trafficin-print'; closing connection.
Feb 13 21:59:54 WARN ( default/core ): no more plugins active. Shutting
down.
Same with json and csv. Don't know why but sometimes it work ...
> /usr/local/sbin/nfacctd -V
NetFlow Accounting Daemon, nfacctd 1.7.0-git (20170924-00)
Arguments:
'--build=x86_64-linux-gnu' '--prefix=/usr/local'
'--includedir=${prefix}/include' '--mandir=${prefix}/share/man'
'--infodir=${prefix}/share/info' '--sysconfdir=/etc'
'--localstatedir=/var' '--libdir=/usr/lib/x86_64-linux-gnu'
'--libexecdir=${prefix}/lib/x86_64-linux-gnu' '--enable-pgsql'
'--enable-mysql' '--enable-sqlite3' '--enable-ipv6' '--enable-64bit'
'--enable-threads' '--enable-jansson' 'build_alias=x86_64-linux-gnu'
'--enable-l2' '--enable-traffic-bins' '--enable-bgp-bins'
'--enable-bmp-bins' '--enable-st-bins'
Libs:
libpcap version 1.6.2
MySQL 5.5.58
PostgreSQL 90415
sqlite3 3.8.7.1
jansson 2.7
System:
Linux 3.16.0-4-amd64 #1 SMP Debian 3.16.43-2+deb8u5 (2017-09-19) x86_64
I was trying to do this, because the plugin pipe is missing data even
with :
plugin_pipe_size: 32768
plugin_buffer_size: 327680
--
FABIEN VINCENT
---___
pmacct-discussion mailing list
http://www.pmacct.net/#mailinglists
Re: [pmacct-discussion] print plugin and swap usage
Hi Pavel, Can we follow-up privately for some further investigation? I'd start with a memory profile, ie. collect every few secs/minute memory usage of every pmacct process, to determine how memory utilization changes over time - and where that leads to. In general i would say: if you keep the daemon running for, say, 24 hours and memory keeps increasing steadily then we have a good lead towards what looks like a memory leak or so. Cheers, Paolo On Tue, Apr 21, 2015 at 05:05:56PM +0200, Pavel Dimow wrote: Hello, I have configured nfacctd to capture cisco nel export from ASR 1000 and save to file every minute via print plugin. Everythig works fine for now but I have some doubts about memory usage. For example when I first start the nfacctd the amount of allocated memory is about 19.8% (print_cache_entries[nat]: 50) but after some time the amount of used memory begins to decline and the usage of swap is going up. Right now for example the amount of used memory is 7.3% (from 19.8) while VmSwap is 1010828 kB (from 0) and it's solely used by nfacctd Print Plugin [nat] Is this some kind of expected behavior or bug? The more scary thing about this is that I am not sure if I have all entries for NEL printed in file or there are some missed one. ___ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists ___ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists
Re: [pmacct-discussion] print plugin and src_as_path
Hi Paolo, thanks for taking care of this! Regards, Mario -Original Message- From: pmacct-discussion [mailto:pmacct-discussion-boun...@pmacct.net] On Behalf Of Paolo Lucente Sent: Mittwoch, 26. Februar 2014 18:49 To: pmacct-discussion@pmacct.net Subject: Re: [pmacct-discussion] print plugin and src_as_path Hi Mario, All, A quick note to say this has now been implemented: http://www.mail-archive.com/pmacct-commits@pmacct.net/msg01006.html Cheers, Paolo ___ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists ___ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists
Re: [pmacct-discussion] print plugin and src_as_path
Hi Mario, All, A quick note to say this has now been implemented: http://www.mail-archive.com/pmacct-commits@pmacct.net/msg01006.html Cheers, Paolo On Thu, Feb 20, 2014 at 04:27:31PM +, Paolo Lucente wrote: Hi Mario, Thanks for bringing this up, as you are scratching on a limbo situation which is good to iron out. My take on src_as_path, src_local_pref, src_std_comms, etc. is that they are all evil since are based on reverse BGP lookups. Hence i've removed mentions from official docs and stopped implementing it on newer plugin (in summary: it is implemented in memory and all SQL plugins; it is not in print, MongoDB and AMQP plugins). Nevertheless people catch these primitives are there in a way or in another and seems they are popular (at least src_as_path is). I reckon the best approach should be: implement them and put a disclaimer: use at your own risk/if you know what you are doing - which was perhaps the missing statement when, back in 2009, during the initial general excitement for implementation of these primitives i got bombed of .. yes, but i know for a fact those flows did not get in through that transit provider. Putting this item on my todo list. Cheers, Paolo On Thu, Feb 20, 2014 at 03:21:35PM +, Jentsch, Mario wrote: Hi, the print plugin doesn't output src_as_path (tested with 1.5.0rc2). I verified a similar setup with the memory plugin successfully and checked the source code, src/print_plugin.c misses src_as_path. Duplicating the lines for as_path and updating them to reflect src_as_path worked for me. Regards, Mario ___ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists ___ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists ___ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists
[pmacct-discussion] print plugin and src_as_path
Hi, the print plugin doesn't output src_as_path (tested with 1.5.0rc2). I verified a similar setup with the memory plugin successfully and checked the source code, src/print_plugin.c misses src_as_path. Duplicating the lines for as_path and updating them to reflect src_as_path worked for me. Regards, Mario ___ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists
Re: [pmacct-discussion] print plugin and src_as_path
Hi Mario, Thanks for bringing this up, as you are scratching on a limbo situation which is good to iron out. My take on src_as_path, src_local_pref, src_std_comms, etc. is that they are all evil since are based on reverse BGP lookups. Hence i've removed mentions from official docs and stopped implementing it on newer plugin (in summary: it is implemented in memory and all SQL plugins; it is not in print, MongoDB and AMQP plugins). Nevertheless people catch these primitives are there in a way or in another and seems they are popular (at least src_as_path is). I reckon the best approach should be: implement them and put a disclaimer: use at your own risk/if you know what you are doing - which was perhaps the missing statement when, back in 2009, during the initial general excitement for implementation of these primitives i got bombed of .. yes, but i know for a fact those flows did not get in through that transit provider. Putting this item on my todo list. Cheers, Paolo On Thu, Feb 20, 2014 at 03:21:35PM +, Jentsch, Mario wrote: Hi, the print plugin doesn't output src_as_path (tested with 1.5.0rc2). I verified a similar setup with the memory plugin successfully and checked the source code, src/print_plugin.c misses src_as_path. Duplicating the lines for as_path and updating them to reflect src_as_path worked for me. Regards, Mario ___ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists ___ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists
