Hi Edvinas,

'pmacctd -V' returns all the libs it is linked against, including
version. There you *should* find an indication the PF_RING-enabled
libpcap is being used.

Paolo
 
On Thu, Jan 03, 2019 at 10:46:55AM +0200, Edvinas K wrote:
> Hello,
> 
> How to check if the PF_RING is in action and active ? some forwarded
> packets counts, or etc ?
> 
> Thanks
> 
> On Thu, Dec 27, 2018 at 3:00 PM Edvinas K <edvinas.em...@gmail.com> wrote:
> 
> > thank you,
> >
> > seems all easy things didin't help.
> >
> > I tried to set up the buffer size in kernel:
> >
> > prod [root@netvpn001prpjay pmacct-1.7.2]# cat
> > /proc/sys/net/core/[rw]mem_max
> > 2000000000
> > 2000000000
> >
> > and then
> >
> > prod [root@netvpn001prpjay pmacct-1.7.2]# cat flowexport.cfg
> >    !
> >    daemonize: no
> >    aggregate: src_host, dst_host, src_port, dst_port, proto, tos
> >    plugins: nfprobe
> >    nfprobe_receiver: 10.3.14.101:2101
> >    nfprobe_version: 9
> >
> >    pmacctd_pipe_size: 2000000000
> >    plugin_pipe_size: 1000000
> >    plugin_buffer_size: 10000
> >
> >    ! nfprobe_engine: 1:1
> >    ! nfprobe_timeouts: tcp=120:maxlife=3600
> >    !
> >    ! networks_file: /path/to/networks.lst
> >    !...
> >
> > maybe after putting plugin_pipe_size and  plugin_buffer_size drops got
> > little bit lower, but still a lot.
> > also noticed strange log message: "INFO ( default/core ): short IPv4
> > packet read (36/38/frags). Snaplen issue ?"
> >
> > I going to try that PF_RING stuff.
> >
> > On Thu, Dec 20, 2018 at 10:08 PM Paolo Lucente <pa...@pmacct.net> wrote:
> >
> >>
> >> Hi Edvinas,
> >>
> >> I wanted to confirm that when you changed pmacctd_pipe_size to 2GB you
> >> ALSO changed /proc/sys/net/core/[rw]mem_max to 2GB and ALSO restarted
> >> pmacctd after having done so.
> >>
> >> Wrt PF_RING: i can't voice since i don't use it myself. While i never
> >> heard any horror story with it (thumbs up!), i think doing a proof of
> >> concept first is always a good idea; this is also to answer your second
> >> question: it will improve things for sure but how much you have to test.
> >>
> >> Another thing you may do is also to increase buffering internal to
> >> pmacct (it may help reduce CPU cycles by the core process and hence help
> >> it process more data), i see that in your config you have NO buffering
> >> enabled. For a quick test you could set:
> >>
> >> plugin_pipe_size: 1000000
> >> plugin_buffer_size: 10000
> >>
> >> And depending if you see any benefits/improvement and if you have memory
> >> you could ramp these values up. Or alternatively you could introduce
> >> ZeroMQ. Again, this is internal queueuing (whereas in my previous email
> >> i was tackling the queueing between kernel and pmacct):
> >>
> >> https://github.com/pmacct/pmacct/blob/master/CONFIG-KEYS#L234-#L292
> >>
> >> Paolo
> >>
> >> On Wed, Dec 19, 2018 at 06:40:14PM +0200, Edvinas K wrote:
> >> > Hello,
> >> >
> >> > How would you recommend to test PF_RING:
> >> >
> >> > Some questions:
> >> >
> >> > Is't safe to install it on production server ?
> >> > Is't possible to hope, that this PF_RING will solve all the discards ?
> >> >
> >> > Thanks
> >> >
> >> > On Tue, Dec 18, 2018 at 5:59 PM Edvinas K <edvinas.em...@gmail.com>
> >> wrote:
> >> >
> >> > > thanks,
> >> > >
> >> > > I tried to change the pipe size. As i noticed my OS (centos) default
> >> and
> >> > > max size are the same:
> >> > >
> >> > > prod [root@netvpn001prpjay pmacct-1.7.2]# cat
> >> > > /proc/sys/net/core/[rw]mem_default
> >> > > 212992
> >> > > 212992
> >> > >
> >> > > prod [root@netvpn001prpjay pmacct-1.7.2]# cat
> >> > > /proc/sys/net/core/[rw]mem_max
> >> > > 212992
> >> > > 212992
> >> > >
> >> > > I tried to set the pmacctd_pipe_size: to 2000000000  and later to
> >> 212992.
> >> > > Seems the drops is still occuring.
> >> > > Tomorrow i will try to look at that PF_RING thing.
> >> > >
> >> > > Thanks
> >> > >
> >> > >
> >> > >
> >> > >
> >> > >
> >> > > On Tue, Dec 18, 2018 at 5:32 PM Paolo Lucente <pa...@pmacct.net>
> >> wrote:
> >> > >
> >> > >>
> >> > >> Hi Edvinas,
> >> > >>
> >> > >> Easier thing first, i recommend to inject some test traffic and see
> >> that
> >> > >> one how it looks like.
> >> > >>
> >> > >> The dropped packets highlight a buffering issue. You could take an
> >> > >> intermediate step and see if enlarging buffers helps. Configure
> >> > >> pmacctd_pipe_size to 2000000000 and follow instructions here for the
> >> > >> /proc files to touch:
> >> > >>
> >> > >> https://github.com/pmacct/pmacct/blob/1.7.2/CONFIG-KEYS#L203-#L216
> >> > >>
> >> > >> If it helps, good. If not: you should really look into one of the
> >> > >> frameworks i was pointing you to in my previous email. PF_RING, for
> >> > >> example, can do sampling and/or balancing. Sampling should not be
> >> done
> >> > >> inside pmacct because the dropped packets are between the kernel and
> >> the
> >> > >> application.
> >> > >>
> >> > >> Paolo
> >> > >>
> >> > >> On Mon, Dec 17, 2018 at 02:52:48PM +0200, Edvinas K wrote:
> >> > >> > Seems there're lots of dropped packets:
> >> > >> >
> >> > >> > prod [root@netvpn001prpjay pmacct-1.7.2]# pmacctd -i ens1f0.432 -f
> >> > >> > flowexport.cfg
> >> > >> > WARN: [flowexport.cfg:2] Invalid value. Ignored.
> >> > >> > INFO ( default/core ): Promiscuous Mode Accounting Daemon, pmacctd
> >> > >> > 1.7.2-git (20181018-00+c3)
> >> > >> > INFO ( default/core ):  '--enable-l2' '--enable-ipv6'
> >> '--enable-64bit'
> >> > >> > '--enable-traffic-bins' '--enable-bgp-bins' '--enable-bmp-bins'
> >> > >> > '--enable-st-bins'
> >> > >> > INFO ( default/core ): Reading configuration file
> >> > >> > '/opt/pmacct-1.7.2/flowexport.cfg'.
> >> > >> > INFO ( default_nfprobe/nfprobe ): NetFlow probe plugin is
> >> originally
> >> > >> based
> >> > >> > on softflowd 0.9.7 software, Copyright 2002 Damien Miller <
> >> > >> d...@mindrot.org>
> >> > >> > All rights reserved.
> >> > >> > INFO ( default_nfprobe/nfprobe ):           TCP timeout: 3600s
> >> > >> > INFO ( default_nfprobe/nfprobe ):  TCP post-RST timeout: 120s
> >> > >> > INFO ( default_nfprobe/nfprobe ):  TCP post-FIN timeout: 300s
> >> > >> > INFO ( default_nfprobe/nfprobe ):           UDP timeout: 300s
> >> > >> > INFO ( default_nfprobe/nfprobe ):          ICMP timeout: 300s
> >> > >> > INFO ( default_nfprobe/nfprobe ):       General timeout: 3600s
> >> > >> > INFO ( default_nfprobe/nfprobe ):      Maximum lifetime: 604800s
> >> > >> > INFO ( default_nfprobe/nfprobe ):       Expiry interval: 60s
> >> > >> > INFO ( default_nfprobe/nfprobe ): Exporting flows to
> >> > >> > [10.3.14.101]:rtcm-sc104
> >> > >> > INFO ( default/core ): [ens1f0.432,0] link type is: 1
> >> > >> > ^C^C^C^C^C^C^C^C
> >> > >> >
> >> > >> > after 1 minute:
> >> > >> >
> >> > >> > WARN ( default_nfprobe/nfprobe ): Shutting down on user request.
> >> > >> > INFO ( default/core ): OK, Exiting ...
> >> > >> > NOTICE ( default/core ): +++
> >> > >> > NOTICE ( default/core ): [ens1f0.432,0] received_packets=3441854
> >> > >> > *dropped_packets=2365166*
> >> > >> >
> >> > >> > About 1GB of traffic is passing through the router where i'm
> >> capturing
> >> > >> the
> >> > >> > packets. Isn't it too much traffic for nfrpobe to process ? CPUs
> >> seems
> >> > >> not
> >> > >> > in 100% usage. We're using  Intel Xeon E5-2620 0 @ 2.00GHz
> >> > >> > <
> >> > >>
> >> http://netmon.adform.com/device/device=531/tab=health/metric=processor/processor_id=1466/
> >> > >> >
> >> > >> > x
> >> > >> > 24.
> >> > >> >
> >> > >> > prod [root@netvpn001prpjay ~]# ps -aux | grep pmacct
> >> > >> > root     41840 30.9  0.0  18964  7760 ?        Rs   Dec14 1309:50
> >> > >> pmacctd:
> >> > >> > Core Process [default]
> >> > >> > root     41841 *68.4%*  0.0  22932  9756 ?        R    Dec14
> >> 2898:29
> >> > >> > pmacctd: Netflow Probe Plugin [default_nfprobe]
> >> > >> > root     41869 32.5  0.0  19360  8128 ?        Ss   Dec14 1378:29
> >> > >> pmacctd:
> >> > >> > Core Process [default]
> >> > >> > root     41870 *67.6%* 0.0  22928  9760 ?        R    Dec14 2865:35
> >> > >> > pmacctd: Netflow Probe Plugin [default_nfprobe]
> >> > >> >
> >> > >> > Before starting with your mentioned 'steroid' things, i would like
> >> to
> >> > >> ask,
> >> > >> > is't really worth to go to that kernel "things", or start with
> >> > >> techniques
> >> > >> > for example like sampling, or like Nikola recommended try to fidle
> >> with
> >> > >> > nfprobe_engine settings ?
> >> > >> >
> >> > >> > Thanks
> >> > >> >
> >> > >> >
> >> > >> >
> >> > >> >
> >> > >> >
> >> > >> >
> >> > >> >
> >> > >> >
> >> > >> >
> >> > >> >
> >> > >> >
> >> > >> >
> >> > >> >
> >> > >> >
> >> > >> >
> >> > >> >
> >> > >> > On Sun, Dec 16, 2018 at 6:25 PM Paolo Lucente <pa...@pmacct.net>
> >> wrote:
> >> > >> >
> >> > >> > >
> >> > >> > > Hi Edvinas,
> >> > >> > >
> >> > >> > > You may want to check whether libpcap is dropping packets on
> >> input to
> >> > >> > > pmacctd. You can achieve that sending a SIGUSR1 and checking the
> >> > >> output
> >> > >> > > in the logfile/syslog/console. You will get something a-la:
> >> > >> > >
> >> > >> > >
> >> https://github.com/pmacct/pmacct/blob/master/docs/SIGNALS#L16-#L34
> >> > >> > >
> >> > >> > > Should amount of dropped packets be non-zero and visibly
> >> increasing
> >> > >> then
> >> > >> > > you may want to put your libpcap on steroids:
> >> > >> > >
> >> > >> > > https://github.com/pmacct/pmacct/blob/master/FAQS#L71-#L101
> >> > >> > >
> >> > >> > > Should, instead, that not be the case, i am unsure and would need
> >> > >> > > further investigation. You could try to produce a controlled
> >> stream of
> >> > >> > > data and sniff nfprobe output. Or collect with a different
> >> software
> >> > >> for
> >> > >> > > a quick counter-test (nfacctd itself or another of your choice).
> >> > >> > >
> >> > >> > > Paolo
> >> > >> > >
> >> > >> > > On Fri, Dec 14, 2018 at 03:02:35PM +0200, Edvinas K wrote:
> >> > >> > > > Thanks, i really appreciate your help.
> >> > >> > > >
> >> > >> > > > Everything seems working OK, on NFSEN (NFDUMP) graphs of flows
> >> > >> statistics
> >> > >> > > > looks good, but the traffic rate Mb/s (45 Mb/s) is somehow 10x
> >> > >> lower than
> >> > >> > > > really is. Maybe some tips to troubleshoot that ?
> >> > >> > > >
> >> > >> > > > [image: image.png]
> >> > >> > > >
> >> > >> > > > Is there any hidden things to check about ?
> >> > >> > > >
> >> > >> > > > My config:
> >> > >> > > >
> >> > >> > > > 1050  pmacctd -i ens1f0.432 -f flowexport.cfg
> >> > >> > > > 1051  pmacctd -i ens1f1.433 -f flowexport.cfg
> >> > >> > > >
> >> > >> > > > cat flowexport.cfg
> >> > >> > > >    !
> >> > >> > > >    daemonize: true
> >> > >> > > >    aggregate: src_host, dst_host, src_port, dst_port, proto,
> >> tos
> >> > >> > > >    plugins: nfprobe
> >> > >> > > >    nfprobe_receiver: 10.3.14.101:2101
> >> > >> > > >    nfprobe_version: 9
> >> > >> > > >    ! nfprobe_engine: 1:1
> >> > >> > > >    ! nfprobe_timeouts: tcp=120:maxlife=3600
> >> > >> > > >    !
> >> > >> > > >    ! networks_file: /path/to/networks.lst
> >> > >> > > >
> >> > >> > > > On Thu, Dec 13, 2018 at 4:32 AM Paolo Lucente <
> >> pa...@pmacct.net>
> >> > >> wrote:
> >> > >> > > >
> >> > >> > > > >
> >> > >> > > > > Hi Nikola,
> >> > >> > > > >
> >> > >> > > > > I see, makes sense. Thanks very much for clarifying.
> >> > >> > > > >
> >> > >> > > > > Paolo
> >> > >> > > > >
> >> > >> > > > > On Wed, Dec 12, 2018 at 06:20:58PM -0800, Nikola Kolev wrote:
> >> > >> > > > > > Hi Paollo,
> >> > >> > > > > >
> >> > >> > > > > > Sorry for being cryptic - what I meant was that I wasn't
> >> able to
> >> > >> > > > > > launch pmacctd/uacctd in a way that it deals with dynamic
> >> > >> interfaces
> >> > >> > > as
> >> > >> > > > > > ppp. Basically I failed to find any reference in the docs
> >> on
> >> > >> how to
> >> > >> > > make
> >> > >> > > > > > it run in such a way, that it collects info from ppp*
> >> (a-la the
> >> > >> ppp+
> >> > >> > > > > > syntax of iptables), without launching a separate pmacctd
> >> > >> instance
> >> > >> > > for
> >> > >> > > > > > each interface, hence the complicated setup with
> >> > >> > > > > > iptables-nflog-uacctd-nfdump.
> >> > >> > > > > >
> >> > >> > > > > > On Thu, 13 Dec 2018 01:35:00 +0000
> >> > >> > > > > > Paolo Lucente <pa...@pmacct.net> wrote:
> >> > >> > > > > >
> >> > >> > > > > > >
> >> > >> > > > > > > Hi Nikola,
> >> > >> > > > > > >
> >> > >> > > > > > > Can you please elaborate a bit more? The cryptic part
> >> for me
> >> > >> is "as
> >> > >> > > > > > > nfacctd is not supporting wildcard addresses to be bound
> >> to".
> >> > >> > > > > > >
> >> > >> > > > > > > Thanks,
> >> > >> > > > > > > Paolo
> >> > >> > > > > > >
> >> > >> > > > > > > On Wed, Dec 12, 2018 at 04:50:33PM -0800, Nikola Kolev
> >> wrote:
> >> > >> > > > > > > > Hey,
> >> > >> > > > > > > >
> >> > >> > > > > > > > If I may add to that:
> >> > >> > > > > > > >
> >> > >> > > > > > > > I'm doing something similar, but in a slightly
> >> different
> >> > >> manner:
> >> > >> > > > > > > >
> >> > >> > > > > > > > as nfacctd is not supporting wildcard addresses to be
> >> bound
> >> > >> to,
> >> > >> > > I'm
> >> > >> > > > > > > > using iptables' rules to export via nflog to uacctd,
> >> which
> >> > >> then
> >> > >> > > can
> >> > >> > > > > > > > send to nfdump. Just food for thought...
> >> > >> > > > > > > >
> >> > >> > > > > > > > On 2018-12-12 14:58, Paolo Lucente wrote:
> >> > >> > > > > > > > >Hi Edvinas,
> >> > >> > > > > > > > >
> >> > >> > > > > > > > >You are looking for the nfprobe plugin. You can
> >> follow the
> >> > >> > > relevant
> >> > >> > > > > > > > >section in the QUICKSTART to get going:
> >> > >> > > > > > > > >
> >> > >> > > > > > > > >
> >> > >> > >
> >> https://github.com/pmacct/pmacct/blob/1.7.2/QUICKSTART#L1167-#L1302
> >> > >> > > > > > > > >
> >> > >> > > > > > > > >Paolo
> >> > >> > > > > > > > >
> >> > >> > > > > > > > >On Wed, Dec 12, 2018 at 03:12:39PM +0200, Edvinas K
> >> wrote:
> >> > >> > > > > > > > >>Hello,
> >> > >> > > > > > > > >>
> >> > >> > > > > > > > >>I managed to run basic pmacct to capture linux router
> >> > >> (FRR)
> >> > >> > > flows
> >> > >> > > > > > > > >>from libcap:
> >> > >> > > > > > > > >>"pmacctd -P print -O formatted -r 10 -i bond0.2170 -c
> >> > >> > > > > > > > >>src_host,dst_host,src_port,dst_port,proto"
> >> > >> > > > > > > > >>
> >> > >> > > > > > > > >>now I need to push all the flows as a netflow format
> >> to
> >> > >> the
> >> > >> > > > > > > > >>netflow collector (nfdump). Could you give me some
> >> advice
> >> > >> how
> >> > >> > > to
> >> > >> > > > > > > > >>configure that ?
> >> > >> > > > > > > > >>Thank you
> >> > >> > > > > > > > >
> >> > >> > > > > > > > >>_______________________________________________
> >> > >> > > > > > > > >>pmacct-discussion mailing list
> >> > >> > > > > > > > >>http://www.pmacct.net/#mailinglists
> >> > >> > > > > > > > >
> >> > >> > > > > > > > >
> >> > >> > > > > > > > >_______________________________________________
> >> > >> > > > > > > > >pmacct-discussion mailing list
> >> > >> > > > > > > > >http://www.pmacct.net/#mailinglists
> >> > >> > > > > > > >
> >> > >> > > > > > > > --
> >> > >> > > > > > > > Nikola
> >> > >> > > > > >
> >> > >> > > > > >
> >> > >> > > > > > --
> >> > >> > > > > > Nikola
> >> > >> > > > >
> >> > >> > > > > _______________________________________________
> >> > >> > > > > pmacct-discussion mailing list
> >> > >> > > > > http://www.pmacct.net/#mailinglists
> >> > >> > > > >
> >> > >> > >
> >> > >> > >
> >> > >> > >
> >> > >>
> >> > >
> >>
> >

_______________________________________________
pmacct-discussion mailing list
http://www.pmacct.net/#mailinglists

Reply via email to