On Mon, Jun 25, 2018 at 6:49 PM, Rasto Rickardt <pho...@axfr.org> wrote:

> Hello,
>
> If there is no more RAM available, i would test sqlite3 plugin, as
> sqlite3 is more suited for limited resources usage. Youl will possibly
> need to change your workflow to export sqlite3 files and load it
> somewhere else, but it should be a lot cheaper memory wise.
>
> Thanx. Seems that I have to stick with mysql at least at this phase.

This one is swapping pretty heavily, i would suggest to set to around 10
> for 4GB RAM.
>
> sysctl vm.swappiness=10 so it will start using swap when available RAM
> is around 400MB.
>
This is already done.


> r.
>
> On 06/25/2018 05:35 PM, Alex K wrote:
> > Thanks Dariush. Appreciate your feedback.
> >
> > I was testing several stripped down kernels by compiling and removing
> > most of unused modules. The gain I had was in the range of few MB.
> > Seems that I have to find a more elegant approach at uacctd
> > configuration since with current setup I am loading several mysql
> > plugins so as to be able to filter traffic with directions, networks and
> > ports.
> > I was testing an edge case where these plugins may rise up to 210
> > instances thus leading to this memory usage. Normally this will not be
> > done so the normal case will be using 2.5 GB of RAM instead of 3.5 GB.
> > Thus in normal cases seems that I am ok.
> >
> > Cheers
> >
> >
> > On Mon, Jun 25, 2018 at 6:07 PM, Dariush Marsh-Mossadeghi
> > <dari...@gravitas.co.uk <mailto:dari...@gravitas.co.uk>> wrote:
> >
> >     OK, so it’s effectively an embedded system scenario, with a fixed
> >     config hardware, or similar
> >     No silver bullets or one-liner fixes here :-\
> >
> >     You’ve a number of options to slim down your memory footprint
> >     - Strip Debian of all the packages you don’t need, learn a lot about
> >     kernel tuning, and tune the kernel to your needs.
> >     - Start looking at one of the more embedded systems oriented
> >     distros, rather than OOTB Debian. Although Debian is skinnier than
> >     most, there’s a lot that can be done to minimise footprint. The fact
> >     that you’ve got a 64bit Debian9 running on it makes me fairly
> >     optimistic that you could run pretty much any Debian derivative.
> >     - Profiling your userspace software stack and see if there are any
> >     dependencies you don’t need, strip them out. Whether you can do this
> >     will very much depend on the modularity of the components.
> >
> >     All of the above lead you down the path of maintaining forks and
> >     compiling your own packages, to a greater or lesser extent. That’s a
> >     maintenance overhead you want to avoid unless you have to.
> >     Having said all that, if you’re looking at deploying thousands of
> >     units, the economics of software maintenance may stack up for you.
> >
> >     HTH
> >     Dariush
> >
> >
> >>     On 25 Jun 2018, at 14:32, Alex K <rightkickt...@gmail.com
> >>     <mailto:rightkickt...@gmail.com>> wrote:
> >>
> >>     Let me change the posting style... :)
> >>
> >>     On Mon, Jun 25, 2018 at 3:51 PM, Dariush
> >>     Marsh-Mossadeghi <dari...@gravitas.co.uk
> >>     <mailto:dari...@gravitas.co.uk>> wrote:
> >>
> >>         OK, so we’re moving from bottom-posting to top-posting…
> >>         that’ll make it interesting for other readers ;-)
> >>
> >>         The output of free doesn’t look desperate, but it is starting
> >>         to look a bit tight.
> >>         You’ve got about a gig of buffer/cache, which the kernel will
> >>         evict if it needs it.
> >>         You’ve got 200M of genuinely free memory.
> >>         What is potentially a little concerning is the gig of swap in
> >>         use, that may or may not be a problem depending on what else
> >>         is running on the box and how it’s memory use varies over time.
> >>
> >>         I’m not intimatley familiar with the internals of uacctd’s
> >>         mysql plugins, but my advice would be:
> >>
> >>         - Monitor the memory usage. If it doesn’t vary much over time
> >>         you could go on for years and be just fine. Keep a close watch
> >>         on swap usage, if that varies a lot or grows over time you’ll
> >>         want to do something about it.
> >>
> >>     Monitoring of last 2 weeks is showing that free mem is fluctuating
> >>     form 50M to 220 MB and swaps remains steady. Thats why I am trying
> >>     to find a solution.
> >>
> >>
> >>         - Read up on OOM Killer, what it does and how it behaves. If
> >>         you see OOM Killer entries in your logs, it’s time to think
> again
> >>
> >>     I am aware of that. thanx
> >>
> >>
> >>         - Can you put some more memory in the box? 8GB of RAM will
> >>         cost you about £60. How much is your time worth ?
> >>
> >>     This is not an option. What I have is 4 GB. This is not just a
> >>     personal project. There are going to be thousands of such
> >>     installations
> >>
> >>     HTH
> >>     Dariush
> >>
> >>
> >>>     On 25 Jun 2018, at 12:32, Alex K <rightkickt...@gmail.com
> >>>     <mailto:rightkickt...@gmail.com>> wrote:
> >>>
> >>>     Thanx for the reply.
> >>>
> >>>     The output of free is the following:
> >>>
> >>>     free
> >>>                 total        used        free        shared
> >>>     buff/cache   available
> >>>     Mem:        4046572     2832576      152012      784240
> >>>     1061984      204248
> >>>     Swap:       3906556     1086080     2820476
> >>>
> >>>     While is as below when stopped:
> >>>
> >>>     free
> >>>                   total        used        free      shared
> >>>     buff/cache   available
> >>>     Mem:        4046572      520352     3223884       14916
> >>>     302336     3286952
> >>>     Swap:       3906556      485040     3421516
> >>>
> >>>     Seems that mysql plugins are reserving quite some memory as they
> >>>     list first in htop when sorted with memory.
> >>>
> >>>     Thanx,
> >>>     Alex
> >>>
> >>>
> >>>     On Mon, Jun 25, 2018 at 2:22 PM, Dariush
> >>>     Marsh-Mossadeghi <dari...@gravitas.co.uk
> >>>     <mailto:dari...@gravitas.co.uk>> wrote:
> >>>
> >>>
> >>>>         On 25 Jun 2018, at 11:54, Alex K <rightkickt...@gmail.com
> >>>>         <mailto:rightkickt...@gmail.com>> wrote:
> >>>>
> >>>>         Hi all,
> >>>>
> >>>>         I have a setup with uacctd monitoring traffic of several
> >>>>         interfaces through NFLOG.
> >>>>         With uacctd stopped I see that the server (a relatively
> >>>>         small device with 4 GB of RAM) consumes 450MB of RAM. Once I
> >>>>         start uacctd the mem usage goes up to 3.5 GB. I am using
> >>>>         mysql plugin and this is running on Debian9 64 bit.
> >>>>
> >>>>         Is there any tweaks I can use to put a limit on the memory
> >>>>         usage of uacctd.
> >>>>
> >>>>         Thanx,
> >>>>         Alex
> >>>>         _______________________________________________
> >>>>         pmacct-discussion mailing list
> >>>>         http://www.pmacct.net/#mailinglists
> >>>>         <http://www.pmacct.net/#mailinglists>
> >>>
> >>>
> >>>         tldr; post the output of the free command when uacctd is
> >>>         running and I’ll do my best to interpret it for you :-)
> >>>
> >>>         Is it _really_ using it, or is it just not completely unused?
> >>>
> >>>         The linux kernel generally does pretty good job of keeping
> >>>         stuff that might be useful in memory, but getting rid of it
> >>>         very very quickly if the space is needed for something else.
> >>>         The challenge you face is that most of the userspace tools
> >>>         come with a long list of caveats about what they appear to
> >>>         report vs what’s really happening, this is mainly due to the
> >>>         way the kernel shares memory between processes, and its use
> >>>         of as much memory as possible for various caches.
> >>>
> >>>         Some threads worth reading if you’re not familiar with the
> >>>         in’s and out’s of linux kernel memory management...
> >>>
> >>>         https://www.linuxatemyram.com/
> >>>         https://stackoverflow.com/questions/4802481/how-to-see-
> top-processes-sorted-by-actual-memory-usage
> >>>         <https://stackoverflow.com/questions/4802481/how-to-see-
> top-processes-sorted-by-actual-memory-usage>
> >>>         https://stackoverflow.com/questions/3784974/want-to-
> know-whether-enough-memory-is-free-on-a-linux-machine-to-deploy-a-new-app/
> >>>         <https://stackoverflow.com/questions/3784974/want-to-
> know-whether-enough-memory-is-free-on-a-linux-machine-to-deploy-a-new-app/
> >
> >>>
> >>>         HTH
> >>>         Dariush
> >>
> >
> >
> >     _______________________________________________
> >     pmacct-discussion mailing list
> >     http://www.pmacct.net/#mailinglists
> >     <http://www.pmacct.net/#mailinglists>
> >
> >
> >
> >
> > _______________________________________________
> > pmacct-discussion mailing list
> > http://www.pmacct.net/#mailinglists
> >
>
> _______________________________________________
> pmacct-discussion mailing list
> http://www.pmacct.net/#mailinglists
>
_______________________________________________
pmacct-discussion mailing list
http://www.pmacct.net/#mailinglists

Reply via email to