Murray S. Kucherawy wrote:
> I've observed a similar thing on all platforms.
>
> To try to come up with an explanation, I ran the filter after compiling it
> with valgrind on Linux and it reported no memory leaks; everything it
> reported was easily explained. Nevertheless, there's no obvious
> justification for the size to which the process gets after running for a
> while.
Some reflexions...
dkim is a multi-threaded application. Each time it begins a new connection, it
creates (or
may create) a new thread and allocates some memory for it. When the thread
finishes, the
allocated memory is again available, but the consummed memory may still be seen
at ps display.
Well, as I said, our bigger server verifies around 50K messages a day. But in
fact it
handles around 300K-400K connections a day (say 350K). The others are rejected
before the
"eom" milter callback.
That said, even if the milter signed only, say, 50K messages a day, 300K
threads where
created, as each thread is created at "connect" callback.
Looking at what happened at our server in the last 24 hours, I see that
connection rate
ranged from around 100 connections per minute to 600 connections per minute.
Adding another information, usually the mean duration of each connection is
something
between 10 and 20 secs (say 15 secs).
Using queueing theory models, the mean number of connections being handled
simultaneously
is, roughly the arrival rate, times the service time. This value corresponds to
the number
of threads in the processus.
So, the mean over all the day, shall be something like
350000/86400 arrivals/sec * 15 secs = ~ 60 threads
But one shall consider the peaks. It's hard to say, as when the connection rate
goes to
the roof, the mean connection handling time shall fall down, as the rate of
connections
being discarded before DATA command shall increase at the same time - so at
these moments
the connection handling time shall also decrease. But the worst case can be
evaluated as
10 arrivals / sec * 15 secs -> ~ 150 threads
Now, Linux consumes more memory per thread than other OS. If I'm right, Linux
allocates,
by default, 10 MB for the stack of each new thread. So, on huge servers, I
should be
astonished seeing dkim consuming some hundreds of MBytes.
Well, this also explains why it's interesting to use the pool of workers
version of libmilter.
Looking back at dkim at our server, I restarted it and, after half hour, it's
still
consumming 7 MB on a sparc solaris computer.
Regards
JM
--
---------------------------------------------------------------
Jose Marcio MARTINS DA CRUZ http://j-chkmail.ensmp.fr
Ecole des Mines de Paris
60, bd Saint Michel 75272 - PARIS CEDEX 06
mailto:[email protected]
------------------------------------------------------------------------------
Register Now for Creativity and Technology (CaT), June 3rd, NYC. CaT
is a gathering of tech-side developers & brand creativity professionals. Meet
the minds behind Google Creative Lab, Visual Complexity, Processing, &
iPhoneDevCamp as they present alongside digital heavyweights like Barbarian
Group, R/GA, & Big Spaceship. http://p.sf.net/sfu/creativitycat-com
_______________________________________________
dkim-milter-discuss mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/dkim-milter-discuss
