Hi Gustaf

Thank you for your replays. I am a bit slow with my responses, sorry about that.

I see the same issue on all servers now - original observation that
some servers are OK was wrong.
This is due to different maxconnections configuration, which shadowed
the memory leak.

Yes - I am using standard memory allocator in Tcl. Would you recommend
to try something different?

Thanks for TclLib explanations - very helpful.

What would you recommend as a replacement for nssession module?
All I need is to keep track on user sessions, being able to set and
read data from the session.
Sessions shall be persistent, survive server restarts.

Thanks,
~ Alex.


On Mon, Mar 17, 2008 at 7:28 AM, Gustaf Neumann <[EMAIL PROTECTED]> wrote:
> Dear Alex,
>
>  Didn't you mention earlier, that the bloat happens only on
>  one of your machines? Was this a wrong indication?
>
>  If you have a well behaved system, you can increase "maxconnections" to
>  e.g. 10000 (the parameter denotes the number of connections to be
>  served by a connection thread before it restarts) and set minthreads to eg.
>  5. This does not solve the memory-leak problem but should
>  slowdown the memory hunger, if connection thread restart
>  causes the growth. I assume, you are using the standard
>  zippy memory allocator from tcl.
>
>  I would not recommend to restart based on the size, since the server
>  will do this at a time you might not want it (e.g. during a demo). A
>  nightly reboot is the better option. The size certainly depends on
>  the amount of tcl code times number of threads and the state management
>  (use of connection-spanning tcl vars, or use of nsv).
>
>  Concerning tcllib and MD5:
>  tcllib contains actually 3 versions of md5:
>   a) a tcl-based code
>   b) a version based on critcl (see tcllib*/md5/md5c.tcl)
>   c) a version based on trf
>
>  During startup, it tries to use the "best" version.
>  (b) and (c) are accelerated versions using c-code, this might
>  explain the speed differences you are experiencing. I would
>  recommend to test first with the tcl-only version and check
>  the memory growth. If it has no effect, it seems safe to
>  use the accelerated versions.
>
>  Concerning xotcl:
>
>   - there is no special handling needed concening memory freeing.
>    as long you take care, that only those objects survive a connection
>    that you want. Typical applications load the frequently uses
>    classes during startup to speed processing of requests.
>
>   - xotcl-core of OpenACS has support to ease deallocation
>    of temporary objects at the end of a connection (e.g.
>    Object o  -destroy_on_cleanup)
>
>   - [Class allinstances] returns the classes. To check the objects,
>    use [Object allinstances]
>
>  Concerning Sig 11:
>  This seems to come from libnssession.so. I am not sure,
>  how frequently this is used (we don't use it). It seems as
>  if the source for the crash is there.
>
>  Is it possible for you to test without nssession (crashes,
>  memory consumption)?
>
>  -gustaf
>
>
>
>
>  > Hello Tom, Gustaf
>  >
>  > I am still trying to pin down the memory leak problem. Could you
>  > answer few more questions, please?
>  >
>  > 1. It looks like that nsd process grows ~1.5 - 2.0 MB every time a
>  > thread dies due to timeout or number of connections (and new one
>  > starts).
>  > Changing configuration just makes the bloating process slower of faster.
>  > I restart the nsd process when it reaches 0.5GB in size, but I hope
>  > that's a reasonable number.
>  > I tried valgrind it, but couldn't make any conclusions. Any better
>  > ideas how to debug it?
>  >
>  > 2. TclLib shall not harm IMHO, it's Tcl- only, no binary code. Please
>  > confirm that.
>  > Besides, I need it only for MD5, and standalone md5 Tcl
>  > implementations for some reason seems to be much slower.
>  > If TclLib is risky, what would be the best package/module for MD5 ?
>  > mhash may be?
>  >
>  > 3. XoTcl - I verified that I do not leave any objects which I created,
>  > they all destroyed. (at least that's what [Class allinstances]
>  > reports)
>  > However, Gustaf noted that it still may need special handling. What
>  > kind of handling? Could you point me to the code in openacs that does
>  > that?
>  > For me it would be a big hit if I need to rewrite without XoTcl, so I
>  > would like to make it working.
>  >
>  > 4. Memory leaks aside, nsd quite frequently dies on Sig 11. Sometimes
>  > just a few seconds after being started.
>  > I remember having similar issues some year+ ago, but those were fixed
>  > quickly. Anyone experiences the same?
>  >
>  > I didnt' debug it in depth yet, that's what gdb said when I quickly tried 
> it:
>  >
>  > Program received signal SIGSEGV, Segmentation fault.
>  > [Switching to Thread -1296041040 (LWP 32284)]
>  > 0xb7338ba5 in Ns_CookieGet () from
>  > /aolserver_bin/aolserver4.5/lib/libnssession.so
>  >
>  > Thanks,
>  > ~ Alex.
>  >
>  >
>  > On Tue, Mar 4, 2008 at 3:29 AM, Gustaf Neumann <[EMAIL PROTECTED]> wrote:
>  >
>  >> Dear Alex,
>  >>
>  >>  The good news is that it bloats only on one server,
>  >>  so the problem must on that configuration.
>  >>
>  >>  There were a couple of memory leaks reported for postgres 7.4,
>  >>  not sure if these hit you (google around):
>  >>  http://www.postgresql.org/docs/7.4/static/release-6-5.html
>  >>
>  >>  do you have a chance to install on the affected machine
>  >>  as well postgres 8.2 (maybe in addition to 7.4)?
>  >>  it would be interesting to see, if you still see a bloat.
>  >>
>  >>  i found a small glitch in your "crude script":
>  >>  The last line of the xotcl install should read:
>  >>
>  >>   execute make install-aol
>  >>
>  >>  instead of
>  >>
>  >>   execute make install
>  >>
>  >>  -gustaf neumann
>  >>
>  >>  Alex schrieb:
>  >>
>  >>
>  >>
>  >>> Tom, Gustaf, All
>  >>>
>  >>  >
>  >>  > Yes, that happens even if the same page is requested.
>  >>  > Could you tell me more about pre-queue filters?
>  >>  >
>  >>  > The interesting thing is that I only see this memory leak issue on one
>  >>  > of my servers.
>  >>  > All servers run x86 Debian, though might be slightly differently
>  >>  > dist-upgraded, Postgres version differs 7.4 vs 8.2
>  >>  > The website code itself, nsd config files are exactly the same on all 
> servers.
>  >>  >
>  >>  > Over the weekend I tried to rebuild tcl, tcllib, xotcl, aolserver.
>  >>  > The details how I did it can be found here
>  >>  > 
> http://alex-andr.org/notes/tiki-index.php?page=AOLServer+setup+with+XOTcl
>  >>  > I used latest versions however, tcl 8.4.18, tcllib 1.10, xotcl 1.6.0
>  >>  >
>  >>  > I even made a crude script that fetches the sources and builds a
>  >>  > stand-alone aolserver installation,
>  >>  > in order to make sure the build results are reproducible.
>  >>  > http://alex-andr.org/notes/tiki-index.php?page=nsd+setup+script
>  >>  >
>  >>  > Still, even when all [hopefully] binaries are being copied to all
>  >>  > servers, only one of them
>  >>  > experience the same issue with huge memory leak on each page request.
>  >>  >
>  >>  > Any insightful thoughts?
>  >>  >
>  >>  > Thanks,
>  >>  > ~ Alex.
>  >>  >
>  >>  >
>  >>  > On Fri, Feb 29, 2008 at 6:23 PM, Tom Jackson <[EMAIL PROTECTED]> wrote:
>  >>  >
>  >>  >> Actually 2MB / request sounds like a massive memory leak. Maybe 
> something
>  >>  >>  isn't getting cleaned up?
>  >>  >>
>  >>  >>  Question: does this happen if you request the same page over and 
> over?
>  >>  >>
>  >>  >>  I discovered one way to get a massive leak in 4.5: register a 
> prequeue filter.
>  >>  >>
>  >>  >>  Note that both XoTCL and tcllib are compiled third party modules, 
> which is
>  >>  >>  usually where you get memory leaks, not in the modules which have 
> been used
>  >>  >>  with AOLserver for years. My guess would be tcllib, or just failing 
> to
>  >>  >>  release something big.
>  >>  >>
>  >>  >>  tom jackson
>  >>  >>
>  >>  >>
>  >>  >>  On Friday 29 February 2008 11:47, Alex wrote:
>  >>  >>  > What would be a "normal" size of 32 bit nsd process? How can it be 
> reduced?
>  >>  >>  > One of my servers only has 1G or ram, and I am forced to restart 
> nsd
>  >>  >>  > every so often, when it uses up almost all the memory.
>  >>  >>  > Running 4.5 + XoTCL + tcllib, it seams to grow up to 2MB on each 
> page
>  >>  >>  > request .
>  >>  >>
>  >>  >>
>  >>  >>
>  >>  >>
>  >>  >> --
>  >>  >>  AOLserver - http://www.aolserver.com/
>  >>  >>
>  >>  >>  To Remove yourself from this list, simply send an email to <[EMAIL 
> PROTECTED]> with the
>  >>  >>  body of "SIGNOFF AOLSERVER" in the email message. You can leave the 
> Subject: field of your email blank.
>  >>  >>
>  >>  >>
>  >>  >
>  >>  >
>  >>  > --
>  >>  > AOLserver - http://www.aolserver.com/
>  >>  >
>  >>  > To Remove yourself from this list, simply send an email to <[EMAIL 
> PROTECTED]> with the
>  >>  > body of "SIGNOFF AOLSERVER" in the email message. You can leave the 
> Subject: field of your email blank.
>  >>  >
>  >>
>  >>
>  >>  --
>  >>  AOLserver - http://www.aolserver.com/
>  >>
>  >>  To Remove yourself from this list, simply send an email to <[EMAIL 
> PROTECTED]> with the
>  >>  body of "SIGNOFF AOLSERVER" in the email message. You can leave the 
> Subject: field of your email blank.
>  >>
>  >>
>
>


--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> 
with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: 
field of your email blank.

Reply via email to