Thanks Bill. It's on an obsolete HPUX system, ancient RH linux 5.x, and a
much more recent CentOS 6.x, as well as Windows 7. Similar problem on all.
Mostly TCP sockets, but also an occasional UDP. I should have mentioned
all that. There are some tiny data structures that persist in memory which
is the reason that it's long running and that the restart solution has been
resisted. I wondered about the socket buffers rather than those tiny data
structures since I imagined that the socket buffers could account for a much
larger memory increment each time than the data structures. But it looks
like they are becoming the prime suspects now. I've pretty much exhausted
the circular refs question, but do keep it in the back of my mind for
another try if all else fails. It may just be time to bite the bullet,
write the data to disk and restart periodically, and give up the fight.
Thanks again.
Jim
-----Original Message-----
From: Bill Ricker
Sent: Thursday, June 02, 2016 10:08 PM
To: Boston PM
Subject: Re: [Boston.pm] perl sockets and memory use
On Thu, Jun 2, 2016 at 12:04 PM, <[email protected]> wrote:
I'm dealing with a very long-running simple real-time program (weeks, even
months) which opens and closes sockets continually to a variety of
locations, for very short transactions. There is very slow growth of
virtual and real memory use (especially the real memory) of this program,
on a variety of *NIX types and apparently proportional to the activity
level, that raises the following questions:
Does it have the ability to restart and continue periodically?
That's the easiest way to give memory back.
- I assume that the kernel has to allocate buffer space for each socket
created, which presumably it charges to the pid of the socket requester
(the problem perl program in this case)?
I would expect both kernel and proc socket buffers would be managed
better but don't know.
- I once ran across a comment somewhere in perldoc or code comments that
when perl requests memory from the system it *never* gets released back to
the system -- perl may reuse it itself after getting it garbage collected,
but it doesn't get returned to the kernel. Could have been a very old
piece of doc -- is this still true?
It's (almost) true of anything built with the C library, which Perl is;
and Perl does NOT try to.
Perl does garbage collect its own heap.
But if virtual memory increases more than real albeit both slowly, that
suggests garbage collection on the heap is not solving all fragmentation
and/or memory leaks.
Perl code can 'leak' memory such that the G/C doesn't free it by creating
circular data structures; solution is weak refs. These leaks could be in
core or libs not just your code.
The underlying C code and runtime can also leak ... and the malloc heap
does NOT g/c; if it stays fragmented, it will have to grow the process.
- Strictly speaking perl isn't directly requesting the buffer memory, just
the socket, so I'm not sure that even if the statement above is true that
the buffer space doesn't go back to the system when the socket closes.
Depends how much socket memory is in kernel vs process ...
But it's possible it's the string copy buffers of processing not the
underthehood sockets and heap fragmentation.
Step one is to find out if it's Perl's heap or the rest of the process that
is growing.
If it's fragmentation in the Perl heap, the classic solution for no-growth,
run-forever is to take over managing whatever object is causing the problem
by allocating a fixed ring buffer of them.
(Applying that process to the C Malloc heap would mean re-writing the
Perl runtime. Nope.)
- Sooooo, it's a long way of asking if closing the socket in perl returns
the buffer memory back to the system?
- And finally, if the buffer memory does stay with the perl pid, is perl
actually able to use that memory later for other purposes (since it seems
like an indirect memory allocation) ?
PS I realize that socket work is not generally done this way (i.e. so
long-running) but would like to better understand the situation before
undertaking restructuring.
and that may require understanding OS and your C lib socket libraries.
This is Unix/Linux with Gnu C ? versions?
Are the sockets TCP, UDP, or Berkley (local) sockets ?
(But i suspect it's Perl &/or Malloc heap fragmentation &/or leakage. You
could re-build Perl from source with an instrumented Malloc to get Malloc
to confess.)--
Bill Ricker
[email protected]
https://www.linkedin.com/in/n1vux
_______________________________________________
Boston-pm mailing list
[email protected]
http://mail.pm.org/mailman/listinfo/boston-pm
_______________________________________________
Boston-pm mailing list
[email protected]
http://mail.pm.org/mailman/listinfo/boston-pm
