On Sunday 14 June 2009 21:03:38 [email protected] wrote:
> Fix for memory leak on client disconnect.
> Specifically ConnectedSocketAdapators and socketobjects
> were not being collected, which resulted in leakage. This
> is now fixed, and confirmed by attaching an interpreter to
> the system and actively rummaging around in the garbage collector
> whilst the server is running.
This is a fairly important bugfix on /trunk, and probably the biggest driver
for a new release. Specifically, for long running servers handling lots of
connections you can see an increase in memory usage proportional to the
number of connections.
Using the new embedded console it's made it relatively easy to identify
repeatable circumstances which make this bug manifest itself, and as a
result, I've spent some time hunting this down and squashing it.
Not only that the console has made it relatively easy to identify whether this
has been successful or not.
trunk/Sketches/MPS/Random/ChatServer.py
Contains a simple chat server, which watches itself, & dumps the contents of
the runqueue when it changes. It listens for connections on port 1234 and
performs a basic echo protocol on that. It also has an embedded python
console on port 8765, thanks to this code:
from Kamaelia.Chassis.ConnectedServer import ServerCore
from Kamaelia.Experimental.PythonInterpreter import InterpreterTransformer
from Kamaelia.Util.PureTransformer import PureTransformer
from Kamaelia.Chassis.Pipeline import Pipeline
import socket
def NetInterpreter(*args, **argv):
return Pipeline(
PureTransformer(lambda x: str(x).rstrip()),
PureTransformer(lambda x: str(x).replace("\r","")),
InterpreterTransformer(),
PureTransformer(lambda x: str(x)+"\r\n>>> "),
)
ServerCore(protocol=NetInterpreter,
socketOptions=(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1),
port=8765).activate()
(This may become a general prefab for use in any system)
Once logged in I can type this:
import gc
import pprint
X=Y=Z=x=y=z=None
gc.collect()
Z=[]
for z in gc.get_objects():
try:
z.__class__
Z.append(z)
except AttributeError:
pass
Then look for specific rogue objects this way:
y=[x for x in Z if "Connected" in x.__class__.__name__]
y=[x for x in Z if "socket" in x.__class__.__name__]
Figure out how many referrers they have:
[ len(gc.get_referrers(x)) for x in y]
And specifically useful determine how many things of each class are
instantiated in the running system:
X = [x.__class__ for x in Z]
y=[x.__name__ for x in X]
names={}
for name in y:
names[name] = names.get(name, 0)+1
pprint.pformat(names)
This last one is particularly useful for identifying patterns of objects which
are leaking, which gives a good clue as to what's going on. You can then also
use this afterwards to confirm success - which I've done.
It was also remarkably interesting to do it this way because you can also
check the referrers of an object, whilst it's running, allowing to check
*other* active stack frames (and what files/line numbers they're in) which
have rogue references.
It's a remarkably odd/amusing thing to do to debug a system by literally
rummaging around inside the running code :)
This also wasn't a theoretical bug, it affects the greylisting server which I
use... :-)
Regards,
Michael.
--
http://yeoldeclue.com/blog
http://twitter.com/kamaelian
http://www.kamaelia.org/Home
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"kamaelia" group.
To post to this group, send email to [email protected]
To unsubscribe from this group, send email to
[email protected]
For more options, visit this group at
http://groups.google.com/group/kamaelia?hl=en
-~----------~----~----~----~------~----~------~--~---
