On 24/01/2013 12:55, Sven Van Caekenberghe wrote:
On 24 Jan 2013, at 13:37, CHRIS BAILEY <[email protected]> wrote:
--- On Thu, 24/1/13, Sven Van Caekenberghe <[email protected]> wrote:
From: Sven Van Caekenberghe <[email protected]>
Subject: Re: [Pharo-users] Memory Leak with Zodiac?
To: "A friendly place where any question about pharo is welcome"
<[email protected]>
Date: Thursday, 24 January, 2013, 10:41
Hi Chris,
On 24 Jan 2013, at 11:02, CHRIS BAILEY <[email protected]>
wrote:
I'm having a little trouble with a possible memory leak
with Zodiac. I'm running Pharo 1.4 Summer on Centos
6.3/Fedora 18 for deploy/dev. The plugin is using
libssl.so.0.9.8, and I also seem to have another version
kicking about which uses libssl.so.6 and am not really sure
where I got that from. It may be an older one from the
http://code.google.com/p/squeakssl/ site. Both have the
same effect, i.e. when running 100 timesRepeat: [ ZnClient
new url: 'https://www.google.com'; get ] VIRT and RES memory both
go up about 70mb, but just using http doesn't accumulate
any. I tried it in Pharo 2.0 which was the same, but the new
NBCog doesn't include the libSqueaKSSL.so so I was only
copying it over.
Has anyone used it successfully in the same
environment? I'll try it on Ubuntu when I get a chance.
Thanks
Chris
Thanks for testing this ;-)
Note that although ZnClient is most often used to do just
one request, it was also designed to be reused to do
multiple request to the same host:port. Even when not taking
care of resource cleanup, for casual use #close to the
socket will eventually be sent where necessary due to
finalization. Helping a bit makes for better performance.
Either one of the following would be better:
100 timesRepeat: [ ZnClient new beOneShot; get: 'https://www.google.com' ].
100 timesRepeat: [ ZnClient new get: 'https://www.google.com'; close ].
| client |
client := ZnClient new.
100 timesRepeat: [ client get: 'https://www.google.com' ].
client close.
Now, Zodiac TLS/SSL Streams are more resource intensive:
they create ZdcPluginSSLSession instances which represent
native native data structures from the OS's SSL libraries.
These are only cleaned up when #destroy is sent, which is
one by ZdcSecureSocketStream>>#close. I didn't write
any explicit code to do this by finalization, so that
probably does not happen. Maybe we should experiment with
adding a #finalize to ZdcPluginSSLSession as an equivalent
to #destroy.
It would be cool if you could test again. Hopefully memory
consumption improves.
Hi Sven, thanks for the quick reply.
Sorry, I didn't consider that when I reproduced it with the simplest example I
could think of! I'm actually using it in the context of a single ZnClient being
held open for an entire Seaside session which talks to an external web service.
I therefore only send a close when the session ends. What solution would you
recommend, as it is quite easy currently to run the image out of memory before
the session gets unregistered.
That is more a question about Seaside session management, which would be
similar to managing database connections. My Seaside knowledge is rusty, but if
I remember correctly, you can arrange for a method to be called on your custom
Seaside session object whenever a session expires, there you could do some
cleanup. Making sure sessions are expired quickly enough is also tuneable in
various ways.
What I have done in some projects is to have a single small pool of ZnClients
ready to talk to a specific service. Each session then calls out to the pool to
access the external service over HTTP(S). Of course, you have to be careful
with multithreaded access, but also with timeouts, resource starvation,
failures and cleanup, unwanted state sharing or caching, etc..
Or you could just try to be aggressive in closing the HTTP(S) clients after
each use. There is not much point in keep those connections open for more than
a couple of seconds as the other end will close them anyway.
Thanks for that, I've just gone with closing it each time and it works
fine now :-) I've had DBXConnections and the http version of ZnClient
working fine with one instance for the session, and such that it gets
closed at unregister but they don't accumulate any memory when used and
left open. Each call was generating about 1mb so it was very easy to run
out of memory even with quite strict session cleanup.
Chris
