Hi-
Just for completeness, it seems that this has been a consistent problem with
StringBuffers, Strings and java.
http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4637640
http://forum.java.sun.com/thread.jspa?threadID=633150&messageID=3668708
I think I have found a way to work around this, but I guess the moral of the
story is watch out for large Strings in java.
James
----- Original Message -----
From: "George Ludwig" <[EMAIL PROTECTED]>
To: "HttpClient User Discussion" <[email protected]>
Sent: Wednesday, March 15, 2006 12:37 PM
Subject: Re: Memory leak using httpclient
On Wed, 2006-03-15 at 11:10 -0500, James Ostheimer
wrote:
> One thing that really did help (the nulled
instances were not being
> collected at all before this) was removing any
stored references to the
> crawler threads. I was keeping a reference to
each running thread in a
> controller class to compute statistics on how well
I was doing (download
> speed). When I removed the reference so that each
thread was completely
> dereferenced (on its own) the memory started going
up much slower.
Keeping references to your threads should not be a
problem unto itself, unless you are not using a thread
pool. If you see the memory use slow down
significantly when you stop doing that, then there are
other issues with how you have designed the units of
work, or the statistics reporting.
I have also design/built a very large distributed
crawler. It can be very painful. If you need some more
advice, just ask.
__________________________________________________
Do You Yahoo!?
Tired of spam? Yahoo! Mail has the best spam protection around
http://mail.yahoo.com
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
