Yes, thanks Steve, I am talking about this issue.
If somebody is interested I put the result of a 12 hours, 20 000 emails
hprof session on ftp.
The zip contains the hprof output at shutdown, the garbage collector log
plus additional data on the run. I have attached the latter to this email
too, with a short evaluation of the measurement at its end. You can download
the entire package here:
http://130.94.18.70/pub/james/james.hprof.rar 4MB
http://130.94.18.70/pub/james/james.hprof.zip 7MB
----- Original Message -----
From: "Steve Short" <[EMAIL PROTECTED]>
To: "James Developers List" <[EMAIL PROTECTED]>
Sent: Sunday, August 11, 2002 4:12 AM
Subject: RE: memory leak in scheduler in 2.0a3, is this fixed?
He's talking about this (copied from a mail I sent to Paul Hammant about
a week ago).:
The DefaultTimeScheduler bug,
http://issues.apache.org/bugzilla/show_bug.cgi?id=9275, has crept up
again in the James developer list and Noel suggested I contact you
regarding a fix for this. Do you have any news ?
Just to overwhelm you with information :),
Original James Dev thread
http://www.mail-archive.com/[email protected]/msg02216.html
Orignal Avalon Dev thread
http://www.mail-archive.com/[email protected]/msg08676.html
New James Dev thread titled Handler timeout/trigger (can't find in
archive to provide URL)
Regards
Steve
> -----Original Message-----
> From: Peter M. Goldstein [mailto:[EMAIL PROTECTED]]
> Sent: Saturday, August 10, 2002 5:26 PM
> To: 'James Developers List'
> Subject: RE: memory leak in scheduler in 2.0a3, is this fixed?
>
>
>
> Hontvari,
>
> Some more detail here would really help. A link to the
> messages in the mail archive you mention would be great.
>
> Whether or not this issue has been addressed, I personally
> don't feel that we're in a position to release a new version.
> There is some substantial bug fixing to be done (closing all
> the open James bugs) as well as some new features that are on
> the immediate horizon (most notably IMAP). We also need to
> validate and roll in all uncommitted patches that have been
> posted to the group (catching up nicely on this one - we've
> got one really big one left AFAIK). I'd say that once those
> tasks are complete we'd be in a position to talk about a new
> version. Of course, that's just my opinion.
>
> --Peter
>
> > -----Original Message-----
> > From: Noel J. Bergman [mailto:[EMAIL PROTECTED]]
> > Sent: Saturday, August 10, 2002 5:13 PM
> > To: James Developers List
> > Subject: RE: memory leak in scheduler in 2.0a3, is this fixed?
> >
> > Which memory leak? We've done a lot of work in the CVS on resource
> leaks.
> > Have we fixed the specific one you spotted? I don't know,
> because it
> is
> > possible we missed one. Can you be more specific?
> >
> > --- Noel
> >
> > -----Original Message-----
> > From: Hontvari Jozsef [mailto:[EMAIL PROTECTED]]
> >
> > There is a memory leak related to SMTPHandler and Avalon scheduler.
> When
> > handling a large number of emails this consumes the entire memory
> after a
> > few hours. Is this fixed in the meantime?
> >
> > I tracked down this leak using hrprof and also read about
> this in the
> mail
> > archive, but I cannot find wheter it is solved or not in the newer
> > version.
> >
> > If it s solved wouldn't it be time to release a new version?
> >
> >
> > --
> > To unsubscribe, e-mail: <mailto:james-dev-
> > [EMAIL PROTECTED]>
> > For additional commands, e-mail: <mailto:james-dev-
> > [EMAIL PROTECTED]>
>
>
>
> --
> To unsubscribe, e-mail:
> <mailto:james-dev-> [EMAIL PROTECTED]>
> For
> additional commands,
> e-mail: <mailto:[EMAIL PROTECTED]>
>
>
--
To unsubscribe, e-mail: <mailto:[EMAIL PROTECTED]>
For additional commands, e-mail: <mailto:[EMAIL PROTECTED]>
CONFIGURATION:
-James 2.0a3 installed as a service, POP3 service disabled
-Windows 2000 Server SP2 (on a very good Internet connection),
-SunJava 1.4.1-beta VM in client mode
(in contrast to server mode, in server mode the service crashed after a few hours)
-Pentium III 800 MHz 128 MB memory
ACTIVITY:
-service had run for 12 hours
-James sent about 8000 emails,
-it received about 8000 email from localhost, and 4000 from remote hosts (bounces)
-incoming emails are only logged and immediately deleted.
-so the total traffic is 20 000 emails, usually 10 kB each
-James was fed by another Java application which had a lower priority,
through SMTP. It would had fed about 1 email / second in optimal circumstances
(single thread, 1 s sleep between each email), but the actual result
was 1 email / 5 second.
-not that hprof also affects the performance
-there were no accumulated backlog (in the spool or outgoing directory)
-CPU consumption: usually 100%, from which 70% is James, 30% is the System process.
The latter is also related to James.
Interestingly simetimes the proc. usage goes down to almost 0% for 10-120 s.
Maybe those stops indicate garbage collection.
After stopping the feed it immediately goes down to 0%, on the other hand
allocated memory doesn't change)
-CPU consumption of the feeder application is practically unmeasurable (a few seconds
total)
EVALUATION:
I used HPjmeter-1.5 (freely downloadable) to show the log in understandable way
Memory leak guesses of HPjmeter-1.5:
james.hprof.txt: Memory Leaks: bytes held
41018960 org.apache.avalon.excalibur.collections.SynchronizedPriorityQueue@b546530
41018944 org.apache.avalon.excalibur.collections.BinaryHeap@b512f00
41018920 java.lang.Object[]@1071cef8
2694640 java.util.Hashtable@b8ea508
2694600 java.util.Hashtable$Entry[]@f934880
264112 java.lang.ref.Finalizer@103a2570
259680 java.util.LinkedHashMap@b485f38
164056 sun.misc.URLClassPath@b74a9a8
The first three rows point to a single problem: the Scheduler block.
Memory allocation hotspots (these are not memory leaks!,
however they are responsible for 60% of the memory allocations in kilobytes):
TRACE 18874:
java.io.BufferedWriter.<init>(BufferedWriter.java:87)
java.io.BufferedWriter.<init>(BufferedWriter.java:70)
java.io.PrintWriter.<init>(PrintWriter.java:103)
org.apache.james.util.InternetPrintWriter.<init>(InternetPrintWriter.java:32)
org.apache.james.smtpserver.SMTPHandler.handleConnection(SMTPHandler.java:136)
org.apache.avalon.cornerstone.blocks.connection.ConnectionRunner.run(Connection.java:167)
org.apache.avalon.excalibur.thread.impl.ExecutableRunnable.execute(ExecutableRunnable.java:47)
org.apache.avalon.excalibur.thread.impl.WorkerThread.run(WorkerThread.java:86)
TRACE 18873:
java.nio.HeapByteBuffer.<init>(HeapByteBuffer.java:39)
java.nio.ByteBuffer.allocate(ByteBuffer.java:304)
sun.nio.cs.StreamEncoder$CharsetSE.<init>(StreamEncoder.java:308)
sun.nio.cs.StreamEncoder$CharsetSE.<init>(StreamEncoder.java:288)
sun.nio.cs.StreamEncoder$CharsetSE.<init>(StreamEncoder.java:272)
sun.nio.cs.StreamEncoder.forOutputStreamWriter(StreamEncoder.java:68)
java.io.OutputStreamWriter.<init>(OutputStreamWriter.java:93)
java.io.PrintWriter.<init>(PrintWriter.java:103)
org.apache.james.util.InternetPrintWriter.<init>(InternetPrintWriter.java:32)
org.apache.james.smtpserver.SMTPHandler.handleConnection(SMTPHandler.java:136)
org.apache.avalon.cornerstone.blocks.connection.ConnectionRunner.run(Connection.java:167)
org.apache.avalon.excalibur.thread.impl.ExecutableRunnable.execute(ExecutableRunnable.java:47)
org.apache.avalon.excalibur.thread.impl.WorkerThread.run(WorkerThread.java:86)
Both of them are related to the following statement:
out = new InternetPrintWriter(socket.getOutputStream(), true);
Garbage collection log:
The last garbage collection log entries:
43161.2: [Full GC 65086K->50779K(65088K), 55.5858317 secs]
43342.2: [Full GC 65088K->48506K(65088K), 110.9842548 secs]
43651.9: [Full GC 65088K->51730K(65088K), 108.7824851 secs]
(The last entry means the collection occured 5 minutes after the previous
one, it took almost 2 minutes and it released 13 MB,
however 52 MB memory was still remained allocated.
--
To unsubscribe, e-mail: <mailto:[EMAIL PROTECTED]>
For additional commands, e-mail: <mailto:[EMAIL PROTECTED]>
