Thank you very much for all the hints.
I'll try checking the points you mentioned later.
However, for test purposes I've moved the database to another PC within
the same network and the problem seems to have gone away!
Other PC (Database server) runs SuSE 9.0 (with MySQL 3.23.55), 2 Pentium
4 processors, 1 GB RAM.
Can this help to pin-point the problem.
I can leave this setup (web server on one PC and that database on
another), but I'm getting the following error once in the while
(normally when there are many hits one after the other).
What can be the problem?
I've tried using the current database driver (com.mysql.jdbc.Driver),
but it didn't solve the problem.
Any ideas?
Thanks a lot,
Roman Zhovtulya
P.S. Here is an exception:
****************
javax.servlet.ServletException: Communication link failure:
java.io.IOException
at
org.apache.jasper.runtime.PageContextImpl.handlePageException(PageContex
tImpl.java:471)
at org.apache.jsp.go$jsp._jspService(go$jsp.java:352)
at
org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:107)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:853)
at
org.apache.jasper.servlet.JspServlet$JspServletWrapper.service(JspServle
t.java:201)
at
org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:381)
at
org.apache.jasper.servlet.JspServlet.service(JspServlet.java:473)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:853)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Applica
tionFilterChain.java:247)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilt
erChain.java:193)
at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValv
e.java:243)
at
org.apache.catalina.core.StandardPipeline.invokeNext(StandardPipeline.ja
va:566)
at
org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:4
72)
at
org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:943)
at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValv
e.java:190)
at
org.apache.catalina.core.StandardPipeline.invokeNext(StandardPipeline.ja
va:566)
at
org.apache.catalina.valves.CertificatesValve.invoke(CertificatesValve.ja
va:246)
at
org.apache.catalina.core.StandardPipeline.invokeNext(StandardPipeline.ja
va:564)
at
org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:4
72)
at
org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:943)
at
org.apache.catalina.core.StandardContext.invoke(StandardContext.java:234
7)
at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java
:180)
at
org.apache.catalina.core.StandardPipeline.invokeNext(StandardPipeline.ja
va:566)
at
org.apache.catalina.valves.ErrorDispatcherValve.invoke(ErrorDispatcherVa
lve.java:170)
at
org.apache.catalina.core.StandardPipeline.invokeNext(StandardPipeline.ja
va:564)
at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java
:170)
at
org.apache.catalina.core.StandardPipeline.invokeNext(StandardPipeline.ja
va:564)
at
org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:468
)
at
org.apache.catalina.core.StandardPipeline.invokeNext(StandardPipeline.ja
va:564)
at
org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:4
72)
at
org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:943)
at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.
java:174)
at
org.apache.catalina.core.StandardPipeline.invokeNext(StandardPipeline.ja
va:566)
at
org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:4
72)
at
org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:943)
at
org.apache.ajp.tomcat4.Ajp13Processor.process(Ajp13Processor.java:458)
at
org.apache.ajp.tomcat4.Ajp13Processor.run(Ajp13Processor.java:551)
at java.lang.Thread.run(Thread.java:536)
root cause
java.sql.SQLException: Communication link failure: java.io.IOException
at org.gjt.mm.mysql.MysqlIO.clearAllReceive(Unknown Source)
at org.gjt.mm.mysql.MysqlIO.sqlQueryDirect(Unknown Source)
at org.gjt.mm.mysql.MysqlIO.sqlQuery(Unknown Source)
at org.gjt.mm.mysql.Connection.execSQL(Unknown Source)
at org.gjt.mm.mysql.Connection.execSQL(Unknown Source)
at org.gjt.mm.mysql.Statement.executeQuery(Unknown Source)
at org.gjt.mm.mysql.jdbc2.Statement.executeQuery(Unknown Source)
at
graduate_school.SQLconnectionBean.sqlQuery(SQLconnectionBean.java:66)
at org.apache.jsp.go$jsp._jspService(go$jsp.java:289)
at
org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:107)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:853)
at
org.apache.jasper.servlet.JspServlet$JspServletWrapper.service(JspServle
t.java:201)
at
org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:381)
at
org.apache.jasper.servlet.JspServlet.service(JspServlet.java:473)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:853)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Applica
tionFilterChain.java:247)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilt
erChain.java:193)
at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValv
e.java:243)
at
org.apache.catalina.core.StandardPipeline.invokeNext(StandardPipeline.ja
va:566)
at
org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:4
72)
at
org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:943)
at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValv
e.java:190)
at
org.apache.catalina.core.StandardPipeline.invokeNext(StandardPipeline.ja
va:566)
at
org.apache.catalina.valves.CertificatesValve.invoke(CertificatesValve.ja
va:246)
at
org.apache.catalina.core.StandardPipeline.invokeNext(StandardPipeline.ja
va:564)
at
org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:4
72)
at
org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:943)
at
org.apache.catalina.core.StandardContext.invoke(StandardContext.java:234
7)
at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java
:180)
at
org.apache.catalina.core.StandardPipeline.invokeNext(StandardPipeline.ja
va:566)
at
org.apache.catalina.valves.ErrorDispatcherValve.invoke(ErrorDispatcherVa
lve.java:170)
at
org.apache.catalina.core.StandardPipeline.invokeNext(StandardPipeline.ja
va:564)
at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java
:170)
at
org.apache.catalina.core.StandardPipeline.invokeNext(StandardPipeline.ja
va:564)
at
org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:468
)
at
org.apache.catalina.core.StandardPipeline.invokeNext(StandardPipeline.ja
va:564)
at
org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:4
72)
at
org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:943)
at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.
java:174)
at
org.apache.catalina.core.StandardPipeline.invokeNext(StandardPipeline.ja
va:566)
at
org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:4
72)
at
org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:943)
at
org.apache.ajp.tomcat4.Ajp13Processor.process(Ajp13Processor.java:458)
at
org.apache.ajp.tomcat4.Ajp13Processor.run(Ajp13Processor.java:551)
at java.lang.Thread.run(Thread.java:536)
****************
-----Original Message-----
From: Steven J. Owens [mailto:[EMAIL PROTECTED]
Sent: Dienstag, 9. November 2004 01:53
To: Tomcat Users List
Cc: [EMAIL PROTECTED]
Subject: Re: How to fix: Tomcat responce time gets to 10-200 seconds
once a day
Roman Zhovtulya [mailto:[EMAIL PROTECTED] asked:
> > Normally the performance is really good, but about once a day it
> > gets really slow (around 1-5 minutes to display the page). It also
> > recovers autocatically to normal response time when you simply wait
> > 5-10 minutes.
On Mon, Nov 08, 2004 at 11:16:55PM -0000, Steve Kirk wrote:
> My favourite answer to any query about a server being painfully slow:
> does the slowdown coincide with a slowdown or outage of your DNS
> service? This can have suprisingly big impact if TC is doing reverse
> lookups of the IP of every request.
>
> At the risk of asking the obvious, are you sure that some other heavy
> process is not kicking off at that time, either on one of the machines
> you mention, or some other machine on the network?
If it's not external load (as SteveK suggests), nor internal load,
it might be resource contention.
1) how many apache processes do you have?
I had a problem on a redhat box, apparently the default apache install
was 10 processes. Needless to say, as soon as the traffic went up a
bit, all 10 processes were busy and subsequent requests had to wait for
a free process - and wait, and wait, and wait...
Does each page load actually load, or does the browser time out while
trying to get a connection?
If the page takes 5 minutes but actually does load (and if the delay is
not because it's trying to shove a lot of data into the page), then it's
probably *not* a lack of apache processes. But it's easy enough to
check on, so check on it anyway.
2) If it's not contention for apache processes, it might be some sort of
contention inside tomcat - either a thread locking issue somewhere, or
getting at the database, or some other resource.
3) It may not be actual load, but more time delays. For example,
anything in your system that hits an outsid server (message queue,
email) may take a couple of seconds to complete. All of those secodns
add up. They may also lead to resource contention.
One of my earlier servlet projects, the "super high reliability" message
queue (MQ Series - this was pre-JMS) performed as advertised... but the
"super high reliability" big iron it was connecting to was shut down for
a couple hours of scheduled maintenance every sunday morning at 1am
(which, of course, I found out about one sunday morning at 2am). The
message queue's default timeout was kind of high (25 seconds), so the
thread just sat there waiting for the answer, holding open one of the
apache processes all the while. Soon enough, all the apache processes
were busy waiting for answers from the tomcat threads, which were
waiting for answers from their message queue requests...
The right way to resolve this sort of problem depends a lot on the
circumstances, so look into that and post here with further questions.
4) Just a wild idea, but check on garbage-collection. Maybe put a
couple of logging calls in the pages, so you can see the memory usage in
the log:
Runtime.getRuntime().maxMemory() // maximum memory allocation
Runtime.getRuntime().totalMemory() // memory actually allocated by JVM
Runtime.getRuntime().freeMemory() // allocated but not in use by
objects
The gotcha to watch out for is that the system will quite happily
allocate memory without doing garbage-collection, until it hits
maxMemory(). Then it'll do garbage collection, but it may take a while.
You can't really eliminate the delay, but you can move it around and/or
spread it out. There are command-line arguments to tweak how garbage
collection is done. At the time I learned this lesson, I don't think
those command-line arguments were available, so I used a call to
System.gc() instead. But I'd strongly suggest looking into them before
you go mucking with System.gc().
System.gc() isn't guaranteed to do anything. Whether it does anything
or not depends on your platform, but it's supposed to sort of "nudge"
the garbage collector and say "now might be a good time to clean up." If
you have any sort of consistent spot in your app where a bunch of
objects become superfluous (in the one where I learned this lesson, it
was an object cache that expired objects every minute), putting in a
call to System.gc() might really even out the memory usage.
5) Do some very rough thumbnail profiling; print in
System.currentTimeInMillis() to the log at a number of strategic points.
Then after a slowdown, look at the log, try to trace a single request
through the process and see where the big delay was, then take a
microscope to that portion of the application.
--
Steven J. Owens
[EMAIL PROTECTED]
"I'm going to make broad, sweeping generalizations and strong,
declarative statements, because otherwise I'll be here all night and
this document will be four times longer and much less fun to read. Take
it all with a grain of salt." - http://darksleep.com/notablog
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
