Paul Procacci wrote:
Hey all,
This question is probably best suited for tomcat-users, but being this
is an a FBSD machine I'd figure I'd ask here first. I have a client
running apache w/ tomcat serving thousands of requests per second. It
works, but with one caveat, pages load extremely slowly. When I first
saw the request, I figured this client was running into some kind of
resource bottleneck, but that's where my problems started. I'm unable
to find any resource that is being starved. I turn to `the mighty
list` for guidance.
This particular machine is running FreeBSD 6.2. It has a bce network
card that currently connected to a GigE switch with a 10G uplink.
Even with all this available bandwidth, this client's application
*currently* is not doing any more than 50Mb/sec. :: Rule out network
congestion.
Moving along to the system ram :: 2 Gigs are in the machine, with very
little usage:
Mem: 724M Active, 874M Inact, 338M Wired, 64M Cache, 112M Buf, 4944K Free
A dual core cpu is in the machine, again with very little usage:
CPU states: 8.6% user, 0.0% nice, 3.9% system, 2.8% interrupt,
84.7% idle
Disk throughput is negligable at this time (50KB/sec :: 3tps).
I did start off mentioning this machine uses tomcat...so let me
continue. We've setup the java process to use libthr via libmap.conf:
#############
cat /etc/libmap.conf
[java]
libpthread.so.2 libthr.so.2
libpthread.so libthr.so
#############
And added the following to rc.conf::
###############
tomcat55_java_opts="-XX:MaxPermSize=512m -Xmx512M"
###############
Apache Configuration:
###################
MaxClients 1024
###################
Tomcat Configuration:
###################
maxThreads="1200"
###################
----------------------------------------------------------------
So onto my question...the slowness being encountered. My initial
thought was the slowness was due to quite possibly the cpu being
hammered by numerous requests. After checking however that's not the
case, then I looked at the networking equipment with my networking
team, that's ok too. (I think) That leaves a) thread contention
somewhere b) apache misconfiguration and c) tomcat configuration.
Both myself and 2 other admins have looked over the apache
configuration and tomcat configuration and we believe that side of
things is probably ok. That's leaves weird contention in the kernel
or userland mutexes or something along those lines.
Here is our current connection count on the external interface ::
################################
sockstat -4cp 80 | wc -l
997
################################
Here is our connection count on localhost (from apache to tomcat
process and vica versa) ::
################################
sockstat -4cp 8009 | wc -l
1679
################################
----------------------------------------------------------------
And finally.....top output:
last pid: 57747; load averages: 0.79, 1.12,
0.87
up 153+17:56:53 18:36:57
<#--snip -->
PID USERNAME PRI NICE SIZE RES STATE C TIME WCPU COMMAND
57460 www 96 0 1385M 432M ucond 0 2:44 2.88% java
57460 www 96 0 1385M 432M ucond 0 2:44 2.73% java
57460 www 96 0 1385M 432M select 0 2:44 0.15% java
57460 www 96 0 1385M 432M select 1 2:44 0.15% java
<#--snip-->
The above lines repeat up to the number of threads we have. `states`
are in ucond and select.
----------------------------------------------------------------
Hopefully I hven't bored anyone....yet, and would appreciate any
guildance. Maybe good tools for debugged kernel threads, or a simple
slap in the face will do nicely.
Thanks in advance,
Paul
_______________________________________________
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to
"[EMAIL PROTECTED]"
List,
Apologies, but I sever my head in shame as I have found the problem.
In one word: KeepAlives.
Thanks!
_______________________________________________
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "[EMAIL PROTECTED]"
