RE: JNDI threads BLOCKED - server falling over

2010-11-09 Thread Gaulin, Mark

Hi
Those threads are not your problem... They represent server threads
that are waiting for someone to open a socket to them.  (JNDI is built
into java; CF might be using it but either way it isn't directly hurting
anything.) 

If your dead/hung servers are using lots of cpu then look for threads
that are not blocked... They will be much more interesting.  If it isn't
a cpu issue then look for threads not blocked on ServerSocket.accept
and you may find a pattern there... Maybe a database or some other kind
of resource, or a java-level synchronization issue where one thread is
waiting for something another thread has (but will never give up).  

Thanks
Mark



-Original Message-
From: O Hilton [mailto:ojh...@gmail.com] 
Sent: Tuesday, November 09, 2010 11:33 AM
To: cf-talk
Subject: JNDI threads BLOCKED - server falling over


Hi everyone, this one has had our collective heads banging against a
wall for longer than any of us would like to admit. Bit of back story,
our site has been having long 'pauses' and sometimes total downtime now
for months and months, sometimes it will be fine for weeks and sometimes
it'll go down every couple of hours. We've looked at every possible
aspect of the code, made numerous improvements, in the process speeding
up our site no end (when its up!) but still it goes down.

Much investigation over much time has lead to the conclusion its
something thread based, most recently we find that as its going down we
have a number of these in the thread dump:

jndi-7 prio=5 tid=1477 BLOCKED
at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:388)
at java.net.ServerSocket.implAccept(ServerSocket.java:453)
at java.net.ServerSocket.accept(ServerSocket.java:421)
at jrun.servlet.network.NetworkService.accept(NetworkService.java:356)
at
jrun.naming.NamingNetworkService.accept(NamingNetworkService.java:115)
at
jrun.naming.NamingNetworkService.createRunnable(NamingNetworkService.jav
a:141)
at
jrunx.scheduler.ThreadPool$ThreadThrottle.createRunnable(ThreadPool.java
:349)
at jrunx.scheduler.WorkerThread.run(WorkerThread.java:62)

Always a 'jndi-X' thread and always blocked, the more there are the
slower/nearer to death the box seems to be. So the question I have yet
to get a decent answer to is, what are these threads? What's creating
them? Why are they waiting (as it looks) for some kind of incoming
connection? and most importantly what the heck can we do about them?!
Any and all suggestions greatly welcomed!

Thanks! 



~|
Order the Adobe Coldfusion Anthology now!
http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion
Archive: 
http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:338997
Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm
Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm


RE: JNDI threads BLOCKED - server falling over

2010-11-09 Thread Brook Davies

Have you tried either fusionReactor or seeFusion? So you can see whats
hanging?

Brook

-Original Message-
From: O Hilton [mailto:ojh...@gmail.com] 
Sent: November-09-10 8:33 AM
To: cf-talk
Subject: JNDI threads BLOCKED - server falling over


Hi everyone, this one has had our collective heads banging against a wall
for longer than any of us would like to admit. Bit of back story, our site
has been having long 'pauses' and sometimes total downtime now for months
and months, sometimes it will be fine for weeks and sometimes it'll go down
every couple of hours. We've looked at every possible aspect of the code,
made numerous improvements, in the process speeding up our site no end (when
its up!) but still it goes down.

Much investigation over much time has lead to the conclusion its something
thread based, most recently we find that as its going down we have a number
of these in the thread dump:

jndi-7 prio=5 tid=1477 BLOCKED
at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:388)
at java.net.ServerSocket.implAccept(ServerSocket.java:453)
at java.net.ServerSocket.accept(ServerSocket.java:421)
at jrun.servlet.network.NetworkService.accept(NetworkService.java:356)
at jrun.naming.NamingNetworkService.accept(NamingNetworkService.java:115)
at
jrun.naming.NamingNetworkService.createRunnable(NamingNetworkService.java:14
1)
at
jrunx.scheduler.ThreadPool$ThreadThrottle.createRunnable(ThreadPool.java:349
)
at jrunx.scheduler.WorkerThread.run(WorkerThread.java:62)

Always a 'jndi-X' thread and always blocked, the more there are the
slower/nearer to death the box seems to be. So the question I have yet to
get a decent answer to is, what are these threads? What's creating them? Why
are they waiting (as it looks) for some kind of incoming connection? and
most importantly what the heck can we do about them?! Any and all
suggestions greatly welcomed!

Thanks! 



~|
Order the Adobe Coldfusion Anthology now!
http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion
Archive: 
http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:339029
Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm
Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm