Lock occures in jdbc driver when a server is unplugged
------------------------------------------------------
Key: SEQUOIA-1047
URL: https://forge.continuent.org/jira/browse/SEQUOIA-1047
Project: Sequoia
Type: Bug
Components: JDBC Driver
Versions: sequoia 2.10.10
Environment: Jboss 4.0.3, sequoia, Mysql
Reporter: Philippe Mioulet
While performing a serie of failover tests we unpluged a sequoia node. We
noticed that our server was blocked(ie it was no longer able to access to the
database).
A jstack of the jboss server allowed us to notice that the connections to the
sequoia driver are blocked. A further analysis of the jstack showed that most
of the driver threads are in the state BLOCKED, ie they are trying to get a
lock on an object, this object being the connection policy.
Here is the 3 steps were threads were blocked and the associated source code :
-
org.continuent.sequoia.driver.connectpolicy.AbstractControllerConnectPolicy.numberOfAliveControllers()
@bci=0, line=251 (Interpreted frame)
249 public synchronized int numberOfAliveControllers()
250 {
251 return aliveControllers.size();
252 }
-
org.continuent.sequoia.driver.connectpolicy.PreferredListConnectPolicy.controllerDown(org.continuent.sequoia.driver.ControllerInfo)
@bci=0, line=145 (Interpreted frame)
143 public synchronized void controllerDown(ControllerInfo controller)
144 {
145 super.controllerDown(controller);
146 while (preferredControllers.remove(controller))
147 ;
148 }
-
org.continuent.sequoia.driver.Driver.connectToNextController(org.continuent.sequoia.driver.SequoiaUrl)
@bci=76, line=488 (Interpreted frame)
456 private Connection connectToNextController(SequoiaUrl sequoiaUrl)
457 throws AuthenticationException, NoMoreControllerException,
458 VirtualDatabaseUnavailableException, GeneralSecurityException
459 {
...
485 synchronized (policy)
486 {
487 // Choose a controller according to the policy
488 newController = policy.getController();
=> One of the sequoia threads that was not blocked was the following :
Thread 32470: (state = IN_NATIVE)
- java.net.PlainSocketImpl.socketConnect(java.net.InetAddress, int,
int) @bci=0 (Interpreted frame)
- java.net.PlainSocketImpl.doConnect(java.net.InetAddress, int, int)
@bci=10, line=333 (Interpreted frame)
- java.net.PlainSocketImpl.connectToAddress(java.net.InetAddress, int,
int) @bci=23, line=195 (Interpreted frame)
- java.net.PlainSocketImpl.connect(java.net.SocketAddress, int)
@bci=71, line=182 (Interpreted frame)
- java.net.SocksSocketImpl.connect(java.net.SocketAddress, int)
@bci=332, line=366 (Interpreted frame)
- java.net.Socket.connect(java.net.SocketAddress, int) @bci=165,
line=519 (Interpreted frame)
- java.net.Socket.connect(java.net.SocketAddress) @bci=3, line=469
(Interpreted frame)
-
org.continuent.sequoia.driver.Driver.connectToNextController(org.continuent.sequoia.driver.SequoiaUrl)
@bci=180, line=523 (Interpreted frame)
456 private Connection connectToNextController(SequoiaUrl sequoiaUrl)
457 throws AuthenticationException, NoMoreControllerException,
458 VirtualDatabaseUnavailableException, GeneralSecurityException
459 {
...
485 synchronized (policy)
486 {
523 socket.connect(newController);
524 }
As you may notice a timeout is not set on the connect. When we do not set it,
java sets it to 0 ie an infinite timeout. As this thread has previously aquired
the lock it is probably this thread that is blocking all the requests. I
suggest a timeout should be set.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
https://forge.continuent.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
_______________________________________________
Sequoia mailing list
[email protected]
https://forge.continuent.org/mailman/listinfo/sequoia
