OS: Solaris 10 JVM: 1.7.0_45 H2: 1.4.181 BoneCP ( Connection Pool ) I am running in Automatic mixed mode with the following connection url... jdbc:h2:../data/audit; COMPRESS=TRUE;AUTOCOMMIT=ON;MVCC=TRUE;DB_CLOSE_ON_EXIT=FALSE;MV_STORE=FALSE;AUTO_SERVER=TRUE;AUTO_SERVER_PORT=8701
Not sure what I can do to help debug this, but this is what happened.. My applications are very active. They are constantly reading and writing to the database. In this case reading and caching small files to the database via a CLOB database type. Simplistic description: One application inserts records to the database ( caches small files to CLOBs ) and the other application reads the CLOBs and process them. The application that is running H2 embedded is random. Today the application just stopped processing out of the blue and jstack indicates that the H2 classes are stuck alternating between a socketRead / socketWrite call. I shutdown the application that was inserting the CLOB records and I let the hung one spin its wheels for an hour. I then I tried to shutdown the application. The application is now hung closing the H2 database ( been another 1.5 hours still shutting down ). Any ideas what could cause this? I will attach three different stack traces from jstack 1. snap shot while application hung ( socketWrite call ). 2. snap shot short time later while application hung ( socketRead call ). 3. snap shot of shutdown classes hung. 1-------------------- > "directory-poller-1" prio=3 tid=0x0087f800 nid=0x53 runnable [0xa4a8f000] > java.lang.Thread.State: RUNNABLE > at java.net.SocketOutputStream.socketWrite0(Native Method) > at > java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:113) > at java.net.SocketOutputStream.write(SocketOutputStream.java:159) > at > java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82) > at > java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140) > - locked <0xbe77ddb0> (a java.io.BufferedOutputStream) > at java.io.DataOutputStream.flush(DataOutputStream.java:123) > at org.h2.value.Transfer.flush(Transfer.java:94) > at org.h2.engine.SessionRemote.done(SessionRemote.java:614) > at > org.h2.engine.SessionRemote.setAutoCommitSend(SessionRemote.java:251) > at > org.h2.engine.SessionRemote.setAutoCommit(SessionRemote.java:226) > at > org.h2.jdbc.JdbcConnection.setAutoCommit(JdbcConnection.java:432) > - locked <0xbe786f10> (a org.h2.jdbc.JdbcConnection) > at > com.jolbox.bonecp.ConnectionHandle.setAutoCommit(ConnectionHandle.java:1292) > at > com.jolbox.bonecp.ConnectionHandle.<init>(ConnectionHandle.java:259) > at > com.jolbox.bonecp.ConnectionHandle.recreateConnectionHandle(ConnectionHandle.java:281) > at > com.jolbox.bonecp.ConnectionHandle.close(ConnectionHandle.java:512) > at com.issinc.cds.db.DBUtils.closeQuietly(DBUtils.java:123) > at > com.issinc.cds.sb.mfwd.impl.FwdMsgQueueManager.getMessage(FwdMsgQueueManager.java:396) > at > com.issinc.cds.sbsend.input.poller.MessageForwardDirPollerRenamer.rename(MessageForwardDirPollerRenamer.java:104) > > > > > 2-------------------- > "directory-poller-1" prio=3 tid=0x0087f800 nid=0x53 runnable [0xa4a8f000] > java.lang.Thread.State: RUNNABLE > at java.net.SocketInputStream.socketRead0(Native Method) > at java.net.SocketInputStream.read(SocketInputStream.java:152) > at java.net.SocketInputStream.read(SocketInputStream.java:122) > at java.io.BufferedInputStream.fill(BufferedInputStream.java:235) > at java.io.BufferedInputStream.read(BufferedInputStream.java:254) > - locked <0xbe770910> (a java.io.BufferedInputStream) > at java.io.DataInputStream.readInt(DataInputStream.java:387) > at org.h2.value.Transfer.readInt(Transfer.java:154) > at org.h2.engine.SessionRemote.done(SessionRemote.java:615) > at org.h2.command.CommandRemote.prepare(CommandRemote.java:68) > at org.h2.command.CommandRemote.<init>(CommandRemote.java:45) > at org.h2.engine.SessionRemote.prepareCommand(SessionRemote.java:492) > - locked <0xbe770718> (a org.h2.engine.SessionRemote) > at org.h2.jdbc.JdbcConnection.prepareCommand(JdbcConnection.java:1189) > at org.h2.jdbc.JdbcStatement.executeQuery(JdbcStatement.java:74) > - locked <0xbe770718> (a org.h2.engine.SessionRemote) > at > com.jolbox.bonecp.StatementHandle.executeQuery(StatementHandle.java:464) > at > com.issinc.cds.sb.mfwd.impl.FwdMsgQueueManager.getMessage(FwdMsgQueueManager.java:356) > at > com.issinc.cds.sbsend.input.poller.MessageForwardDirPollerRenamer.rename(MessageForwardDirPollerRenamer.java:104) > > > > > > 3-------------------- > "H2 TCP Server (tcp://172.16.1.14:7801) thread" daemon prio=3 > tid=0x00e9d800 nid=0x77 runnable [0xa3aff000] > java.lang.Thread.State: RUNNABLE > at java.net.SocketInputStream.socketRead0(Native Method) > at java.net.SocketInputStream.read(SocketInputStream.java:152) > at java.net.SocketInputStream.read(SocketInputStream.java:122) > at java.io.BufferedInputStream.fill(BufferedInputStream.java:235) > at java.io.BufferedInputStream.read(BufferedInputStream.java:254) > - locked <0xc38a2af8> (a java.io.BufferedInputStream) > at java.io.DataInputStream.readInt(DataInputStream.java:387) > at org.h2.value.Transfer.readInt(Transfer.java:154) > at org.h2.server.TcpServerThread.process(TcpServerThread.java:257) > at org.h2.server.TcpServerThread.run(TcpServerThread.java:159) > at java.lang.Thread.run(Thread.java:744) > > "H2 TCP Server (tcp://172.16.1.14:7801) thread" daemon prio=3 > tid=0x00b22c00 nid=0x75 runnable [0xa3b8f000] > java.lang.Thread.State: RUNNABLE > at java.net.SocketInputStream.socketRead0(Native Method) > at java.net.SocketInputStream.read(SocketInputStream.java:152) > at java.net.SocketInputStream.read(SocketInputStream.java:122) > at java.io.BufferedInputStream.fill(BufferedInputStream.java:235) > at java.io.BufferedInputStream.read(BufferedInputStream.java:254) > - locked <0xc38a24d0> (a java.io.BufferedInputStream) > at java.io.DataInputStream.readInt(DataInputStream.java:387) > at org.h2.value.Transfer.readInt(Transfer.java:154) > at org.h2.server.TcpServerThread.process(TcpServerThread.java:257) > at org.h2.server.TcpServerThread.run(TcpServerThread.java:159) > at java.lang.Thread.run(Thread.java:744) > > "H2 TCP Server (tcp://172.16.1.14:7801) thread" daemon prio=3 > tid=0x00b22800 nid=0x73 runnable [0xa3c1f000] > java.lang.Thread.State: RUNNABLE > at java.net.SocketInputStream.socketRead0(Native Method) > at java.net.SocketInputStream.read(SocketInputStream.java:152) > at java.net.SocketInputStream.read(SocketInputStream.java:122) > at java.io.BufferedInputStream.fill(BufferedInputStream.java:235) > at java.io.BufferedInputStream.read(BufferedInputStream.java:254) > - locked <0xc38ef390> (a java.io.BufferedInputStream) > at java.io.DataInputStream.readInt(DataInputStream.java:387) > at org.h2.value.Transfer.readInt(Transfer.java:154) > at org.h2.server.TcpServerThread.process(TcpServerThread.java:257) > at org.h2.server.TcpServerThread.run(TcpServerThread.java:159) > at java.lang.Thread.run(Thread.java:744) > > "H2 TCP Server (tcp://172.16.1.14:7801) thread" daemon prio=3 > tid=0x0136c400 nid=0x71 runnable [0xa3caf000] > java.lang.Thread.State: RUNNABLE > at java.net.SocketInputStream.socketRead0(Native Method) > at java.net.SocketInputStream.read(SocketInputStream.java:152) > at java.net.SocketInputStream.read(SocketInputStream.java:122) > at java.io.BufferedInputStream.fill(BufferedInputStream.java:235) > at java.io.BufferedInputStream.read(BufferedInputStream.java:254) > - locked <0xc3887f70> (a java.io.BufferedInputStream) > at java.io.DataInputStream.readInt(DataInputStream.java:387) > at org.h2.value.Transfer.readInt(Transfer.java:154) > at org.h2.server.TcpServerThread.process(TcpServerThread.java:257) > at org.h2.server.TcpServerThread.run(TcpServerThread.java:159) > at java.lang.Thread.run(Thread.java:744) > > "H2 Log Writer MSGFWD" daemon prio=3 tid=0x0136b800 nid=0x6f in > Object.wait() [0xa3d3f000] > java.lang.Thread.State: TIMED_WAITING (on object monitor) > at java.lang.Object.wait(Native Method) > - waiting on <0xc385d370> (a org.h2.store.WriterThread) > at org.h2.store.WriterThread.run(WriterThread.java:102) > - locked <0xc385d370> (a org.h2.store.WriterThread) > at java.lang.Thread.run(Thread.java:744) > > "H2 TCP Server (tcp://172.16.1.14:7801)" daemon prio=3 tid=0x006c1400 > nid=0x6d runnable [0xa3dcf000] > java.lang.Thread.State: RUNNABLE > at java.net.PlainSocketImpl.socketAccept(Native Method) > at > java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:398) > at java.net.ServerSocket.implAccept(ServerSocket.java:530) > at java.net.ServerSocket.accept(ServerSocket.java:498) > at org.h2.server.TcpServer.listen(TcpServer.java:250) > at org.h2.tools.Server.run(Server.java:568) > at java.lang.Thread.run(Thread.java:744) > > "H2 File Lock Watchdog /opt/cds/dataSyncPlus_h2/data/audit/msgfwd.lock.db" > daemon prio=3 tid=0x00c32000 nid=0x6c waiting on condition [0xa416f000] > java.lang.Thread.State: TIMED_WAITING (sleeping) > at java.lang.Thread.sleep(Native Method) > at org.h2.store.FileLock.run(FileLock.java:516) > at java.lang.Thread.run(Thread.java:744) > > "directory-poller-1" prio=3 tid=0x0087f800 nid=0x53 runnable [0xa4a8f000] > java.lang.Thread.State: RUNNABLE > at java.net.SocketOutputStream.socketWrite0(Native Method) > at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:113) > at java.net.SocketOutputStream.write(SocketOutputStream.java:159) > at > java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82) > at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140) > - locked <0xc38a32b8> (a java.io.BufferedOutputStream) > at java.io.DataOutputStream.flush(DataOutputStream.java:123) > at org.h2.value.Transfer.flush(Transfer.java:94) > at org.h2.engine.SessionRemote.done(SessionRemote.java:614) > at org.h2.command.CommandRemote.prepare(CommandRemote.java:68) > at org.h2.command.CommandRemote.<init>(CommandRemote.java:45) > at org.h2.engine.SessionRemote.prepareCommand(SessionRemote.java:492) > - locked <0xbe74f160> (a org.h2.engine.SessionRemote) > at org.h2.jdbc.JdbcConnection.prepareCommand(JdbcConnection.java:1189) > at org.h2.jdbc.JdbcStatement.executeQuery(JdbcStatement.java:74) > - locked <0xbe74f160> (a org.h2.engine.SessionRemote) > at > com.jolbox.bonecp.StatementHandle.executeQuery(StatementHandle.java:464) > at > com.issinc.cds.sb.mfwd.impl.FwdMsgQueueManager.getMessage(FwdMsgQueueManager.java:356) > at > com.issinc.cds.sbsend.input.poller.MessageForwardDirPollerRenamer.rename(MessageForwardDirPollerRenamer.java:104) > -- You received this message because you are subscribed to the Google Groups "H2 Database" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/h2-database. For more options, visit https://groups.google.com/d/optout.
